i.MX Media Driver
Subsystems (11)
Steve Longerbeam
maintainerv8 cover i.MX Media Driver
In version 8:
- Switched to v4l2_fwnode APIs.
- Always pass a valid CSI id to ipu_set_ic_src_mux() in imx-ic-prp, even if the IC is receiving from the VDIC. The reason is due to a bug in the i.MX6 reference manual: from experiment it is determined that the CSI id select bit in IPU_CONF register selects which CSI is routed to either the VDIC or the IC, and is independent of whether the IC is set to receive from a CSI or the VDIC. Sugested by Marek Vasut <[email protected]>.
- ov5640: propagate error codes from all i2c register accesses. Sugested by Sakari Ailus <[email protected]>.
- ov5640: drop the entity stream count check in ov5640_s_stream(). Sugested by Sakari Ailus <[email protected]>.
- ov5640: Fix manual exposure control. The manual exposure setting (in line periods) cannot exceed the max exposure value in registers {0x380E, 0x380F} + {0x350C,0x350D}, however the max eposure value was not being calcualted correctly.
- ov5640: the video mode register tables require auto gain/exposure be disabled before programming the register set. However auto gain/exp was being disabled by direct register write. This caused the auto gain/exp control values to be inconsistent with the actual hardware setting. Fixed by going through v4l2-ctrl when disabling auto gain/exp.
- ov5640: converted virtual channel macro to a module parameter, default to channel 0.
- ov5640: override the v4l2-ctl lock to use the ov5640 subdev driver's lock. Sugested by Sakari Ailus <[email protected]>.
- ov5640: switch to unit-less V4L2_CID_EXPOSURE. Using V4L2_CID_EXPOSURE_ABSOLUTE will require converting from 100-usec units to line periods and vice-versa. Sugested by Sakari Ailus and Pavel Machek <[email protected]>.
- ov5640: drop dangling regulator_bulk_disable() from probe/remove. Sugested by Sakari Ailus <[email protected]>.
- FIM: move input capture channel selection out of the device-tree and make this a V4L2 control. In order to support attaching a FIM to prpencvf, the FIM cannot have any device-tree configuration, because prpencvf has no device node. The FIM now is completely configurable via its V4L2 controls.
- FIM: drop imx_media_fim_set_power(), and move the input capture channel request to imx_media_fim_set_stream(). This allows to drop csi_s_power() as well, since the latter only called imx_media_fim_set_power().
- FIM: add a spinlock to protect the frame_interval_monitor() from the setting of new control values. The frame_interval_monitor() is called from interrupt context so a spinlock must be used.
- Updated to version 8 video-mux patchset from Philipp Zabel <[email protected]>.
Marek Vasut (1):
media: imx: Drop warning upon multiple S_STREAM disable calls
Philipp Zabel (8):
dt-bindings: Add bindings for video-multiplexer device
ARM: dts: imx6qdl: add multiplexer controls
ARM: dts: imx6qdl: Add video multiplexers, mipi_csi, and their
connections
add mux and video interface bridge entity functions
platform: add video-multiplexer subdevice driver
media: imx: csi: increase burst size for YUV formats
media: imx: csi: add frame skipping support
media: imx: csi: add sink selection rectangles
Russell King (3):
media: imx: csi: add support for bayer formats
media: imx: csi: add frame size/interval enumeration
media: imx: capture: add frame sizes/interval enumeration
Steve Longerbeam (22):
[media] dt-bindings: Add bindings for i.MX media driver [media] dt/bindings: Add bindings for OV5640
ARM: dts: imx6qdl: Add compatible, clocks, irqs to MIPI CSI-2 node
ARM: dts: imx6qdl: add capture-subsystem device
ARM: dts: imx6qdl-sabrelite: remove erratum ERR006687 workaround
ARM: dts: imx6-sabrelite: add OV5642 and OV5640 camera sensors
ARM: dts: imx6-sabresd: add OV5642 and OV5640 camera sensors
ARM: dts: imx6-sabreauto: create i2cmux for i2c3
ARM: dts: imx6-sabreauto: add reset-gpios property for max7310_b
ARM: dts: imx6-sabreauto: add pinctrl for gpt input capture
ARM: dts: imx6-sabreauto: add the ADV7180 video decoder [media] add Omnivision OV5640 sensor driver
media: Add userspace header file for i.MX
media: Add i.MX media core driver
media: imx: Add a TODO file
media: imx: Add Capture Device Interface
media: imx: Add CSI subdev driver
media: imx: Add VDIC subdev driver
media: imx: Add IC subdev drivers
media: imx: Add MIPI CSI-2 Receiver subdev driver
media: imx: set and propagate default field, colorimetry
ARM: imx_v6_v7_defconfig: Enable staging video4linux drivers
.../devicetree/bindings/media/i2c/ov5640.txt | 45 + Documentation/devicetree/bindings/media/imx.txt | 47 + .../devicetree/bindings/media/video-mux.txt | 60 + Documentation/media/uapi/mediactl/media-types.rst | 21 + Documentation/media/v4l-drivers/imx.rst | 614 +++++ arch/arm/boot/dts/imx6dl-sabrelite.dts | 5 + arch/arm/boot/dts/imx6dl-sabresd.dts | 5 + arch/arm/boot/dts/imx6dl.dtsi | 189 ++ arch/arm/boot/dts/imx6q-sabrelite.dts | 5 + arch/arm/boot/dts/imx6q-sabresd.dts | 5 + arch/arm/boot/dts/imx6q.dtsi | 125 ++ arch/arm/boot/dts/imx6qdl-sabreauto.dtsi | 136 +- arch/arm/boot/dts/imx6qdl-sabrelite.dtsi | 152 +- arch/arm/boot/dts/imx6qdl-sabresd.dtsi | 114 +- arch/arm/boot/dts/imx6qdl.dtsi | 20 +- arch/arm/configs/imx_v6_v7_defconfig | 11 + drivers/media/i2c/Kconfig | 10 + drivers/media/i2c/Makefile | 1 + drivers/media/i2c/ov5640.c | 2344 ++++++++++++++++++++ drivers/media/platform/Kconfig | 6 + drivers/media/platform/Makefile | 2 + drivers/media/platform/video-mux.c | 334 +++ drivers/staging/media/Kconfig | 2 + drivers/staging/media/Makefile | 1 + drivers/staging/media/imx/Kconfig | 21 + drivers/staging/media/imx/Makefile | 12 + drivers/staging/media/imx/TODO | 23 + drivers/staging/media/imx/imx-ic-common.c | 113 + drivers/staging/media/imx/imx-ic-prp.c | 518 +++++ drivers/staging/media/imx/imx-ic-prpencvf.c | 1309 +++++++++++ drivers/staging/media/imx/imx-ic.h | 38 + drivers/staging/media/imx/imx-media-capture.c | 775 +++++++ drivers/staging/media/imx/imx-media-csi.c | 1816 +++++++++++++++ drivers/staging/media/imx/imx-media-dev.c | 666 ++++++ drivers/staging/media/imx/imx-media-fim.c | 494 +++++ drivers/staging/media/imx/imx-media-internal-sd.c | 349 +++ drivers/staging/media/imx/imx-media-of.c | 270 +++ drivers/staging/media/imx/imx-media-utils.c | 896 ++++++++ drivers/staging/media/imx/imx-media-vdic.c | 1009 +++++++++ drivers/staging/media/imx/imx-media.h | 325 +++ drivers/staging/media/imx/imx6-mipi-csi2.c | 698 ++++++ include/linux/imx-media.h | 29 + include/media/imx.h | 15 + include/uapi/linux/media.h | 6 + include/uapi/linux/v4l2-controls.h | 4 + 45 files changed, 13613 insertions(+), 27 deletions(-) create mode 100644 Documentation/devicetree/bindings/media/i2c/ov5640.txt create mode 100644 Documentation/devicetree/bindings/media/imx.txt create mode 100644 Documentation/devicetree/bindings/media/video-mux.txt create mode 100644 Documentation/media/v4l-drivers/imx.rst create mode 100644 drivers/media/i2c/ov5640.c create mode 100644 drivers/media/platform/video-mux.c create mode 100644 drivers/staging/media/imx/Kconfig create mode 100644 drivers/staging/media/imx/Makefile create mode 100644 drivers/staging/media/imx/TODO create mode 100644 drivers/staging/media/imx/imx-ic-common.c create mode 100644 drivers/staging/media/imx/imx-ic-prp.c create mode 100644 drivers/staging/media/imx/imx-ic-prpencvf.c create mode 100644 drivers/staging/media/imx/imx-ic.h create mode 100644 drivers/staging/media/imx/imx-media-capture.c create mode 100644 drivers/staging/media/imx/imx-media-csi.c create mode 100644 drivers/staging/media/imx/imx-media-dev.c create mode 100644 drivers/staging/media/imx/imx-media-fim.c create mode 100644 drivers/staging/media/imx/imx-media-internal-sd.c create mode 100644 drivers/staging/media/imx/imx-media-of.c create mode 100644 drivers/staging/media/imx/imx-media-utils.c create mode 100644 drivers/staging/media/imx/imx-media-vdic.c create mode 100644 drivers/staging/media/imx/imx-media.h create mode 100644 drivers/staging/media/imx/imx6-mipi-csi2.c create mode 100644 include/linux/imx-media.h create mode 100644 include/media/imx.h
Signature · 1 line
2.7.4
Steve Longerbeam
maintainerPATCH v8 1/34 dt-bindings: Add bindings for video-multiplexer device
From: Philipp Zabel <[email protected]>
Add bindings documentation for the video multiplexer device.
--- .../devicetree/bindings/media/video-mux.txt | 60 ++++++++++++++++++++++ 1 file changed, 60 insertions(+) create mode 100644 Documentation/devicetree/bindings/media/video-mux.txt
Documentation/devicetree/bindings/media/video-mux.txt new file +60 -0
@@ -0,0 +1,60 @@ +Video Multiplexer +================= + +Video multiplexers allow to select between multiple input ports. Video received +on the active input port is passed through to the output port. Muxes described +by this binding are controlled by a multiplexer controller that is described by +the bindings in Documentation/devicetree/bindings/mux/mux-controller.txt + +Required properties: +- compatible : should be "video-mux" +- mux-controls : mux controller node to use for operating the mux +- #address-cells: should be <1> +- #size-cells: should be <0> +- port@*: at least three port nodes containing endpoints connecting to the + source and sink devices according to of_graph bindings. The last port is + the output port, all others are inputs. + +Optionally, #address-cells, #size-cells, and port nodes can be grouped under a +ports node as described in Documentation/devicetree/bindings/graph.txt. + +Example: + + mux: mux-controller { + compatible = "gpio-mux"; + #mux-control-cells = <0>; + + mux-gpios = <&gpio1 15 GPIO_ACTIVE_HIGH>; + }; + + video-mux { + compatible = "video-mux"; + mux-controls = <&mux>; + #address-cells = <1>; + #size-cells = <0>; + + port at 0 { + reg = <0>; + + mux_in0: endpoint { + remote-endpoint = <&video_source0_out>; + }; + }; + + port at 1 { + reg = <1>; + + mux_in1: endpoint { + remote-endpoint = <&video_source1_out>; + }; + }; + + port at 2 { + reg = <2>; + + mux_out: endpoint { + remote-endpoint = <&capture_interface_in>; + }; + }; + }; +};
Signature · 1 line
2.7.4
Steve Longerbeam
maintainerPATCH v8 2/34 dt-bindings: Add bindings for i.MX media driver
In reply to Steve LongerbeamAdd bindings documentation for the i.MX media driver.
--- Documentation/devicetree/bindings/media/imx.txt | 47 +++++++++++++++++++++++++ 1 file changed, 47 insertions(+) create mode 100644 Documentation/devicetree/bindings/media/imx.txt
Documentation/devicetree/bindings/media/imx.txt new file +47 -0
@@ -0,0 +1,47 @@ +Freescale i.MX Media Video Device +================================= + +Video Media Controller node +--------------------------- + +This is the media controller node for video capture support. It is a +virtual device that lists the camera serial interface nodes that the +media device will control. + +Required properties: +- compatible : "fsl,imx-capture-subsystem"; +- ports : Should contain a list of phandles pointing to camera + sensor interface ports of IPU devices + +example: + +capture-subsystem { + compatible = "fsl,imx-capture-subsystem"; + ports = <&ipu1_csi0>, <&ipu1_csi1>; +}; + + +mipi_csi2 node +-------------- + +This is the device node for the MIPI CSI-2 Receiver, required for MIPI +CSI-2 sensors. + +Required properties: +- compatible : "fsl,imx6-mipi-csi2", "snps,dw-mipi-csi2"; +- reg : physical base address and length of the register set; +- clocks : the MIPI CSI-2 receiver requires three clocks: hsi_tx + (the D-PHY clock), video_27m (D-PHY PLL reference + clock), and eim_podf; +- clock-names : must contain "dphy", "ref", "pix"; +- port@* : five port nodes must exist, containing endpoints + connecting to the source and sink devices according to + of_graph bindings. The first port is an input port, + connecting with a MIPI CSI-2 source, and ports 1 + through 4 are output ports connecting with parallel + bus sink endpoint nodes and correspond to the four + MIPI CSI-2 virtual channel outputs. + +Optional properties: +- interrupts : must contain two level-triggered interrupts, + in order: 100 and 101;
Signature · 1 line
2.7.4
Steve Longerbeam
maintainerRe: [PATCH v8 02/34] [media] dt-bindings: Add bindings for i.MX media driver
Hi Rob, Mark,
Are there any remaining technical issues with this binding doc? At this point an Ack from you is the only thing holding up merge of the imx-media driver.
Thanks,
Steve
Quoted 52 lines · Steve Longerbeam
Add bindings documentation for the i.MX media driver.
Signed-off-by Steve Longerbeam <[email protected]>--- Documentation/devicetree/bindings/media/imx.txt | 47 +++++++++++++++++++++++++ 1 file changed, 47 insertions(+)create mode 100644
Documentation/devicetree/bindings/media/imx.txtDocumentation/devicetree/bindings/media/imx.txt new file +37
-0@@ -0,0 +1,47 @@ +Freescale i.MX Media Video Device +================================= + +Video Media Controller node +--------------------------- + +This is the media controller node for video capture support. It is a +virtual device that lists the camera serial interface nodes that the +media device will control. + +Required properties: +- compatible : "fsl,imx-capture-subsystem"; +- ports : Should contain a list of phandles pointing to camera + sensor interface ports of IPU devices + +example: + +capture-subsystem { + compatible = "fsl,imx-capture-subsystem"; + ports = <&ipu1_csi0>, <&ipu1_csi1>; +}; + + +mipi_csi2 node +-------------- + +This is the device node for the MIPI CSI-2 Receiver, required for MIPI +CSI-2 sensors. + +Required properties: +- compatible : "fsl,imx6-mipi-csi2", "snps,dw-mipi-csi2"; +- reg : physical base address and length of the register set; +- clocks : the MIPI CSI-2 receiver requires three clocks: hsi_tx + (the D-PHY clock), video_27m (D-PHY PLL reference + clock), and eim_podf; +- clock-names : must contain "dphy", "ref", "pix"; +- port@* : five port nodes must exist, containing endpoints
Documentation/devicetree/bindings/media/imx.txt line 47
Quoted 10 lines
Documentation/devicetree/bindings/media/imx.txt new file +10
-0+ connecting to the source and sink devices according to + of_graph bindings. The first port is an input port, + connecting with a MIPI CSI-2 source, and ports 1 + through 4 are output ports connecting with parallel + bus sink endpoint nodes and correspond to the four + MIPI CSI-2 virtual channel outputs. + +Optional properties: +- interrupts : must contain two level-triggered interrupts, + in order: 100 and 101;
--
To unsubscribe from this list: send the line "unsubscribe devicetree" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Steve Longerbeam
maintainerRe: [PATCH v8 02/34] [media] dt-bindings: Add bindings for i.MX media driver
Quoted 7 lines · Steve Longerbeam
Hi Rob, Mark,
Are there any remaining technical issues with this binding doc? At this point an Ack from you is the only thing holding up merge of the imx-media driver.
Note that the Synopsys core in the i.MX6 is a differently configured Synopsys core from the core as described in the bindings at [1].
Russell King provided more information on the differences between these cores at [2]. They are essentially different devices.
So perhaps the "snps,dw-mipi-csi2" compatibility needs to be removed from this binding, for now, until this driver is moved to drivers/media/ and is made compatible with other MIPI CSI-2 Synopsys cores with different configurations.
[1] http://patchwork.ozlabs.org/patch/736177/
[2] <[email protected]>
Quoted 65 lines · Steve Longerbeam
Quoted 63 lines
Add bindings documentation for the i.MX media driver.
Signed-off-by Steve Longerbeam <[email protected]>--- Documentation/devicetree/bindings/media/imx.txt | 47 +++++++++++++++++++++++++ 1 file changed, 47 insertions(+)create mode 100644
Documentation/devicetree/bindings/media/imx.txtDocumentation/devicetree/bindings/media/imx.txt new file +47
-0@@ -0,0 +1,47 @@ +Freescale i.MX Media Video Device +================================= + +Video Media Controller node +--------------------------- + +This is the media controller node for video capture support. It is a +virtual device that lists the camera serial interface nodes that the +media device will control. + +Required properties: +- compatible : "fsl,imx-capture-subsystem"; +- ports : Should contain a list of phandles pointing to camera + sensor interface ports of IPU devices + +example: + +capture-subsystem { + compatible = "fsl,imx-capture-subsystem"; + ports = <&ipu1_csi0>, <&ipu1_csi1>; +}; + + +mipi_csi2 node +-------------- + +This is the device node for the MIPI CSI-2 Receiver, required for MIPI +CSI-2 sensors. + +Required properties: +- compatible : "fsl,imx6-mipi-csi2", "snps,dw-mipi-csi2"; +- reg : physical base address and length of the register set; +- clocks : the MIPI CSI-2 receiver requires three clocks: hsi_tx + (the D-PHY clock), video_27m (D-PHY PLL reference + clock), and eim_podf; +- clock-names : must contain "dphy", "ref", "pix"; +- port@* : five port nodes must exist, containing endpoints + connecting to the source and sink devices according to + of_graph bindings. The first port is an input port, + connecting with a MIPI CSI-2 source, and ports 1 + through 4 are output ports connecting with parallel + bus sink endpoint nodes and correspond to the four + MIPI CSI-2 virtual channel outputs. + +Optional properties: +- interrupts : must contain two level-triggered interrupts, + in order: 100 and 101;
Re: [PATCH v8 02/34] [media] dt-bindings: Add bindings for i.MX media driver
Quoted 14 lines · Steve Longerbeam
Quoted 7 lines · Steve Longerbeam
Hi Rob, Mark,
Are there any remaining technical issues with this binding doc? At this point an Ack from you is the only thing holding up merge of the imx-media driver.
Note that the Synopsys core in the i.MX6 is a differently configured Synopsys core from the core as described in the bindings at [1].
Quoted 7 lines
Russell King provided more information on the differences between these cores at [2]. They are essentially different devices.
So perhaps the "snps,dw-mipi-csi2" compatibility needs to be removed from this binding, for now, until this driver is moved to
drivers/media/and is made compatible with other MIPI CSI-2 Synopsys cores with different configurations.
Yes, it probably should be dropped. With that,
Steve Longerbeam
maintainerPATCH v9 2/34 dt-bindings: Add bindings for i.MX media driver
In reply to Steve LongerbeamAdd bindings documentation for the i.MX media driver.
---
Changes since v8 [1]:
- expand on description of the MIPI CSI-2 IP core in i.MX6, and drop "snps,dw-mipi-csi2" compatibility for now.
[1] https://patchwork.linuxtv.org/patch/41717/
--- Documentation/devicetree/bindings/media/imx.txt | 53 +++++++++++++++++++++++++ 1 file changed, 53 insertions(+) create mode 100644 Documentation/devicetree/bindings/media/imx.txt
Documentation/devicetree/bindings/media/imx.txt new file +53 -0
@@ -0,0 +1,53 @@ +Freescale i.MX Media Video Device +================================= + +Video Media Controller node +--------------------------- + +This is the media controller node for video capture support. It is a +virtual device that lists the camera serial interface nodes that the +media device will control. + +Required properties: +- compatible : "fsl,imx-capture-subsystem"; +- ports : Should contain a list of phandles pointing to camera + sensor interface ports of IPU devices + +example: + +capture-subsystem { + compatible = "fsl,imx-capture-subsystem"; + ports = <&ipu1_csi0>, <&ipu1_csi1>; +}; + + +mipi_csi2 node +-------------- + +This is the device node for the MIPI CSI-2 Receiver core in the i.MX +SoC. This is a Synopsys Designware MIPI CSI-2 host controller core +combined with a D-PHY core mixed into the same register block. In +addition this device consists of an i.MX-specific "CSI2IPU gasket" +glue logic, also controlled from the same register block. The CSI2IPU +gasket demultiplexes the four virtual channel streams from the host +controller's 32-bit output image bus onto four 16-bit parallel busses +to the i.MX IPU CSIs. + +Required properties: +- compatible : "fsl,imx6-mipi-csi2"; +- reg : physical base address and length of the register set; +- clocks : the MIPI CSI-2 receiver requires three clocks: hsi_tx + (the D-PHY clock), video_27m (D-PHY PLL reference + clock), and eim_podf; +- clock-names : must contain "dphy", "ref", "pix"; +- port@* : five port nodes must exist, containing endpoints + connecting to the source and sink devices according to + of_graph bindings. The first port is an input port, + connecting with a MIPI CSI-2 source, and ports 1 + through 4 are output ports connecting with parallel + bus sink endpoint nodes and correspond to the four + MIPI CSI-2 virtual channel outputs. + +Optional properties: +- interrupts : must contain two level-triggered interrupts, + in order: 100 and 101;
Signature · 6 lines
2.7.4 -- To unsubscribe from this list: send the line "unsubscribe devicetree" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Steve Longerbeam
maintainerPATCH v8 3/34 dt/bindings: Add bindings for OV5640
In reply to Steve LongerbeamAdd device tree binding documentation for the OV5640 camera sensor.
--- .../devicetree/bindings/media/i2c/ov5640.txt | 45 ++++++++++++++++++++++ 1 file changed, 45 insertions(+) create mode 100644 Documentation/devicetree/bindings/media/i2c/ov5640.txt
Documentation/devicetree/bindings/media/i2c/ov5640.txt new file +45 -0
@@ -0,0 +1,45 @@ +* Omnivision OV5640 MIPI CSI-2 sensor + +Required Properties: +- compatible: should be "ovti,ov5640" +- clocks: reference to the xclk input clock. +- clock-names: should be "xclk". +- DOVDD-supply: Digital I/O voltage supply, 1.8 volts +- AVDD-supply: Analog voltage supply, 2.8 volts +- DVDD-supply: Digital core voltage supply, 1.5 volts + +Optional Properties: +- reset-gpios: reference to the GPIO connected to the reset pin, if any. + This is an active low signal to the OV5640. +- powerdown-gpios: reference to the GPIO connected to the powerdown pin, + if any. This is an active high signal to the OV5640. + +The device node must contain one 'port' child node for its digital output +video port, in accordance with the video interface bindings defined in +Documentation/devicetree/bindings/media/video-interfaces.txt. + +Example: + +&i2c1 { + ov5640: camera at 3c { + compatible = "ovti,ov5640"; + pinctrl-names = "default"; + pinctrl-0 = <&pinctrl_ov5640>; + reg = <0x3c>; + clocks = <&clks IMX6QDL_CLK_CKO>; + clock-names = "xclk"; + DOVDD-supply = <&vgen4_reg>; /* 1.8v */ + AVDD-supply = <&vgen3_reg>; /* 2.8v */ + DVDD-supply = <&vgen2_reg>; /* 1.5v */ + powerdown-gpios = <&gpio1 19 GPIO_ACTIVE_HIGH>; + reset-gpios = <&gpio1 20 GPIO_ACTIVE_LOW>; + + port { + ov5640_to_mipi_csi2: endpoint { + remote-endpoint = <&mipi_csi2_from_ov5640>; + clock-lanes = <0>; + data-lanes = <1 2>; + }; + }; + }; +};
Signature · 1 line
2.7.4
Steve Longerbeam
maintainerPATCH v8 4/34 ARM: dts: imx6qdl: add multiplexer controls
In reply to Steve LongerbeamFrom: Philipp Zabel <[email protected]>
The IOMUXC General Purpose Register space contains various bitfields that control video bus multiplexers. Describe them using a mmio-mux node. The placement of the IPU CSI video mux controls differs between i.MX6D/Q and i.MX6S/DL.
--- arch/arm/boot/dts/imx6dl.dtsi | 10 ++++++++++ arch/arm/boot/dts/imx6q.dtsi | 10 ++++++++++ arch/arm/boot/dts/imx6qdl.dtsi | 7 ++++++- 3 files changed, 26 insertions(+), 1 deletion(-)
arch/arm/boot/dts/imx6dl.dtsi +10 -0
@@ -181,6 +181,16 @@ "di0", "di1"; }; +&mux { + mux-reg-masks = <0x34 0x00000007>, /* IPU_CSI0_MUX */ + <0x34 0x00000038>, /* IPU_CSI1_MUX */ + <0x0c 0x0000000c>, /* HDMI_MUX_CTL */ + <0x0c 0x000000c0>, /* LVDS0_MUX_CTL */ + <0x0c 0x00000300>, /* LVDS1_MUX_CTL */ + <0x28 0x00000003>, /* DCIC1_MUX_CTL */ + <0x28 0x0000000c>; /* DCIC2_MUX_CTL */ +}; + &vpu { compatible = "fsl,imx6dl-vpu", "cnm,coda960"; };
arch/arm/boot/dts/imx6q.dtsi +10 -0
@@ -332,6 +332,16 @@ }; }; +&mux { + mux-reg-masks = <0x04 0x00080000>, /* MIPI_IPU1_MUX */ + <0x04 0x00100000>, /* MIPI_IPU2_MUX */ + <0x0c 0x0000000c>, /* HDMI_MUX_CTL */ + <0x0c 0x000000c0>, /* LVDS0_MUX_CTL */ + <0x0c 0x00000300>, /* LVDS1_MUX_CTL */ + <0x28 0x00000003>, /* DCIC1_MUX_CTL */ + <0x28 0x0000000c>; /* DCIC2_MUX_CTL */ +}; + &vpu { compatible = "fsl,imx6q-vpu", "cnm,coda960"; };
arch/arm/boot/dts/imx6qdl.dtsi +6 -1
@@ -808,8 +808,13 @@ }; gpr: iomuxc-gpr at 020e0000 { - compatible = "fsl,imx6q-iomuxc-gpr", "syscon"; + compatible = "fsl,imx6q-iomuxc-gpr", "syscon", "simple-mfd"; reg = <0x020e0000 0x38>; + + mux: mux-controller { + compatible = "mmio-mux"; + #mux-control-cells = <1>; + }; }; iomuxc: iomuxc at 020e0000 {
Signature · 1 line
2.7.4
Steve Longerbeam
maintainerPATCH v8 5/34 ARM: dts: imx6qdl: Add compatible, clocks, irqs to MIPI CSI-2 node
In reply to Steve LongerbeamAdd to the MIPI CSI2 receiver node: compatible strings, interrupt sources, and clocks.
--- arch/arm/boot/dts/imx6qdl.dtsi | 7 +++++++ 1 file changed, 7 insertions(+)
arch/arm/boot/dts/imx6qdl.dtsi +7 -0
@@ -1140,7 +1140,14 @@ }; mipi_csi: mipi at 021dc000 { + compatible = "fsl,imx6-mipi-csi2", "snps,dw-mipi-csi2"; reg = <0x021dc000 0x4000>; + interrupts = <0 100 0x04>, <0 101 0x04>; + clocks = <&clks IMX6QDL_CLK_HSI_TX>, + <&clks IMX6QDL_CLK_VIDEO_27M>, + <&clks IMX6QDL_CLK_EIM_PODF>; + clock-names = "dphy", "ref", "pix"; + status = "disabled"; }; mipi_dsi: mipi at 021e0000 {
Signature · 1 line
2.7.4
Steve Longerbeam
maintainerPATCH v8 6/34 ARM: dts: imx6qdl: Add video multiplexers, mipi_csi, and their connections
In reply to Steve LongerbeamFrom: Philipp Zabel <[email protected]>
This patch adds the device tree graph connecting the input multiplexers to the IPU CSIs and the MIPI-CSI2 gasket on i.MX6. The MIPI_IPU multiplexers are added as children of the iomuxc-gpr syscon device node. On i.MX6Q/D two two-input multiplexers in front of IPU1 CSI0 and IPU2 CSI1 allow to select between CSI0/1 parallel input pads and the MIPI CSI-2 virtual channels 0/3.
On i.MX6DL/S two five-input multiplexers in front of IPU1 CSI0 and IPU1 CSI1 allow to select between CSI0/1 parallel input pads and any of the four MIPI CSI-2 virtual channels.
- Removed some dangling/unused endpoints (ipu2_csi0_from_csi2ipu)
- Renamed the mipi virtual channel endpoint labels, from "mipi_csiX_..." to "mipi_vcX...".
- Added input endpoint anchors to the video muxes for the connections from parallel sensors.
--- arch/arm/boot/dts/imx6dl.dtsi | 174 +++++++++++++++++++++++++++++++++++++++++ arch/arm/boot/dts/imx6q.dtsi | 110 ++++++++++++++++++++++++++ arch/arm/boot/dts/imx6qdl.dtsi | 6 ++ 3 files changed, 290 insertions(+)
arch/arm/boot/dts/imx6dl.dtsi +174 -0
@@ -164,6 +164,116 @@ <&iomuxc 9 207 1>, <&iomuxc 10 206 1>, <&iomuxc 11 133 3>; }; +&gpr { + ipu1_csi0_mux: ipu1_csi0_mux at 34 { + compatible = "video-mux"; + mux-controls = <&mux 0>; + #address-cells = <1>; + #size-cells = <0>; + + port at 0 { + reg = <0>; + + ipu1_csi0_mux_from_mipi_vc0: endpoint { + remote-endpoint = <&mipi_vc0_to_ipu1_csi0_mux>; + }; + }; + + port at 1 { + reg = <1>; + + ipu1_csi0_mux_from_mipi_vc1: endpoint { + remote-endpoint = <&mipi_vc1_to_ipu1_csi0_mux>; + }; + }; + + port at 2 { + reg = <2>; + + ipu1_csi0_mux_from_mipi_vc2: endpoint { + remote-endpoint = <&mipi_vc2_to_ipu1_csi0_mux>; + }; + }; + + port at 3 { + reg = <3>; + + ipu1_csi0_mux_from_mipi_vc3: endpoint { + remote-endpoint = <&mipi_vc3_to_ipu1_csi0_mux>; + }; + }; + + port at 4 { + reg = <4>; + + ipu1_csi0_mux_from_parallel_sensor: endpoint { + }; + }; + + port at 5 { + reg = <5>; + + ipu1_csi0_mux_to_ipu1_csi0: endpoint { + remote-endpoint = <&ipu1_csi0_from_ipu1_csi0_mux>; + }; + }; + }; + + ipu1_csi1_mux: ipu1_csi1_mux at 34 { + compatible = "video-mux"; + mux-controls = <&mux 1>; + #address-cells = <1>; + #size-cells = <0>; + + port at 0 { + reg = <0>; + + ipu1_csi1_mux_from_mipi_vc0: endpoint { + remote-endpoint = <&mipi_vc0_to_ipu1_csi1_mux>; + }; + }; + + port at 1 { + reg = <1>; + + ipu1_csi1_mux_from_mipi_vc1: endpoint { + remote-endpoint = <&mipi_vc1_to_ipu1_csi1_mux>; + }; + }; + + port at 2 { + reg = <2>; + + ipu1_csi1_mux_from_mipi_vc2: endpoint { + remote-endpoint = <&mipi_vc2_to_ipu1_csi1_mux>; + }; + }; + + port at 3 { + reg = <3>; + + ipu1_csi1_mux_from_mipi_vc3: endpoint { + remote-endpoint = <&mipi_vc3_to_ipu1_csi1_mux>; + }; + }; + + port at 4 { + reg = <4>; + + ipu1_csi1_mux_from_parallel_sensor: endpoint { + }; + }; + + port at 5 { + reg = <5>; + + ipu1_csi1_mux_to_ipu1_csi1: endpoint { + remote-endpoint = <&ipu1_csi1_from_ipu1_csi1_mux>; + }; + }; + }; +}; + &gpt { compatible = "fsl,imx6dl-gpt"; };
@@ -172,6 +282,12 @@ compatible = "fsl,imx6dl-hdmi"; }; +&ipu1_csi1 { + ipu1_csi1_from_ipu1_csi1_mux: endpoint { + remote-endpoint = <&ipu1_csi1_mux_to_ipu1_csi1>; + }; +}; + &ldb { clocks = <&clks IMX6QDL_CLK_LDB_DI0_SEL>, <&clks IMX6QDL_CLK_LDB_DI1_SEL>, <&clks IMX6QDL_CLK_IPU1_DI0_SEL>, <&clks IMX6QDL_CLK_IPU1_DI1_SEL>,
@@ -181,6 +297,64 @@ "di0", "di1"; }; +&mipi_csi { + port at 1 { + reg = <1>; + #address-cells = <1>; + #size-cells = <0>; + + mipi_vc0_to_ipu1_csi0_mux: endpoint at 0 { + remote-endpoint = <&ipu1_csi0_mux_from_mipi_vc0>; + }; + + mipi_vc0_to_ipu1_csi1_mux: endpoint at 1 { + remote-endpoint = <&ipu1_csi1_mux_from_mipi_vc0>; + }; + }; + + port at 2 { + reg = <2>; + #address-cells = <1>; + #size-cells = <0>; + + mipi_vc1_to_ipu1_csi0_mux: endpoint at 0 { + remote-endpoint = <&ipu1_csi0_mux_from_mipi_vc1>; + }; + + mipi_vc1_to_ipu1_csi1_mux: endpoint at 1 { + remote-endpoint = <&ipu1_csi1_mux_from_mipi_vc1>; + }; + }; + + port at 3 { + reg = <3>; + #address-cells = <1>; + #size-cells = <0>; + + mipi_vc2_to_ipu1_csi0_mux: endpoint at 0 { + remote-endpoint = <&ipu1_csi0_mux_from_mipi_vc2>; + }; + + mipi_vc2_to_ipu1_csi1_mux: endpoint at 1 { + remote-endpoint = <&ipu1_csi1_mux_from_mipi_vc2>; + }; + }; + + port at 4 { + reg = <4>; + #address-cells = <1>; + #size-cells = <0>; + + mipi_vc3_to_ipu1_csi0_mux: endpoint at 0 { + remote-endpoint = <&ipu1_csi0_mux_from_mipi_vc3>; + }; + + mipi_vc3_to_ipu1_csi1_mux: endpoint at 1 { + remote-endpoint = <&ipu1_csi1_mux_from_mipi_vc3>; + }; + }; +}; + &mux { mux-reg-masks = <0x34 0x00000007>, /* IPU_CSI0_MUX */ <0x34 0x00000038>, /* IPU_CSI1_MUX */
arch/arm/boot/dts/imx6q.dtsi +110 -0
138 lines not on this page · raw message
arch/arm/boot/dts/imx6qdl.dtsi +6 -0
@@ -1142,6 +1142,8 @@ mipi_csi: mipi at 021dc000 { compatible = "fsl,imx6-mipi-csi2", "snps,dw-mipi-csi2"; reg = <0x021dc000 0x4000>; + #address-cells = <1>; + #size-cells = <0>; interrupts = <0 100 0x04>, <0 101 0x04>; clocks = <&clks IMX6QDL_CLK_HSI_TX>, <&clks IMX6QDL_CLK_VIDEO_27M>,
@@ -1249,6 +1251,10 @@ ipu1_csi0: port at 0 { reg = <0>; + + ipu1_csi0_from_ipu1_csi0_mux: endpoint { + remote-endpoint = <&ipu1_csi0_mux_to_ipu1_csi0>; + }; }; ipu1_csi1: port at 1 {
Signature · 1 line
2.7.4
Steve Longerbeam
maintainerPATCH v8 7/34 ARM: dts: imx6qdl: add capture-subsystem device
In reply to Steve Longerbeam--- arch/arm/boot/dts/imx6dl.dtsi | 5 +++++ arch/arm/boot/dts/imx6q.dtsi | 5 +++++ 2 files changed, 10 insertions(+)
arch/arm/boot/dts/imx6dl.dtsi +5 -0
@@ -100,6 +100,11 @@ }; }; + capture-subsystem { + compatible = "fsl,imx-capture-subsystem"; + ports = <&ipu1_csi0>, <&ipu1_csi1>; + }; + display-subsystem { compatible = "fsl,imx-display-subsystem"; ports = <&ipu1_di0>, <&ipu1_di1>;
arch/arm/boot/dts/imx6q.dtsi +5 -0
@@ -206,6 +206,11 @@ }; }; + capture-subsystem { + compatible = "fsl,imx-capture-subsystem"; + ports = <&ipu1_csi0>, <&ipu1_csi1>, <&ipu2_csi0>, <&ipu2_csi1>; + }; + display-subsystem { compatible = "fsl,imx-display-subsystem"; ports = <&ipu1_di0>, <&ipu1_di1>, <&ipu2_di0>, <&ipu2_di1>;
Signature · 1 line
2.7.4
Steve Longerbeam
maintainerPATCH v8 8/34 ARM: dts: imx6qdl-sabrelite: remove erratum ERR006687 workaround
In reply to Steve LongerbeamThere is a pin conflict with GPIO_6. This pin functions as a power input pin to the OV5642 camera sensor, but ENET uses it as the h/w workaround for erratum ERR006687, to wake-up the ARM cores on normal RX and TX packet done events. So we need to remove the h/w workaround to support the OV5642. The result is that the CPUidle driver will no longer allow entering the deep idle states on the sabrelite.
This is a partial revert of
commit 6261c4c8f13e ("ARM: dts: imx6qdl-sabrelite: use GPIO_6 for FEC
interrupt.")
commit a28eeb43ee57 ("ARM: dts: imx6: tag boards that have the HW workaround
for ERR006687")
--- arch/arm/boot/dts/imx6qdl-sabrelite.dtsi | 4 ---- 1 file changed, 4 deletions(-)
arch/arm/boot/dts/imx6qdl-sabrelite.dtsi +0 -4
@@ -270,9 +270,6 @@ txd1-skew-ps = <0>; txd2-skew-ps = <0>; txd3-skew-ps = <0>; - interrupts-extended = <&gpio1 6 IRQ_TYPE_LEVEL_HIGH>, - <&intc 0 119 IRQ_TYPE_LEVEL_HIGH>; - fsl,err006687-workaround-present; status = "okay"; };
@@ -373,7 +370,6 @@ MX6QDL_PAD_RGMII_RX_CTL__RGMII_RX_CTL 0x1b030 /* Phy reset */ MX6QDL_PAD_EIM_D23__GPIO3_IO23 0x000b0 - MX6QDL_PAD_GPIO_6__ENET_IRQ 0x000b1 >; };
Signature · 1 line
2.7.4
Steve Longerbeam
maintainerPATCH v8 9/34 ARM: dts: imx6-sabrelite: add OV5642 and OV5640 camera sensors
In reply to Steve LongerbeamAdds the OV5642 parallel-bus sensor, and the OV5640 MIPI CSI-2 sensor. Both hang off the same i2c2 bus, so they require different (and non- default) i2c slave addresses.
The OV5642 connects to the parallel-bus mux input port on ipu1_csi0_mux.
The OV5640 connects to the input port on the MIPI CSI-2 receiver on mipi_csi.
The OV5642 node is disabled temporarily while the subdev driver is cleaned up and submitted later.
--- arch/arm/boot/dts/imx6dl-sabrelite.dts | 5 ++ arch/arm/boot/dts/imx6q-sabrelite.dts | 5 ++ arch/arm/boot/dts/imx6qdl-sabrelite.dtsi | 148 +++++++++++++++++++++++++++++++ 3 files changed, 158 insertions(+)
arch/arm/boot/dts/imx6dl-sabrelite.dts +5 -0
@@ -48,3 +48,8 @@ model = "Freescale i.MX6 DualLite SABRE Lite Board"; compatible = "fsl,imx6dl-sabrelite", "fsl,imx6dl"; }; + +&ipu1_csi1_from_ipu1_csi1_mux { + clock-lanes = <0>; + data-lanes = <1 2>; +};
arch/arm/boot/dts/imx6q-sabrelite.dts +5 -0
@@ -52,3 +52,8 @@ &sata { status = "okay"; }; + +&ipu1_csi1_from_mipi_vc1 { + clock-lanes = <0>; + data-lanes = <1 2>; +};
arch/arm/boot/dts/imx6qdl-sabrelite.dtsi +148 -0
187 lines not on this page · raw message
Signature · 1 line
2.7.4
Steve Longerbeam
maintainerPATCH v8 10/34 ARM: dts: imx6-sabresd: add OV5642 and OV5640 camera sensors
In reply to Steve LongerbeamEnables the OV5642 parallel-bus sensor, and the OV5640 MIPI CSI-2 sensor.
The OV5642 connects to the parallel-bus mux input port on ipu1_csi0_mux.
The OV5640 connects to the input port on the MIPI CSI-2 receiver on mipi_csi.
Until the OV5652 sensor module compatible with the SabreSD becomes available for testing, the ov5642 node is currently disabled.
--- arch/arm/boot/dts/imx6dl-sabresd.dts | 5 ++ arch/arm/boot/dts/imx6q-sabresd.dts | 5 ++ arch/arm/boot/dts/imx6qdl-sabresd.dtsi | 114 ++++++++++++++++++++++++++++++++- 3 files changed, 123 insertions(+), 1 deletion(-)
arch/arm/boot/dts/imx6dl-sabresd.dts +5 -0
@@ -15,3 +15,8 @@ model = "Freescale i.MX6 DualLite SABRE Smart Device Board"; compatible = "fsl,imx6dl-sabresd", "fsl,imx6dl"; }; + +&ipu1_csi1_from_ipu1_csi1_mux { + clock-lanes = <0>; + data-lanes = <1 2>; +};
arch/arm/boot/dts/imx6q-sabresd.dts +5 -0
@@ -23,3 +23,8 @@ &sata { status = "okay"; }; + +&ipu1_csi1_from_mipi_vc1 { + clock-lanes = <0>; + data-lanes = <1 2>; +};
arch/arm/boot/dts/imx6qdl-sabresd.dtsi +113 -1
144 lines not on this page · raw message
Signature · 1 line
2.7.4
Steve Longerbeam
maintainerPATCH v8 11/34 ARM: dts: imx6-sabreauto: create i2cmux for i2c3
In reply to Steve LongerbeamThe sabreauto uses a steering pin to select between the SDA signal on i2c3 bus, and a data-in pin for an SPI NOR chip. Use i2cmux to control this steering pin. Idle state of the i2cmux selects SPI NOR. This is not a classic way to use i2cmux, since one side of the mux selects something other than an i2c bus, but it works and is probably the cleanest solution. Note that if one thread is attempting to access SPI NOR while another thread is accessing i2c3, the SPI NOR access will fail since the i2cmux has selected the SDA pin rather than SPI NOR data-in. This couldn't be avoided in any case, the board is not designed to allow concurrent i2c3 and SPI NOR functions (and the default device-tree does not enable SPI NOR anyway).
Devices hanging off i2c3 should now be defined under i2cmux, so that the steering pin can be properly controlled to access those devices. The port expanders (MAX7310) are thus moved into i2cmux.
--- arch/arm/boot/dts/imx6qdl-sabreauto.dtsi | 65 +++++++++++++++++++++----------- 1 file changed, 44 insertions(+), 21 deletions(-)
arch/arm/boot/dts/imx6qdl-sabreauto.dtsi +44 -21
@@ -108,6 +108,44 @@ default-brightness-level = <7>; status = "okay"; }; + + i2cmux { + compatible = "i2c-mux-gpio"; + #address-cells = <1>; + #size-cells = <0>; + pinctrl-names = "default"; + pinctrl-0 = <&pinctrl_i2c3mux>; + mux-gpios = <&gpio5 4 0>; + i2c-parent = <&i2c3>; + idle-state = <0>; + + i2c at 1 { + #address-cells = <1>; + #size-cells = <0>; + reg = <1>; + + max7310_a: gpio at 30 { + compatible = "maxim,max7310"; + reg = <0x30>; + gpio-controller; + #gpio-cells = <2>; + }; + + max7310_b: gpio at 32 { + compatible = "maxim,max7310"; + reg = <0x32>; + gpio-controller; + #gpio-cells = <2>; + }; + + max7310_c: gpio at 34 { + compatible = "maxim,max7310"; + reg = <0x34>; + gpio-controller; + #gpio-cells = <2>; + }; + }; + }; }; &clks {
@@ -290,27 +328,6 @@ pinctrl-names = "default"; pinctrl-0 = <&pinctrl_i2c3>; status = "okay"; - - max7310_a: gpio at 30 { - compatible = "maxim,max7310"; - reg = <0x30>; - gpio-controller; - #gpio-cells = <2>; - }; - - max7310_b: gpio at 32 { - compatible = "maxim,max7310"; - reg = <0x32>; - gpio-controller; - #gpio-cells = <2>; - }; - - max7310_c: gpio at 34 { - compatible = "maxim,max7310"; - reg = <0x34>; - gpio-controller; - #gpio-cells = <2>; - }; }; &iomuxc {
@@ -418,6 +435,12 @@ >; }; + pinctrl_i2c3mux: i2c3muxgrp { + fsl,pins = < + MX6QDL_PAD_EIM_A24__GPIO5_IO04 0x0b0b1 + >; + }; + pinctrl_pwm3: pwm1grp { fsl,pins = < MX6QDL_PAD_SD4_DAT1__PWM3_OUT 0x1b0b1
Signature · 1 line
2.7.4
Steve Longerbeam
maintainerPATCH v8 12/34 ARM: dts: imx6-sabreauto: add reset-gpios property for max7310_b
In reply to Steve LongerbeamThe reset pin to the port expander chip (MAX7310) is controlled by a gpio, so define a reset-gpios property to control it. There are three MAX7310's on the SabreAuto CPU card (max7310_[abc]), but all use the same pin for their reset. Since all can't acquire the same pin, assign it to max7310_b, that chip is needed by more functions (usb and adv7180).
--- arch/arm/boot/dts/imx6qdl-sabreauto.dtsi | 9 +++++++++ 1 file changed, 9 insertions(+)
arch/arm/boot/dts/imx6qdl-sabreauto.dtsi +9 -0
@@ -136,6 +136,9 @@ reg = <0x32>; gpio-controller; #gpio-cells = <2>; + pinctrl-names = "default"; + pinctrl-0 = <&pinctrl_max7310>; + reset-gpios = <&gpio1 15 GPIO_ACTIVE_LOW>; }; max7310_c: gpio at 34 {
@@ -441,6 +444,12 @@ >; }; + pinctrl_max7310: max7310grp { + fsl,pins = < + MX6QDL_PAD_SD2_DAT0__GPIO1_IO15 0x1b0b0 + >; + }; + pinctrl_pwm3: pwm1grp { fsl,pins = < MX6QDL_PAD_SD4_DAT1__PWM3_OUT 0x1b0b1
Signature · 1 line
2.7.4
Steve Longerbeam
maintainerPATCH v8 13/34 ARM: dts: imx6-sabreauto: add pinctrl for gpt input capture
In reply to Steve LongerbeamAdd pinctrl groups for both GPT input capture channels.
--- arch/arm/boot/dts/imx6qdl-sabreauto.dtsi | 12 ++++++++++++ 1 file changed, 12 insertions(+)
arch/arm/boot/dts/imx6qdl-sabreauto.dtsi +12 -0
@@ -456,6 +456,18 @@ >; }; + pinctrl_gpt_input_capture0: gptinputcapture0grp { + fsl,pins = < + MX6QDL_PAD_SD1_DAT0__GPT_CAPTURE1 0x1b0b0 + >; + }; + + pinctrl_gpt_input_capture1: gptinputcapture1grp { + fsl,pins = < + MX6QDL_PAD_SD1_DAT1__GPT_CAPTURE2 0x1b0b0 + >; + }; + pinctrl_spdif: spdifgrp { fsl,pins = < MX6QDL_PAD_KEY_COL3__SPDIF_IN 0x1b0b0
Signature · 1 line
2.7.4
Steve Longerbeam
maintainerPATCH v8 14/34 ARM: dts: imx6-sabreauto: add the ADV7180 video decoder
In reply to Steve LongerbeamEnables the ADV7180 decoder sensor. The ADV7180 connects to the parallel-bus mux input on ipu1_csi0_mux.
The ADV7180 power pin is via max7310_b port expander.
- Use IRQ_TYPE_LEVEL_LOW instead of 0x8 for interrupt type for clarity.
- For 8-bit parallel IPU1-CSI0 bus connection only data[12-19] are used.
--- arch/arm/boot/dts/imx6qdl-sabreauto.dtsi | 50 ++++++++++++++++++++++++++++++++ 1 file changed, 50 insertions(+)
arch/arm/boot/dts/imx6qdl-sabreauto.dtsi +50 -0
@@ -124,6 +124,21 @@ #size-cells = <0>; reg = <1>; + adv7180: camera at 21 { + compatible = "adi,adv7180"; + reg = <0x21>; + powerdown-gpios = <&max7310_b 2 GPIO_ACTIVE_LOW>; + interrupt-parent = <&gpio1>; + interrupts = <27 IRQ_TYPE_LEVEL_LOW>; + + port { + adv7180_to_ipu1_csi0_mux: endpoint { + remote-endpoint = <&ipu1_csi0_mux_from_parallel_sensor>; + bus-width = <8>; + }; + }; + }; + max7310_a: gpio at 30 { compatible = "maxim,max7310"; reg = <0x30>;
@@ -151,6 +166,25 @@ }; }; +&ipu1_csi0_from_ipu1_csi0_mux { + bus-width = <8>; +}; + +&ipu1_csi0_mux_from_parallel_sensor { + remote-endpoint = <&adv7180_to_ipu1_csi0_mux>; + bus-width = <8>; +}; + +&ipu1_csi0 { + pinctrl-names = "default"; + pinctrl-0 = <&pinctrl_ipu1_csi0>; + + /* enable frame interval monitor on this port */ + fim { + status = "okay"; + }; +}; + &clks { assigned-clocks = <&clks IMX6QDL_PLL4_BYPASS_SRC>, <&clks IMX6QDL_PLL4_BYPASS>,
@@ -444,6 +478,22 @@ >; }; + pinctrl_ipu1_csi0: ipu1csi0grp { + fsl,pins = < + MX6QDL_PAD_CSI0_DAT12__IPU1_CSI0_DATA12 0x1b0b0 + MX6QDL_PAD_CSI0_DAT13__IPU1_CSI0_DATA13 0x1b0b0 + MX6QDL_PAD_CSI0_DAT14__IPU1_CSI0_DATA14 0x1b0b0 + MX6QDL_PAD_CSI0_DAT15__IPU1_CSI0_DATA15 0x1b0b0 + MX6QDL_PAD_CSI0_DAT16__IPU1_CSI0_DATA16 0x1b0b0 + MX6QDL_PAD_CSI0_DAT17__IPU1_CSI0_DATA17 0x1b0b0 + MX6QDL_PAD_CSI0_DAT18__IPU1_CSI0_DATA18 0x1b0b0 + MX6QDL_PAD_CSI0_DAT19__IPU1_CSI0_DATA19 0x1b0b0 + MX6QDL_PAD_CSI0_PIXCLK__IPU1_CSI0_PIXCLK 0x1b0b0 + MX6QDL_PAD_CSI0_MCLK__IPU1_CSI0_HSYNC 0x1b0b0 + MX6QDL_PAD_CSI0_VSYNC__IPU1_CSI0_VSYNC 0x1b0b0 + >; + }; + pinctrl_max7310: max7310grp { fsl,pins = < MX6QDL_PAD_SD2_DAT0__GPIO1_IO15 0x1b0b0
Signature · 1 line
2.7.4
Re: [PATCH v8 14/34] ARM: dts: imx6-sabreauto: add the ADV7180 video decoder
Quoted 52 lines · Steve Longerbeam
Enables the ADV7180 decoder sensor. The ADV7180 connects to the parallel-bus mux input on
ipu1_csi0_mux.The ADV7180 power pin is via
max7310_bport expander.Signed-off-by Steve Longerbeam <[email protected]>- Use
IRQ_TYPE_LEVEL_LOWinstead of0x8for interrupt type for clarity.
- For 8-bit parallel IPU1-CSI0 bus connection only data[12-19] are used.Signed-off-by Tim Harvey <[email protected]>--- arch/arm/boot/dts/imx6qdl-sabreauto.dtsi | 50 ++++++++++++++++++++++++++++++++ 1 file changed, 50 insertions(+)arch/arm/boot/dts/imx6qdl-sabreauto.dtsi +22
-0@@ -124,6 +124,21 @@ #size-cells = <0>; reg = <1>; + adv7180: camera at 21 { + compatible = "adi,adv7180"; + reg = <0x21>; + powerdown-gpios = <&max7310_b 2 GPIO_ACTIVE_LOW>; + interrupt-parent = <&gpio1>; + interrupts = <27 IRQ_TYPE_LEVEL_LOW>; + + port { + adv7180_to_ipu1_csi0_mux: endpoint { + remote-endpoint = <&ipu1_csi0_mux_from_parallel_sensor>; + bus-width = <8>; + }; + }; + }; + max7310_a: gpio at 30 { compatible = "maxim,max7310"; reg = <0x30>;@@ -151,6 +166,25 @@ }; }; +&ipu1_csi0_from_ipu1_csi0_mux { + bus-width = <8>; +}; + +&ipu1_csi0_mux_from_parallel_sensor { + remote-endpoint = <&adv7180_to_ipu1_csi0_mux>; + bus-width = <8>;
arch/arm/boot/dts/imx6qdl-sabreauto.dtsi line 185
Quoted 10 lines
arch/arm/boot/dts/imx6qdl-sabreauto.dtsi +10
-0+}; + +&ipu1_csi0 { + pinctrl-names = "default"; + pinctrl-0 = <&pinctrl_ipu1_csi0>; + + /* enable frame interval monitor on this port */ + fim { + status = "okay"; + };
Steve,
You need to remove the fim node now that you've moved this to V4L2 controls.
Regards,
Tim
Steve Longerbeam
maintainerRe: [PATCH v8 14/34] ARM: dts: imx6-sabreauto: add the ADV7180 video decoder
Quoted 6 lines · Tim Harvey
Steve,
You need to remove the fim node now that you've moved this to V4L2 controls.
Yep, I caught this just after sending the v8 patchset. I'll send a v9 of this patch.
Steve
Re: [PATCH v8 14/34] ARM: dts: imx6-sabreauto: add the ADV7180 video decoder
Quoted 5 lines · Steve Longerbeam
Quoted 2 lines · Tim Harvey
Quoted 7 lines
Quoted 4 lines
Steve,
You need to remove the fim node now that you've moved this to V4L2 controls.
Yep, I caught this just after sending the v8 patchset. I'll send a v9 of this patch.
This needs ack from devicetree people, then it can be merged. Can you be a bit more forceful getting the ack?
I don't think it makes sense to resubmit v9 before that. This is not a rocket science.
Thanks,
Pavel
Signature · 9 lines
(english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: Digital signature URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20170609/c0586c4b/attachment.sig>
Steve Longerbeam
maintainerRe: [PATCH v8 14/34] ARM: dts: imx6-sabreauto: add the ADV7180 video decoder
Quoted 8 lines · Pavel Machek
Quoted 7 lines · Steve Longerbeam
Quoted 4 lines · Tim Harvey
Steve,
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
You need to remove the fim node now that you've moved this to V4L2 controls.
Yep, I caught this just after sending the v8 patchset. I'll send a v9 of this patch.
This needs ack from devicetree people, then it can be merged. Can you be a bit more forceful getting the ack?
OK, I need an Ack please, he said, in a forceful way. :)
In fact Ack's are needed for all the changes to dts sources, patches 4-14.
Quoted 3 lines
I don't think it makes sense to resubmit v9 before that. This is not a rocket science.
I guess that makes sense, I'll wait for Ack's from all these patches before submitting the entire patchset as v9.
Steve
Re: [PATCH v8 14/34] ARM: dts: imx6-sabreauto: add the ADV7180 video decoder
Hi!
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 4 lines
Steve,
You need to remove the fim node now that you've moved this to V4L2 controls.
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
Yep, I caught this just after sending the v8 patchset. I'll send a v9 of this patch.
This needs ack from devicetree people, then it can be merged. Can you be a bit more forceful getting the ack?
OK, I need an Ack please, he said, in a forceful way. :)
I'd tune the force up a tiny bit more. This is not FreeBSD ;-). You can read some emails from Linus for inspiration. Or drink few beers and look at Al Viro's emails.
Quoted 2 lines
In fact Ack's are needed for all the changes to dts sources, patches 4-14.
Actually, are they? Those should not need acks from device tree people, just acks from ARM people, and those are easier to get... in fact they should not need any acks, you should just send them to arm people and get them merged.
1-4 is just a documentation, and you have acks there. (Except 2?) That's ready to be merged, probably via the media tree? Just make it clear you want it merged.
15,16 should be ready to. Media tree, too, I guess.
drivers/staging is greg. Advantage is these do _not_ need to go after the dts changes. It is a driver. Actually I'd normally add dts support after the driver. So you can push it now.
Quoted 5 lines
Quoted 2 lines
I don't think it makes sense to resubmit v9 before that. This is not a rocket science.
I guess that makes sense, I'll wait for Ack's from all these patches before submitting the entire patchset as v9.
You may want to split the series, according to mainainters, or just ask the maintainers to merge the relevant parts. I believe most of it can be pushed now.
Good luck,
Pavel
Signature · 9 lines
(english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: Digital signature URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20170610/5fa97090/attachment-0001.sig>
Steve Longerbeam
maintainerRe: [PATCH v8 14/34] ARM: dts: imx6-sabreauto: add the ADV7180 video decoder
Quoted 19 lines · Pavel Machek
Hi!
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 4 lines
Steve,
You need to remove the fim node now that you've moved this to V4L2 controls.
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
Yep, I caught this just after sending the v8 patchset. I'll send a v9 of this patch.
This needs ack from devicetree people, then it can be merged. Can you be a bit more forceful getting the ack?
OK, I need an Ack please, he said, in a forceful way. :)
I'd tune the force up a tiny bit more. This is not FreeBSD ;-). You can read some emails from Linus for inspiration. Or drink few beers and look at Al Viro's emails.
Quoted 7 lines
Quoted 2 lines
In fact Ack's are needed for all the changes to dts sources, patches 4-14.
Actually, are they? Those should not need acks from device tree people, just acks from ARM people, and those are easier to get... in fact they should not need any acks, you should just send them to arm people and get them merged.
Hans said he prefers to have the dts patches as part of the whole set rather than submitted separately. But I did add --to's for the ARM people for 4-14.
Quoted 4 lines
1-4 is just a documentation, and you have acks there. (Except 2?) That's ready to be merged, probably via the media tree? Just make it clear you want it merged.
Yes, 1-3 now have Acks (binding docs).
Quoted 2 lines
15,16 should be ready to. Media tree, too, I guess.
Yes, those have Acks (video-mux entity functions and subdev driver).
Quoted 4 lines
drivers/stagingis greg. Advantage is these do _not_ need to go after the dts changes. It is a driver. Actually I'd normally add dts support after the driver. So you can push it now.
Right, Hans agrees, except that for staging drivers we don't really need an Ack from Greg.
Quoted 4 lines
Quoted 4 lines
Quoted 3 lines
I don't think it makes sense to resubmit v9 before that. This is not a rocket science.
Quoted 8 lines
Quoted 2 lines
I guess that makes sense, I'll wait for Ack's from all these patches before submitting the entire patchset as v9.
You may want to split the series, according to mainainters, or just ask the maintainers to merge the relevant parts. I believe most of it can be pushed now.
Good luck,
Thanks!
Steve
Steve Longerbeam
maintainerPATCH v9 14/34 ARM: dts: imx6-sabreauto: add the ADV7180 video decoder
In reply to Steve LongerbeamEnables the ADV7180 decoder sensor. The ADV7180 connects to the parallel-bus mux input on ipu1_csi0_mux.
The ADV7180 power pin is via max7310_b port expander.
- Use IRQ_TYPE_LEVEL_LOW instead of 0x8 for interrupt type for clarity.
- For 8-bit parallel IPU1-CSI0 bus connection only data[12-19] are used.
---
Changes since v8 [1]:
- drop no-longer-applicable fim node from ipu1_csi0 port.
[1] https://patchwork.linuxtv.org/patch/41705/
--- arch/arm/boot/dts/imx6qdl-sabreauto.dtsi | 45 ++++++++++++++++++++++++++++++++ 1 file changed, 45 insertions(+)
arch/arm/boot/dts/imx6qdl-sabreauto.dtsi +45 -0
@@ -124,6 +124,21 @@ #size-cells = <0>; reg = <1>; + adv7180: camera at 21 { + compatible = "adi,adv7180"; + reg = <0x21>; + powerdown-gpios = <&max7310_b 2 GPIO_ACTIVE_LOW>; + interrupt-parent = <&gpio1>; + interrupts = <27 IRQ_TYPE_LEVEL_LOW>; + + port { + adv7180_to_ipu1_csi0_mux: endpoint { + remote-endpoint = <&ipu1_csi0_mux_from_parallel_sensor>; + bus-width = <8>; + }; + }; + }; + max7310_a: gpio at 30 { compatible = "maxim,max7310"; reg = <0x30>;
@@ -151,6 +166,20 @@ }; }; +&ipu1_csi0_from_ipu1_csi0_mux { + bus-width = <8>; +}; + +&ipu1_csi0_mux_from_parallel_sensor { + remote-endpoint = <&adv7180_to_ipu1_csi0_mux>; + bus-width = <8>; +}; + +&ipu1_csi0 { + pinctrl-names = "default"; + pinctrl-0 = <&pinctrl_ipu1_csi0>; +}; + &clks { assigned-clocks = <&clks IMX6QDL_PLL4_BYPASS_SRC>, <&clks IMX6QDL_PLL4_BYPASS>,
@@ -444,6 +473,22 @@ >; }; + pinctrl_ipu1_csi0: ipu1csi0grp { + fsl,pins = < + MX6QDL_PAD_CSI0_DAT12__IPU1_CSI0_DATA12 0x1b0b0 + MX6QDL_PAD_CSI0_DAT13__IPU1_CSI0_DATA13 0x1b0b0 + MX6QDL_PAD_CSI0_DAT14__IPU1_CSI0_DATA14 0x1b0b0 + MX6QDL_PAD_CSI0_DAT15__IPU1_CSI0_DATA15 0x1b0b0 + MX6QDL_PAD_CSI0_DAT16__IPU1_CSI0_DATA16 0x1b0b0 + MX6QDL_PAD_CSI0_DAT17__IPU1_CSI0_DATA17 0x1b0b0 + MX6QDL_PAD_CSI0_DAT18__IPU1_CSI0_DATA18 0x1b0b0 + MX6QDL_PAD_CSI0_DAT19__IPU1_CSI0_DATA19 0x1b0b0 + MX6QDL_PAD_CSI0_PIXCLK__IPU1_CSI0_PIXCLK 0x1b0b0 + MX6QDL_PAD_CSI0_MCLK__IPU1_CSI0_HSYNC 0x1b0b0 + MX6QDL_PAD_CSI0_VSYNC__IPU1_CSI0_VSYNC 0x1b0b0 + >; + }; + pinctrl_max7310: max7310grp { fsl,pins = < MX6QDL_PAD_SD2_DAT0__GPIO1_IO15 0x1b0b0
Signature · 1 line
2.7.4
Steve Longerbeam
maintainerPATCH v8 15/34 add mux and video interface bridge entity functions
In reply to Steve LongerbeamFrom: Philipp Zabel <[email protected]>
Add two new media entity function definitions for video multiplexers and video interface bridges.
- renamed MEDIA_ENT_F_MUX to MEDIA_ENT_F_VID_MUX
--- Documentation/media/uapi/mediactl/media-types.rst | 21 +++++++++++++++++++++ include/uapi/linux/media.h | 6 ++++++ 2 files changed, 27 insertions(+)
Documentation/media/uapi/mediactl/media-types.rst +21 -0
@@ -299,6 +299,27 @@ Types and flags used to represent the media graph elements received on its sink pad and outputs the statistics data on its source pad. + - .. row 29 + + .. _MEDIA-ENT-F-VID-MUX: + + - ``MEDIA_ENT_F_VID_MUX`` + + - Video multiplexer. An entity capable of multiplexing must have at + least two sink pads and one source pad, and must pass the video + frame(s) received from the active sink pad to the source pad. + + - .. row 30 + + .. _MEDIA-ENT-F-VID-IF-BRIDGE: + + - ``MEDIA_ENT_F_VID_IF_BRIDGE`` + + - Video interface bridge. A video interface bridge entity must have at + least one sink pad and at least one source pad. It receives video + frames on its sink pad from an input video bus of one type (HDMI, eDP, + MIPI CSI-2, ...), and outputs them on its source pad to an output + video bus of another type (eDP, MIPI CSI-2, parallel, ...). .. tabularcolumns:: |p{5.5cm}|p{12.0cm}|
include/uapi/linux/media.h +6 -0
@@ -105,6 +105,12 @@ struct media_device_info {#define MEDIA_ENT_F_PROC_VIDEO_STATISTICS (MEDIA_ENT_F_BASE + 0x4006)/* +* Switch and bridge entitites +*/ +#define MEDIA_ENT_F_VID_MUX (MEDIA_ENT_F_BASE + 0x5001) +#define MEDIA_ENT_F_VID_IF_BRIDGE (MEDIA_ENT_F_BASE + 0x5002) + +/* * Connectors */ /* It is a responsibility of the entity drivers to add connectors and links */
Signature · 1 line
2.7.4
Steve Longerbeam
maintainerPATCH v8 16/34 platform: add video-multiplexer subdevice driver
In reply to Steve LongerbeamFrom: Philipp Zabel <[email protected]>
This driver can handle SoC internal and external video bus multiplexers, controlled by mux controllers provided by the mux controller framework, such as MMIO register bitfields or GPIOs. The subdevice passes through the mbus configuration of the active input to the output side.
Since the mux framework is not yet merged, this driver contains temporary mmio-mux support to work without the framework. The driver should be converted to use the multiplexer API once the "mux: minimal mux subsystem" and "mux: mmio-based syscon mux controller" patches are merged.
--- drivers/media/platform/Kconfig | 6 + drivers/media/platform/Makefile | 2 + drivers/media/platform/video-mux.c | 334 +++++++++++++++++++++++++++++++++++++ 3 files changed, 342 insertions(+) create mode 100644 drivers/media/platform/video-mux.c
drivers/media/platform/Kconfig +6 -0
@@ -74,6 +74,12 @@ config VIDEO_M32R_AR_M64278 To compile this driver as a module, choose M here: the module will be called arv. +config VIDEO_MUX + tristate "Video Multiplexer" + depends on OF && VIDEO_V4L2_SUBDEV_API && MEDIA_CONTROLLER + help + This driver provides support for N:1 video bus multiplexers. + config VIDEO_OMAP3 tristate "OMAP 3 Camera support" depends on VIDEO_V4L2 && I2C && VIDEO_V4L2_SUBDEV_API && ARCH_OMAP3
drivers/media/platform/Makefile +2 -0
@@ -28,6 +28,8 @@ obj-$(CONFIG_VIDEO_SH_VEU) += sh_veu.o obj-$(CONFIG_VIDEO_MEM2MEM_DEINTERLACE) += m2m-deinterlace.o +obj-$(CONFIG_VIDEO_MUX) += video-mux.o + obj-$(CONFIG_VIDEO_S3C_CAMIF) += s3c-camif/ obj-$(CONFIG_VIDEO_SAMSUNG_EXYNOS4_IS) += exynos4-is/ obj-$(CONFIG_VIDEO_SAMSUNG_S5P_JPEG) += s5p-jpeg/
drivers/media/platform/video-mux.c new file +334 -0
334 lines not on this page · raw message
Signature · 1 line
2.7.4
Steve Longerbeam
maintainerPATCH v8 17/34 add Omnivision OV5640 sensor driver
In reply to Steve LongerbeamThis driver is based on ov5640_mipi.c from Freescale imx_3.10.17_1.0.0_beta branch, modified heavily to bring forward to latest interfaces and code cleanup.
--- drivers/media/i2c/Kconfig | 10 + drivers/media/i2c/Makefile | 1 + drivers/media/i2c/ov5640.c | 2344 ++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 2355 insertions(+) create mode 100644 drivers/media/i2c/ov5640.c
drivers/media/i2c/Kconfig +10 -0
@@ -548,6 +548,16 @@ config VIDEO_OV2659 To compile this driver as a module, choose M here: the module will be called ov2659. +config VIDEO_OV5640 + tristate "OmniVision OV5640 sensor support" + depends on OF + depends on GPIOLIB && VIDEO_V4L2 && I2C && VIDEO_V4L2_SUBDEV_API + depends on MEDIA_CAMERA_SUPPORT + select V4L2_FWNODE + ---help--- + This is a Video4Linux2 sensor-level driver for the Omnivision + OV5640 camera sensor with a MIPI CSI-2 interface. + config VIDEO_OV5645 tristate "OmniVision OV5645 sensor support" depends on OF
drivers/media/i2c/Makefile +1 -0
@@ -58,6 +58,7 @@ obj-$(CONFIG_VIDEO_SONY_BTF_MPX) += sony-btf-mpx.o obj-$(CONFIG_VIDEO_UPD64031A) += upd64031a.o obj-$(CONFIG_VIDEO_UPD64083) += upd64083.o obj-$(CONFIG_VIDEO_OV2640) += ov2640.o +obj-$(CONFIG_VIDEO_OV5640) += ov5640.o obj-$(CONFIG_VIDEO_OV5645) += ov5645.o obj-$(CONFIG_VIDEO_OV5647) += ov5647.o obj-$(CONFIG_VIDEO_OV7640) += ov7640.o
drivers/media/i2c/ov5640.c new file +2344 -0
2,344 lines not on this page · raw message
Signature · 1 line
2.7.4
Re: [PATCH v8 17/34] [media] add Omnivision OV5640 sensor driver
Hi Steve,
On Wed, Jun 07, 2017 at 11:33:56AM -0700, Steve Longerbeam wrote:
...
drivers/media/i2c/ov5640.c line 109
Quoted 8 lines
drivers/media/i2c/ov5640.c new file +8
-0+/* +* FIXME: remove this when a subdev API becomes available +* to set the MIPI CSI-2 virtual channel. +*/ +static unsigned int virtual_channel; +module_param (virtual_channel, int,0 ); +MODULE_PARM_DESC (virtual_channel, +"MIPI CSI-2 virtual channel (0..3), default 0" );
If this works for you it's a good workaround. I hope we can address the matter properly soon --- it's related to data types, embedded data etc.
For the patch:
Signature · 4 lines
Regards, Sakari Ailus e-mail: sakari.ailus at iki.fi XMPP: sailus at retiisi.org.uk
Steve Longerbeam
maintainerPATCH v8 18/34 media: Add userspace header file for i.MX
In reply to Steve LongerbeamThis adds a header file for use by userspace programs wanting to interact with the i.MX media driver. It defines custom events and v4l2 controls for the i.MX v4l2 subdevices.
--- include/linux/imx-media.h | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) create mode 100644 include/linux/imx-media.h
include/linux/imx-media.h new file +29 -0
@@ -0,0 +1,29 @@ +/* +* Copyright (c) 2014-2017 Mentor Graphics Inc. +* +* This program is free software; you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published by the +* Free Software Foundation; either version 2 of the +* License, or (at your option) any later version +*/ + +#ifndef __LINUX_IMX_MEDIA_H__ +#define __LINUX_IMX_MEDIA_H__ + +/* +* events from the subdevs +*/ +#define V4L2_EVENT_IMX_CLASS V4L2_EVENT_PRIVATE_START +#define V4L2_EVENT_IMX_FRAME_INTERVAL_ERROR (V4L2_EVENT_IMX_CLASS + 1) + +enum imx_ctrl_id { +V4L2_CID_IMX_FIM_ENABLE = (V4L2_CID_USER_IMX_BASE +0 ), +V4L2_CID_IMX_FIM_NUM , +V4L2_CID_IMX_FIM_TOLERANCE_MIN , +V4L2_CID_IMX_FIM_TOLERANCE_MAX , +V4L2_CID_IMX_FIM_NUM_SKIP , +V4L2_CID_IMX_FIM_ICAP_EDGE , +V4L2_CID_IMX_FIM_ICAP_CHANNEL , +}; + +#endif
Signature · 1 line
2.7.4
Steve Longerbeam
maintainerPATCH v8 19/34 media: Add i.MX media core driver
In reply to Steve LongerbeamAdd the core media driver for i.MX SOC.
Switch from the v4l2_of_ APIs to the v4l2_fwnode_ APIs.
Add the bayer formats to imx-media's list of supported pixel and bus formats.
--- Documentation/media/v4l-drivers/imx.rst | 614 ++++++++++++++++ drivers/staging/media/Kconfig | 2 + drivers/staging/media/Makefile | 1 + drivers/staging/media/imx/Kconfig | 7 + drivers/staging/media/imx/Makefile | 5 + drivers/staging/media/imx/imx-media-dev.c | 666 +++++++++++++++++ drivers/staging/media/imx/imx-media-fim.c | 494 +++++++++++++ drivers/staging/media/imx/imx-media-internal-sd.c | 349 +++++++++ drivers/staging/media/imx/imx-media-of.c | 270 +++++++ drivers/staging/media/imx/imx-media-utils.c | 834 ++++++++++++++++++++++ drivers/staging/media/imx/imx-media.h | 323 +++++++++ include/media/imx.h | 15 + include/uapi/linux/v4l2-controls.h | 4 + 13 files changed, 3584 insertions(+) create mode 100644 Documentation/media/v4l-drivers/imx.rst create mode 100644 drivers/staging/media/imx/Kconfig create mode 100644 drivers/staging/media/imx/Makefile create mode 100644 drivers/staging/media/imx/imx-media-dev.c create mode 100644 drivers/staging/media/imx/imx-media-fim.c create mode 100644 drivers/staging/media/imx/imx-media-internal-sd.c create mode 100644 drivers/staging/media/imx/imx-media-of.c create mode 100644 drivers/staging/media/imx/imx-media-utils.c create mode 100644 drivers/staging/media/imx/imx-media.h create mode 100644 include/media/imx.h
Documentation/media/v4l-drivers/imx.rst new file +614 -0
@@ -0,0 +1,614 @@ +i.MX Video Capture Driver +========================= + +Introduction +------------ + +The Freescale i.MX5/6 contains an Image Processing Unit (IPU), which +handles the flow of image frames to and from capture devices and +display devices. + +For image capture, the IPU contains the following internal subunits: + +- Image DMA Controller (IDMAC) +- Camera Serial Interface (CSI) +- Image Converter (IC) +- Sensor Multi-FIFO Controller (SMFC) +- Image Rotator (IRT) +- Video De-Interlacing or Combining Block (VDIC) + +The IDMAC is the DMA controller for transfer of image frames to and from +memory. Various dedicated DMA channels exist for both video capture and +display paths. During transfer, the IDMAC is also capable of vertical +image flip, 8x8 block transfer (see IRT description), pixel component +re-ordering (for example UYVY to YUYV) within the same colorspace, and +even packed <--> planar conversion. It can also perform a simple +de-interlacing by interleaving even and odd lines during transfer +(without motion compensation which requires the VDIC). + +The CSI is the backend capture unit that interfaces directly with +camera sensors over Parallel, BT.656/1120, and MIPI CSI-2 busses. + +The IC handles color-space conversion, resizing (downscaling and +upscaling), horizontal flip, and 90/270 degree rotation operations. + +There are three independent "tasks" within the IC that can carry out +conversions concurrently: pre-process encoding, pre-process viewfinder, +and post-processing. Within each task, conversions are split into three +sections: downsizing section, main section (upsizing, flip, colorspace +conversion, and graphics plane combining), and rotation section. + +The IPU time-shares the IC task operations. The time-slice granularity +is one burst of eight pixels in the downsizing section, one image line +in the main processing section, one image frame in the rotation section. + +The SMFC is composed of four independent FIFOs that each can transfer +captured frames from sensors directly to memory concurrently via four +IDMAC channels. + +The IRT carries out 90 and 270 degree image rotation operations. The +rotation operation is carried out on 8x8 pixel blocks at a time. This +operation is supported by the IDMAC which handles the 8x8 block transfer +along with block reordering, in coordination with vertical flip. + +The VDIC handles the conversion of interlaced video to progressive, with +support for different motion compensation modes (low, medium, and high +motion). The deinterlaced output frames from the VDIC can be sent to the +IC pre-process viewfinder task for further conversions. The VDIC also +contains a Combiner that combines two image planes, with alpha blending +and color keying. + +In addition to the IPU internal subunits, there are also two units +outside the IPU that are also involved in video capture on i.MX: + +- MIPI CSI-2 Receiver for camera sensors with the MIPI CSI-2 bus + interface. This is a Synopsys DesignWare core. +- Two video multiplexers for selecting among multiple sensor inputs + to send to a CSI. + +For more info, refer to the latest versions of the i.MX5/6 reference +manuals [#f1]_ and [#f2]_. + + +Features +-------- + +Some of the features of this driver include: + +- Many different pipelines can be configured via media controller API, + that correspond to the hardware video capture pipelines supported in + the i.MX. + +- Supports parallel, BT.565, and MIPI CSI-2 interfaces. + +- Concurrent independent streams, by configuring pipelines to multiple + video capture interfaces using independent entities. + +- Scaling, color-space conversion, horizontal and vertical flip, and + image rotation via IC task subdevs. + +- Many pixel formats supported (RGB, packed and planar YUV, partial + planar YUV). + +- The VDIC subdev supports motion compensated de-interlacing, with three + motion compensation modes: low, medium, and high motion. Pipelines are + defined that allow sending frames to the VDIC subdev directly from the + CSI. There is also support in the future for sending frames to the + VDIC from memory buffers via a output/mem2mem devices. + +- Includes a Frame Interval Monitor (FIM) that can correct vertical sync + problems with the ADV718x video decoders. + + +Entities +-------- + +imx6-mipi-csi2 +-------------- + +This is the MIPI CSI-2 receiver entity. It has one sink pad to receive +the MIPI CSI-2 stream (usually from a MIPI CSI-2 camera sensor). It has +four source pads, corresponding to the four MIPI CSI-2 demuxed virtual +channel outputs. Multpiple source pads can be enabled to independently +stream from multiple virtual channels. + +This entity actually consists of two sub-blocks. One is the MIPI CSI-2 +core. This is a Synopsys Designware MIPI CSI-2 core. The other sub-block +is a "CSI-2 to IPU gasket". The gasket acts as a demultiplexer of the +four virtual channels streams, providing four separate parallel buses +containing each virtual channel that are routed to CSIs or video +multiplexers as described below. + +On i.MX6 solo/dual-lite, all four virtual channel buses are routed to +two video multiplexers. Both CSI0 and CSI1 can receive any virtual +channel, as selected by the video multiplexers. + +On i.MX6 Quad, virtual channel 0 is routed to IPU1-CSI0 (after selected +by a video mux), virtual channels 1 and 2 are hard-wired to IPU1-CSI1 +and IPU2-CSI0, respectively, and virtual channel 3 is routed to +IPU2-CSI1 (again selected by a video mux). + +ipuX_csiY_mux +------------- + +These are the video multiplexers. They have two or more sink pads to +select from either camera sensors with a parallel interface, or from +MIPI CSI-2 virtual channels from imx6-mipi-csi2 entity. They have a +single source pad that routes to a CSI (ipuX_csiY entities). + +On i.MX6 solo/dual-lite, there are two video mux entities. One sits +in front of IPU1-CSI0 to select between a parallel sensor and any of +the four MIPI CSI-2 virtual channels (a total of five sink pads). The +other mux sits in front of IPU1-CSI1, and again has five sink pads to +select between a parallel sensor and any of the four MIPI CSI-2 virtual +channels. + +On i.MX6 Quad, there are two video mux entities. One sits in front of +IPU1-CSI0 to select between a parallel sensor and MIPI CSI-2 virtual +channel 0 (two sink pads). The other mux sits in front of IPU2-CSI1 to +select between a parallel sensor and MIPI CSI-2 virtual channel 3 (two +sink pads). + +ipuX_csiY +--------- + +These are the CSI entities. They have a single sink pad receiving from +either a video mux or from a MIPI CSI-2 virtual channel as described +above. + +This entity has two source pads. The first source pad can link directly +to the ipuX_vdic entity or the ipuX_ic_prp entity, using hardware links +that require no IDMAC memory buffer transfer. + +When the direct source pad is routed to the ipuX_ic_prp entity, frames +from the CSI can be processed by one or both of the IC pre-processing +tasks. + +When the direct source pad is routed to the ipuX_vdic entity, the VDIC +will carry out motion-compensated de-interlace using "high motion" mode +(see description of ipuX_vdic entity). + +The second source pad sends video frames directly to memory buffers +via the SMFC and an IDMAC channel, bypassing IC pre-processing. This +source pad is routed to a capture device node, with a node name of the +format "ipuX_csiY capture". + +Note that since the IDMAC source pad makes use of an IDMAC channel, it +can do pixel reordering within the same colorspace. For example, the +sink pad can take UYVY2X8, but the IDMAC source pad can output YUYV2X8. +If the sink pad is receiving YUV, the output at the capture device can +also be converted to a planar YUV format such as YUV420. + +It will also perform simple de-interlace without motion compensation, +which is activated if the sink pad's field type is an interlaced type, +and the IDMAC source pad field type is set to none. + +This subdev can generate the following event when enabling the second +IDMAC source pad: + +- V4L2_EVENT_IMX_FRAME_INTERVAL_ERROR + +The user application can subscribe to this event from the ipuX_csiY +subdev node. This event is generated by the Frame Interval Monitor +(see below for more on the FIM). + +Cropping in ipuX_csiY +--------------------- + +The CSI supports cropping the incoming raw sensor frames. This is +implemented in the ipuX_csiY entities at the sink pad, using the +crop selection subdev API. + +The CSI also supports fixed divide-by-two downscaling indepently in +width and height. This is implemented in the ipuX_csiY entities at +the sink pad, using the compose selection subdev API. + +The output rectangle at the ipuX_csiY source pad is the same as +the compose rectangle at the sink pad. So the source pad rectangle +cannot be negotiated, it must be set using the compose selection +API at sink pad (if /w downscale is desired, otherwise source pad +rectangle is equal to incoming rectangle). + +To give an example of crop and /2 downscale, this will crop a +1280x960 input frame to 640x480, and then /2 downscale in both +dimensions to 320x240 (assumes ipu1_csi0 is linked to ipu1_csi0_mux): + +media-ctl -V "'ipu1_csi0_mux':2[fmt:UYVY2X8/1280x960]" +media-ctl -V "'ipu1_csi0':0[crop:(0,0)/640x480]" +media-ctl -V "'ipu1_csi0':0[compose:(0,0)/320x240]" + +Frame Skipping in ipuX_csiY +--------------------------- + +The CSI supports frame rate decimation, via frame skipping. Frame +rate decimation is specified by setting the frame intervals at +sink and source pads. The ipuX_csiY entity then applies the best +frame skip setting to the CSI to achieve the desired frame rate +at the source pad. + +The following example reduces an assumed incoming 60 Hz frame +rate by half at the IDMAC output source pad: + +media-ctl -V "'ipu1_csi0':0[fmt:UYVY2X8/640x480@1/60]" +media-ctl -V "'ipu1_csi0':2[fmt:UYVY2X8/640x480@1/30]" + +Frame Interval Monitor in ipuX_csiY +----------------------------------- + +The adv718x decoders can occasionally send corrupt fields during +NTSC/PAL signal re-sync (too little or too many video lines). When +this happens, the IPU triggers a mechanism to re-establish vertical +sync by adding 1 dummy line every frame, which causes a rolling effect +from image to image, and can last a long time before a stable image is +recovered. Or sometimes the mechanism doesn't work at all, causing a +permanent split image (one frame contains lines from two consecutive +captured images). + +From experiment it was found that during image rolling, the frame +intervals (elapsed time between two EOF's) drop below the nominal +value for the current standard, by about one frame time (60 usec), +and remain at that value until rolling stops. + +While the reason for this observation isn't known (the IPU dummy +line mechanism should show an increase in the intervals by 1 line +time every frame, not a fixed value), we can use it to detect the +corrupt fields using a frame interval monitor. If the FIM detects a +bad frame interval, the ipuX_csiY subdev will send the event +V4L2_EVENT_IMX_FRAME_INTERVAL_ERROR. Userland can register with +the FIM event notification on the ipuX_csiY subdev device node. +Userland can issue a streaming restart when this event is received +to correct the rolling/split image. + +The ipuX_csiY subdev includes custom controls to tweak some dials for +FIM. If one of these controls is changed during streaming, the FIM will +be reset and will continue at the new settings. + +- V4L2_CID_IMX_FIM_ENABLE + +Enable/disable the FIM. + +- V4L2_CID_IMX_FIM_NUM + +How many frame interval measurements to average before comparing against +the nominal frame interval reported by the sensor. This can reduce noise +caused by interrupt latency. + +- V4L2_CID_IMX_FIM_TOLERANCE_MIN + +If the averaged intervals fall outside nominal by this amount, in +microseconds, the V4L2_EVENT_IMX_FRAME_INTERVAL_ERROR event is sent. + +- V4L2_CID_IMX_FIM_TOLERANCE_MAX + +If any intervals are higher than this value, those samples are +discarded and do not enter into the average. This can be used to +discard really high interval errors that might be due to interrupt +latency from high system load. + +- V4L2_CID_IMX_FIM_NUM_SKIP + +How many frames to skip after a FIM reset or stream restart before +FIM begins to average intervals. + +- V4L2_CID_IMX_FIM_ICAP_CHANNEL +- V4L2_CID_IMX_FIM_ICAP_EDGE + +These controls will configure an input capture channel as the method +for measuring frame intervals. This is superior to the default method +of measuring frame intervals via EOF interrupt, since it is not subject +to uncertainty errors introduced by interrupt latency. + +Input capture requires hardware support. A VSYNC signal must be routed +to one of the i.MX6 input capture channel pads. + +V4L2_CID_IMX_FIM_ICAP_CHANNEL configures which i.MX6 input capture +channel to use. This must be 0 or 1. + +V4L2_CID_IMX_FIM_ICAP_EDGE configures which signal edge will trigger +input capture events. By default the input capture method is disabled +with a value of IRQ_TYPE_NONE. Set this control to IRQ_TYPE_EDGE_RISING, +IRQ_TYPE_EDGE_FALLING, or IRQ_TYPE_EDGE_BOTH to enable input capture, +triggered on the given signal edge(s). + +When input capture is disabled, frame intervals will be measured via +EOF interrupt. + + +ipuX_vdic +--------- + +The VDIC carries out motion compensated de-interlacing, with three +motion compensation modes: low, medium, and high motion. The mode is +specified with the menu control V4L2_CID_DEINTERLACING_MODE. It has +two sink pads and a single source pad. + +The direct sink pad receives from an ipuX_csiY direct pad. With this +link the VDIC can only operate in high motion mode. + +When the IDMAC sink pad is activated, it receives from an output +or mem2mem device node. With this pipeline, it can also operate +in low and medium modes, because these modes require receiving +frames from memory buffers. Note that an output or mem2mem device +is not implemented yet, so this sink pad currently has no links. + +The source pad routes to the IC pre-processing entity ipuX_ic_prp. + +ipuX_ic_prp +----------- + +This is the IC pre-processing entity. It acts as a router, routing +data from its sink pad to one or both of its source pads. + +It has a single sink pad. The sink pad can receive from the ipuX_csiY +direct pad, or from ipuX_vdic. + +This entity has two source pads. One source pad routes to the +pre-process encode task entity (ipuX_ic_prpenc), the other to the +pre-process viewfinder task entity (ipuX_ic_prpvf). Both source pads +can be activated at the same time if the sink pad is receiving from +ipuX_csiY. Only the source pad to the pre-process viewfinder task entity +can be activated if the sink pad is receiving from ipuX_vdic (frames +from the VDIC can only be processed by the pre-process viewfinder task). + +ipuX_ic_prpenc +-------------- + +This is the IC pre-processing encode entity. It has a single sink +pad from ipuX_ic_prp, and a single source pad. The source pad is +routed to a capture device node, with a node name of the format +"ipuX_ic_prpenc capture". + +This entity performs the IC pre-process encode task operations: +color-space conversion, resizing (downscaling and upscaling), +horizontal and vertical flip, and 90/270 degree rotation. Flip +and rotation are provided via standard V4L2 controls. + +Like the ipuX_csiY IDMAC source, it can also perform simple de-interlace +without motion compensation, and pixel reordering. + +ipuX_ic_prpvf +------------- + +This is the IC pre-processing viewfinder entity. It has a single sink +pad from ipuX_ic_prp, and a single source pad. The source pad is routed +to a capture device node, with a node name of the format +"ipuX_ic_prpvf capture". + +It is identical in operation to ipuX_ic_prpenc, with the same resizing +and CSC operations and flip/rotation controls. It will receive and +process de-interlaced frames from the ipuX_vdic if ipuX_ic_prp is +receiving from ipuX_vdic. + +Like the ipuX_csiY IDMAC source, it can perform simple de-interlace +without motion compensation. However, note that if the ipuX_vdic is +included in the pipeline (ipuX_ic_prp is receiving from ipuX_vdic), +it's not possible to use simple de-interlace in ipuX_ic_prpvf, since +the ipuX_vdic has already carried out de-interlacing (with motion +compensation) and therefore the field type output from ipuX_ic_prp can +only be none. + +Capture Pipelines +----------------- + +The following describe the various use-cases supported by the pipelines. + +The links shown do not include the backend sensor, video mux, or mipi +csi-2 receiver links. This depends on the type of sensor interface +(parallel or mipi csi-2). So these pipelines begin with: + +sensor -> ipuX_csiY_mux -> ... + +for parallel sensors, or: + +sensor -> imx6-mipi-csi2 -> (ipuX_csiY_mux) -> ... + +for mipi csi-2 sensors. The imx6-mipi-csi2 receiver may need to route +to the video mux (ipuX_csiY_mux) before sending to the CSI, depending +on the mipi csi-2 virtual channel, hence ipuX_csiY_mux is shown in +parenthesis. + +Unprocessed Video Capture: +-------------------------- + +Send frames directly from sensor to camera device interface node, with +no conversions, via ipuX_csiY IDMAC source pad: + +-> ipuX_csiY:2 -> ipuX_csiY capture + +IC Direct Conversions: +---------------------- + +This pipeline uses the preprocess encode entity to route frames directly +from the CSI to the IC, to carry out scaling up to 1024x1024 resolution, +CSC, flipping, and image rotation: + +-> ipuX_csiY:1 -> 0:ipuX_ic_prp:1 -> 0:ipuX_ic_prpenc:1 -> + ipuX_ic_prpenc capture + +Motion Compensated De-interlace: +-------------------------------- + +This pipeline routes frames from the CSI direct pad to the VDIC entity to +support motion-compensated de-interlacing (high motion mode only), +scaling up to 1024x1024, CSC, flip, and rotation: + +-> ipuX_csiY:1 -> 0:ipuX_vdic:2 -> 0:ipuX_ic_prp:2 -> + 0:ipuX_ic_prpvf:1 -> ipuX_ic_prpvf capture + + +Usage Notes +----------- + +To aid in configuration and for backward compatibility with V4L2 +applications that access controls only from video device nodes, the +capture device interfaces inherit controls from the active entities +in the current pipeline, so controls can be accessed either directly +from the subdev or from the active capture device interface. For +example, the FIM controls are available either from the ipuX_csiY +subdevs or from the active capture device. + +The following are specific usage notes for the Sabre* reference +boards: + + +SabreLite with OV5642 and OV5640 +-------------------------------- + +This platform requires the OmniVision OV5642 module with a parallel +camera interface, and the OV5640 module with a MIPI CSI-2 +interface. Both modules are available from Boundary Devices: + +https://boundarydevices.com/product/nit6x_5mp +https://boundarydevices.com/product/nit6x_5mp_mipi + +Note that if only one camera module is available, the other sensor +node can be disabled in the device tree. + +The OV5642 module is connected to the parallel bus input on the i.MX +internal video mux to IPU1 CSI0. It's i2c bus connects to i2c bus 2. + +The MIPI CSI-2 OV5640 module is connected to the i.MX internal MIPI CSI-2 +receiver, and the four virtual channel outputs from the receiver are +routed as follows: vc0 to the IPU1 CSI0 mux, vc1 directly to IPU1 CSI1, +vc2 directly to IPU2 CSI0, and vc3 to the IPU2 CSI1 mux. The OV5640 is +also connected to i2c bus 2 on the SabreLite, therefore the OV5642 and +OV5640 must not share the same i2c slave address. + +The following basic example configures unprocessed video capture +pipelines for both sensors. The OV5642 is routed to ipu1_csi0, and +the OV5640, transmitting on MIPI CSI-2 virtual channel 1 (which is +imx6-mipi-csi2 pad 2), is routed to ipu1_csi1. Both sensors are +configured to output 640x480, and the OV5642 outputs YUYV2X8, the +OV5640 UYVY2X8: + +.. code-block:: none + + # Setup links for OV5642 + media-ctl -l "'ov5642 1-0042':0 -> 'ipu1_csi0_mux':1[1]" + media-ctl -l "'ipu1_csi0_mux':2 -> 'ipu1_csi0':0[1]" + media-ctl -l "'ipu1_csi0':2 -> 'ipu1_csi0 capture':0[1]" + # Setup links for OV5640 + media-ctl -l "'ov5640 1-0040':0 -> 'imx6-mipi-csi2':0[1]" + media-ctl -l "'imx6-mipi-csi2':2 -> 'ipu1_csi1':0[1]" + media-ctl -l "'ipu1_csi1':2 -> 'ipu1_csi1 capture':0[1]" + # Configure pads for OV5642 pipeline + media-ctl -V "'ov5642 1-0042':0 [fmt:YUYV2X8/640x480 field:none]" + media-ctl -V "'ipu1_csi0_mux':2 [fmt:YUYV2X8/640x480 field:none]" + media-ctl -V "'ipu1_csi0':2 [fmt:AYUV32/640x480 field:none]" + # Configure pads for OV5640 pipeline + media-ctl -V "'ov5640 1-0040':0 [fmt:UYVY2X8/640x480 field:none]" + media-ctl -V "'imx6-mipi-csi2':2 [fmt:UYVY2X8/640x480 field:none]" + media-ctl -V "'ipu1_csi1':2 [fmt:AYUV32/640x480 field:none]" + +Streaming can then begin independently on the capture device nodes +"ipu1_csi0 capture" and "ipu1_csi1 capture". The v4l2-ctl tool can +be used to select any supported YUV pixelformat on the capture device +nodes, including planar. + +SabreAuto with ADV7180 decoder +------------------------------ + +On the SabreAuto, an on-board ADV7180 SD decoder is connected to the +parallel bus input on the internal video mux to IPU1 CSI0. + +The following example configures a pipeline to capture from the ADV7180 +video decoder, assuming NTSC 720x480 input signals, with Motion +Compensated de-interlacing. Pad field types assume the adv7180 outputs +"interlaced". $outputfmt can be any format supported by the ipu1_ic_prpvf +entity at its output pad: + +.. code-block:: none + + # Setup links + media-ctl -l "'adv7180 3-0021':0 -> 'ipu1_csi0_mux':1[1]" + media-ctl -l "'ipu1_csi0_mux':2 -> 'ipu1_csi0':0[1]" + media-ctl -l "'ipu1_csi0':1 -> 'ipu1_vdic':0[1]" + media-ctl -l "'ipu1_vdic':2 -> 'ipu1_ic_prp':0[1]" + media-ctl -l "'ipu1_ic_prp':2 -> 'ipu1_ic_prpvf':0[1]" + media-ctl -l "'ipu1_ic_prpvf':1 -> 'ipu1_ic_prpvf capture':0[1]" + # Configure pads + media-ctl -V "'adv7180 3-0021':0 [fmt:UYVY2X8/720x480]" + media-ctl -V "'ipu1_csi0_mux':2 [fmt:UYVY2X8/720x480 field:interlaced]" + media-ctl -V "'ipu1_csi0':1 [fmt:AYUV32/720x480 field:interlaced]" + media-ctl -V "'ipu1_vdic':2 [fmt:AYUV32/720x480 field:none]" + media-ctl -V "'ipu1_ic_prp':2 [fmt:AYUV32/720x480 field:none]" + media-ctl -V "'ipu1_ic_prpvf':1 [fmt:$outputfmt field:none]" + +Streaming can then begin on the capture device node at +"ipu1_ic_prpvf capture". The v4l2-ctl tool can be used to select any +supported YUV or RGB pixelformat on the capture device node. + +This platform accepts Composite Video analog inputs to the ADV7180 on +Ain1 (connector J42). + +SabreSD with MIPI CSI-2 OV5640 +------------------------------ + +Similarly to SabreLite, the SabreSD supports a parallel interface +OV5642 module on IPU1 CSI0, and a MIPI CSI-2 OV5640 module. The OV5642 +connects to i2c bus 1 and the OV5640 to i2c bus 2. + +The device tree for SabreSD includes OF graphs for both the parallel +OV5642 and the MIPI CSI-2 OV5640, but as of this writing only the MIPI +CSI-2 OV5640 has been tested, so the OV5642 node is currently disabled. +The OV5640 module connects to MIPI connector J5 (sorry I don't have the +compatible module part number or URL). + +The following example configures a direct conversion pipeline to capture +from the OV5640, transmitting on MIPI CSI-2 virtual channel 1. $sensorfmt +can be any format supported by the OV5640. $sensordim is the frame +dimension part of $sensorfmt (minus the mbus pixel code). $outputfmt can +be any format supported by the ipu1_ic_prpenc entity at its output pad: + +.. code-block:: none + + # Setup links + media-ctl -l "'ov5640 1-003c':0 -> 'imx6-mipi-csi2':0[1]" + media-ctl -l "'imx6-mipi-csi2':2 -> 'ipu1_csi1':0[1]" + media-ctl -l "'ipu1_csi1':1 -> 'ipu1_ic_prp':0[1]" + media-ctl -l "'ipu1_ic_prp':1 -> 'ipu1_ic_prpenc':0[1]" + media-ctl -l "'ipu1_ic_prpenc':1 -> 'ipu1_ic_prpenc capture':0[1]" + # Configure pads + media-ctl -V "'ov5640 1-003c':0 [fmt:$sensorfmt field:none]" + media-ctl -V "'imx6-mipi-csi2':2 [fmt:$sensorfmt field:none]" + media-ctl -V "'ipu1_csi1':1 [fmt:AYUV32/$sensordim field:none]" + media-ctl -V "'ipu1_ic_prp':1 [fmt:AYUV32/$sensordim field:none]" + media-ctl -V "'ipu1_ic_prpenc':1 [fmt:$outputfmt field:none]" + +Streaming can then begin on "ipu1_ic_prpenc capture" node. The v4l2-ctl +tool can be used to select any supported YUV or RGB pixelformat on the +capture device node. + + +Known Issues +------------ + +1. When using 90 or 270 degree rotation control at capture resolutions + near the IC resizer limit of 1024x1024, and combined with planar + pixel formats (YUV420, YUV422p), frame capture will often fail with + no end-of-frame interrupts from the IDMAC channel. To work around + this, use lower resolution and/or packed formats (YUYV, RGB3, etc.) + when 90 or 270 rotations are needed. + + +File list +--------- + +drivers/staging/media/imx/ +include/media/imx.h +include/linux/imx-media.h + +References +---------- + +.. [#f1] http://www.nxp.com/assets/documents/data/en/reference-manuals/IMX6DQRM.pdf +.. [#f2] http://www.nxp.com/assets/documents/data/en/reference-manuals/IMX6SDLRM.pdf + + +Authors +------- +Steve Longerbeam <[email protected]> +Philipp Zabel <[email protected]> +Russell King <[email protected]> + +Copyright (C) 2012-2017 Mentor Graphics Inc.
drivers/staging/media/Kconfig +2 -0
@@ -27,6 +27,8 @@ source "drivers/staging/media/cxd2099/Kconfig" source "drivers/staging/media/davinci_vpfe/Kconfig" +source "drivers/staging/media/imx/Kconfig" + source "drivers/staging/media/omap4iss/Kconfig" # Keep LIRC at the end, as it has sub-menus
drivers/staging/media/Makefile +1 -0
@@ -1,5 +1,6 @@ obj-$(CONFIG_I2C_BCM2048) += bcm2048/ obj-$(CONFIG_DVB_CXD2099) += cxd2099/ +obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx/ obj-$(CONFIG_LIRC_STAGING) += lirc/ obj-$(CONFIG_VIDEO_DM365_VPFE) += davinci_vpfe/ obj-$(CONFIG_VIDEO_OMAP4) += omap4iss/
drivers/staging/media/imx/Kconfig new file +7 -0
@@ -0,0 +1,7 @@ +config VIDEO_IMX_MEDIA + tristate "i.MX5/6 V4L2 media core driver" + depends on MEDIA_CONTROLLER && VIDEO_V4L2 && ARCH_MXC && IMX_IPUV3_CORE + select V4L2_FWNODE + ---help--- + Say yes here to enable support for video4linux media controller + driver for the i.MX5/6 SOC.
drivers/staging/media/imx/Makefile new file +5 -0
@@ -0,0 +1,5 @@ +imx-media-objs := imx-media-dev.o imx-media-internal-sd.o imx-media-of.o +imx-media-common-objs := imx-media-utils.o imx-media-fim.o + +obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media.o +obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media-common.o
drivers/staging/media/imx/imx-media-dev.c new file +666 -0
666 lines not on this page · raw message
drivers/staging/media/imx/imx-media-fim.c new file +494 -0
494 lines not on this page · raw message
drivers/staging/media/imx/imx-media-internal-sd.c new file +349 -0
349 lines not on this page · raw message
drivers/staging/media/imx/imx-media-of.c new file +270 -0
270 lines not on this page · raw message
drivers/staging/media/imx/imx-media-utils.c new file +834 -0
581 lines not on this page · raw message
drivers/staging/media/imx/imx-media.h new file +323 -0
323 lines not on this page · raw message
The rest of this diff is not shown. View the raw message.
Signature · 1 line
2.7.4
Part of the diff is not shown here. View the raw message.
Philipp Zabel
maintainerRe: [PATCH v8 19/34] media: Add i.MX media core driver
Quoted 13 lines · Steve Longerbeam
Add the core media driver for i.MX SOC.
Signed-off-by Steve Longerbeam <[email protected]>Switch from the
v4l2_of_APIs to thev4l2_fwnode_APIs.Signed-off-by Philipp Zabel <[email protected]>Add the bayer formats to imx-media's list of supported pixel and bus formats.
Signed-off-by Russell King <[email protected]>---
[...]
Quoted 6 lines
drivers/staging/media/imx/imx-media-dev.c new file +0
-0@@ -0,0 +1,666 @@
[...]
Quoted 7 lines
drivers/staging/media/imx/imx-media-dev.c new file +7
-0+/* +* adds given video device to given imx-media source pad vdev list. +* Continues upstream from the pad entity's sink pads. +*/ +static int imx_media_add_vdev_to_pad (struct imx_media_dev *imxmd, +struct imx_media_video_dev *vdev, +struct media_pad *srcpad)
drivers/staging/media/imx/imx-media-dev.c line 343
Quoted 10 lines
drivers/staging/media/imx/imx-media-dev.c new file +10
-0+{ +struct media_entity *entity = srcpad->entity; +struct imx_media_subdev *imxsd; +struct imx_media_pad *imxpad; +struct media_link *link; +struct v4l2_subdev *sd; +int i, vdev_idx, ret; + +if (!is_media_entity_v4l2_subdev (entity)) +return -EINVAL ;
Could we make this return 0, to just skip non-v4l2_subdev entities? Currently, imx_media_probe_complete silently fails with this -EINVAL if there is a tvp5150 connected due to the separate media entities that the tvp5150 driver creates for the input connectors (Composite0, for example).
regards
Philipp
Steve Longerbeam
maintainerRe: [PATCH v8 19/34] media: Add i.MX media core driver
Quoted 37 lines · Philipp Zabel
Quoted 13 lines · Steve Longerbeam
Add the core media driver for i.MX SOC.
Signed-off-by Steve Longerbeam <[email protected]>Switch from the
v4l2_of_APIs to thev4l2_fwnode_APIs.Signed-off-by Philipp Zabel <[email protected]>Add the bayer formats to imx-media's list of supported pixel and bus formats.
Signed-off-by Russell King <[email protected]>---
[...]
Quoted 6 lines
drivers/staging/media/imx/imx-media-dev.c new file +0
-0@@ -0,0 +1,666 @@[...]
Quoted 15 lines
drivers/staging/media/imx/imx-media-dev.c new file +15
-0+/* +* adds given video device to given imx-media source pad vdev list. +* Continues upstream from the pad entity's sink pads. +*/ +static int imx_media_add_vdev_to_pad (struct imx_media_dev *imxmd, +struct imx_media_video_dev *vdev, +struct media_pad *srcpad) +{ +struct media_entity *entity = srcpad->entity; +struct imx_media_subdev *imxsd; +struct imx_media_pad *imxpad; +struct media_link *link; +struct v4l2_subdev *sd; +int i, vdev_idx, ret; +
Quoted 9 lines
drivers/staging/media/imx/imx-media-dev.c
Quoted 2 lines
drivers/staging/media/imx/imx-media-dev.c new file +2
-0+if (!is_media_entity_v4l2_subdev (entity)) +return -EINVAL ;Could we make this return 0, to just skip non-v4l2_subdev entities? Currently,
imx_media_probe_completesilently fails with this-EINVALif there is a tvp5150 connected due to the separate media entities that the tvp5150 driver creates for the input connectors (Composite0, for example).
Right, I've made that change.
Steve
Steve Longerbeam
maintainerPATCH v9 19/34 media: Add i.MX media core driver
In reply to Steve LongerbeamAdd the core media driver for i.MX SOC.
Switch from the v4l2_of_ APIs to the v4l2_fwnode_ APIs.
Add the bayer formats to imx-media's list of supported pixel and bus formats.
---
Changes since v8 [1]:
- don't return error if the entity given to imx_media_add_vdev_to_pad() is not a v4l2-subdev, just skip it.
[1] https://patchwork.linuxtv.org/patch/41703/
--- Documentation/media/v4l-drivers/imx.rst | 614 ++++++++++++++++ drivers/staging/media/Kconfig | 2 + drivers/staging/media/Makefile | 1 + drivers/staging/media/imx/Kconfig | 7 + drivers/staging/media/imx/Makefile | 5 + drivers/staging/media/imx/imx-media-dev.c | 667 +++++++++++++++++ drivers/staging/media/imx/imx-media-fim.c | 494 +++++++++++++ drivers/staging/media/imx/imx-media-internal-sd.c | 349 +++++++++ drivers/staging/media/imx/imx-media-of.c | 270 +++++++ drivers/staging/media/imx/imx-media-utils.c | 834 ++++++++++++++++++++++ drivers/staging/media/imx/imx-media.h | 323 +++++++++ include/media/imx.h | 15 + include/uapi/linux/v4l2-controls.h | 4 + 13 files changed, 3585 insertions(+) create mode 100644 Documentation/media/v4l-drivers/imx.rst create mode 100644 drivers/staging/media/imx/Kconfig create mode 100644 drivers/staging/media/imx/Makefile create mode 100644 drivers/staging/media/imx/imx-media-dev.c create mode 100644 drivers/staging/media/imx/imx-media-fim.c create mode 100644 drivers/staging/media/imx/imx-media-internal-sd.c create mode 100644 drivers/staging/media/imx/imx-media-of.c create mode 100644 drivers/staging/media/imx/imx-media-utils.c create mode 100644 drivers/staging/media/imx/imx-media.h create mode 100644 include/media/imx.h
Documentation/media/v4l-drivers/imx.rst new file +614 -0
@@ -0,0 +1,614 @@ +i.MX Video Capture Driver +========================= + +Introduction +------------ + +The Freescale i.MX5/6 contains an Image Processing Unit (IPU), which +handles the flow of image frames to and from capture devices and +display devices. + +For image capture, the IPU contains the following internal subunits: + +- Image DMA Controller (IDMAC) +- Camera Serial Interface (CSI) +- Image Converter (IC) +- Sensor Multi-FIFO Controller (SMFC) +- Image Rotator (IRT) +- Video De-Interlacing or Combining Block (VDIC) + +The IDMAC is the DMA controller for transfer of image frames to and from +memory. Various dedicated DMA channels exist for both video capture and +display paths. During transfer, the IDMAC is also capable of vertical +image flip, 8x8 block transfer (see IRT description), pixel component +re-ordering (for example UYVY to YUYV) within the same colorspace, and +even packed <--> planar conversion. It can also perform a simple +de-interlacing by interleaving even and odd lines during transfer +(without motion compensation which requires the VDIC). + +The CSI is the backend capture unit that interfaces directly with +camera sensors over Parallel, BT.656/1120, and MIPI CSI-2 busses. + +The IC handles color-space conversion, resizing (downscaling and +upscaling), horizontal flip, and 90/270 degree rotation operations. + +There are three independent "tasks" within the IC that can carry out +conversions concurrently: pre-process encoding, pre-process viewfinder, +and post-processing. Within each task, conversions are split into three +sections: downsizing section, main section (upsizing, flip, colorspace +conversion, and graphics plane combining), and rotation section. + +The IPU time-shares the IC task operations. The time-slice granularity +is one burst of eight pixels in the downsizing section, one image line +in the main processing section, one image frame in the rotation section. + +The SMFC is composed of four independent FIFOs that each can transfer +captured frames from sensors directly to memory concurrently via four +IDMAC channels. + +The IRT carries out 90 and 270 degree image rotation operations. The +rotation operation is carried out on 8x8 pixel blocks at a time. This +operation is supported by the IDMAC which handles the 8x8 block transfer +along with block reordering, in coordination with vertical flip. + +The VDIC handles the conversion of interlaced video to progressive, with +support for different motion compensation modes (low, medium, and high +motion). The deinterlaced output frames from the VDIC can be sent to the +IC pre-process viewfinder task for further conversions. The VDIC also +contains a Combiner that combines two image planes, with alpha blending +and color keying. + +In addition to the IPU internal subunits, there are also two units +outside the IPU that are also involved in video capture on i.MX: + +- MIPI CSI-2 Receiver for camera sensors with the MIPI CSI-2 bus + interface. This is a Synopsys DesignWare core. +- Two video multiplexers for selecting among multiple sensor inputs + to send to a CSI. + +For more info, refer to the latest versions of the i.MX5/6 reference +manuals [#f1]_ and [#f2]_. + + +Features +-------- + +Some of the features of this driver include: + +- Many different pipelines can be configured via media controller API, + that correspond to the hardware video capture pipelines supported in + the i.MX. + +- Supports parallel, BT.565, and MIPI CSI-2 interfaces. + +- Concurrent independent streams, by configuring pipelines to multiple + video capture interfaces using independent entities. + +- Scaling, color-space conversion, horizontal and vertical flip, and + image rotation via IC task subdevs. + +- Many pixel formats supported (RGB, packed and planar YUV, partial + planar YUV). + +- The VDIC subdev supports motion compensated de-interlacing, with three + motion compensation modes: low, medium, and high motion. Pipelines are + defined that allow sending frames to the VDIC subdev directly from the + CSI. There is also support in the future for sending frames to the + VDIC from memory buffers via a output/mem2mem devices. + +- Includes a Frame Interval Monitor (FIM) that can correct vertical sync + problems with the ADV718x video decoders. + + +Entities +-------- + +imx6-mipi-csi2 +-------------- + +This is the MIPI CSI-2 receiver entity. It has one sink pad to receive +the MIPI CSI-2 stream (usually from a MIPI CSI-2 camera sensor). It has +four source pads, corresponding to the four MIPI CSI-2 demuxed virtual +channel outputs. Multpiple source pads can be enabled to independently +stream from multiple virtual channels. + +This entity actually consists of two sub-blocks. One is the MIPI CSI-2 +core. This is a Synopsys Designware MIPI CSI-2 core. The other sub-block +is a "CSI-2 to IPU gasket". The gasket acts as a demultiplexer of the +four virtual channels streams, providing four separate parallel buses +containing each virtual channel that are routed to CSIs or video +multiplexers as described below. + +On i.MX6 solo/dual-lite, all four virtual channel buses are routed to +two video multiplexers. Both CSI0 and CSI1 can receive any virtual +channel, as selected by the video multiplexers. + +On i.MX6 Quad, virtual channel 0 is routed to IPU1-CSI0 (after selected +by a video mux), virtual channels 1 and 2 are hard-wired to IPU1-CSI1 +and IPU2-CSI0, respectively, and virtual channel 3 is routed to +IPU2-CSI1 (again selected by a video mux). + +ipuX_csiY_mux +------------- + +These are the video multiplexers. They have two or more sink pads to +select from either camera sensors with a parallel interface, or from +MIPI CSI-2 virtual channels from imx6-mipi-csi2 entity. They have a +single source pad that routes to a CSI (ipuX_csiY entities). + +On i.MX6 solo/dual-lite, there are two video mux entities. One sits +in front of IPU1-CSI0 to select between a parallel sensor and any of +the four MIPI CSI-2 virtual channels (a total of five sink pads). The +other mux sits in front of IPU1-CSI1, and again has five sink pads to +select between a parallel sensor and any of the four MIPI CSI-2 virtual +channels. + +On i.MX6 Quad, there are two video mux entities. One sits in front of +IPU1-CSI0 to select between a parallel sensor and MIPI CSI-2 virtual +channel 0 (two sink pads). The other mux sits in front of IPU2-CSI1 to +select between a parallel sensor and MIPI CSI-2 virtual channel 3 (two +sink pads). + +ipuX_csiY +--------- + +These are the CSI entities. They have a single sink pad receiving from +either a video mux or from a MIPI CSI-2 virtual channel as described +above. + +This entity has two source pads. The first source pad can link directly +to the ipuX_vdic entity or the ipuX_ic_prp entity, using hardware links +that require no IDMAC memory buffer transfer. + +When the direct source pad is routed to the ipuX_ic_prp entity, frames +from the CSI can be processed by one or both of the IC pre-processing +tasks. + +When the direct source pad is routed to the ipuX_vdic entity, the VDIC +will carry out motion-compensated de-interlace using "high motion" mode +(see description of ipuX_vdic entity). + +The second source pad sends video frames directly to memory buffers +via the SMFC and an IDMAC channel, bypassing IC pre-processing. This +source pad is routed to a capture device node, with a node name of the +format "ipuX_csiY capture". + +Note that since the IDMAC source pad makes use of an IDMAC channel, it +can do pixel reordering within the same colorspace. For example, the +sink pad can take UYVY2X8, but the IDMAC source pad can output YUYV2X8. +If the sink pad is receiving YUV, the output at the capture device can +also be converted to a planar YUV format such as YUV420. + +It will also perform simple de-interlace without motion compensation, +which is activated if the sink pad's field type is an interlaced type, +and the IDMAC source pad field type is set to none. + +This subdev can generate the following event when enabling the second +IDMAC source pad: + +- V4L2_EVENT_IMX_FRAME_INTERVAL_ERROR + +The user application can subscribe to this event from the ipuX_csiY +subdev node. This event is generated by the Frame Interval Monitor +(see below for more on the FIM). + +Cropping in ipuX_csiY +--------------------- + +The CSI supports cropping the incoming raw sensor frames. This is +implemented in the ipuX_csiY entities at the sink pad, using the +crop selection subdev API. + +The CSI also supports fixed divide-by-two downscaling indepently in +width and height. This is implemented in the ipuX_csiY entities at +the sink pad, using the compose selection subdev API. + +The output rectangle at the ipuX_csiY source pad is the same as +the compose rectangle at the sink pad. So the source pad rectangle +cannot be negotiated, it must be set using the compose selection +API at sink pad (if /2 downscale is desired, otherwise source pad +rectangle is equal to incoming rectangle). + +To give an example of crop and /2 downscale, this will crop a +1280x960 input frame to 640x480, and then /2 downscale in both +dimensions to 320x240 (assumes ipu1_csi0 is linked to ipu1_csi0_mux): + +media-ctl -V "'ipu1_csi0_mux':2[fmt:UYVY2X8/1280x960]" +media-ctl -V "'ipu1_csi0':0[crop:(0,0)/640x480]" +media-ctl -V "'ipu1_csi0':0[compose:(0,0)/320x240]" + +Frame Skipping in ipuX_csiY +--------------------------- + +The CSI supports frame rate decimation, via frame skipping. Frame +rate decimation is specified by setting the frame intervals at +sink and source pads. The ipuX_csiY entity then applies the best +frame skip setting to the CSI to achieve the desired frame rate +at the source pad. + +The following example reduces an assumed incoming 60 Hz frame +rate by half at the IDMAC output source pad: + +media-ctl -V "'ipu1_csi0':0[fmt:UYVY2X8/640x480@1/60]" +media-ctl -V "'ipu1_csi0':2[fmt:UYVY2X8/640x480@1/30]" + +Frame Interval Monitor in ipuX_csiY +----------------------------------- + +The adv718x decoders can occasionally send corrupt fields during +NTSC/PAL signal re-sync (too little or too many video lines). When +this happens, the IPU triggers a mechanism to re-establish vertical +sync by adding 1 dummy line every frame, which causes a rolling effect +from image to image, and can last a long time before a stable image is +recovered. Or sometimes the mechanism doesn't work at all, causing a +permanent split image (one frame contains lines from two consecutive +captured images). + +From experiment it was found that during image rolling, the frame +intervals (elapsed time between two EOF's) drop below the nominal +value for the current standard, by about one frame time (60 usec), +and remain at that value until rolling stops. + +While the reason for this observation isn't known (the IPU dummy +line mechanism should show an increase in the intervals by 1 line +time every frame, not a fixed value), we can use it to detect the +corrupt fields using a frame interval monitor. If the FIM detects a +bad frame interval, the ipuX_csiY subdev will send the event +V4L2_EVENT_IMX_FRAME_INTERVAL_ERROR. Userland can register with +the FIM event notification on the ipuX_csiY subdev device node. +Userland can issue a streaming restart when this event is received +to correct the rolling/split image. + +The ipuX_csiY subdev includes custom controls to tweak some dials for +FIM. If one of these controls is changed during streaming, the FIM will +be reset and will continue at the new settings. + +- V4L2_CID_IMX_FIM_ENABLE + +Enable/disable the FIM. + +- V4L2_CID_IMX_FIM_NUM + +How many frame interval measurements to average before comparing against +the nominal frame interval reported by the sensor. This can reduce noise +caused by interrupt latency. + +- V4L2_CID_IMX_FIM_TOLERANCE_MIN + +If the averaged intervals fall outside nominal by this amount, in +microseconds, the V4L2_EVENT_IMX_FRAME_INTERVAL_ERROR event is sent. + +- V4L2_CID_IMX_FIM_TOLERANCE_MAX + +If any intervals are higher than this value, those samples are +discarded and do not enter into the average. This can be used to +discard really high interval errors that might be due to interrupt +latency from high system load. + +- V4L2_CID_IMX_FIM_NUM_SKIP + +How many frames to skip after a FIM reset or stream restart before +FIM begins to average intervals. + +- V4L2_CID_IMX_FIM_ICAP_CHANNEL +- V4L2_CID_IMX_FIM_ICAP_EDGE + +These controls will configure an input capture channel as the method +for measuring frame intervals. This is superior to the default method +of measuring frame intervals via EOF interrupt, since it is not subject +to uncertainty errors introduced by interrupt latency. + +Input capture requires hardware support. A VSYNC signal must be routed +to one of the i.MX6 input capture channel pads. + +V4L2_CID_IMX_FIM_ICAP_CHANNEL configures which i.MX6 input capture +channel to use. This must be 0 or 1. + +V4L2_CID_IMX_FIM_ICAP_EDGE configures which signal edge will trigger +input capture events. By default the input capture method is disabled +with a value of IRQ_TYPE_NONE. Set this control to IRQ_TYPE_EDGE_RISING, +IRQ_TYPE_EDGE_FALLING, or IRQ_TYPE_EDGE_BOTH to enable input capture, +triggered on the given signal edge(s). + +When input capture is disabled, frame intervals will be measured via +EOF interrupt. + + +ipuX_vdic +--------- + +The VDIC carries out motion compensated de-interlacing, with three +motion compensation modes: low, medium, and high motion. The mode is +specified with the menu control V4L2_CID_DEINTERLACING_MODE. It has +two sink pads and a single source pad. + +The direct sink pad receives from an ipuX_csiY direct pad. With this +link the VDIC can only operate in high motion mode. + +When the IDMAC sink pad is activated, it receives from an output +or mem2mem device node. With this pipeline, it can also operate +in low and medium modes, because these modes require receiving +frames from memory buffers. Note that an output or mem2mem device +is not implemented yet, so this sink pad currently has no links. + +The source pad routes to the IC pre-processing entity ipuX_ic_prp. + +ipuX_ic_prp +----------- + +This is the IC pre-processing entity. It acts as a router, routing +data from its sink pad to one or both of its source pads. + +It has a single sink pad. The sink pad can receive from the ipuX_csiY +direct pad, or from ipuX_vdic. + +This entity has two source pads. One source pad routes to the +pre-process encode task entity (ipuX_ic_prpenc), the other to the +pre-process viewfinder task entity (ipuX_ic_prpvf). Both source pads +can be activated at the same time if the sink pad is receiving from +ipuX_csiY. Only the source pad to the pre-process viewfinder task entity +can be activated if the sink pad is receiving from ipuX_vdic (frames +from the VDIC can only be processed by the pre-process viewfinder task). + +ipuX_ic_prpenc +-------------- + +This is the IC pre-processing encode entity. It has a single sink +pad from ipuX_ic_prp, and a single source pad. The source pad is +routed to a capture device node, with a node name of the format +"ipuX_ic_prpenc capture". + +This entity performs the IC pre-process encode task operations: +color-space conversion, resizing (downscaling and upscaling), +horizontal and vertical flip, and 90/270 degree rotation. Flip +and rotation are provided via standard V4L2 controls. + +Like the ipuX_csiY IDMAC source, it can also perform simple de-interlace +without motion compensation, and pixel reordering. + +ipuX_ic_prpvf +------------- + +This is the IC pre-processing viewfinder entity. It has a single sink +pad from ipuX_ic_prp, and a single source pad. The source pad is routed +to a capture device node, with a node name of the format +"ipuX_ic_prpvf capture". + +It is identical in operation to ipuX_ic_prpenc, with the same resizing +and CSC operations and flip/rotation controls. It will receive and +process de-interlaced frames from the ipuX_vdic if ipuX_ic_prp is +receiving from ipuX_vdic. + +Like the ipuX_csiY IDMAC source, it can perform simple de-interlace +without motion compensation. However, note that if the ipuX_vdic is +included in the pipeline (ipuX_ic_prp is receiving from ipuX_vdic), +it's not possible to use simple de-interlace in ipuX_ic_prpvf, since +the ipuX_vdic has already carried out de-interlacing (with motion +compensation) and therefore the field type output from ipuX_ic_prp can +only be none. + +Capture Pipelines +----------------- + +The following describe the various use-cases supported by the pipelines. + +The links shown do not include the backend sensor, video mux, or mipi +csi-2 receiver links. This depends on the type of sensor interface +(parallel or mipi csi-2). So these pipelines begin with: + +sensor -> ipuX_csiY_mux -> ... + +for parallel sensors, or: + +sensor -> imx6-mipi-csi2 -> (ipuX_csiY_mux) -> ... + +for mipi csi-2 sensors. The imx6-mipi-csi2 receiver may need to route +to the video mux (ipuX_csiY_mux) before sending to the CSI, depending +on the mipi csi-2 virtual channel, hence ipuX_csiY_mux is shown in +parenthesis. + +Unprocessed Video Capture: +-------------------------- + +Send frames directly from sensor to camera device interface node, with +no conversions, via ipuX_csiY IDMAC source pad: + +-> ipuX_csiY:2 -> ipuX_csiY capture + +IC Direct Conversions: +---------------------- + +This pipeline uses the preprocess encode entity to route frames directly +from the CSI to the IC, to carry out scaling up to 1024x1024 resolution, +CSC, flipping, and image rotation: + +-> ipuX_csiY:1 -> 0:ipuX_ic_prp:1 -> 0:ipuX_ic_prpenc:1 -> + ipuX_ic_prpenc capture + +Motion Compensated De-interlace: +-------------------------------- + +This pipeline routes frames from the CSI direct pad to the VDIC entity to +support motion-compensated de-interlacing (high motion mode only), +scaling up to 1024x1024, CSC, flip, and rotation: + +-> ipuX_csiY:1 -> 0:ipuX_vdic:2 -> 0:ipuX_ic_prp:2 -> + 0:ipuX_ic_prpvf:1 -> ipuX_ic_prpvf capture + + +Usage Notes +----------- + +To aid in configuration and for backward compatibility with V4L2 +applications that access controls only from video device nodes, the +capture device interfaces inherit controls from the active entities +in the current pipeline, so controls can be accessed either directly +from the subdev or from the active capture device interface. For +example, the FIM controls are available either from the ipuX_csiY +subdevs or from the active capture device. + +The following are specific usage notes for the Sabre* reference +boards: + + +SabreLite with OV5642 and OV5640 +-------------------------------- + +This platform requires the OmniVision OV5642 module with a parallel +camera interface, and the OV5640 module with a MIPI CSI-2 +interface. Both modules are available from Boundary Devices: + +https://boundarydevices.com/product/nit6x_5mp +https://boundarydevices.com/product/nit6x_5mp_mipi + +Note that if only one camera module is available, the other sensor +node can be disabled in the device tree. + +The OV5642 module is connected to the parallel bus input on the i.MX +internal video mux to IPU1 CSI0. It's i2c bus connects to i2c bus 2. + +The MIPI CSI-2 OV5640 module is connected to the i.MX internal MIPI CSI-2 +receiver, and the four virtual channel outputs from the receiver are +routed as follows: vc0 to the IPU1 CSI0 mux, vc1 directly to IPU1 CSI1, +vc2 directly to IPU2 CSI0, and vc3 to the IPU2 CSI1 mux. The OV5640 is +also connected to i2c bus 2 on the SabreLite, therefore the OV5642 and +OV5640 must not share the same i2c slave address. + +The following basic example configures unprocessed video capture +pipelines for both sensors. The OV5642 is routed to ipu1_csi0, and +the OV5640, transmitting on MIPI CSI-2 virtual channel 1 (which is +imx6-mipi-csi2 pad 2), is routed to ipu1_csi1. Both sensors are +configured to output 640x480, and the OV5642 outputs YUYV2X8, the +OV5640 UYVY2X8: + +.. code-block:: none + + # Setup links for OV5642 + media-ctl -l "'ov5642 1-0042':0 -> 'ipu1_csi0_mux':1[1]" + media-ctl -l "'ipu1_csi0_mux':2 -> 'ipu1_csi0':0[1]" + media-ctl -l "'ipu1_csi0':2 -> 'ipu1_csi0 capture':0[1]" + # Setup links for OV5640 + media-ctl -l "'ov5640 1-0040':0 -> 'imx6-mipi-csi2':0[1]" + media-ctl -l "'imx6-mipi-csi2':2 -> 'ipu1_csi1':0[1]" + media-ctl -l "'ipu1_csi1':2 -> 'ipu1_csi1 capture':0[1]" + # Configure pads for OV5642 pipeline + media-ctl -V "'ov5642 1-0042':0 [fmt:YUYV2X8/640x480 field:none]" + media-ctl -V "'ipu1_csi0_mux':2 [fmt:YUYV2X8/640x480 field:none]" + media-ctl -V "'ipu1_csi0':2 [fmt:AYUV32/640x480 field:none]" + # Configure pads for OV5640 pipeline + media-ctl -V "'ov5640 1-0040':0 [fmt:UYVY2X8/640x480 field:none]" + media-ctl -V "'imx6-mipi-csi2':2 [fmt:UYVY2X8/640x480 field:none]" + media-ctl -V "'ipu1_csi1':2 [fmt:AYUV32/640x480 field:none]" + +Streaming can then begin independently on the capture device nodes +"ipu1_csi0 capture" and "ipu1_csi1 capture". The v4l2-ctl tool can +be used to select any supported YUV pixelformat on the capture device +nodes, including planar. + +SabreAuto with ADV7180 decoder +------------------------------ + +On the SabreAuto, an on-board ADV7180 SD decoder is connected to the +parallel bus input on the internal video mux to IPU1 CSI0. + +The following example configures a pipeline to capture from the ADV7180 +video decoder, assuming NTSC 720x480 input signals, with Motion +Compensated de-interlacing. Pad field types assume the adv7180 outputs +"interlaced". $outputfmt can be any format supported by the ipu1_ic_prpvf +entity at its output pad: + +.. code-block:: none + + # Setup links + media-ctl -l "'adv7180 3-0021':0 -> 'ipu1_csi0_mux':1[1]" + media-ctl -l "'ipu1_csi0_mux':2 -> 'ipu1_csi0':0[1]" + media-ctl -l "'ipu1_csi0':1 -> 'ipu1_vdic':0[1]" + media-ctl -l "'ipu1_vdic':2 -> 'ipu1_ic_prp':0[1]" + media-ctl -l "'ipu1_ic_prp':2 -> 'ipu1_ic_prpvf':0[1]" + media-ctl -l "'ipu1_ic_prpvf':1 -> 'ipu1_ic_prpvf capture':0[1]" + # Configure pads + media-ctl -V "'adv7180 3-0021':0 [fmt:UYVY2X8/720x480]" + media-ctl -V "'ipu1_csi0_mux':2 [fmt:UYVY2X8/720x480 field:interlaced]" + media-ctl -V "'ipu1_csi0':1 [fmt:AYUV32/720x480 field:interlaced]" + media-ctl -V "'ipu1_vdic':2 [fmt:AYUV32/720x480 field:none]" + media-ctl -V "'ipu1_ic_prp':2 [fmt:AYUV32/720x480 field:none]" + media-ctl -V "'ipu1_ic_prpvf':1 [fmt:$outputfmt field:none]" + +Streaming can then begin on the capture device node at +"ipu1_ic_prpvf capture". The v4l2-ctl tool can be used to select any +supported YUV or RGB pixelformat on the capture device node. + +This platform accepts Composite Video analog inputs to the ADV7180 on +Ain1 (connector J42). + +SabreSD with MIPI CSI-2 OV5640 +------------------------------ + +Similarly to SabreLite, the SabreSD supports a parallel interface +OV5642 module on IPU1 CSI0, and a MIPI CSI-2 OV5640 module. The OV5642 +connects to i2c bus 1 and the OV5640 to i2c bus 2. + +The device tree for SabreSD includes OF graphs for both the parallel +OV5642 and the MIPI CSI-2 OV5640, but as of this writing only the MIPI +CSI-2 OV5640 has been tested, so the OV5642 node is currently disabled. +The OV5640 module connects to MIPI connector J5 (sorry I don't have the +compatible module part number or URL). + +The following example configures a direct conversion pipeline to capture +from the OV5640, transmitting on MIPI CSI-2 virtual channel 1. $sensorfmt +can be any format supported by the OV5640. $sensordim is the frame +dimension part of $sensorfmt (minus the mbus pixel code). $outputfmt can +be any format supported by the ipu1_ic_prpenc entity at its output pad: + +.. code-block:: none + + # Setup links + media-ctl -l "'ov5640 1-003c':0 -> 'imx6-mipi-csi2':0[1]" + media-ctl -l "'imx6-mipi-csi2':2 -> 'ipu1_csi1':0[1]" + media-ctl -l "'ipu1_csi1':1 -> 'ipu1_ic_prp':0[1]" + media-ctl -l "'ipu1_ic_prp':1 -> 'ipu1_ic_prpenc':0[1]" + media-ctl -l "'ipu1_ic_prpenc':1 -> 'ipu1_ic_prpenc capture':0[1]" + # Configure pads + media-ctl -V "'ov5640 1-003c':0 [fmt:$sensorfmt field:none]" + media-ctl -V "'imx6-mipi-csi2':2 [fmt:$sensorfmt field:none]" + media-ctl -V "'ipu1_csi1':1 [fmt:AYUV32/$sensordim field:none]" + media-ctl -V "'ipu1_ic_prp':1 [fmt:AYUV32/$sensordim field:none]" + media-ctl -V "'ipu1_ic_prpenc':1 [fmt:$outputfmt field:none]" + +Streaming can then begin on "ipu1_ic_prpenc capture" node. The v4l2-ctl +tool can be used to select any supported YUV or RGB pixelformat on the +capture device node. + + +Known Issues +------------ + +1. When using 90 or 270 degree rotation control at capture resolutions + near the IC resizer limit of 1024x1024, and combined with planar + pixel formats (YUV420, YUV422p), frame capture will often fail with + no end-of-frame interrupts from the IDMAC channel. To work around + this, use lower resolution and/or packed formats (YUYV, RGB3, etc.) + when 90 or 270 rotations are needed. + + +File list +--------- + +drivers/staging/media/imx/ +include/media/imx.h +include/linux/imx-media.h + +References +---------- + +.. [#f1] http://www.nxp.com/assets/documents/data/en/reference-manuals/IMX6DQRM.pdf +.. [#f2] http://www.nxp.com/assets/documents/data/en/reference-manuals/IMX6SDLRM.pdf + + +Authors +------- +Steve Longerbeam <[email protected]> +Philipp Zabel <[email protected]> +Russell King <[email protected]> + +Copyright (C) 2012-2017 Mentor Graphics Inc.
drivers/staging/media/Kconfig +2 -0
@@ -27,6 +27,8 @@ source "drivers/staging/media/cxd2099/Kconfig" source "drivers/staging/media/davinci_vpfe/Kconfig" +source "drivers/staging/media/imx/Kconfig" + source "drivers/staging/media/omap4iss/Kconfig" # Keep LIRC at the end, as it has sub-menus
drivers/staging/media/Makefile +1 -0
@@ -1,5 +1,6 @@ obj-$(CONFIG_I2C_BCM2048) += bcm2048/ obj-$(CONFIG_DVB_CXD2099) += cxd2099/ +obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx/ obj-$(CONFIG_LIRC_STAGING) += lirc/ obj-$(CONFIG_VIDEO_DM365_VPFE) += davinci_vpfe/ obj-$(CONFIG_VIDEO_OMAP4) += omap4iss/
drivers/staging/media/imx/Kconfig new file +7 -0
@@ -0,0 +1,7 @@ +config VIDEO_IMX_MEDIA + tristate "i.MX5/6 V4L2 media core driver" + depends on MEDIA_CONTROLLER && VIDEO_V4L2 && ARCH_MXC && IMX_IPUV3_CORE + select V4L2_FWNODE + ---help--- + Say yes here to enable support for video4linux media controller + driver for the i.MX5/6 SOC.
drivers/staging/media/imx/Makefile new file +5 -0
@@ -0,0 +1,5 @@ +imx-media-objs := imx-media-dev.o imx-media-internal-sd.o imx-media-of.o +imx-media-common-objs := imx-media-utils.o imx-media-fim.o + +obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media.o +obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media-common.o
drivers/staging/media/imx/imx-media-dev.c new file +667 -0
667 lines not on this page · raw message
drivers/staging/media/imx/imx-media-fim.c new file +494 -0
494 lines not on this page · raw message
drivers/staging/media/imx/imx-media-internal-sd.c new file +349 -0
349 lines not on this page · raw message
drivers/staging/media/imx/imx-media-of.c new file +270 -0
270 lines not on this page · raw message
drivers/staging/media/imx/imx-media-utils.c new file +834 -0
580 lines not on this page · raw message
drivers/staging/media/imx/imx-media.h new file +323 -0
323 lines not on this page · raw message
The rest of this diff is not shown. View the raw message.
Signature · 1 line
2.7.4
Part of the diff is not shown here. View the raw message.
Steve Longerbeam
maintainerPATCH v8 20/34 media: imx: Add a TODO file
In reply to Steve LongerbeamAdd a TODO file.
--- drivers/staging/media/imx/TODO | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) create mode 100644 drivers/staging/media/imx/TODO
drivers/staging/media/imx/TODO new file +23 -0
@@ -0,0 +1,23 @@ + +- Clean up and move the ov5642 subdev driver to drivers/media/i2c, or + merge support for OV5642 into drivers/media/i2c/ov5640.c, and create + the binding docs for it. + +- The Frame Interval Monitor could be exported to v4l2-core for + general use. + +- At driver load time, the device-tree node that is the original source + (the "sensor"), is parsed to record its media bus configuration, and + this info is required in imx-media-csi.c to setup the CSI. + Laurent Pinchart argues that instead the CSI subdev should call its + neighbor's g_mbus_config op (which should be propagated if necessary) + to get this info. However Hans Verkuil is planning to remove the + g_mbus_config op. For now this driver uses the parsed DT mbus config + method until this issue is resolved. + +- This media driver supports inheriting V4L2 controls to the + video capture devices, from the subdevices in the capture device's + pipeline. The controls for each capture device are updated in the + link_notify callback when the pipeline is modified. It should be + decided whether this feature is useful enough to make it generally + available by exporting to v4l2-core.
Signature · 1 line
2.7.4
Steve Longerbeam
maintainerPATCH v8 21/34 media: imx: Add Capture Device Interface
In reply to Steve LongerbeamThis is the capture device interface driver that provides the v4l2 user interface. Frames can be received from various sources:
- directly from CSI for capturing unconverted images directly from camera sensors.
- from the IC pre-process encode task.
- from the IC pre-process viewfinder task.
--- drivers/staging/media/imx/Makefile | 1 + drivers/staging/media/imx/imx-media-capture.c | 702 ++++++++++++++++++++++++++ 2 files changed, 703 insertions(+) create mode 100644 drivers/staging/media/imx/imx-media-capture.c
drivers/staging/media/imx/Makefile +1 -0
@@ -3,3 +3,4 @@ imx-media-common-objs := imx-media-utils.o imx-media-fim.o obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media.o obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media-common.o +obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media-capture.o
drivers/staging/media/imx/imx-media-capture.c new file +702 -0
702 lines not on this page · raw message
Signature · 1 line
2.7.4
Steve Longerbeam
maintainerPATCH v8 22/34 media: imx: Add CSI subdev driver
In reply to Steve LongerbeamThis is a media entity subdevice for the i.MX Camera Sensor Interface module.
- Added support for negotiation of frame intervals.
- Fixed cropping rectangle negotiation at input and output pads.
- Added support for /2 downscaling, if the output pad dimension(s) are 1/2 the crop dimension(s) at csi_setup() time.
--- drivers/staging/media/imx/Kconfig | 14 + drivers/staging/media/imx/Makefile | 2 + drivers/staging/media/imx/imx-media-csi.c | 1472 +++++++++++++++++++++++++++++ 3 files changed, 1488 insertions(+) create mode 100644 drivers/staging/media/imx/imx-media-csi.c
drivers/staging/media/imx/Kconfig +14 -0
@@ -5,3 +5,17 @@ config VIDEO_IMX_MEDIA ---help--- Say yes here to enable support for video4linux media controller driver for the i.MX5/6 SOC. + +if VIDEO_IMX_MEDIA +menu "i.MX5/6 Media Sub devices" + +config VIDEO_IMX_CSI + tristate "i.MX5/6 Camera Sensor Interface driver" + depends on VIDEO_IMX_MEDIA && VIDEO_DEV && I2C + select VIDEOBUF2_DMA_CONTIG + default y + ---help--- + A video4linux camera sensor interface driver for i.MX5/6. + +endmenu +endif
drivers/staging/media/imx/Makefile +2 -0
@@ -4,3 +4,5 @@ imx-media-common-objs := imx-media-utils.o imx-media-fim.o obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media.o obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media-common.o obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media-capture.o + +obj-$(CONFIG_VIDEO_IMX_CSI) += imx-media-csi.o
drivers/staging/media/imx/imx-media-csi.c new file +1472 -0
1,472 lines not on this page · raw message
Signature · 1 line
2.7.4
Steve Longerbeam
maintainerPATCH v8 23/34 media: imx: Add VDIC subdev driver
In reply to Steve LongerbeamThis is a media entity subdevice driver for the i.MX Video De-Interlacing or Combining Block. So far this entity does not implement the Combining function but only motion compensated deinterlacing. Video frames are received from the CSI and are routed to the IC PRPVF entity.
--- drivers/staging/media/imx/Makefile | 1 + drivers/staging/media/imx/imx-media-vdic.c | 1004 ++++++++++++++++++++++++++++ 2 files changed, 1005 insertions(+) create mode 100644 drivers/staging/media/imx/imx-media-vdic.c
drivers/staging/media/imx/Makefile +1 -0
@@ -4,5 +4,6 @@ imx-media-common-objs := imx-media-utils.o imx-media-fim.o obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media.o obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media-common.o obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media-capture.o +obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media-vdic.o obj-$(CONFIG_VIDEO_IMX_CSI) += imx-media-csi.o
drivers/staging/media/imx/imx-media-vdic.c new file +1004 -0
1,004 lines not on this page · raw message
Signature · 1 line
2.7.4
Steve Longerbeam
maintainerPATCH v8 24/34 media: imx: Add IC subdev drivers
In reply to Steve LongerbeamThis is a set of three media entity subdevice drivers for the i.MX Image Converter:
- Pre-process Router: Takes input frames from CSI0, CSI1, or VDIC. Two output pads enable either or both of the preprocess tasks below. If the input is from one of the CSIs, both proprocess task links can be enabled to process frames from that CSI simultaneously. If the input is the VDIC, only the Pre-processing Viewfinder task link can be enabled.
- Pre-processing Encode task: video frames are routed directly from the CSI and can be scaled, color-space converted, and rotated. Scaled output is limited to 1024x1024 resolution. Output frames are routed to the prpenc capture device.
- Pre-processing Viewfinder task: this task can perform the same conversions as the pre-process encode task, but in addition can be used for hardware motion compensated deinterlacing. Frames can come either directly from the CSI or from the VDIC. Scaled output is limited to 1024x1024 resolution. Output frames are routed to the prpvf capture device.
--- drivers/staging/media/imx/Makefile | 2 + drivers/staging/media/imx/imx-ic-common.c | 113 +++ drivers/staging/media/imx/imx-ic-prp.c | 513 +++++++++++ drivers/staging/media/imx/imx-ic-prpencvf.c | 1299 +++++++++++++++++++++++++++ drivers/staging/media/imx/imx-ic.h | 38 + 5 files changed, 1965 insertions(+) create mode 100644 drivers/staging/media/imx/imx-ic-common.c create mode 100644 drivers/staging/media/imx/imx-ic-prp.c create mode 100644 drivers/staging/media/imx/imx-ic-prpencvf.c create mode 100644 drivers/staging/media/imx/imx-ic.h
drivers/staging/media/imx/Makefile +2 -0
@@ -1,9 +1,11 @@ imx-media-objs := imx-media-dev.o imx-media-internal-sd.o imx-media-of.o imx-media-common-objs := imx-media-utils.o imx-media-fim.o +imx-media-ic-objs := imx-ic-common.o imx-ic-prp.o imx-ic-prpencvf.o obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media.o obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media-common.o obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media-capture.o obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media-vdic.o +obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media-ic.o obj-$(CONFIG_VIDEO_IMX_CSI) += imx-media-csi.o
drivers/staging/media/imx/imx-ic-common.c new file +113 -0
113 lines not on this page · raw message
drivers/staging/media/imx/imx-ic-prp.c new file +513 -0
513 lines not on this page · raw message
drivers/staging/media/imx/imx-ic-prpencvf.c new file +1299 -0
1,299 lines not on this page · raw message
drivers/staging/media/imx/imx-ic.h new file +38 -0
@@ -0,0 +1,38 @@ +/* +* V4L2 Image Converter Subdev for Freescale i.MX5/6 SOC +* +* Copyright (c) 2016 Mentor Graphics Inc. +* +* This program is free software; you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published by +* the Free Software Foundation; either version 2 of the License, or +* (at your option) any later version. +*/ +#ifndef _IMX_IC_H +#define _IMX_IC_H + +#include <media/v4l2-subdev.h> + +struct imx_ic_priv { +struct device *dev; +struct v4l2_subdev sd; +int ipu_id; +int task_id; +void *prp_priv; +void *task_priv; +}; + +struct imx_ic_ops { +const struct v4l2_subdev_ops *subdev_ops; +const struct v4l2_subdev_internal_ops *internal_ops; +const struct media_entity_operations *entity_ops; + +int (*init)(struct imx_ic_priv *ic_priv); +void (*remove)(struct imx_ic_priv *ic_priv); +}; + +extern struct imx_ic_ops imx_ic_prp_ops; +extern struct imx_ic_ops imx_ic_prpencvf_ops; +extern struct imx_ic_ops imx_ic_pp_ops; + +#endif
Signature · 1 line
2.7.4
Steve Longerbeam
maintainerPATCH v8 25/34 media: imx: Add MIPI CSI-2 Receiver subdev driver
In reply to Steve LongerbeamAdds MIPI CSI-2 Receiver subdev driver. This subdev is required for sensors with a MIPI CSI2 interface.
- Switch from the v4l2_of_ APIs to the v4l2_fwnode_ APIs.
- Add the function csi2ipu_gasket_init() to initialize the gasket at s_power(ON). The gasket needs to be programmed with the correct color component ordering to handle UYVY vs. YUYV ordered mbus formats from sensors. Note that the description of the CSI2IPU_GASKET register in the i.MX6 reference manual is wrong w.r.t bit 2 (the manual refers to this register as CSI2_SW_RST): setting bit 2 selects YUYV order, not UYVY.
--- drivers/staging/media/imx/Makefile | 1 + drivers/staging/media/imx/imx6-mipi-csi2.c | 697 +++++++++++++++++++++++++++++ 2 files changed, 698 insertions(+) create mode 100644 drivers/staging/media/imx/imx6-mipi-csi2.c
drivers/staging/media/imx/Makefile +1 -0
@@ -9,3 +9,4 @@ obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media-vdic.o obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media-ic.o obj-$(CONFIG_VIDEO_IMX_CSI) += imx-media-csi.o +obj-$(CONFIG_VIDEO_IMX_CSI) += imx6-mipi-csi2.o
drivers/staging/media/imx/imx6-mipi-csi2.c new file +697 -0
697 lines not on this page · raw message
Signature · 1 line
2.7.4
Steve Longerbeam
maintainerPATCH v8 26/34 media: imx: csi: add support for bayer formats
In reply to Steve LongerbeamFrom: Russell King <[email protected]>
Bayer formats must be treated as generic data and passthrough mode must be used. Add the correct setup for these formats.
- added check to csi_link_validate() to verify that destination is IDMAC output pad when passthrough conditions exist: bayer formats and 16-bit parallel buses.
--- drivers/staging/media/imx/imx-media-csi.c | 74 ++++++++++++++++++++++++------- 1 file changed, 57 insertions(+), 17 deletions(-)
drivers/staging/media/imx/imx-media-csi.c +57 -17
@@ -287,10 +287,11 @@ static int csi_idmac_setup_channel(struct csi_priv *priv)struct imx_media_video_dev *vdev = priv->vdev;struct v4l2_fwnode_endpoint *sensor_ep;struct v4l2_mbus_framefmt *infmt; -unsigned int burst_size;struct ipu_image image; +u32 passthrough_bits;dma_addr_t phys[2 ];bool passthrough; +u32 burst_size;int ret; infmt = &priv->format_mbus[CSI_SINK_PAD ];
@@ -308,24 +309,52 @@ static int csi_idmac_setup_channel(struct csi_priv *priv) image.phys0 = phys[0 ]; image.phys1 = phys[1 ]; - ret =ipu_cpmem_set_image (priv->idmac_ch, &image); -if (ret) - goto unsetup_vb2; - - burst_size = (image.pix.width &0xf ) ?8 :16 ; - -ipu_cpmem_set_burstsize (priv->idmac_ch, burst_size); -/* -* If the sensor uses 16-bit parallel CSI bus, we must handle -* the data internally in the IPU as 16-bit generic, aka -* passthrough mode. +* Check for conditions that require the IPU to handle the +* data internally as generic data, aka passthrough mode: +* - raw bayer formats +* - the sensor bus is 16-bit parallel */ - passthrough = (sensor_ep->bus_type !=V4L2_MBUS_CSI2 && - sensor_ep->bus.parallel.bus_width >=16 ); +switch (image.pix.pixelformat) { +case V4L2_PIX_FMT_SBGGR8 : +case V4L2_PIX_FMT_SGBRG8 : +case V4L2_PIX_FMT_SGRBG8 : +case V4L2_PIX_FMT_SRGGB8 : + burst_size =8 ; + passthrough = true; + passthrough_bits =8 ; +break ; +case V4L2_PIX_FMT_SBGGR16 : +case V4L2_PIX_FMT_SGBRG16 : +case V4L2_PIX_FMT_SGRBG16 : +case V4L2_PIX_FMT_SRGGB16 : + burst_size =4 ; + passthrough = true; + passthrough_bits =16 ; +break ; +default : + burst_size = (image.pix.width &0xf ) ?8 :16 ; + passthrough = (sensor_ep->bus_type !=V4L2_MBUS_CSI2 && + sensor_ep->bus.parallel.bus_width >=16 ); + passthrough_bits =16 ; +break ; + } -if (passthrough) -ipu_cpmem_set_format_passthrough (priv->idmac_ch,16 ); +if (passthrough) { +ipu_cpmem_set_resolution (priv->idmac_ch, image.rect.width, + image.rect.height); +ipu_cpmem_set_stride (priv->idmac_ch, image.pix.bytesperline); +ipu_cpmem_set_buffer (priv->idmac_ch,0 , image.phys0); +ipu_cpmem_set_buffer (priv->idmac_ch,1 , image.phys1); +ipu_cpmem_set_format_passthrough (priv->idmac_ch, + passthrough_bits); + }else { + ret =ipu_cpmem_set_image (priv->idmac_ch, &image); +if (ret) + goto unsetup_vb2; + } + +ipu_cpmem_set_burstsize (priv->idmac_ch, burst_size);/* * Set the channel for the direct CSI-->memory via SMFC
@@ -799,6 +828,7 @@ static int csi_link_validate(struct v4l2_subdev *sd, {struct csi_priv *priv =v4l2_get_subdevdata (sd);struct v4l2_fwnode_endpoint *sensor_ep; +const struct imx_media_pixfmt *incc;struct imx_media_subdev *sensor;bool is_csi2;int ret;
@@ -819,6 +849,16 @@ static int csi_link_validate(struct v4l2_subdev *sd, priv->sensor = sensor; sensor_ep = &priv->sensor->sensor_ep; is_csi2 = (sensor_ep->bus_type ==V4L2_MBUS_CSI2 ); + incc = priv->cc[CSI_SINK_PAD ]; + +if (priv->dest !=IPU_CSI_DEST_IDMAC && + (incc->bayer || (!is_csi2 && + sensor_ep->bus.parallel.bus_width >=16 ))) { +v4l2_err (&priv->sd, +"bayer/16-bit parallel buses must go to IDMAC pad\n" ); + ret = -EINVAL ; + goto out; + }if (is_csi2) {int vc_num =0 ;
@@ -843,7 +883,7 @@ static int csi_link_validate(struct v4l2_subdev *sd,/* select either parallel or MIPI-CSI2 as input to CSI */ ipu_set_csi_src_mux (priv->ipu, priv->csi_id, is_csi2); - +out:mutex_unlock (&priv->lock);return ret; }
Signature · 1 line
2.7.4
Steve Longerbeam
maintainerPATCH v8 27/34 media: imx: csi: increase burst size for YUV formats
In reply to Steve LongerbeamFrom: Philipp Zabel <[email protected]>
The IDMAC supports burst sizes of up to 32 pixels for interleaved YUV formats and up to 64 pixels for planar YUV formats.
--- drivers/staging/media/imx/imx-media-csi.c | 17 +++++++++++++++++ 1 file changed, 17 insertions(+)
drivers/staging/media/imx/imx-media-csi.c +17 -0
@@ -332,6 +332,23 @@ static int csi_idmac_setup_channel(struct csi_priv *priv) passthrough = true; passthrough_bits =16 ;break ; +case V4L2_PIX_FMT_YUV420 : +case V4L2_PIX_FMT_NV12 : + burst_size = (image.pix.width &0x3f ) ? + ((image.pix.width &0x1f ) ? + ((image.pix.width &0xf ) ?8 :16 ) :32 ) :64 ; + passthrough = (sensor_ep->bus_type !=V4L2_MBUS_CSI2 && + sensor_ep->bus.parallel.bus_width >=16 ); + passthrough_bits =16 ; +break ; +case V4L2_PIX_FMT_YUYV : +case V4L2_PIX_FMT_UYVY : + burst_size = (image.pix.width &0x1f ) ? + ((image.pix.width &0xf ) ?8 :16 ) :32 ; + passthrough = (sensor_ep->bus_type !=V4L2_MBUS_CSI2 && + sensor_ep->bus.parallel.bus_width >=16 ); + passthrough_bits =16 ; +break ;default : burst_size = (image.pix.width &0xf ) ?8 :16 ; passthrough = (sensor_ep->bus_type != V4L2_MBUS_CSI2 &&
Signature · 1 line
2.7.4
Steve Longerbeam
maintainerPATCH v8 28/34 media: imx: csi: add frame skipping support
In reply to Steve LongerbeamFrom: Philipp Zabel <[email protected]>
The CSI can skip any out of up to 6 input frames, allowing to reduce the frame rate at the output pads by small fractions.
--- drivers/staging/media/imx/imx-media-csi.c | 167 ++++++++++++++++++++++++++---- 1 file changed, 146 insertions(+), 21 deletions(-)
drivers/staging/media/imx/imx-media-csi.c +146 -21
@@ -2,6 +2,7 @@* V4L2 Capture CSI Subdev for Freescale i.MX5/6 SOC * * Copyright (c) 2014-2017 Mentor Graphics Inc. +* Copyright (C) 2017 Pengutronix, Philipp Zabel <[email protected]> * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by
@@ -9,6 +10,7 @@* (at your option) any later version. */ #include <linux/delay.h> +#include <linux/gcd.h> #include <linux/interrupt.h> #include <linux/module.h> #include <linux/platform_device.h>
@@ -41,6 +43,18 @@#define H_ALIGN 1/* multiple of 2 lines */ #define S_ALIGN 1/* multiple of 2 */ +/* +* struct csi_skip_desc - CSI frame skipping descriptor +* @keep - number of frames kept per max_ratio frames +* @max_ratio - width of skip_smfc, written to MAX_RATIO bitfield +* @skip_smfc - skip pattern written to the SKIP_SMFC bitfield +*/ +struct csi_skip_desc { +u8 keep; +u8 max_ratio; +u8 skip_smfc; +}; +struct csi_priv {struct device *dev;struct ipu_soc *ipu;
@@ -64,8 +78,9 @@ struct csi_priv {struct v4l2_mbus_framefmt format_mbus[CSI_NUM_PADS ];const struct imx_media_pixfmt *cc[CSI_NUM_PADS ]; -struct v4l2_fract frame_interval; +struct v4l2_fract frame_interval[CSI_NUM_PADS ];struct v4l2_rect crop; +const struct csi_skip_desc *skip;/* active vb2 buffers to send to video dev sink */ struct imx_media_buffer *active_vb2_buf[2 ];
@@ -580,6 +595,10 @@ static int csi_setup(struct csi_priv *priv)ipu_csi_set_dest (priv->csi, priv->dest); +if (priv->dest ==IPU_CSI_DEST_IDMAC ) +ipu_csi_set_skip_smfc (priv->csi, priv->skip->skip_smfc, + priv->skip->max_ratio -1 ,0 ); +ipu_csi_dump (priv->csi);return 0 ;
@@ -587,6 +606,7 @@ static int csi_setup(struct csi_priv *priv)static int csi_start (struct csi_priv *priv) { +struct v4l2_fract *output_fi, *input_fi;u32 bad_frames =0 ;int ret;
@@ -595,10 +615,12 @@ static int csi_start(struct csi_priv *priv)return -EINVAL ; } + output_fi = &priv->frame_interval[priv->active_output_pad]; + input_fi = &priv->frame_interval[CSI_SINK_PAD ]; + ret =v4l2_subdev_call (priv->sensor->sd, sensor, g_skip_frames, &bad_frames);if (!ret && bad_frames) { -struct v4l2_fract *fi = &priv->frame_interval;u32 delay_usec;/*
@@ -609,8 +631,8 @@ static int csi_start(struct csi_priv *priv)* to lose vert/horiz sync. */ delay_usec =DIV_ROUND_UP_ULL ( - (u64 )USEC_PER_SEC * fi->numerator * bad_frames, - fi->denominator); + (u64 )USEC_PER_SEC * input_fi->numerator * bad_frames, + input_fi->denominator);usleep_range (delay_usec, delay_usec +1000 ); }
@@ -626,8 +648,7 @@ static int csi_start(struct csi_priv *priv)/* start the frame interval monitor */ if (priv->fim && priv->dest ==IPU_CSI_DEST_IDMAC ) { - ret =imx_media_fim_set_stream (priv->fim, &priv->frame_interval, - true); + ret =imx_media_fim_set_stream (priv->fim, output_fi, true);if (ret) goto idmac_stop; }
@@ -642,8 +663,7 @@ static int csi_start(struct csi_priv *priv) fim_off:if (priv->fim && priv->dest ==IPU_CSI_DEST_IDMAC ) -imx_media_fim_set_stream (priv->fim, &priv->frame_interval, - false); +imx_media_fim_set_stream (priv->fim,NULL , false); idmac_stop:if (priv->dest ==IPU_CSI_DEST_IDMAC )csi_idmac_stop (priv);
@@ -657,14 +677,85 @@ static void csi_stop(struct csi_priv *priv)/* stop the frame interval monitor */ if (priv->fim) -imx_media_fim_set_stream (priv->fim, - &priv->frame_interval, - false); +imx_media_fim_set_stream (priv->fim,NULL , false); }ipu_csi_disable (priv->csi); } +static const struct csi_skip_desc csi_skip[12 ] = { + {1 ,1 ,0x00 },/* Keep all frames */ + {5 ,6 ,0x10 },/* Skip every sixth frame */ + {4 ,5 ,0x08 },/* Skip every fifth frame */ + {3 ,4 ,0x04 },/* Skip every fourth frame */ + {2 ,3 ,0x02 },/* Skip every third frame */ + {3 ,5 ,0x0a },/* Skip frames 1 and 3 of every 5 */ + {1 ,2 ,0x01 },/* Skip every second frame */ + {2 ,5 ,0x0b },/* Keep frames 1 and 4 of every 5 */ + {1 ,3 ,0x03 },/* Keep one in three frames */ + {1 ,4 ,0x07 },/* Keep one in four frames */ + {1 ,5 ,0x0f },/* Keep one in five frames */ + {1 ,6 ,0x1f },/* Keep one in six frames */ +}; + +static void csi_apply_skip_interval (const struct csi_skip_desc *skip, +struct v4l2_fract *interval) +{ +unsigned int div; + + interval->numerator *= skip->max_ratio; + interval->denominator *= skip->keep; + +/* Reduce fraction to lowest terms */ + div =gcd (interval->numerator, interval->denominator); +if (div >1 ) { + interval->numerator /= div; + interval->denominator /= div; + } +} + +/* +* Find the skip pattern to produce the output frame interval closest to the +* requested one, for the given input frame interval. Updates the output frame +* interval to the exact value. +*/ +static const struct csi_skip_desc *csi_find_best_skip (struct v4l2_fract *in, +struct v4l2_fract *out) +{ +const struct csi_skip_desc *skip = &csi_skip[0 ], *best_skip = skip; +u32 min_err =UINT_MAX ; +u64 want_us; +int i; + +/* Default to 1:1 ratio */ +if (out->numerator ==0 || out->denominator ==0 || + in->numerator ==0 || in->denominator ==0 ) { + *out = *in; +return best_skip; + } + + want_us =div_u64 ((u64 )USEC_PER_SEC * out->numerator, out->denominator); + +/* Find the reduction closest to the requested time per frame */ +for (i =0 ; i <ARRAY_SIZE (csi_skip); i++, skip++) { +u64 tmp, err; + + tmp =div_u64 ((u64 )USEC_PER_SEC * in->numerator * + skip->max_ratio, in->denominator * skip->keep); + + err =abs ((s64 )tmp - want_us); +if (err < min_err) { + min_err = err; + best_skip = skip; + } + } + + *out = *in; +csi_apply_skip_interval (best_skip, out); + +return best_skip; +} +/* * V4L2 subdev operations. */
@@ -674,8 +765,13 @@ static int csi_g_frame_interval(struct v4l2_subdev *sd, {struct csi_priv *priv =v4l2_get_subdevdata (sd); +if (fi->pad >=CSI_NUM_PADS ) +return -EINVAL ; +mutex_lock (&priv->lock); - fi->interval = priv->frame_interval; + + fi->interval = priv->frame_interval[fi->pad]; +mutex_unlock (&priv->lock);return 0 ;
@@ -685,18 +781,44 @@ static int csi_s_frame_interval(struct v4l2_subdev *sd, struct v4l2_subdev_frame_interval *fi) {struct csi_priv *priv =v4l2_get_subdevdata (sd); +struct v4l2_fract *input_fi; +int ret =0 ;mutex_lock (&priv->lock); -/* Output pads mirror active input pad, no limits on input pads */ -if (fi->pad ==CSI_SRC_PAD_IDMAC || fi->pad ==CSI_SRC_PAD_DIRECT ) - fi->interval = priv->frame_interval; + input_fi = &priv->frame_interval[CSI_SINK_PAD ]; - priv->frame_interval = fi->interval; +switch (fi->pad) { +case CSI_SINK_PAD : +/* No limits on input frame interval */ +/* Reset output intervals and frame skipping ratio to 1:1 */ + priv->frame_interval[CSI_SRC_PAD_IDMAC ] = fi->interval; + priv->frame_interval[CSI_SRC_PAD_DIRECT ] = fi->interval; + priv->skip = &csi_skip[0 ]; +break ; +case CSI_SRC_PAD_IDMAC : +/* +* frame interval at IDMAC output pad depends on input +* interval, modified by frame skipping. +*/ + priv->skip =csi_find_best_skip (input_fi, &fi->interval); +break ; +case CSI_SRC_PAD_DIRECT : +/* +* frame interval at DIRECT output pad is same as input +* interval. +*/ + fi->interval = *input_fi; +break ; +default : + ret = -EINVAL ; + goto out; + } + priv->frame_interval[fi->pad] = fi->interval; +out:mutex_unlock (&priv->lock); - -return 0 ; +return ret; }static int csi_s_stream (struct v4l2_subdev *sd,int enable)
@@ -1333,11 +1455,14 @@ static int csi_registered(struct v4l2_subdev *sd) &priv->cc[i]);if (ret) goto put_csi; + +/* init default frame interval */ + priv->frame_interval[i].numerator =1 ; + priv->frame_interval[i].denominator =30 ; } -/* init default frame interval */ - priv->frame_interval.numerator =1 ; - priv->frame_interval.denominator =30 ; +/* disable frame skipping */ + priv->skip = &csi_skip[0 ]; priv->fim =imx_media_fim_init (&priv->sd);if (IS_ERR (priv->fim)) {
Signature · 1 line
2.7.4
Steve Longerbeam
maintainerPATCH v8 29/34 media: imx: csi: add sink selection rectangles
In reply to Steve LongerbeamFrom: Philipp Zabel <[email protected]>
Move the crop rectangle to the sink pad and add a sink compose rectangle to configure scaling. Also propagate rectangles from sink pad to crop rectangle, to compose rectangle, and to the source pads both in ACTIVE and TRY variants of set_fmt/selection, and initialize the default crop and compose rectangles.
--- drivers/staging/media/imx/imx-media-csi.c | 153 +++++++++++++++++++++++------- 1 file changed, 117 insertions(+), 36 deletions(-)
drivers/staging/media/imx/imx-media-csi.c +117 -36
@@ -80,6 +80,7 @@ struct csi_priv {const struct imx_media_pixfmt *cc[CSI_NUM_PADS ];struct v4l2_fract frame_interval[CSI_NUM_PADS ];struct v4l2_rect crop; +struct v4l2_rect compose;const struct csi_skip_desc *skip;/* active vb2 buffers to send to video dev sink */
@@ -588,8 +589,8 @@ static int csi_setup(struct csi_priv *priv)ipu_csi_set_window (priv->csi, &priv->crop);ipu_csi_set_downsize (priv->csi, - priv->crop.width ==2 * outfmt->width, - priv->crop.height ==2 * outfmt->height); + priv->crop.width ==2 * priv->compose.width, + priv->crop.height ==2 * priv->compose.height);ipu_csi_init_interface (priv->csi, &sensor_mbus_cfg, &if_fmt);
@@ -1047,6 +1048,17 @@ __csi_get_crop(struct csi_priv *priv, struct v4l2_subdev_pad_config *cfg,return &priv->crop; } +static struct v4l2_rect * +__csi_get_compose (struct csi_priv *priv,struct v4l2_subdev_pad_config *cfg, +enum v4l2_subdev_format_whence which) +{ +if (which ==V4L2_SUBDEV_FORMAT_TRY ) +return v4l2_subdev_get_try_compose (&priv->sd, cfg, +CSI_SINK_PAD ); +else +return &priv->compose; +} + static void csi_try_crop(struct csi_priv *priv, struct v4l2_rect *crop, struct v4l2_subdev_pad_config *cfg,
@@ -1156,6 +1168,7 @@ static void csi_try_fmt(struct csi_priv *priv,struct v4l2_subdev_pad_config *cfg, struct v4l2_subdev_format *sdformat, struct v4l2_rect *crop, + struct v4l2_rect *compose, const struct imx_media_pixfmt **cc) { const struct imx_media_pixfmt *incc;
@@ -1170,15 +1183,8 @@ static void csi_try_fmt(struct csi_priv *priv, incc =imx_media_find_mbus_format (infmt->code,CS_SEL_ANY , true); -if (sdformat->format.width < crop->width *3 /4 ) - sdformat->format.width = crop->width /2 ; -else - sdformat->format.width = crop->width; - -if (sdformat->format.height < crop->height *3 /4 ) - sdformat->format.height = crop->height /2 ; -else - sdformat->format.height = crop->height; + sdformat->format.width = compose->width; + sdformat->format.height = compose->height;if (incc->bayer) { sdformat->format.code = infmt->code;
@@ -1214,11 +1220,17 @@ static void csi_try_fmt(struct csi_priv *priv,v4l_bound_align_image (&sdformat->format.width,MIN_W ,MAX_W ,W_ALIGN , &sdformat->format.height,MIN_H ,MAX_H ,H_ALIGN ,S_ALIGN ); + +/* Reset crop and compose rectangles */ crop->left =0 ; crop->top =0 ; crop->width = sdformat->format.width; crop->height = sdformat->format.height;csi_try_crop (priv, crop, cfg, &sdformat->format, sensor); + compose->left =0 ; + compose->top =0 ; + compose->width = crop->width; + compose->height = crop->height; *cc =imx_media_find_mbus_format (sdformat->format.code,CS_SEL_ANY , true);
@@ -1243,7 +1255,7 @@ static int csi_set_fmt(struct v4l2_subdev *sd,struct imx_media_subdev *sensor;struct v4l2_pix_format vdev_fmt;struct v4l2_mbus_framefmt *fmt; -struct v4l2_rect *crop; +struct v4l2_rect *crop, *compose;int ret =0 ;if (sdformat->pad >=CSI_NUM_PADS )
@@ -1263,8 +1275,9 @@ static int csi_set_fmt(struct v4l2_subdev *sd, } crop =__csi_get_crop (priv, cfg, sdformat->which); + compose =__csi_get_compose (priv, cfg, sdformat->which); -csi_try_fmt (priv, sensor, cfg, sdformat, crop, &cc); +csi_try_fmt (priv, sensor, cfg, sdformat, crop, compose, &cc); fmt =__csi_get_fmt (priv, cfg, sdformat->pad, sdformat->which); *fmt = sdformat->format;
@@ -1281,7 +1294,8 @@ static int csi_set_fmt(struct v4l2_subdev *sd, format.pad = pad; format.which = sdformat->which; format.format = sdformat->format; -csi_try_fmt (priv, sensor, cfg, &format, crop, &outcc); +csi_try_fmt (priv, sensor, cfg, &format,NULL , compose, + &outcc); outfmt =__csi_get_fmt (priv, cfg, pad, sdformat->which); *outfmt = format.format;
@@ -1315,16 +1329,17 @@ static int csi_get_selection(struct v4l2_subdev *sd, {struct csi_priv *priv =v4l2_get_subdevdata (sd);struct v4l2_mbus_framefmt *infmt; -struct v4l2_rect *crop; +struct v4l2_rect *crop, *compose;int ret =0 ; -if (sel->pad >=CSI_NUM_PADS || sel->pad ==CSI_SINK_PAD ) +if (sel->pad !=CSI_SINK_PAD )return -EINVAL ;mutex_lock (&priv->lock); infmt =__csi_get_fmt (priv, cfg,CSI_SINK_PAD , sel->which); crop =__csi_get_crop (priv, cfg, sel->which); + compose =__csi_get_compose (priv, cfg, sel->which);switch (sel->target) {case V4L2_SEL_TGT_CROP_BOUNDS :
@@ -1336,6 +1351,15 @@ static int csi_get_selection(struct v4l2_subdev *sd,case V4L2_SEL_TGT_CROP : sel->r = *crop;break ; +case V4L2_SEL_TGT_COMPOSE_BOUNDS : + sel->r.left =0 ; + sel->r.top =0 ; + sel->r.width = crop->width; + sel->r.height = crop->height; +break ; +case V4L2_SEL_TGT_COMPOSE : + sel->r = *compose; +break ;default : ret = -EINVAL ; }
@@ -1344,19 +1368,34 @@ static int csi_get_selection(struct v4l2_subdev *sd,return ret; } +static int csi_set_scale (u32 *compose,u32 crop,u32 flags) +{ +if ((flags & (V4L2_SEL_FLAG_LE |V4L2_SEL_FLAG_GE )) == + (V4L2_SEL_FLAG_LE |V4L2_SEL_FLAG_GE ) && + *compose != crop && *compose != crop /2 ) +return -ERANGE ; + +if (*compose <= crop /2 || + (*compose < crop *3 /4 && !(flags &V4L2_SEL_FLAG_GE )) || + (*compose < crop && (flags &V4L2_SEL_FLAG_LE ))) + *compose = crop /2 ; +else + *compose = crop; + +return 0 ; +} +static int csi_set_selection (struct v4l2_subdev *sd,struct v4l2_subdev_pad_config *cfg,struct v4l2_subdev_selection *sel) {struct csi_priv *priv =v4l2_get_subdevdata (sd);struct v4l2_mbus_framefmt *infmt; +struct v4l2_rect *crop, *compose;struct imx_media_subdev *sensor; -struct v4l2_rect *crop;int pad, ret =0 ; -if (sel->pad >=CSI_NUM_PADS || - sel->pad ==CSI_SINK_PAD || - sel->target !=V4L2_SEL_TGT_CROP ) +if (sel->pad !=CSI_SINK_PAD )return -EINVAL ; sensor =imx_media_find_sensor (priv->md, &priv->sd.entity);
@@ -1374,30 +1413,66 @@ static int csi_set_selection(struct v4l2_subdev *sd, infmt =__csi_get_fmt (priv, cfg,CSI_SINK_PAD , sel->which); crop =__csi_get_crop (priv, cfg, sel->which); + compose =__csi_get_compose (priv, cfg, sel->which); -/* -* Modifying the crop rectangle always changes the format on the source -* pad. If the KEEP_CONFIG flag is set, just return the current crop -* rectangle. -*/ -if (sel->flags &V4L2_SEL_FLAG_KEEP_CONFIG ) { - sel->r = priv->crop; -if (sel->which ==V4L2_SUBDEV_FORMAT_TRY ) - *crop = sel->r; - goto out; - } +switch (sel->target) { +case V4L2_SEL_TGT_CROP : +/* +* Modifying the crop rectangle always changes the format on +* the source pads. If the KEEP_CONFIG flag is set, just return +* the current crop rectangle. +*/ +if (sel->flags &V4L2_SEL_FLAG_KEEP_CONFIG ) { + sel->r = priv->crop; +if (sel->which ==V4L2_SUBDEV_FORMAT_TRY ) + *crop = sel->r; + goto out; + } + +csi_try_crop (priv, &sel->r, cfg, infmt, sensor); + + *crop = sel->r; + +/* Reset scaling to 1:1 */ + compose->width = crop->width; + compose->height = crop->height; +break ; +case V4L2_SEL_TGT_COMPOSE : +/* +* Modifying the compose rectangle always changes the format on +* the source pads. If the KEEP_CONFIG flag is set, just return +* the current compose rectangle. +*/ +if (sel->flags &V4L2_SEL_FLAG_KEEP_CONFIG ) { + sel->r = priv->compose; +if (sel->which ==V4L2_SUBDEV_FORMAT_TRY ) + *compose = sel->r; + goto out; + } -csi_try_crop (priv, &sel->r, cfg, infmt, sensor); + sel->r.left =0 ; + sel->r.top =0 ; + ret =csi_set_scale (&sel->r.width, crop->width, sel->flags); +if (ret) + goto out; + ret =csi_set_scale (&sel->r.height, crop->height, sel->flags); +if (ret) + goto out; - *crop = sel->r; + *compose = sel->r; +break ; +default : + ret = -EINVAL ; + goto out; + } -/* Update the source pad formats */ +/* Reset source pads to sink compose rectangle */ for (pad =CSI_SINK_PAD +1 ; pad <CSI_NUM_PADS ; pad++) {struct v4l2_mbus_framefmt *outfmt; outfmt =__csi_get_fmt (priv, cfg, pad, sel->which); - outfmt->width = crop->width; - outfmt->height = crop->height; + outfmt->width = compose->width; + outfmt->height = compose->height; } out:
@@ -1464,6 +1539,12 @@ static int csi_registered(struct v4l2_subdev *sd)/* disable frame skipping */ priv->skip = &csi_skip[0 ]; +/* init default crop and compose rectangle sizes */ + priv->crop.width =640 ; + priv->crop.height =480 ; + priv->compose.width =640 ; + priv->compose.height =480 ; + priv->fim =imx_media_fim_init (&priv->sd);if (IS_ERR (priv->fim)) { ret =PTR_ERR (priv->fim);
Signature · 1 line
2.7.4
Steve Longerbeam
maintainerPATCH v8 30/34 media: imx: csi: add frame size/interval enumeration
In reply to Steve LongerbeamFrom: Russell King <[email protected]>
Add frame size and frame interval enumeration to CSI.
CSI can downscale the image independently horizontally and vertically by a factor of two, which enumerates to four different frame sizes at the output pads. The input pad supports a range of frame sizes.
CSI can also drop frames, resulting in frame rate reduction, so enumerate the resulting possible output frame rates.
--- drivers/staging/media/imx/imx-media-csi.c | 70 +++++++++++++++++++++++++++++++ 1 file changed, 70 insertions(+)
drivers/staging/media/imx/imx-media-csi.c +70 -0
@@ -1138,6 +1138,74 @@ static int csi_enum_mbus_code(struct v4l2_subdev *sd,return ret; } +static int csi_enum_frame_size (struct v4l2_subdev *sd, +struct v4l2_subdev_pad_config *cfg, +struct v4l2_subdev_frame_size_enum *fse) +{ +struct csi_priv *priv =v4l2_get_subdevdata (sd); +struct v4l2_rect *crop; +int ret =0 ; + +if (fse->pad >=CSI_NUM_PADS || + fse->index > (fse->pad ==CSI_SINK_PAD ?0 :3 )) +return -EINVAL ; + +mutex_lock (&priv->lock); + +if (fse->pad ==CSI_SINK_PAD ) { + fse->min_width =MIN_W ; + fse->max_width =MAX_W ; + fse->min_height =MIN_H ; + fse->max_height =MAX_H ; + }else { + crop =__csi_get_crop (priv, cfg, fse->which); + + fse->min_width = fse->max_width = fse->index &1 ? + crop->width /2 : crop->width; + fse->min_height = fse->max_height = fse->index &2 ? + crop->height /2 : crop->height; + } + +mutex_unlock (&priv->lock); +return ret; +} + +static int csi_enum_frame_interval (struct v4l2_subdev *sd, +struct v4l2_subdev_pad_config *cfg, +struct v4l2_subdev_frame_interval_enum *fie) +{ +struct csi_priv *priv =v4l2_get_subdevdata (sd); +struct v4l2_fract *input_fi; +struct v4l2_rect *crop; +int ret =0 ; + +if (fie->pad >=CSI_NUM_PADS || + fie->index >= (fie->pad !=CSI_SRC_PAD_IDMAC ? +1 :ARRAY_SIZE (csi_skip))) +return -EINVAL ; + +mutex_lock (&priv->lock); + + input_fi = &priv->frame_interval[CSI_SINK_PAD ]; + crop =__csi_get_crop (priv, cfg, fie->which); + +if ((fie->width != crop->width && fie->width != crop->width /2 ) || + (fie->height != crop->height && fie->height != crop->height /2 )) { + ret = -EINVAL ; + goto out; + } + + fie->interval = *input_fi; + +if (fie->pad ==CSI_SRC_PAD_IDMAC ) +csi_apply_skip_interval (&csi_skip[fie->index], + &fie->interval); + +out: +mutex_unlock (&priv->lock); +return ret; +} +static int csi_get_fmt (struct v4l2_subdev *sd,struct v4l2_subdev_pad_config *cfg,struct v4l2_subdev_format *sdformat)
@@ -1605,6 +1673,8 @@ static const struct v4l2_subdev_video_ops csi_video_ops = {static const struct v4l2_subdev_pad_ops csi_pad_ops = { .enum_mbus_code = csi_enum_mbus_code, + .enum_frame_size = csi_enum_frame_size, + .enum_frame_interval = csi_enum_frame_interval, .get_fmt = csi_get_fmt, .set_fmt = csi_set_fmt, .get_selection = csi_get_selection,
Signature · 1 line
2.7.4
Steve Longerbeam
maintainerPATCH v8 31/34 media: imx: capture: add frame sizes/interval enumeration
In reply to Steve LongerbeamFrom: Russell King <[email protected]>
Add support for enumerating frame sizes and frame intervals from the first subdev via the V4L2 interfaces.
--- drivers/staging/media/imx/imx-media-capture.c | 73 +++++++++++++++++++++++++++ 1 file changed, 73 insertions(+)
drivers/staging/media/imx/imx-media-capture.c +73 -0
@@ -81,6 +81,76 @@ static int vidioc_querycap(struct file *file, void *fh,return 0 ; } +static int capture_enum_framesizes (struct file *file,void *fh, +struct v4l2_frmsizeenum *fsize) +{ +struct capture_priv *priv =video_drvdata (file); +const struct imx_media_pixfmt *cc; +struct v4l2_subdev_frame_size_enum fse = { + .index = fsize->index, + .pad = priv->src_sd_pad, + .which =V4L2_SUBDEV_FORMAT_ACTIVE , + }; +int ret; + + cc =imx_media_find_format (fsize->pixel_format,CS_SEL_ANY , true); +if (!cc) +return -EINVAL ; + + fse.code = cc->codes[0 ]; + + ret =v4l2_subdev_call (priv->src_sd, pad, enum_frame_size,NULL , &fse); +if (ret) +return ret; + +if (fse.min_width == fse.max_width && + fse.min_height == fse.max_height) { + fsize->type =V4L2_FRMSIZE_TYPE_DISCRETE ; + fsize->discrete.width = fse.min_width; + fsize->discrete.height = fse.min_height; + }else { + fsize->type =V4L2_FRMSIZE_TYPE_CONTINUOUS ; + fsize->stepwise.min_width = fse.min_width; + fsize->stepwise.max_width = fse.max_width; + fsize->stepwise.min_height = fse.min_height; + fsize->stepwise.max_height = fse.max_height; + fsize->stepwise.step_width =1 ; + fsize->stepwise.step_height =1 ; + } + +return 0 ; +} + +static int capture_enum_frameintervals (struct file *file,void *fh, +struct v4l2_frmivalenum *fival) +{ +struct capture_priv *priv =video_drvdata (file); +const struct imx_media_pixfmt *cc; +struct v4l2_subdev_frame_interval_enum fie = { + .index = fival->index, + .pad = priv->src_sd_pad, + .width = fival->width, + .height = fival->height, + .which =V4L2_SUBDEV_FORMAT_ACTIVE , + }; +int ret; + + cc =imx_media_find_format (fival->pixel_format,CS_SEL_ANY , true); +if (!cc) +return -EINVAL ; + + fie.code = cc->codes[0 ]; + + ret =v4l2_subdev_call (priv->src_sd, pad, enum_frame_interval,NULL , &fie); +if (ret) +return ret; + + fival->type =V4L2_FRMIVAL_TYPE_DISCRETE ; + fival->discrete = fie.interval; + +return 0 ; +} +static int capture_enum_fmt_vid_cap (struct file *file,void *fh,struct v4l2_fmtdesc *f) {
@@ -269,6 +339,9 @@ static int capture_s_parm(struct file *file, void *fh,static const struct v4l2_ioctl_ops capture_ioctl_ops = { .vidioc_querycap = vidioc_querycap, + .vidioc_enum_framesizes = capture_enum_framesizes, + .vidioc_enum_frameintervals = capture_enum_frameintervals, + .vidioc_enum_fmt_vid_cap = capture_enum_fmt_vid_cap, .vidioc_g_fmt_vid_cap = capture_g_fmt_vid_cap, .vidioc_try_fmt_vid_cap = capture_try_fmt_vid_cap,
Signature · 1 line
2.7.4
Steve Longerbeam
maintainerPATCH v8 32/34 media: imx: set and propagate default field, colorimetry
In reply to Steve LongerbeamThis patch adds a call to imx_media_fill_default_mbus_fields() in the *_try_fmt() functions at the sink pads, to set empty field order and colorimetry parameters.
If the field order is set to ANY, choose the currently set field order at the sink pad. If the colorspace is set to DEFAULT, choose the current colorspace at the sink pad. If any of xfer_func, ycbcr_enc or quantization are set to DEFAULT, either choose the current sink pad setting, or the default setting for the new colorspace, if non-DEFAULT colorspace was given.
If a format is destined to be routed through the Image Converter, fixed quantization and Y`CbCr encoding must be set.
Colorimetry is also propagated from sink to source pads anywhere this has not already been done.
--- drivers/staging/media/imx/imx-ic-prp.c | 10 +++-- drivers/staging/media/imx/imx-ic-prpencvf.c | 17 ++++++-- drivers/staging/media/imx/imx-media-csi.c | 14 ++++++- drivers/staging/media/imx/imx-media-utils.c | 62 +++++++++++++++++++++++++++++ drivers/staging/media/imx/imx-media-vdic.c | 8 +++- drivers/staging/media/imx/imx-media.h | 4 +- 6 files changed, 103 insertions(+), 12 deletions(-)
drivers/staging/media/imx/imx-ic-prp.c +7 -3
@@ -166,8 +166,8 @@ static int prp_set_fmt(struct v4l2_subdev *sd, struct v4l2_subdev_format *sdformat) {struct prp_priv *priv =sd_to_priv (sd); +struct v4l2_mbus_framefmt *fmt, *infmt;const struct imx_media_pixfmt *cc; -struct v4l2_mbus_framefmt *fmt;int ret =0 ;u32 code;
@@ -181,6 +181,8 @@ static int prp_set_fmt(struct v4l2_subdev *sd, goto out; } + infmt =__prp_get_fmt (priv, cfg,PRP_SINK_PAD , sdformat->which); +switch (sdformat->pad) {case PRP_SINK_PAD : v4l_bound_align_image(&sdformat->format.width, MIN_W, MAX_W,
@@ -194,12 +196,14 @@ static int prp_set_fmt(struct v4l2_subdev *sd, cc =imx_media_find_ipu_format (code,CS_SEL_ANY ); sdformat->format.code = cc->codes[0 ]; } + +imx_media_fill_default_mbus_fields (&sdformat->format, infmt, + true);break ;case PRP_SRC_PAD_PRPENC :case PRP_SRC_PAD_PRPVF :/* Output pads mirror input pad */ - fmt =__prp_get_fmt (priv, cfg,PRP_SINK_PAD , sdformat->which); - sdformat->format = *fmt; + sdformat->format = *infmt;break ; }
drivers/staging/media/imx/imx-ic-prpencvf.c +13 -4
@@ -805,6 +805,8 @@ static void prp_try_fmt(struct prp_priv *priv,struct v4l2_subdev_format *sdformat, const struct imx_media_pixfmt **cc) { + struct v4l2_mbus_framefmt *infmt; + *cc =imx_media_find_ipu_format (sdformat->format.code,CS_SEL_ANY );if (!*cc) {u32 code;
@@ -814,22 +816,29 @@ static void prp_try_fmt(struct prp_priv *priv, sdformat->format.code = (*cc)->codes[0 ]; } -if (sdformat->pad ==PRPENCVF_SRC_PAD ) { -struct v4l2_mbus_framefmt *infmt = -__prp_get_fmt (priv, cfg,PRPENCVF_SINK_PAD , - sdformat->which); + infmt =__prp_get_fmt (priv, cfg,PRPENCVF_SINK_PAD , sdformat->which); +if (sdformat->pad ==PRPENCVF_SRC_PAD ) {if (sdformat->format.field !=V4L2_FIELD_NONE ) sdformat->format.field = infmt->field;prp_bound_align_output (&sdformat->format, infmt, priv->rot_mode); + +/* propagate colorimetry from sink */ + sdformat->format.colorspace = infmt->colorspace; + sdformat->format.xfer_func = infmt->xfer_func; + sdformat->format.quantization = infmt->quantization; + sdformat->format.ycbcr_enc = infmt->ycbcr_enc; }else {v4l_bound_align_image (&sdformat->format.width,MIN_W_SINK ,MAX_W_SINK ,W_ALIGN_SINK , &sdformat->format.height,MIN_H_SINK ,MAX_H_SINK ,H_ALIGN_SINK ,S_ALIGN ); + +imx_media_fill_default_mbus_fields (&sdformat->format, infmt, + true); } }
drivers/staging/media/imx/imx-media-csi.c +12 -2
@@ -1243,11 +1243,11 @@ static void csi_try_fmt(struct csi_priv *priv,struct v4l2_mbus_framefmt *infmt;u32 code; + infmt =__csi_get_fmt (priv, cfg,CSI_SINK_PAD , sdformat->which); +switch (sdformat->pad) {case CSI_SRC_PAD_DIRECT :case CSI_SRC_PAD_IDMAC : - infmt =__csi_get_fmt (priv, cfg,CSI_SINK_PAD , - sdformat->which); incc =imx_media_find_mbus_format (infmt->code,CS_SEL_ANY , true);
@@ -1283,6 +1283,12 @@ static void csi_try_fmt(struct csi_priv *priv, sdformat->format.field = (infmt->height ==480 ) ?V4L2_FIELD_SEQ_TB :V4L2_FIELD_SEQ_BT ; } + +/* propagate colorimetry from sink */ + sdformat->format.colorspace = infmt->colorspace; + sdformat->format.xfer_func = infmt->xfer_func; + sdformat->format.quantization = infmt->quantization; + sdformat->format.ycbcr_enc = infmt->ycbcr_enc;break ;case CSI_SINK_PAD : v4l_bound_align_image(&sdformat->format.width, MIN_W, MAX_W,
@@ -1309,6 +1315,10 @@ static void csi_try_fmt(struct csi_priv *priv, CS_SEL_ANY, false); sdformat->format.code = (*cc)->codes[0 ]; } + +imx_media_fill_default_mbus_fields ( + &sdformat->format, infmt, + priv->active_output_pad ==CSI_SRC_PAD_DIRECT );break ; } }
drivers/staging/media/imx/imx-media-utils.c +62 -0
68 lines not on this page · raw message
drivers/staging/media/imx/imx-media-vdic.c +6 -2
@@ -603,10 +603,11 @@ static void vdic_try_fmt(struct vdic_priv *priv, sdformat->format.code = (*cc)->codes[0 ]; } + infmt =__vdic_get_fmt (priv, cfg, priv->active_input_pad, + sdformat->which); +switch (sdformat->pad) {case VDIC_SRC_PAD_DIRECT : - infmt =__vdic_get_fmt (priv, cfg, priv->active_input_pad, - sdformat->which); sdformat->format = *infmt;/* output is always progressive! */ sdformat->format.field =V4L2_FIELD_NONE ;
@@ -618,6 +619,9 @@ static void vdic_try_fmt(struct vdic_priv *priv, &sdformat->format.height, MIN_H, MAX_H_VDIC, H_ALIGN, S_ALIGN); +imx_media_fill_default_mbus_fields (&sdformat->format, infmt, + true); +/* input must be interlaced! Choose SEQ_TB if not */ if (!V4L2_FIELD_HAS_BOTH (sdformat->format.field)) sdformat->format.field =V4L2_FIELD_SEQ_TB ;
drivers/staging/media/imx/imx-media.h +3 -1
@@ -209,7 +209,9 @@ int imx_media_enum_ipu_format(u32 *code, u32 index, enum codespace_sel cs_sel);int imx_media_init_mbus_fmt (struct v4l2_mbus_framefmt *mbus,u32 width,u32 height,u32 code,u32 field,const struct imx_media_pixfmt **cc); - +void imx_media_fill_default_mbus_fields (struct v4l2_mbus_framefmt *tryfmt, +struct v4l2_mbus_framefmt *fmt, +bool ic_route);int imx_media_mbus_fmt_to_pix_fmt (struct v4l2_pix_format *pix,struct v4l2_mbus_framefmt *mbus,const struct imx_media_pixfmt *cc);
Signature · 1 line
2.7.4
Steve Longerbeam
maintainerPATCH v8 33/34 media: imx: Drop warning upon multiple S_STREAM disable calls
In reply to Steve LongerbeamFrom: Marek Vasut <[email protected]>
Calling S_STREAM OFF multiple times on a video device is valid, although dubious, practice. Instead of warning about it and setting stream count lower than zero, just ignore the subsequent S_STREAM calls and correct the stream count to zero.
--- drivers/staging/media/imx/imx-ic-prp.c | 3 ++- drivers/staging/media/imx/imx-ic-prpencvf.c | 3 ++- drivers/staging/media/imx/imx-media-csi.c | 3 ++- drivers/staging/media/imx/imx-media-vdic.c | 3 ++- drivers/staging/media/imx/imx6-mipi-csi2.c | 3 ++- 5 files changed, 10 insertions(+), 5 deletions(-)
drivers/staging/media/imx/imx-ic-prp.c +2 -1
@@ -391,7 +391,8 @@ static int prp_s_stream(struct v4l2_subdev *sd, int enable) update_count: priv->stream_count += enable ?1 :-1 ; -WARN_ON (priv->stream_count <0 ); +if (priv->stream_count <0 ) + priv->stream_count =0 ; out:mutex_unlock (&priv->lock);return ret;
drivers/staging/media/imx/imx-ic-prpencvf.c +2 -1
@@ -1140,7 +1140,8 @@ static int prp_s_stream(struct v4l2_subdev *sd, int enable) update_count: priv->stream_count += enable ?1 :-1 ; -WARN_ON (priv->stream_count <0 ); +if (priv->stream_count <0 ) + priv->stream_count =0 ; out:mutex_unlock (&priv->lock);return ret;
drivers/staging/media/imx/imx-media-csi.c +2 -1
@@ -863,7 +863,8 @@ static int csi_s_stream(struct v4l2_subdev *sd, int enable) update_count: priv->stream_count += enable ?1 :-1 ; -WARN_ON (priv->stream_count <0 ); +if (priv->stream_count <0 ) + priv->stream_count =0 ; out:mutex_unlock (&priv->lock);return ret;
drivers/staging/media/imx/imx-media-vdic.c +2 -1
@@ -536,7 +536,8 @@ static int vdic_s_stream(struct v4l2_subdev *sd, int enable) update_count: priv->stream_count += enable ?1 :-1 ; -WARN_ON (priv->stream_count <0 ); +if (priv->stream_count <0 ) + priv->stream_count =0 ; out:mutex_unlock (&priv->lock);return ret;
drivers/staging/media/imx/imx6-mipi-csi2.c +2 -1
@@ -397,7 +397,8 @@ static int csi2_s_stream(struct v4l2_subdev *sd, int enable) update_count: csi2->stream_count += enable ?1 :-1 ; -WARN_ON (csi2->stream_count <0 ); +if (csi2->stream_count <0 ) + csi2->stream_count =0 ; out:mutex_unlock (&csi2->lock);return ret;
Signature · 1 line
2.7.4
Steve Longerbeam
maintainerPATCH v8 34/34 ARM: imx_v6_v7_defconfig: Enable staging video4linux drivers
In reply to Steve LongerbeamEnable i.MX v4l2 media staging driver. For video capture on i.MX, the video multiplexer subdev is required. On the SabreAuto, the ADV7180 video decoder is required along with i2c-mux-gpio. The Sabrelite and SabreSD require the OV5640 and the SabreLite requires PWM clocks for the OV5640.
Increase max zoneorder to allow larger video buffer allocations.
--- arch/arm/configs/imx_v6_v7_defconfig | 11 +++++++++++ 1 file changed, 11 insertions(+)
arch/arm/configs/imx_v6_v7_defconfig +11 -0
@@ -51,6 +51,7 @@ CONFIG_PREEMPT_VOLUNTARY=y CONFIG_AEABI=y CONFIG_HIGHMEM=y CONFIG_CMA=y +CONFIG_FORCE_MAX_ZONEORDER=14 CONFIG_CMDLINE="noinitrd console=ttymxc0,115200" CONFIG_KEXEC=y CONFIG_CPU_FREQ=y
@@ -183,6 +184,7 @@ CONFIG_SERIAL_FSL_LPUART=y CONFIG_SERIAL_FSL_LPUART_CONSOLE=y # CONFIG_I2C_COMPAT is not set CONFIG_I2C_CHARDEV=y +CONFIG_I2C_MUX=y CONFIG_I2C_MUX_GPIO=y # CONFIG_I2C_HELPER_AUTO is not set CONFIG_I2C_ALGOPCF=m
@@ -223,14 +225,20 @@ CONFIG_REGULATOR_PFUZE100=y CONFIG_MEDIA_SUPPORT=y CONFIG_MEDIA_CAMERA_SUPPORT=y CONFIG_MEDIA_RC_SUPPORT=y +CONFIG_MEDIA_CONTROLLER=y +CONFIG_VIDEO_V4L2_SUBDEV_API=y CONFIG_RC_DEVICES=y CONFIG_IR_GPIO_CIR=y CONFIG_MEDIA_USB_SUPPORT=y CONFIG_USB_VIDEO_CLASS=m CONFIG_V4L_PLATFORM_DRIVERS=y +CONFIG_VIDEO_MUX=y CONFIG_SOC_CAMERA=y CONFIG_V4L_MEM2MEM_DRIVERS=y CONFIG_VIDEO_CODA=y +# CONFIG_MEDIA_SUBDRV_AUTOSELECT is not set +CONFIG_VIDEO_ADV7180=m +CONFIG_VIDEO_OV5640=m CONFIG_SOC_CAMERA_OV2640=y CONFIG_IMX_IPUV3_CORE=y CONFIG_DRM=y
@@ -340,6 +348,9 @@ CONFIG_FSL_EDMA=y CONFIG_IMX_SDMA=y CONFIG_MXS_DMA=y CONFIG_STAGING=y +CONFIG_STAGING_MEDIA=y +CONFIG_VIDEO_IMX_MEDIA=y +CONFIG_COMMON_CLK_PWM=y CONFIG_IIO=y CONFIG_VF610_ADC=y CONFIG_MPL3115=y
Signature · 1 line
2.7.4
We're still waiting for an Ack for patch 02/34, right?
Other than that everything is ready AFAICT.
Regards,
Hans
Quoted 175 lines · Steve Longerbeam
In version 8:
- Switched to
v4l2_fwnodeAPIs.- Always pass a valid CSI id to
ipu_set_ic_src_mux()in imx-ic-prp, even if the IC is receiving from the VDIC. The reason is due to a bug in the i.MX6 reference manual: from experiment it is determined that the CSI id select bit inIPU_CONFregister selects which CSI is routed to either the VDIC or the IC, and is independent of whether the IC is set to receive from a CSI or the VDIC. Sugested by Marek Vasut <[email protected]>.- ov5640: propagate error codes from all i2c register accesses. Sugested by Sakari Ailus <[email protected]>.
- ov5640: drop the entity stream count check in
ov5640_s_stream(). Sugested by Sakari Ailus <[email protected]>.- ov5640: Fix manual exposure control. The manual exposure setting (in line periods) cannot exceed the max exposure value in registers {
0x380E,0x380F} + {0x350C,0x350D}, however the max eposure value was not being calcualted correctly.- ov5640: the video mode register tables require auto gain/exposure be disabled before programming the register set. However auto gain/exp was being disabled by direct register write. This caused the auto gain/exp control values to be inconsistent with the actual hardware setting. Fixed by going through v4l2-ctrl when disabling auto gain/exp.
- ov5640: converted virtual channel macro to a module parameter, default to channel 0.
- ov5640: override the v4l2-ctl lock to use the ov5640 subdev driver's lock. Sugested by Sakari Ailus <[email protected]>.
- ov5640: switch to unit-less
V4L2_CID_EXPOSURE. UsingV4L2_CID_EXPOSURE_ABSOLUTEwill require converting from 100-usec units to line periods and vice-versa. Sugested by Sakari Ailus and Pavel Machek <[email protected]>.- ov5640: drop dangling
regulator_bulk_disable()from probe/remove. Sugested by Sakari Ailus <[email protected]>.- FIM: move input capture channel selection out of the device-tree and make this a V4L2 control. In order to support attaching a FIM to prpencvf, the FIM cannot have any device-tree configuration, because prpencvf has no device node. The FIM now is completely configurable via its V4L2 controls.
- FIM: drop
imx_media_fim_set_power(), and move the input capture channel request toimx_media_fim_set_stream(). This allows to dropcsi_s_power()as well, since the latter only calledimx_media_fim_set_power().- FIM: add a spinlock to protect the
frame_interval_monitor()from the setting of new control values. Theframe_interval_monitor()is called from interrupt context so a spinlock must be used.- Updated to version 8 video-mux patchset from Philipp Zabel <[email protected]>.
Marek Vasut (1):
media: imx: Drop warning upon multipleS_STREAMdisable callsPhilipp Zabel (8):
dt-bindings: Add bindings for video-multiplexer device
ARM: dts: imx6qdl: add multiplexer controls
ARM: dts: imx6qdl: Add video multiplexers,mipi_csi, and their
connections
add mux and video interface bridge entity functions
platform: add video-multiplexer subdevice driver
media: imx: csi: increase burst size for YUV formats
media: imx: csi: add frame skipping support
media: imx: csi: add sink selection rectanglesRussell King (3):
media: imx: csi: add support for bayer formats
media: imx: csi: add frame size/interval enumeration
media: imx: capture: add frame sizes/interval enumerationSteve Longerbeam (22):
[media] dt-bindings: Add bindings for i.MX media driver [media] dt/bindings: Add bindings for OV5640
ARM: dts: imx6qdl: Add compatible, clocks, irqs to MIPI CSI-2 node
ARM: dts: imx6qdl: add capture-subsystem device
ARM: dts: imx6qdl-sabrelite: remove erratum ERR006687 workaround
ARM: dts: imx6-sabrelite: add OV5642 and OV5640 camera sensors
ARM: dts: imx6-sabresd: add OV5642 and OV5640 camera sensors
ARM: dts: imx6-sabreauto: create i2cmux for i2c3
ARM: dts: imx6-sabreauto: add reset-gpios property formax7310_b
ARM: dts: imx6-sabreauto: add pinctrl for gpt input capture
ARM: dts: imx6-sabreauto: add the ADV7180 video decoder [media] add Omnivision OV5640 sensor driver
media: Add userspace header file for i.MX
media: Add i.MX media core driver
media: imx: Add a TODO file
media: imx: Add Capture Device Interface
media: imx: Add CSI subdev driver
media: imx: Add VDIC subdev driver
media: imx: Add IC subdev drivers
media: imx: Add MIPI CSI-2 Receiver subdev driver
media: imx: set and propagate default field, colorimetry
ARM:imx_v6_v7_defconfig: Enable staging video4linux drivers.../devicetree/bindings/media/i2c/ov5640.txt | 45 + Documentation/devicetree/bindings/media/imx.txt | 47 + .../devicetree/bindings/media/video-mux.txt | 60 + Documentation/media/uapi/mediactl/media-types.rst | 21 + Documentation/media/v4l-drivers/imx.rst | 614 +++++ arch/arm/boot/dts/imx6dl-sabrelite.dts | 5 + arch/arm/boot/dts/imx6dl-sabresd.dts | 5 + arch/arm/boot/dts/imx6dl.dtsi | 189 ++ arch/arm/boot/dts/imx6q-sabrelite.dts | 5 + arch/arm/boot/dts/imx6q-sabresd.dts | 5 + arch/arm/boot/dts/imx6q.dtsi | 125 ++ arch/arm/boot/dts/imx6qdl-sabreauto.dtsi | 136 +- arch/arm/boot/dts/imx6qdl-sabrelite.dtsi | 152 +- arch/arm/boot/dts/imx6qdl-sabresd.dtsi | 114 +- arch/arm/boot/dts/imx6qdl.dtsi | 20 +- arch/arm/configs/imx_v6_v7_defconfig | 11 + drivers/media/i2c/Kconfig | 10 + drivers/media/i2c/Makefile | 1 + drivers/media/i2c/ov5640.c | 2344 ++++++++++++++++++++ drivers/media/platform/Kconfig | 6 + drivers/media/platform/Makefile | 2 + drivers/media/platform/video-mux.c | 334 +++ drivers/staging/media/Kconfig | 2 + drivers/staging/media/Makefile | 1 + drivers/staging/media/imx/Kconfig | 21 + drivers/staging/media/imx/Makefile | 12 + drivers/staging/media/imx/TODO | 23 + drivers/staging/media/imx/imx-ic-common.c | 113 + drivers/staging/media/imx/imx-ic-prp.c | 518 +++++ drivers/staging/media/imx/imx-ic-prpencvf.c | 1309 +++++++++++ drivers/staging/media/imx/imx-ic.h | 38 + drivers/staging/media/imx/imx-media-capture.c | 775 +++++++ drivers/staging/media/imx/imx-media-csi.c | 1816 +++++++++++++++ drivers/staging/media/imx/imx-media-dev.c | 666 ++++++ drivers/staging/media/imx/imx-media-fim.c | 494 +++++ drivers/staging/media/imx/imx-media-internal-sd.c | 349 +++ drivers/staging/media/imx/imx-media-of.c | 270 +++ drivers/staging/media/imx/imx-media-utils.c | 896 ++++++++ drivers/staging/media/imx/imx-media-vdic.c | 1009 +++++++++ drivers/staging/media/imx/imx-media.h | 325 +++ drivers/staging/media/imx/imx6-mipi-csi2.c | 698 ++++++ include/linux/imx-media.h | 29 + include/media/imx.h | 15 + include/uapi/linux/media.h | 6 + include/uapi/linux/v4l2-controls.h | 4 + 45 files changed, 13613 insertions(+), 27 deletions(-) create mode 100644 Documentation/devicetree/bindings/media/i2c/ov5640.txt create mode 100644 Documentation/devicetree/bindings/media/imx.txt create mode 100644 Documentation/devicetree/bindings/media/video-mux.txt create mode 100644 Documentation/media/v4l-drivers/imx.rst create mode 100644 drivers/media/i2c/ov5640.c create mode 100644 drivers/media/platform/video-mux.c create mode 100644 drivers/staging/media/imx/Kconfig create mode 100644 drivers/staging/media/imx/Makefile create mode 100644 drivers/staging/media/imx/TODO create mode 100644 drivers/staging/media/imx/imx-ic-common.c create mode 100644 drivers/staging/media/imx/imx-ic-prp.c create mode 100644 drivers/staging/media/imx/imx-ic-prpencvf.c create mode 100644 drivers/staging/media/imx/imx-ic.h create mode 100644 drivers/staging/media/imx/imx-media-capture.c create mode 100644 drivers/staging/media/imx/imx-media-csi.c create mode 100644 drivers/staging/media/imx/imx-media-dev.c create mode 100644 drivers/staging/media/imx/imx-media-fim.c create mode 100644 drivers/staging/media/imx/imx-media-internal-sd.c create mode 100644 drivers/staging/media/imx/imx-media-of.c create mode 100644 drivers/staging/media/imx/imx-media-utils.c create mode 100644 drivers/staging/media/imx/imx-media-vdic.c create mode 100644 drivers/staging/media/imx/imx-media.h create mode 100644 drivers/staging/media/imx/imx6-mipi-csi2.c create mode 100644 include/linux/imx-media.h create mode 100644 include/media/imx.h
Steve Longerbeam
maintainerQuoted 2 lines · Hans Verkuil
We're still waiting for an Ack for patch 02/34, right?
Hi Hans, Yes still waiting for an ack for the imx-media bindings.
Quoted 1 line
Other than that everything is ready AFAICT.
Agreed.
Steve
Quoted 182 lines
Regards,
Hans
Quoted 175 lines · Steve Longerbeam
In version 8:
- Switched to
v4l2_fwnodeAPIs.- Always pass a valid CSI id to
ipu_set_ic_src_mux()in imx-ic-prp, even if the IC is receiving from the VDIC. The reason is due to a bug in the i.MX6 reference manual: from experiment it is determined that the CSI id select bit inIPU_CONFregister selects which CSI is routed to either the VDIC or the IC, and is independent of whether the IC is set to receive from a CSI or the VDIC. Sugested by Marek Vasut <[email protected]>.- ov5640: propagate error codes from all i2c register accesses. Sugested by Sakari Ailus <[email protected]>.
- ov5640: drop the entity stream count check in
ov5640_s_stream(). Sugested by Sakari Ailus <[email protected]>.- ov5640: Fix manual exposure control. The manual exposure setting (in line periods) cannot exceed the max exposure value in registers {
0x380E,0x380F} + {0x350C,0x350D}, however the max eposure value was not being calcualted correctly.- ov5640: the video mode register tables require auto gain/exposure be disabled before programming the register set. However auto gain/exp was being disabled by direct register write. This caused the auto gain/exp control values to be inconsistent with the actual hardware setting. Fixed by going through v4l2-ctrl when disabling auto gain/exp.
- ov5640: converted virtual channel macro to a module parameter, default to channel 0.
- ov5640: override the v4l2-ctl lock to use the ov5640 subdev driver's lock. Sugested by Sakari Ailus <[email protected]>.
- ov5640: switch to unit-less
V4L2_CID_EXPOSURE. UsingV4L2_CID_EXPOSURE_ABSOLUTEwill require converting from 100-usec units to line periods and vice-versa. Sugested by Sakari Ailus and Pavel Machek <[email protected]>.- ov5640: drop dangling
regulator_bulk_disable()from probe/remove. Sugested by Sakari Ailus <[email protected]>.- FIM: move input capture channel selection out of the device-tree and make this a V4L2 control. In order to support attaching a FIM to prpencvf, the FIM cannot have any device-tree configuration, because prpencvf has no device node. The FIM now is completely configurable via its V4L2 controls.
- FIM: drop
imx_media_fim_set_power(), and move the input capture channel request toimx_media_fim_set_stream(). This allows to dropcsi_s_power()as well, since the latter only calledimx_media_fim_set_power().- FIM: add a spinlock to protect the
frame_interval_monitor()from the setting of new control values. Theframe_interval_monitor()is called from interrupt context so a spinlock must be used.- Updated to version 8 video-mux patchset from Philipp Zabel <[email protected]>.
Marek Vasut (1):
media: imx: Drop warning upon multipleS_STREAMdisable callsPhilipp Zabel (8):
dt-bindings: Add bindings for video-multiplexer device
ARM: dts: imx6qdl: add multiplexer controls
ARM: dts: imx6qdl: Add video multiplexers,mipi_csi, and their
connections
add mux and video interface bridge entity functions
platform: add video-multiplexer subdevice driver
media: imx: csi: increase burst size for YUV formats
media: imx: csi: add frame skipping support
media: imx: csi: add sink selection rectanglesRussell King (3):
media: imx: csi: add support for bayer formats
media: imx: csi: add frame size/interval enumeration
media: imx: capture: add frame sizes/interval enumerationSteve Longerbeam (22):
[media] dt-bindings: Add bindings for i.MX media driver [media] dt/bindings: Add bindings for OV5640
ARM: dts: imx6qdl: Add compatible, clocks, irqs to MIPI CSI-2 node
ARM: dts: imx6qdl: add capture-subsystem device
ARM: dts: imx6qdl-sabrelite: remove erratum ERR006687 workaround
ARM: dts: imx6-sabrelite: add OV5642 and OV5640 camera sensors
ARM: dts: imx6-sabresd: add OV5642 and OV5640 camera sensors
ARM: dts: imx6-sabreauto: create i2cmux for i2c3
ARM: dts: imx6-sabreauto: add reset-gpios property formax7310_b
ARM: dts: imx6-sabreauto: add pinctrl for gpt input capture
ARM: dts: imx6-sabreauto: add the ADV7180 video decoder [media] add Omnivision OV5640 sensor driver
media: Add userspace header file for i.MX
media: Add i.MX media core driver
media: imx: Add a TODO file
media: imx: Add Capture Device Interface
media: imx: Add CSI subdev driver
media: imx: Add VDIC subdev driver
media: imx: Add IC subdev drivers
media: imx: Add MIPI CSI-2 Receiver subdev driver
media: imx: set and propagate default field, colorimetry
ARM:imx_v6_v7_defconfig: Enable staging video4linux drivers.../devicetree/bindings/media/i2c/ov5640.txt | 45 + Documentation/devicetree/bindings/media/imx.txt | 47 + .../devicetree/bindings/media/video-mux.txt | 60 + Documentation/media/uapi/mediactl/media-types.rst | 21 + Documentation/media/v4l-drivers/imx.rst | 614 +++++ arch/arm/boot/dts/imx6dl-sabrelite.dts | 5 + arch/arm/boot/dts/imx6dl-sabresd.dts | 5 + arch/arm/boot/dts/imx6dl.dtsi | 189 ++ arch/arm/boot/dts/imx6q-sabrelite.dts | 5 + arch/arm/boot/dts/imx6q-sabresd.dts | 5 + arch/arm/boot/dts/imx6q.dtsi | 125 ++ arch/arm/boot/dts/imx6qdl-sabreauto.dtsi | 136 +- arch/arm/boot/dts/imx6qdl-sabrelite.dtsi | 152 +- arch/arm/boot/dts/imx6qdl-sabresd.dtsi | 114 +- arch/arm/boot/dts/imx6qdl.dtsi | 20 +- arch/arm/configs/imx_v6_v7_defconfig | 11 + drivers/media/i2c/Kconfig | 10 + drivers/media/i2c/Makefile | 1 + drivers/media/i2c/ov5640.c | 2344 ++++++++++++++++++++ drivers/media/platform/Kconfig | 6 + drivers/media/platform/Makefile | 2 + drivers/media/platform/video-mux.c | 334 +++ drivers/staging/media/Kconfig | 2 + drivers/staging/media/Makefile | 1 + drivers/staging/media/imx/Kconfig | 21 + drivers/staging/media/imx/Makefile | 12 + drivers/staging/media/imx/TODO | 23 + drivers/staging/media/imx/imx-ic-common.c | 113 + drivers/staging/media/imx/imx-ic-prp.c | 518 +++++ drivers/staging/media/imx/imx-ic-prpencvf.c | 1309 +++++++++++ drivers/staging/media/imx/imx-ic.h | 38 + drivers/staging/media/imx/imx-media-capture.c | 775 +++++++ drivers/staging/media/imx/imx-media-csi.c | 1816 +++++++++++++++ drivers/staging/media/imx/imx-media-dev.c | 666 ++++++ drivers/staging/media/imx/imx-media-fim.c | 494 +++++ drivers/staging/media/imx/imx-media-internal-sd.c | 349 +++ drivers/staging/media/imx/imx-media-of.c | 270 +++ drivers/staging/media/imx/imx-media-utils.c | 896 ++++++++ drivers/staging/media/imx/imx-media-vdic.c | 1009 +++++++++ drivers/staging/media/imx/imx-media.h | 325 +++ drivers/staging/media/imx/imx6-mipi-csi2.c | 698 ++++++ include/linux/imx-media.h | 29 + include/media/imx.h | 15 + include/uapi/linux/media.h | 6 + include/uapi/linux/v4l2-controls.h | 4 + 45 files changed, 13613 insertions(+), 27 deletions(-)create mode 100644
Documentation/devicetree/bindings/media/i2c/ov5640.txtcreate mode 100644Documentation/devicetree/bindings/media/imx.txtcreate mode 100644Documentation/devicetree/bindings/media/video-mux.txtcreate mode 100644Documentation/media/v4l-drivers/imx.rstcreate mode 100644drivers/media/i2c/ov5640.c
create mode 100644drivers/media/platform/video-mux.c
create mode 100644drivers/staging/media/imx/Kconfig
create mode 100644drivers/staging/media/imx/Makefile
create mode 100644drivers/staging/media/imx/TODO
create mode 100644drivers/staging/media/imx/imx-ic-common.ccreate mode 100644drivers/staging/media/imx/imx-ic-prp.ccreate mode 100644drivers/staging/media/imx/imx-ic-prpencvf.ccreate mode 100644drivers/staging/media/imx/imx-ic.h
create mode 100644drivers/staging/media/imx/imx-media-capture.ccreate mode 100644drivers/staging/media/imx/imx-media-csi.ccreate mode 100644drivers/staging/media/imx/imx-media-dev.ccreate mode 100644drivers/staging/media/imx/imx-media-fim.ccreate mode 100644drivers/staging/media/imx/imx-media-internal-sd.ccreate mode 100644drivers/staging/media/imx/imx-media-of.ccreate mode 100644drivers/staging/media/imx/imx-media-utils.ccreate mode 100644drivers/staging/media/imx/imx-media-vdic.ccreate mode 100644drivers/staging/media/imx/imx-media.hcreate mode 100644drivers/staging/media/imx/imx6-mipi-csi2.ccreate mode 100644include/linux/imx-media.h
create mode 100644include/media/imx.h
Steve Longerbeam
maintainerQuoted 2 lines · Hans Verkuil
We're still waiting for an Ack for patch 02/34, right?
Hi Hans, Rub has provided an Ack for patch 2.
Quoted 2 lines
Other than that everything is ready AFAICT.
But as Pavel pointed out, in fact we are missing many Acks still, for all of the dts source changes (patches 4-14), as well as really everything else (imx-media staging driver patches).
Steve
Steve Longerbeam
maintainerQuoted 7 lines · Steve Longerbeam
Quoted 2 lines · Hans Verkuil
We're still waiting for an Ack for patch 02/34, right?
Hi Hans, Rub
damn, I really need to proof-read before hitting send. "Rob" (sorry Rob!).
Steve
Quoted 8 lines · Steve Longerbeam
Quoted 2 lines · Hans Verkuil
We're still waiting for an Ack for patch 02/34, right?
Hi Hans, Rub has provided an Ack for patch 2.
Quoted 7 lines
Quoted 2 lines
Other than that everything is ready AFAICT.
But as Pavel pointed out, in fact we are missing many Acks still, for all of the dts source changes (patches 4-14), as well as really everything else (imx-media staging driver patches).
No Acks needed for the staging part. It's staging, so not held to the same standards as non-staging parts. That doesn't mean Acks aren't welcome, of course.
You don't need Greg's Ack for staging/media either, patches there go in via us (generally at least) and we handle those, not Greg.
Regards,
Hans
Hi!
Quoted 8 lines
Quoted 7 lines
Quoted 2 lines
Other than that everything is ready AFAICT.
But as Pavel pointed out, in fact we are missing many Acks still, for all of the dts source changes (patches 4-14), as well as really everything else (imx-media staging driver patches).
Quoted 6 lines
No Acks needed for the staging part. It's staging, so not held to the same standards as non-staging parts. That doesn't mean Acks aren't welcome, of course.
You don't need Greg's Ack for staging/media either, patches there go in via us (generally at least) and we handle those, not Greg.
Ok, good.
Can you just apply the patches? This is staging, they can be reviewed/fixed there -- as expected for staging. They are already way beyond staging quality, we don't want to get to "series v17" here, and repeatedly sending them over the email does not really do them any good.
Thanks,
Pavel
Signature · 9 lines
(english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: Digital signature URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20170610/ae3ad840/attachment.sig>
Quoted 12 lines · Hans Verkuil
Quoted 11 lines · Steve Longerbeam
Quoted 2 lines · Hans Verkuil
We're still waiting for an Ack for patch 02/34, right?
Hi Hans, Rub has provided an Ack for patch 2.
Quoted 2 lines
Other than that everything is ready AFAICT.
Quoted 8 lines
Quoted 4 lines
But as Pavel pointed out, in fact we are missing many Acks still, for all of the dts source changes (patches 4-14), as well as really everything else (imx-media staging driver patches).
No Acks needed for the staging part. It's staging, so not held to the same standards as non-staging parts. That doesn't mean Acks aren't welcome, of course.
Acks are wanted for particular i.MX DTS changes including device tree binding descriptions.
Shawn, please bless the series.
Quoted 4 lines
You don't need Greg's Ack for staging/media either, patches there go in via us (generally at least) and we handle those, not Greg.
--
With best wishes,
Vladimir
Steve Longerbeam
maintainerQuoted 14 lines · Vladimir Zapolskiy
Quoted 13 lines · Hans Verkuil
Quoted 12 lines · Steve Longerbeam
Quoted 2 lines · Hans Verkuil
We're still waiting for an Ack for patch 02/34, right?
Hi Hans, Rub has provided an Ack for patch 2.
Quoted 2 lines
Other than that everything is ready AFAICT.
But as Pavel pointed out, in fact we are missing many Acks still, for all of the dts source changes (patches 4-14), as well as really everything else (imx-media staging driver patches).
Quoted 8 lines
Quoted 3 lines
No Acks needed for the staging part. It's staging, so not held to the same standards as non-staging parts. That doesn't mean Acks aren't welcome, of course.
Acks are wanted for particular i.MX DTS changes including device tree binding descriptions.
Shawn, please bless the series.
I second that request!
There are a couple minor update to the ARM dts patches for imx6, I will post a new series for them. But there should not be anything of much controversy in these ARM dts patches.
Steve
Shawn Guo: your attetion is needed here Re: [PATCH v8 00/34] i.MX Media Driver
In reply to Vladimir ZapolskiyHi!
Quoted 5 lines
Quoted 5 lines
Quoted 4 lines
But as Pavel pointed out, in fact we are missing many Acks still, for all of the dts source changes (patches 4-14), as well as really everything else (imx-media staging driver patches).
Quoted 8 lines
Quoted 3 lines
No Acks needed for the staging part. It's staging, so not held to the same standards as non-staging parts. That doesn't mean Acks aren't welcome, of course.
Acks are wanted for particular i.MX DTS changes including device tree binding descriptions.
Shawn, please bless the series.
Hmm. I changed the subject to grab Shawn's attetion.
But his acks should not be needed for forward progress. Yes, it would be good, but he does not react -- so just reorder the series so that dts changes come last, then apply the parts you can apply: driver can go in.
And actually... if maintainer does not respond at all, there are ways to deal with that, too...
Best regards,
Pavel
Signature · 9 lines
(english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: Digital signature URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20170620/f6680805/attachment.sig>
Fabio Estevam
reviewerRe: Shawn Guo: your attetion is needed here Re: [PATCH v8 00/34] i.MX Media Driver
Quoted 9 lines · Pavel Machek
Hmm. I changed the subject to grab Shawn's attetion.
But his acks should not be needed for forward progress. Yes, it would be good, but he does not react -- so just reorder the series so that dts changes come last, then apply the parts you can apply: driver can go in.
And actually... if maintainer does not respond at all, there are ways to deal with that, too...
Shawn has already applied the dts part of the series and they show up in linux-next.
Re: Shawn Guo: your attetion is needed here Re: [PATCH v8 00/34] i.MX Media Driver
Quoted 9 lines · Fabio Estevam
Quoted 7 lines · Pavel Machek
Hmm. I changed the subject to grab Shawn's attetion.
But his acks should not be needed for forward progress. Yes, it would be good, but he does not react -- so just reorder the series so that dts changes come last, then apply the parts you can apply: driver can go in.
Quoted 5 lines
Quoted 2 lines
And actually... if maintainer does not respond at all, there are ways to deal with that, too...
Shawn has already applied the dts part of the series and they show up in linux-next.
Aha, sorry about the noise. I see videomux parts being merged by Mauro. Good :-).
Pavel
Signature · 9 lines
(english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: Digital signature URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20170620/61ce5724/attachment.sig>
Hi Steve, Philipp,
While preparing the pull request I noticed that the MAINTAINERS file wasn't updated. Steve, can you post a patch adding entries for the imx and ov5640 driver? Philipp, can you do the same for the video mux? I assume you're the maintainer for this?
Thanks!
I also made it possible to compile-test this driver with this patch:
---
drivers/staging/media/imx/Kconfig +2 -1
@@ -1,6 +1,7 @@ config VIDEO_IMX_MEDIA tristate "i.MX5/6 V4L2 media core driver" - depends on MEDIA_CONTROLLER && VIDEO_V4L2 && ARCH_MXC && IMX_IPUV3_CORE + depends on MEDIA_CONTROLLER && VIDEO_V4L2 + depends on (ARCH_MXC && IMX_IPUV3_CORE) || COMPILE_TEST select V4L2_FWNODE ---help--- Say yes here to enable support for video4linux media controller
drivers/staging/media/imx/imx-media-csi.c +1 -0
@@ -13,6 +13,7 @@#include <linux/gcd.h> #include <linux/interrupt.h> #include <linux/module.h> +#include <linux/pinctrl/consumer.h> #include <linux/platform_device.h> #include <media/v4l2-ctrls.h> #include <media/v4l2-device.h>
Steve, if you're OK with that I was planning to just modify your original patch rather than adding another patch on top.
Regards,
Hans
Quoted 18 lines · Hans Verkuil
Hi Steve, Philipp,
While preparing the pull request I noticed that the
MAINTAINERSfile wasn't updated. Steve, can you post a patch adding entries for the imx and ov5640 driver? Philipp, can you do the same for the video mux? I assume you're the maintainer for this?Thanks!
I also made it possible to compile-test this driver with this patch:
Signed-off-by Hans Verkuil <[email protected]>---
drivers/staging/media/imx/Kconfig +0
-0
drivers/staging/media/imx/Kconfig line 7
Quoted 8 lines
drivers/staging/media/imx/Kconfig +2
-1@@ -1,6 +1,7 @@ config VIDEO_IMX_MEDIA tristate "i.MX5/6 V4L2 media core driver" - depends on MEDIA_CONTROLLER && VIDEO_V4L2 && ARCH_MXC && IMX_IPUV3_CORE + depends on MEDIA_CONTROLLER && VIDEO_V4L2 + depends on (ARCH_MXC && IMX_IPUV3_CORE) || COMPILE_TEST select V4L2_FWNODE ---help--- Say yes here to enable support for video4linux media controller
^^^^
This is too optimistic, it fails on IPU code. Ignore this chunk.
Hans
Quoted 20 lines
drivers/staging/media/imx/imx-media-csi.c +1
-0@@ -13,6 +13,7 @@#include <linux/gcd.h> #include <linux/interrupt.h> #include <linux/module.h> +#include <linux/pinctrl/consumer.h> #include <linux/platform_device.h> #include <media/v4l2-ctrls.h> #include <media/v4l2-device.h> Steve, if you're OK with that I was planning to just modify your original patch rather than adding another patch on top.
Regards,
Hans