Skip to contentSkip to conversation

IMX219 MIPI Sensor (meda-tree) with vaniila I.MX6Q media drivers

DISCUSSION Михайлов Алексей Анатольевич 4 participants 8 messages Mar 26, 2020 – Mar 28, 2020 Media (V4L/DVB)

lore
Share & tools
lore
#0 lore raw

Hi!

We build custom CPU Module with NXP/Freescale IMX6QuadPlus CPU.I use latest stable kernel from kernel.org. This time kernel version 5.5.11. Also I connect to I.MX MIPI cameras from RaspberryPI (Rev 2.1 with Sony IMX219). For IMX219 used actual driver from [1]. Usersapce based on Gentoo Linux, have media-utils version 1.2.1, v4l2-utils version 1.18.0, gstreamer version 1.14.5 with v4l2 plugins. Also Wayland version 1.17 based graphics with XWayland.

Camera write in DTB:
==== cut: DTB fragments ====
/ {
[skiped]

	imx219_clk: camera-clk {
		compatible = "fixed-clock";
		#clock-cells = <0>;
		clock-frequency = <24000000>;
	};

	imx219_1v2_reg: cam1v2_regulator {
		compatible = "regulator-fixed";
		regulator-name = "IMX219_1V2";
		regulator-min-microvolt = <1200000>;
		regulator-max-microvolt = <1200000>;
		vin-supply = <&p3v3_reg>;
		regulator-always-on;
	};

	imx219_1v8_reg: cam1v8_regulator {
		compatible = "regulator-fixed";
		regulator-name = "IMX219_1V8";
		regulator-min-microvolt = <1800000>;
		regulator-max-microvolt = <1800000>;
		vin-supply = <&p3v3_reg>;
		regulator-always-on;
	};

	imx219_2v8_reg: cam2v8_regulator {
		compatible = "regulator-fixed";
		regulator-name = "IMX219_2V8";
		regulator-min-microvolt = <2800000>;
		regulator-max-microvolt = <2800000>;
		vin-supply = <&p3v3_reg>;
		regulator-always-on;
	};
[skiped]
csi_i2c: i2c-mux@1 { /* CSI camera */
	#address-cells = <1>;
	#size-cells = <0>;
	reg = <1>;
	sensor@10 {	/* Raspberry Camera V2 */
		compatible = "sony,imx219";
		reg = <0x10>;
		#address-cells = <1>;
		#size-cells = <0>;
		clocks = <&imx219_clk>;
		clock-names = "xclk";
		DOVDD-supply = <&imx219_1v8_reg>; /* 1.8v */
		AVDD-supply = <&imx219_2v8_reg>;  /* 2.8v */
		DVDD-supply = <&imx219_1v2_reg>;  /* 1.2v */

		port {
			csi_sensor_out: endpoint {
				remote-endpoint = <&csi_port_in>;
				link-frequencies = /bits/ 64 <456000000>;
				clock-lanes = <0>;
				data-lanes = <1 2>;
			};
		};
	};
};
[skiped]
&mipi_csi {
	status = "okay";

	port@0 {
		reg = <0>;
		csi_port_in: endpoint {
			remote-endpoint = <&csi_sensor_out>;
			clock-lanes = <0>;
			data-lanes = <1 2>;
		};
	};
};

[skiped]
==== cut: DTB fragments ====

I use script for init connected camera

==== cut: Camera init script ===
#!/bin/bash

# sensor output format and resolutions
# RaspberryPI Camera rev 2.1 (Sony I.MX219)
I_FORMAT=SRGGB10_1X10
I_RESOLUTION=1920x1080
CROP=(0,0)/640x480
# capture format and resolution
O_FORMAT=AYUV32
O_RESOLUTION=640x480
# viewport format and resolution
V_FORMAT=AYUV32
V_RESOLUTION=640x480

# Reset all media links
media-ctl -r

# Sersor to IPU and PRP path
# RaspberryPI Camera rev 2.1 (Sony I.MX219) media-ctl -l "'imx219 9-0010':0 -> 'imx6-mipi-csi2':0[1]" media-ctl -l "'imx6-mipi-csi2':2 -> 'ipu1_csi1':0[1]" media-ctl -l "'ipu1_csi1':1 -> 'ipu1_ic_prp':0[1]"

# media-ctl -l "'ipu1_csi1':2 -> 'ipu1_csi1 capture':0[1]" # /dev/video3 (unused, unprocessed)
# IPU to capture
media-ctl -l "'ipu1_ic_prp':1 -> 'ipu1_ic_prpenc':0[1]"
media-ctl -l "'ipu1_ic_prpenc':1 -> 'ipu1_ic_prpenc capture':0[1]" # /dev/video1
# IPU to viewport
media-ctl -l "'ipu1_ic_prp':2 -> 'ipu1_ic_prpvf':0[1]"
media-ctl -l "'ipu1_ic_prpvf':1 -> 'ipu1_ic_prpvf capture':0[1]" # /dev/video2

# RaspberryPI Camera rev 2.1 (Sony I.MX219)
media-ctl -V "'imx219 9-0010':0 [fmt:${I_FORMAT}/${I_RESOLUTION} field:none]" media-ctl -V "'imx6-mipi-csi2':2 [fmt:${I_FORMAT}/${I_RESOLUTION} field:none]" media-ctl -V "'ipu1_csi1':0 [crop:${CROP}]"
media-ctl -V "'ipu1_csi1':1 [fmt:${I_FORMAT}/${O_RESOLUTION} field:none]"

media-ctl -V "'ipu1_ic_prp':1 [fmt:${O_FORMAT}/${O_RESOLUTION} field:none]" media-ctl -V "'ipu1_ic_prpenc':1 [fmt:${O_FORMAT}/${O_RESOLUTION} field:none]"

media-ctl -V "'ipu1_ic_prp':2 [fmt:${V_FORMAT}/${V_RESOLUTION} field:none]" media-ctl -V "'ipu1_ic_prpvf':1 [fmt:${V_FORMAT}/${V_RESOLUTION} field:none]" ==== cut: Camera init script ===

I try start system with V2.1 camera and start capture frames:
[...]

# ./camera_init.sh
# gst-launch-1.0 -v v4l2src device=/dev/video2 ! fakesink
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
/GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = video/x-raw, format=(string)YUY2, framerate=(fraction)30000/1001, width=(int)640, height=(int)480, colorimetry=(string)2:4:7:1, interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstFakeSink:fakesink0.GstPad:sink: caps = video/x-raw, format=(string)YUY2, framerate=(fraction)30000/1001, width=(int)640, height=(int)480, colorimetry=(string)2:4:7:1, interlace-mode=(string)progressive
New clock: GstSystemClock
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Failed to allocate required memory.
Additional debug info:
/var/tmp/portage/media-plugins/gst-plugins-v4l2-1.14.5/work/gst-plugins-good-1.14.5/sys/v4l2/gstv4l2src.c(656): gst_v4l2src_decide_allocation (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
Buffer pool activation failed
Execution ended after 0:00:00.014952667
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...
# media-ctl -p
Media controller API version 5.5.13

Media device information
------------------------
driver imx-media
model imx-media
serial
bus info
hw revision 0x0
driver version 5.5.13

Device topology
[...]
- entity 15: ipu1_ic_prp (3 pads, 5 links)
type V4L2 subdev subtype Unknown flags 0
device node name /dev/v4l-subdev2
pad0: Sink
[fmt:AYUV8_1X32/640x480@1/30 field:none colorspace:srgb
xfer:srgb ycbcr:601 quantization:full-range]

                <- "ipu1_csi0":1 []
                <- "ipu1_vdic":2 []
                <- "ipu1_csi1":1 [ENABLED]
        pad1: Source
                [fmt:AYUV8_1X32/640x480@1/30 field:none colorspace:srgb 
xfer:srgb ycbcr:601 quantization:full-range]
                -> "ipu1_ic_prpenc":0 [ENABLED]
        pad2: Source
                [fmt:AYUV8_1X32/640x480@1/30 field:none colorspace:srgb 
xfer:srgb ycbcr:601 quantization:full-range]
                -> "ipu1_ic_prpvf":0 [ENABLED]

- entity 19: ipu1_ic_prpenc (2 pads, 2 links)
type V4L2 subdev subtype Unknown flags 0
device node name /dev/v4l-subdev3
pad0: Sink
[fmt:AYUV8_1X32/640x480@1/30 field:none colorspace:srgb
xfer:srgb ycbcr:601 quantization:full-range]

                <- "ipu1_ic_prp":1 [ENABLED]
        pad1: Source
                [fmt:AYUV8_1X32/640x480@1/30 field:none colorspace:srgb 
xfer:srgb ycbcr:601 quantization:lim-range]
                -> "ipu1_ic_prpenc capture":0 [ENABLED]

- entity 22: ipu1_ic_prpenc capture (1 pad, 1 link) type Node subtype V4L flags 0 device node name /dev/video1
pad0: Sink
<- "ipu1_ic_prpenc":1 [ENABLED]

- entity 28: ipu1_ic_prpvf (2 pads, 2 links)
type V4L2 subdev subtype Unknown flags 0 device node name /dev/v4l-subdev4
pad0: Sink
[fmt:AYUV8_1X32/640x480@1001/30000 field:none
colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range]

                <- "ipu1_ic_prp":2 [ENABLED]
        pad1: Source
                [fmt:AYUV8_1X32/640x480@1001/30000 field:none 
colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range]
                -> "ipu1_ic_prpvf capture":0 [ENABLED]

- entity 31: ipu1_ic_prpvf capture (1 pad, 1 link) type Node subtype V4L flags 0 device node name /dev/video2
pad0: Sink
<- "ipu1_ic_prpvf":1 [ENABLED]

- entity 47: ipu1_csi1 (3 pads, 4 links)
type V4L2 subdev subtype Unknown flags 0 device node name /dev/v4l-subdev5
pad0: Sink
[fmt:SRGGB10_1X10/1920x1080@1/30 field:none
colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range

                 crop.bounds:(0,0)/1920x1080
                 crop:(0,0)/640x480
                 compose.bounds:(0,0)/640x480
                 compose:(0,0)/640x480]
                <- "imx6-mipi-csi2":2 [ENABLED]
        pad1: Source
                [fmt:SRGGB10_1X10/640x480@1/30 field:none 
colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range]
                -> "ipu1_ic_prp":0 [ENABLED]
                -> "ipu1_vdic":0 []
        pad2: Source
                [fmt:SRGGB10_1X10/640x480@1/30 field:none 
colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range]
                -> "ipu1_csi1 capture":0 []

[...]
- entity 121: imx6-mipi-csi2 (5 pads, 5 links)
type V4L2 subdev subtype Unknown flags 0 device node name /dev/v4l-subdev12
pad0: Sink
[fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb
xfer:srgb ycbcr:601 quantization:full-range]

                <- "imx219 9-0010":0 [ENABLED]
        pad1: Source
                [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb 
xfer:srgb ycbcr:601 quantization:full-range]
                -> "ipu1_csi0_mux":0 []
        pad2: Source
                [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb 
xfer:srgb ycbcr:601 quantization:full-range]
                -> "ipu1_csi1":0 [ENABLED]
        pad3: Source
                [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb 
xfer:srgb ycbcr:601 quantization:full-range]
                -> "ipu2_csi0":0 []
        pad4: Source
                [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb 
xfer:srgb ycbcr:601 quantization:full-range]
                -> "ipu2_csi1_mux":0 []

[...]
- entity 135: imx219 9-0010 (1 pad, 1 link)
type V4L2 subdev subtype Sensor flags 0 device node name /dev/v4l-subdev15
pad0: Source
[fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb
xfer:srgb ycbcr:601 quantization:full-range]

                -> "imx6-mipi-csi2":0 [ENABLED]
localhost ~ # dmesg -c
[...]
[   51.941858] ipu1_ic_prpvf: pipeline start failed with -32

localhost ~ #

I think EPIPE (-32) caused by link between pad1 ipu1_csi1 (fmt:SRGGB10_1X10/640x480@1/30) and pad0 ipu1_ic_prp (fmt:AYUV8_1X32/640x480@1/30) - format mismatch. But I don't know how fix this trouble. Theory, I also have RaspberryPI camera rev 1.3 with OmniVision OV5647 camera. Unfortunately, she did not work either.

Anybody can help me?

[1] https://git.linuxtv.org/media_tree.git/tree/drivers/media/i2c/imx219.c

#1 lore raw

Adding Steve and Philipp in case they have some ideas.

Quoted 306 lines · Михайлов Алексей Анатольевич

Hi!

We build custom CPU Module with NXP/Freescale IMX6QuadPlus CPU.I use latest stable kernel from kernel.org. This time kernel version 5.5.11. Also I connect to I.MX MIPI cameras from RaspberryPI (Rev 2.1 with Sony IMX219). For IMX219 used actual driver from [1]. Usersapce based on Gentoo Linux, have media-utils version 1.2.1, v4l2-utils version 1.18.0, gstreamer version 1.14.5 with v4l2 plugins. Also Wayland version 1.17 based graphics with XWayland.

Camera write in DTB:
==== cut: DTB fragments ====
/ {
[skiped]

        imx219_clk: camera-clk {
                compatible = "fixed-clock";
                #clock-cells = <0>;
                clock-frequency = <24000000>;
        };

        imx219_1v2_reg: cam1v2_regulator {
                compatible = "regulator-fixed";
                regulator-name = "IMX219_1V2";
                regulator-min-microvolt = <1200000>;
                regulator-max-microvolt = <1200000>;
                vin-supply = <&p3v3_reg>;
                regulator-always-on;
        };

        imx219_1v8_reg: cam1v8_regulator {
                compatible = "regulator-fixed";
                regulator-name = "IMX219_1V8";
                regulator-min-microvolt = <1800000>;
                regulator-max-microvolt = <1800000>;
                vin-supply = <&p3v3_reg>;
                regulator-always-on;
        };

        imx219_2v8_reg: cam2v8_regulator {
                compatible = "regulator-fixed";
                regulator-name = "IMX219_2V8";
                regulator-min-microvolt = <2800000>;
                regulator-max-microvolt = <2800000>;
                vin-supply = <&p3v3_reg>;
                regulator-always-on;
        };
[skiped]
csi_i2c: i2c-mux@1 { /* CSI camera */
        #address-cells = <1>;
        #size-cells = <0>;
        reg = <1>;
        sensor@10 {     /* Raspberry Camera V2 */
                compatible = "sony,imx219";
                reg = <0x10>;
                #address-cells = <1>;
                #size-cells = <0>;
                clocks = <&imx219_clk>;
                clock-names = "xclk";
                DOVDD-supply = <&imx219_1v8_reg>; /* 1.8v */
                AVDD-supply = <&imx219_2v8_reg>;  /* 2.8v */
                DVDD-supply = <&imx219_1v2_reg>;  /* 1.2v */

                port {
                        csi_sensor_out: endpoint {
                                remote-endpoint = <&csi_port_in>;
                                link-frequencies = /bits/ 64 <456000000>;
                                clock-lanes = <0>;
                                data-lanes = <1 2>;
                        };
                };
        };
};
[skiped]
&mipi_csi {
        status = "okay";

        port@0 {
                reg = <0>;
                csi_port_in: endpoint {
                        remote-endpoint = <&csi_sensor_out>;
                        clock-lanes = <0>;
                        data-lanes = <1 2>;
                };
        };
};

[skiped]
==== cut: DTB fragments ====

I use script for init connected camera

==== cut: Camera init script ===
#!/bin/bash

# sensor output format and resolutions
# RaspberryPI Camera rev 2.1 (Sony I.MX219)
I_FORMAT=SRGGB10_1X10
I_RESOLUTION=1920x1080
CROP=(0,0)/640x480
# capture format and resolution
O_FORMAT=AYUV32
O_RESOLUTION=640x480
# viewport format and resolution
V_FORMAT=AYUV32
V_RESOLUTION=640x480

# Reset all media links
media-ctl -r

# Sersor to IPU and PRP path
# RaspberryPI Camera rev 2.1 (Sony I.MX219) media-ctl -l "'imx219 9-0010':0 -> 'imx6-mipi-csi2':0[1]" media-ctl -l "'imx6-mipi-csi2':2 -> 'ipu1_csi1':0[1]" media-ctl -l "'ipu1_csi1':1 -> 'ipu1_ic_prp':0[1]"

# media-ctl -l "'ipu1_csi1':2 -> 'ipu1_csi1 capture':0[1]" # /dev/video3
(unused, unprocessed)
# IPU to capture
media-ctl -l "'ipu1_ic_prp':1 -> 'ipu1_ic_prpenc':0[1]"
media-ctl -l "'ipu1_ic_prpenc':1 -> 'ipu1_ic_prpenc capture':0[1]" #
/dev/video1
# IPU to viewport
media-ctl -l "'ipu1_ic_prp':2 -> 'ipu1_ic_prpvf':0[1]"
media-ctl -l "'ipu1_ic_prpvf':1 -> 'ipu1_ic_prpvf capture':0[1]" #
/dev/video2

# RaspberryPI Camera rev 2.1 (Sony I.MX219)
media-ctl -V "'imx219 9-0010':0 [fmt:${I_FORMAT}/${I_RESOLUTION} field:none]"
media-ctl -V "'imx6-mipi-csi2':2 [fmt:${I_FORMAT}/${I_RESOLUTION} field:none]"
media-ctl -V "'ipu1_csi1':0 [crop:${CROP}]"
media-ctl -V "'ipu1_csi1':1 [fmt:${I_FORMAT}/${O_RESOLUTION} field:none]"

media-ctl -V "'ipu1_ic_prp':1 [fmt:${O_FORMAT}/${O_RESOLUTION} field:none]" media-ctl -V "'ipu1_ic_prpenc':1 [fmt:${O_FORMAT}/${O_RESOLUTION} field:none]"

media-ctl -V "'ipu1_ic_prp':2 [fmt:${V_FORMAT}/${V_RESOLUTION} field:none]" media-ctl -V "'ipu1_ic_prpvf':1 [fmt:${V_FORMAT}/${V_RESOLUTION} field:none]"
==== cut: Camera init script ===

I try start system with V2.1 camera and start capture frames:
[...]

# ./camera_init.sh
# gst-launch-1.0 -v v4l2src device=/dev/video2 ! fakesink
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
/GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps =
video/x-raw, format=(string)YUY2, framerate=(fraction)30000/1001,
width=(int)640, height=(int)480, colorimetry=(string)2:4:7:1,
interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstFakeSink:fakesink0.GstPad:sink: caps =
video/x-raw, format=(string)YUY2, framerate=(fraction)30000/1001,
width=(int)640, height=(int)480, colorimetry=(string)2:4:7:1,
interlace-mode=(string)progressive
New clock: GstSystemClock
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Failed
to allocate required memory.
Additional debug info:
/var/tmp/portage/media-plugins/gst-plugins-v4l2-1.14.5/work/gst-plugins-good-1.14.5/sys/v4l2/gstv4l2src.c(656):
gst_v4l2src_decide_allocation ():
/GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
Buffer pool activation failed
Execution ended after 0:00:00.014952667
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...
# media-ctl -p
Media controller API version 5.5.13

Media device information
------------------------
driver imx-media
model imx-media
serial
bus info
hw revision 0x0
driver version 5.5.13

Device topology
[...]
- entity 15: ipu1_ic_prp (3 pads, 5 links)
type V4L2 subdev subtype Unknown flags 0 device node name /dev/v4l-subdev2
pad0: Sink
[fmt:AYUV8_1X32/640x480@1/30 field:none colorspace:srgb
xfer:srgb ycbcr:601 quantization:full-range]

                 <- "ipu1_csi0":1 []
                 <- "ipu1_vdic":2 []
                 <- "ipu1_csi1":1 [ENABLED]
         pad1: Source
                 [fmt:AYUV8_1X32/640x480@1/30 field:none colorspace:srgb
xfer:srgb ycbcr:601 quantization:full-range]
                 -> "ipu1_ic_prpenc":0 [ENABLED]
         pad2: Source
                 [fmt:AYUV8_1X32/640x480@1/30 field:none colorspace:srgb
xfer:srgb ycbcr:601 quantization:full-range]
                 -> "ipu1_ic_prpvf":0 [ENABLED]

- entity 19: ipu1_ic_prpenc (2 pads, 2 links)
type V4L2 subdev subtype Unknown flags 0 device node name /dev/v4l-subdev3
pad0: Sink
[fmt:AYUV8_1X32/640x480@1/30 field:none colorspace:srgb
xfer:srgb ycbcr:601 quantization:full-range]

                 <- "ipu1_ic_prp":1 [ENABLED]
         pad1: Source
                 [fmt:AYUV8_1X32/640x480@1/30 field:none colorspace:srgb
xfer:srgb ycbcr:601 quantization:lim-range]
                 -> "ipu1_ic_prpenc capture":0 [ENABLED]

- entity 22: ipu1_ic_prpenc capture (1 pad, 1 link) type Node subtype V4L flags 0 device node name /dev/video1
pad0: Sink
<- "ipu1_ic_prpenc":1 [ENABLED]

- entity 28: ipu1_ic_prpvf (2 pads, 2 links)
type V4L2 subdev subtype Unknown flags 0 device node name /dev/v4l-subdev4
pad0: Sink
[fmt:AYUV8_1X32/640x480@1001/30000 field:none
colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range]

                 <- "ipu1_ic_prp":2 [ENABLED]
         pad1: Source
                 [fmt:AYUV8_1X32/640x480@1001/30000 field:none
colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range]
                 -> "ipu1_ic_prpvf capture":0 [ENABLED]

- entity 31: ipu1_ic_prpvf capture (1 pad, 1 link) type Node subtype V4L flags 0 device node name /dev/video2
pad0: Sink
<- "ipu1_ic_prpvf":1 [ENABLED]

- entity 47: ipu1_csi1 (3 pads, 4 links)
type V4L2 subdev subtype Unknown flags 0 device node name /dev/v4l-subdev5
pad0: Sink
[fmt:SRGGB10_1X10/1920x1080@1/30 field:none
colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range

                  crop.bounds:(0,0)/1920x1080
                  crop:(0,0)/640x480
                  compose.bounds:(0,0)/640x480
                  compose:(0,0)/640x480]
                 <- "imx6-mipi-csi2":2 [ENABLED]
         pad1: Source
                 [fmt:SRGGB10_1X10/640x480@1/30 field:none
colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range]
                 -> "ipu1_ic_prp":0 [ENABLED]
                 -> "ipu1_vdic":0 []
         pad2: Source
                 [fmt:SRGGB10_1X10/640x480@1/30 field:none
colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range]
                 -> "ipu1_csi1 capture":0 []

[...]
- entity 121: imx6-mipi-csi2 (5 pads, 5 links)
type V4L2 subdev subtype Unknown flags 0 device node name /dev/v4l-subdev12
pad0: Sink
[fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb
xfer:srgb ycbcr:601 quantization:full-range]

                 <- "imx219 9-0010":0 [ENABLED]
         pad1: Source
                 [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb
xfer:srgb ycbcr:601 quantization:full-range]
                 -> "ipu1_csi0_mux":0 []
         pad2: Source
                 [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb
xfer:srgb ycbcr:601 quantization:full-range]
                 -> "ipu1_csi1":0 [ENABLED]
         pad3: Source
                 [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb
xfer:srgb ycbcr:601 quantization:full-range]
                 -> "ipu2_csi0":0 []
         pad4: Source
                 [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb
xfer:srgb ycbcr:601 quantization:full-range]
                 -> "ipu2_csi1_mux":0 []

[...]
- entity 135: imx219 9-0010 (1 pad, 1 link)
type V4L2 subdev subtype Sensor flags 0 device node name /dev/v4l-subdev15
pad0: Source
[fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb
xfer:srgb ycbcr:601 quantization:full-range]

                 -> "imx6-mipi-csi2":0 [ENABLED]
localhost ~ # dmesg -c
[...]
[   51.941858] ipu1_ic_prpvf: pipeline start failed with -32

localhost ~ #

I think EPIPE (-32) caused by link between pad1 ipu1_csi1 (fmt:SRGGB10_1X10/640x480@1/30) and pad0 ipu1_ic_prp (fmt:AYUV8_1X32/640x480@1/30) - format mismatch. But I don't know how fix this trouble. Theory, I also have RaspberryPI camera rev 1.3 with OmniVision OV5647 camera. Unfortunately, she did not work either.

Anybody can help me?

[1] https://git.linuxtv.org/media_tree.git/tree/drivers/media/i2c/imx219.c

#2 lore raw

Hi,

Quoted 123 lines · Fabio Estevam

Adding Steve and Philipp in case they have some ideas.

Quoted 118 lines · Михайлов Алексей Анатольевич

Hi!

We build custom CPU Module with NXP/Freescale IMX6QuadPlus CPU.I use latest stable kernel from kernel.org. This time kernel version 5.5.11. Also I connect to I.MX MIPI cameras from RaspberryPI (Rev 2.1 with Sony IMX219). For IMX219 used actual driver from [1]. Usersapce based on Gentoo Linux, have media-utils version 1.2.1, v4l2-utils version 1.18.0, gstreamer version 1.14.5 with v4l2 plugins. Also Wayland version 1.17 based graphics with XWayland.

Camera write in DTB:
==== cut: DTB fragments ====
/ {
[skiped]

        imx219_clk: camera-clk {
                compatible = "fixed-clock";
                #clock-cells = <0>;
                clock-frequency = <24000000>;
        };

        imx219_1v2_reg: cam1v2_regulator {
                compatible = "regulator-fixed";
                regulator-name = "IMX219_1V2";
                regulator-min-microvolt = <1200000>;
                regulator-max-microvolt = <1200000>;
                vin-supply = <&p3v3_reg>;
                regulator-always-on;
        };

        imx219_1v8_reg: cam1v8_regulator {
                compatible = "regulator-fixed";
                regulator-name = "IMX219_1V8";
                regulator-min-microvolt = <1800000>;
                regulator-max-microvolt = <1800000>;
                vin-supply = <&p3v3_reg>;
                regulator-always-on;
        };

        imx219_2v8_reg: cam2v8_regulator {
                compatible = "regulator-fixed";
                regulator-name = "IMX219_2V8";
                regulator-min-microvolt = <2800000>;
                regulator-max-microvolt = <2800000>;
                vin-supply = <&p3v3_reg>;
                regulator-always-on;
        };
[skiped]
csi_i2c: i2c-mux@1 { /* CSI camera */
        #address-cells = <1>;
        #size-cells = <0>;
        reg = <1>;
        sensor@10 {     /* Raspberry Camera V2 */
                compatible = "sony,imx219";
                reg = <0x10>;
                #address-cells = <1>;
                #size-cells = <0>;
                clocks = <&imx219_clk>;
                clock-names = "xclk";
                DOVDD-supply = <&imx219_1v8_reg>; /* 1.8v */
                AVDD-supply = <&imx219_2v8_reg>;  /* 2.8v */
                DVDD-supply = <&imx219_1v2_reg>;  /* 1.2v */

                port {
                        csi_sensor_out: endpoint {
                                remote-endpoint = <&csi_port_in>;
                                link-frequencies = /bits/ 64 <456000000>;
                                clock-lanes = <0>;
                                data-lanes = <1 2>;
                        };
                };
        };
};
[skiped]
&mipi_csi {
        status = "okay";

        port@0 {
                reg = <0>;
                csi_port_in: endpoint {
                        remote-endpoint = <&csi_sensor_out>;
                        clock-lanes = <0>;
                        data-lanes = <1 2>;
                };
        };
};

[skiped]
==== cut: DTB fragments ====

I use script for init connected camera

==== cut: Camera init script ===
#!/bin/bash

# sensor output format and resolutions
# RaspberryPI Camera rev 2.1 (Sony I.MX219)
I_FORMAT=SRGGB10_1X10
I_RESOLUTION=1920x1080
CROP=(0,0)/640x480
# capture format and resolution
O_FORMAT=AYUV32
O_RESOLUTION=640x480
# viewport format and resolution
V_FORMAT=AYUV32
V_RESOLUTION=640x480

# Reset all media links
media-ctl -r

# Sersor to IPU and PRP path
# RaspberryPI Camera rev 2.1 (Sony I.MX219) media-ctl -l "'imx219 9-0010':0 -> 'imx6-mipi-csi2':0[1]" media-ctl -l "'imx6-mipi-csi2':2 -> 'ipu1_csi1':0[1]" media-ctl -l "'ipu1_csi1':1 -> 'ipu1_ic_prp':0[1]"

# media-ctl -l "'ipu1_csi1':2 -> 'ipu1_csi1 capture':0[1]" # /dev/video3
(unused, unprocessed)
      ^

The IPUv3 IC hardware does not support processing Bayer formats. Those can only be written straight to RAM using this path.

regards
Philipp

#3 lore raw

Hi!

Ok. This time I try to check hardware and not need image processing. I uncomment this line in script:

[...]
- entity 47: ipu1_csi1 (3 pads, 4 links)
type V4L2 subdev subtype Unknown flags 0 device node name /dev/v4l-subdev5
pad0: Sink
[fmt:SRGGB10_1X10/1920x1080@1/30 field:none
colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range

                 crop.bounds:(0,0)/1920x1080
                 crop:(0,0)/640x480
                 compose.bounds:(0,0)/640x480
                 compose:(0,0)/640x480]
                <- "imx6-mipi-csi2":2 [ENABLED]
        pad1: Source
                [fmt:SRGGB10_1X10/640x480@1/30 field:none 
colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range]
                -> "ipu1_ic_prp":0 []
                -> "ipu1_vdic":0 []
        pad2: Source
                [fmt:SRGGB10_1X10/640x480@1/30 field:none 
colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range]
                -> "ipu1_csi1 capture":0 [ENABLED]

- entity 51: ipu1_csi1 capture (1 pad, 1 link)
type Node subtype V4L flags 0
device node name /dev/video3
pad0: Sink
<- "ipu1_csi1":2 [ENABLED]
[...]

- entity 121: imx6-mipi-csi2 (5 pads, 5 links)
type V4L2 subdev subtype Unknown flags 0 device node name /dev/v4l-subdev12
pad0: Sink
[fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb
xfer:srgb ycbcr:601 quantization:full-range]

                <- "imx219 9-0010":0 [ENABLED]
        pad1: Source
                [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb 
xfer:srgb ycbcr:601 quantization:full-range]
                -> "ipu1_csi0_mux":0 []
        pad2: Source
                [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb 
xfer:srgb ycbcr:601 quantization:full-range]
                -> "ipu1_csi1":0 [ENABLED]
        pad3: Source
                [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb 
xfer:srgb ycbcr:601 quantization:full-range]
                -> "ipu2_csi0":0 []
        pad4: Source
                [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb 
xfer:srgb ycbcr:601 quantization:full-range]
                -> "ipu2_csi1_mux":0 []

- entity 135: imx219 9-0010 (1 pad, 1 link)
type V4L2 subdev subtype Sensor flags 0 device node name /dev/v4l-subdev15
pad0: Source
[fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb
xfer:srgb ycbcr:601 quantization:full-range]

                -> "imx6-mipi-csi2":0 [ENABLED]

[...]
localhost ~ # gst-launch-1.0 v4l2src device=/dev/video3 ! fakesink
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.
Additional debug info:
/var/tmp/portage/media-libs/gstreamer-1.14.5/work/gstreamer-1.14.5/libs/gst/base/gstbasesrc.c(3055): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)
Execution ended after 0:00:00.000606666
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...
localhost ~ # dmesg -c

Another error, but some result. I think fakesink must got Bayer format (as specific /dev/null for v4l2).

Quoted 5 lines
Quoted 2 lines
Quoted 2 lines
# media-ctl -l "'ipu1_csi1':2 -> 'ipu1_csi1 capture':0[1]" # /dev/video3
(unused, unprocessed)
       ^

The IPUv3 IC hardware does not support processing Bayer formats. Those can only be written straight to RAM using this path.

#4 lore raw

Hi Alex,

Quoted 4 lines · Alex Mihaylov

Hi!

Ok. This time I try to check hardware and not need image processing. I uncomment this line in script:

Quoted 11 lines

[...]
- entity 47: ipu1_csi1 (3 pads, 4 links)
             type V4L2 subdev subtype Unknown flags 0
             device node name /dev/v4l-subdev5
        pad0: Sink
                [fmt:SRGGB10_1X10/1920x1080@1/30 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range crop.bounds:(0,0)/1920x1080
                 crop:(0,0)/640x480
                 compose.bounds:(0,0)/640x480
                 compose:(0,0)/640x480]
                <- "imx6-mipi-csi2":2 [ENABLED]

This is virtual channel 1 output from the imx6-mipi-csi2 receiver. Make sure the imx219 is indeed transmitting on virtual channel 1.

Quoted 8 lines

pad1: Source
                [fmt:SRGGB10_1X10/640x480@1/30 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range] -> "ipu1_ic_prp":0 []
                -> "ipu1_vdic":0 []
        pad2: Source
                [fmt:SRGGB10_1X10/640x480@1/30 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range] -> "ipu1_csi1 capture":0 [ENABLED]

Quoted 5 lines

- entity 51: ipu1_csi1 capture (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video3
        pad0: Sink
                <- "ipu1_csi1":2 [ENABLED]

What is the format on /dev/video3 ? Please provide output from 'v4l2-ctl -d3 -V'

Quoted 44 lines

[...]

- entity 121: imx6-mipi-csi2 (5 pads, 5 links)
              type V4L2 subdev subtype Unknown flags 0
              device node name /dev/v4l-subdev12
        pad0: Sink
                [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range] <- "imx219 9-0010":0 [ENABLED]
        pad1: Source
                [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range] -> "ipu1_csi0_mux":0 []
        pad2: Source
                [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range] -> "ipu1_csi1":0 [ENABLED]
        pad3: Source
                [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range] -> "ipu2_csi0":0 []
        pad4: Source
                [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range] -> "ipu2_csi1_mux":0 []

- entity 135: imx219 9-0010 (1 pad, 1 link)
              type V4L2 subdev subtype Sensor flags 0
              device node name /dev/v4l-subdev15
        pad0: Source
                [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range] -> "imx6-mipi-csi2":0 [ENABLED]

[...]
localhost ~ # gst-launch-1.0 v4l2src device=/dev/video3 ! fakesink
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.
Additional debug info:
/var/tmp/portage/media-libs/gstreamer-1.14.5/work/gstreamer-1.14.5/libs/gst/base/gstbasesrc.c(3055): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)
Execution ended after 0:00:00.000606666
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...
localhost ~ # dmesg -c

Quoted 1 line

Another error, but some result. I think fakesink must got Bayer format (as specific /dev/null for v4l2).

Can you try simply:

localhost ~ # v4l2-ctl -d3 --stream-mmap

and let me know if you receive frames.

Steve

Quoted 7 lines
Quoted 7 lines
Quoted 4 lines
Quoted 4 lines

# media-ctl -l "'ipu1_csi1':2 -> 'ipu1_csi1 capture':0[1]" # /dev/video3
(unused, unprocessed)

       ^
The IPUv3 IC hardware does not support processing Bayer formats. Those can only be written straight to RAM using this path.

#5 lore raw

Good day Steve,

I apologize. Due to the incestive reasons, we were quarantined. Alas, all work is postponed for at least a week.

Quoted 12 lines
Quoted 11 lines

[...]
- entity 47: ipu1_csi1 (3 pads, 4 links)
             type V4L2 subdev subtype Unknown flags 0
             device node name /dev/v4l-subdev5
        pad0: Sink
                [fmt:SRGGB10_1X10/1920x1080@1/30 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range crop.bounds:(0,0)/1920x1080
                 crop:(0,0)/640x480
                 compose.bounds:(0,0)/640x480
                 compose:(0,0)/640x480]
                <- "imx6-mipi-csi2":2 [ENABLED]

Quoted 1 line

This is virtual channel 1 output from the imx6-mipi-csi2 receiver. Make sure the imx219 is indeed transmitting on virtual channel 1.

Khm... I belive my schematechnics. Electrical connection present. Osciloscope show clock and data if I write stream enable command via I2C to IMX219. Does it make sense to do additional checks? What kind?

Quoted 8 lines
Quoted 5 lines

- entity 51: ipu1_csi1 capture (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video3
        pad0: Sink
                <- "ipu1_csi1":2 [ENABLED]

What is the format on /dev/video3 ? Please provide output from 'v4l2-ctl -d3 -V'

Alas, this requires physical access to the board.

Quoted 4 lines
Quoted 1 line

I think fakesink must got Bayer format (as specific /dev/null for v4l2).

Can you try simply:
localhost ~ # v4l2-ctl -d3 --stream-mmap
and let me know if you receive frames.

Thanks. So far, I have not had to deal with the Linux media subsystem. I only understand the design and hierarchy of drivers. Obviously, it is far from always successful.

#6 lore raw
In reply to Михайлов Алексей Анатольевич
Quoted 51 lines · Михайлов Алексей Анатольевич

Hi!

We build custom CPU Module with NXP/Freescale IMX6QuadPlus CPU.I use latest stable kernel from kernel.org. This time kernel version 5.5.11. Also I connect to I.MX MIPI cameras from RaspberryPI (Rev 2.1 with Sony IMX219). For IMX219 used actual driver from [1]. Usersapce based on Gentoo Linux, have media-utils version 1.2.1, v4l2-utils version 1.18.0, gstreamer version 1.14.5 with v4l2 plugins. Also Wayland version 1.17 based graphics with XWayland.

Camera write in DTB:
==== cut: DTB fragments ====
/ {
[skiped]

        imx219_clk: camera-clk {
                compatible = "fixed-clock";
                #clock-cells = <0>;
                clock-frequency = <24000000>;
        };

        imx219_1v2_reg: cam1v2_regulator {
                compatible = "regulator-fixed";
                regulator-name = "IMX219_1V2";
                regulator-min-microvolt = <1200000>;
                regulator-max-microvolt = <1200000>;
                vin-supply = <&p3v3_reg>;
                regulator-always-on;
        };

        imx219_1v8_reg: cam1v8_regulator {
                compatible = "regulator-fixed";
                regulator-name = "IMX219_1V8";
                regulator-min-microvolt = <1800000>;
                regulator-max-microvolt = <1800000>;
                vin-supply = <&p3v3_reg>;
                regulator-always-on;
        };

        imx219_2v8_reg: cam2v8_regulator {
                compatible = "regulator-fixed";
                regulator-name = "IMX219_2V8";
                regulator-min-microvolt = <2800000>;
                regulator-max-microvolt = <2800000>;
                vin-supply = <&p3v3_reg>;
                regulator-always-on;
        };
[skiped]
csi_i2c: i2c-mux@1 { /* CSI camera */
        #address-cells = <1>;
        #size-cells = <0>;
Quoted 8 lines
        reg = <1>;
        sensor@10 {     /* Raspberry Camera V2 */
                compatible = "sony,imx219";
                reg = <0x10>;
                #address-cells = <1>;
                #size-cells = <0>;
                clocks = <&imx219_clk>;
                clock-names = "xclk";

No clock-names is described in Documentation/devicetree/bindings/media/i2c/imx219.yaml

Quoted 3 lines
                DOVDD-supply = <&imx219_1v8_reg>; /* 1.8v */
                AVDD-supply = <&imx219_2v8_reg>;  /* 2.8v */
                DVDD-supply = <&imx219_1v2_reg>;  /* 1.2v */

These supplies names do not match the ones described in:
Documentation/devicetree/bindings/media/i2c/imx219.yaml

#7 lore raw

Hi!

Oooppps! My mistake. Before you answer in logs:

localhost ~ # dmesg | grep -i imx219
IMX219_1V2: supplied by P3V3
IMX219_1V8: supplied by P3V3
IMX219_2V8: supplied by P3V3
imx219 9-0010: 9-0010 supply VANA not found, using dummy regulator imx219 9-0010: 9-0010 supply VDIG not found, using dummy regulator imx219 9-0010: 9-0010 supply VDDL not found, using dummy regulator
imx-media: imx219 9-0010:0 -> imx6-mipi-csi2:0 localhost ~ #

P3V3 always on regulator, so all 3 regs always on. On Raspberry Camera V2.1 all of them enabled by single GPIO. This GPIO on my schematic pull-up to 3,3V. So for first step I fix DTB
- clock-names = "xclk";
- DOVDD-supply = <&imx219_1v8_reg>; /* 1.8v */
- AVDD-supply = <&imx219_2v8_reg>; /* 2.8v */
- DVDD-supply = <&imx219_1v2_reg>; /* 1.2v */
+ VDIG-supply = <&imx219_1v8_reg>; /* 1.8v */
+ VANA-supply = <&imx219_2v8_reg>; /* 2.8v */
+ VDDL-supply = <&imx219_1v2_reg>; /* 1.2v */

Result:
localhost ~ # dmesg | grep -i imx219

[    0.102656] IMX219_1V2: supplied by P3V3
[    0.102737] IMX219_1V8: supplied by P3V3
[    0.102798] IMX219_2V8: supplied by P3V3
[   13.851525] imx-media: imx219 9-0010:0 -> imx6-mipi-csi2:0

localhost ~ #

No visible changes from original message.

Next step. Our schematic have separate, controlled by GPIO, voltage switch for raspberry camera 3,3V supply. I try use them:

@@ -111,7 +108,6 @@ p3v3_cam_reg: p3v3_cam_regulator {
regulator-max-microvolt = <3300000>;
vin-supply = <&p3v3_reg>;
gpio = <&gpio2 25 GPIO_ACTIVE_LOW>;
- regulator-always-on; /* XP25: force on (1-2)/normal (2-3) */ };

also in all IMX219 regulators fix vin-supply and disable always-on mode
- vin-supply = <&p3v3_reg>;
- regulator-always-on;
+ vin-supply = <&p3v3_cam_reg>;

I see more noice about regulators in log, but capture still not start

localhost ~ # dmesg | grep -i imx219

[    0.103533] IMX219_1V2: supplied by P3V3_CAM
[    0.103612] IMX219_1V8: supplied by P3V3_CAM
[    0.103672] IMX219_2V8: supplied by P3V3_CAM
[   13.668314] imx-media: imx219 9-0010:0 -> imx6-mipi-csi2:0
[   35.671233] IMX219_1V2: disabling
[   35.671246] IMX219_1V8: disabling
[   35.671255] IMX219_2V8: disabling

localhost ~ # ./camera_init.sh
localhost ~ # gst-launch-1.0 v4l2src device=/dev/video2 ! fakesink
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Failed to allocate required memory.
Additional debug info:
/var/tmp/portage/media-plugins/gst-plugins-v4l2-1.14.5/work/gst-plugins-good-1.14.5/sys/v4l2/gstv4l2src.c(656): gst_v4l2src_decide_allocation (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
Buffer pool activation failed
Execution ended after 0:00:00.017843333
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...
#

So, I fix cosmetic with DTS. But, unfortunately, this not fix troubles.

Quoted 11 lines

No clock-names is described in Documentation/devicetree/bindings/media/i2c/imx219.yaml

Quoted 3 lines
                 DOVDD-supply = <&imx219_1v8_reg>; /* 1.8v */
                 AVDD-supply = <&imx219_2v8_reg>;  /* 2.8v */
                 DVDD-supply = <&imx219_1v2_reg>;  /* 1.2v */

These supplies names do not match the ones described in:
Documentation/devicetree/bindings/media/i2c/imx219.yaml