Skip to contentSkip to conversation

media: imx.rst: Fix the MIPI CSI-2 virtual channel

PATCH Superseded Fabio Estevam 2 participants 15 messages Apr 8, 2020

Subsystems (3)

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Fabio Estevam

author reviewer
#0 lore raw

PATCH 1/4 media: imx.rst: Fix the MIPI CSI-2 virtual channel

The current instructions for imx6q-sabresd do not lead to functional capture on OV5640 MIPI CSI-2.

The reason for this, as explained by Steve Longerbeam, is that OV5640 by default transmits on virtual channel 0, not channel 1 as is given in the instructions.

Adapt the instructions to use virtual channel 0 so that a working camera setup can be achieved on imx6q-sabresd.

---
Hi Steve,

I don't have access to a imx6q sabrelite with camera, but I think it needs fixing as well.

 Documentation/media/v4l-drivers/imx.rst | 23 ++++++++++++-----------
 1 file changed, 12 insertions(+), 11 deletions(-)
Documentation/media/v4l-drivers/imx.rst +12 -11
index 1246573c1019..7c3afd32e456 100644
@@ -645,25 +645,26 @@ The OV5640 module connects to MIPI connector J5 (sorry I don't have the
 compatible module part number or URL).
 
 The following example configures a direct conversion pipeline to capture
-from the OV5640, transmitting on MIPI CSI-2 virtual channel 1. $sensorfmt
-can be any format supported by the OV5640. $sensordim is the frame
-dimension part of $sensorfmt (minus the mbus pixel code). $outputfmt can
-be any format supported by the ipu1_ic_prpenc entity at its output pad:
+from the OV5640, transmitting on MIPI CSI-2 virtual channel 0:
 
 .. code-block:: none
 
    # Setup links
    media-ctl -l "'ov5640 1-003c':0 -> 'imx6-mipi-csi2':0[1]"
-   media-ctl -l "'imx6-mipi-csi2':2 -> 'ipu1_csi1':0[1]"
-   media-ctl -l "'ipu1_csi1':1 -> 'ipu1_ic_prp':0[1]"
+   media-ctl -l "'imx6-mipi-csi2':1 -> 'ipu1_csi0_mux':0[1]"
+   media-ctl -l "'ipu1_csi0_mux':2 -> 'ipu1_csi0':0[1]"
+   media-ctl -l "'ipu1_csi0':1 -> 'ipu1_ic_prp':0[1]"
    media-ctl -l "'ipu1_ic_prp':1 -> 'ipu1_ic_prpenc':0[1]"
    media-ctl -l "'ipu1_ic_prpenc':1 -> 'ipu1_ic_prpenc capture':0[1]"
    # Configure pads
-   media-ctl -V "'ov5640 1-003c':0 [fmt:$sensorfmt field:none]"
-   media-ctl -V "'imx6-mipi-csi2':2 [fmt:$sensorfmt field:none]"
-   media-ctl -V "'ipu1_csi1':1 [fmt:AYUV32/$sensordim field:none]"
-   media-ctl -V "'ipu1_ic_prp':1 [fmt:AYUV32/$sensordim field:none]"
-   media-ctl -V "'ipu1_ic_prpenc':1 [fmt:$outputfmt field:none]"
+   media-ctl -V "'ov5640 1-003c':0 [fmt:UYVY2X8/640x480]"
+   media-ctl -V "'imx6-mipi-csi2':1 [fmt:UYVY2X8/640x480]"
+   media-ctl -V "'ipu1_csi0_mux':2 [fmt:UYVY2X8/640x480]"
+   media-ctl -V "'ipu1_csi0':0 [fmt:UYVY2X8/640x480]"
+   media-ctl -V "'ipu1_csi0':1 [fmt:AYUV32/640x480]"
+   media-ctl -V "'ipu1_ic_prp':1 [fmt:AYUV32/640x480]"
+   media-ctl -V "'ipu1_ic_prpenc':0 [fmt:AYUV32/640x480]"
+   media-ctl -V "'ipu1_ic_prpenc':1 [fmt:AYUV32/640x480]"
 
 Streaming can then begin on "ipu1_ic_prpenc capture" node. The v4l2-ctl
 tool can be used to select any supported YUV or RGB pixelformat on the
Signature · 1 line
2.17.1
author reviewer
#1 lore raw

PATCH 2/4 media: imx.rst: Provide an example for unprocessed video capture

The current example for imx6q-sabresd is for a direct conversion pipeline.

Provide an extra example using unprocessed video capture for completeness.

---
 Documentation/media/v4l-drivers/imx.rst | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)
Documentation/media/v4l-drivers/imx.rst +20 -0
index 7c3afd32e456..3b4767f5f422 100644
@@ -670,6 +670,26 @@ Streaming can then begin on "ipu1_ic_prpenc capture" node. The v4l2-ctl
 tool can be used to select any supported YUV or RGB pixelformat on the
 capture device node.
 
+The following example configures unprocessed video capture pipeline to
+capture from the OV5640, transmitting on MIPI CSI-2 virtual channel 0:
+
+.. code-block:: none
+
+   # Setup links
+   media-ctl -l "'ov5640 1-003c':0 -> 'imx6-mipi-csi2':0[1]"
+   media-ctl -l "'imx6-mipi-csi2':1 -> 'ipu1_csi0_mux':0[1]"
+   media-ctl -l "'ipu1_csi0_mux':2 -> 'ipu1_csi0':0[1]"
+   media-ctl -l "'ipu1_csi0':2 -> 'ipu1_csi0 capture':0[1]"
+   # Configure pads
+   media-ctl -V "'ov5640 1-003c':0 [fmt:UYVY2X8/640x480]"
+   media-ctl -V "'imx6-mipi-csi2':1 [fmt:UYVY2X8/640x480]"
+   media-ctl -V "'ipu1_csi0_mux':0 [fmt:UYVY2X8/640x480]"
+   media-ctl -V "'ipu1_csi0':0 [fmt:AYUV32/640x480]"
+
+Streaming can then begin on "ipu1_csi0 capture" node. The v4l2-ctl
+tool can be used to select any supported YUV or RGB pixelformat on the
+capture device node.
+
 
 Known Issues
 ------------
Signature · 1 line
2.17.1
Reviewed-by on v1 2/4
#2 lore raw

Re: [PATCH 2/4] media: imx.rst: Provide an example for unprocessed video capture

Quoted 5 lines · Fabio Estevam

The current example for imx6q-sabresd is for a direct conversion pipeline.

Provide an extra example using unprocessed video capture for completeness.

Quoted 35 lines
---
  Documentation/media/v4l-drivers/imx.rst | 20 ++++++++++++++++++++
  1 file changed, 20 insertions(+)
Documentation/media/v4l-drivers/imx.rst +20 -0
index 7c3afd32e456..3b4767f5f422 100644
@@ -670,6 +670,26 @@ Streaming can then begin on "ipu1_ic_prpenc capture" node. The v4l2-ctl
  tool can be used to select any supported YUV or RGB pixelformat on the
  capture device node.
 
+The following example configures unprocessed video capture pipeline to
+capture from the OV5640, transmitting on MIPI CSI-2 virtual channel 0:
+
+.. code-block:: none
+
+   # Setup links
+   media-ctl -l "'ov5640 1-003c':0 -> 'imx6-mipi-csi2':0[1]"
+   media-ctl -l "'imx6-mipi-csi2':1 -> 'ipu1_csi0_mux':0[1]"
+   media-ctl -l "'ipu1_csi0_mux':2 -> 'ipu1_csi0':0[1]"
+   media-ctl -l "'ipu1_csi0':2 -> 'ipu1_csi0 capture':0[1]"
+   # Configure pads
+   media-ctl -V "'ov5640 1-003c':0 [fmt:UYVY2X8/640x480]"
+   media-ctl -V "'imx6-mipi-csi2':1 [fmt:UYVY2X8/640x480]"
+   media-ctl -V "'ipu1_csi0_mux':0 [fmt:UYVY2X8/640x480]"
+   media-ctl -V "'ipu1_csi0':0 [fmt:AYUV32/640x480]"
+
+Streaming can then begin on "ipu1_csi0 capture" node. The v4l2-ctl
+tool can be used to select any supported YUV or RGB pixelformat on the
+capture device node.
+
 
  Known Issues
  ------------
author reviewer
#3 lore raw

PATCH 3/4 media: imx.rst: Provide the OV560 module part number

In reply to Fabio Estevam

In order to improve the documentation, provide the OV5640 MIPI module part number that is used on the imx6q-sabresd board.

---
 Documentation/media/v4l-drivers/imx.rst | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
Documentation/media/v4l-drivers/imx.rst +2 -2
index 3b4767f5f422..37b944f12d93 100644
@@ -641,8 +641,8 @@ connects to i2c bus 1 and the OV5640 to i2c bus 2.
 The device tree for SabreSD includes OF graphs for both the parallel
 OV5642 and the MIPI CSI-2 OV5640, but as of this writing only the MIPI
 CSI-2 OV5640 has been tested, so the OV5642 node is currently disabled.
-The OV5640 module connects to MIPI connector J5 (sorry I don't have the
-compatible module part number or URL).
+The OV5640 module connects to MIPI connector J5. The NXP part number
+for the OV5640 module that connects to the SabreSD board is H120729.
 
 The following example configures a direct conversion pipeline to capture
 from the OV5640, transmitting on MIPI CSI-2 virtual channel 0:
Signature · 1 line
2.17.1
Reviewed-by on v1 3/4
#4 lore raw

Re: [PATCH 3/4] media: imx.rst: Provide the OV560 module part number

Quoted 4 lines · Fabio Estevam

In order to improve the documentation, provide the OV5640 MIPI module part number that is used on the imx6q-sabresd board.

Quoted 19 lines
---
  Documentation/media/v4l-drivers/imx.rst | 4 ++--
  1 file changed, 2 insertions(+), 2 deletions(-)
Documentation/media/v4l-drivers/imx.rst +2 -2
index 3b4767f5f422..37b944f12d93 100644
@@ -641,8 +641,8 @@ connects to i2c bus 1 and the OV5640 to i2c bus 2.
  The device tree for SabreSD includes OF graphs for both the parallel
  OV5642 and the MIPI CSI-2 OV5640, but as of this writing only the MIPI
  CSI-2 OV5640 has been tested, so the OV5642 node is currently disabled.
-The OV5640 module connects to MIPI connector J5 (sorry I don't have the
-compatible module part number or URL).
+The OV5640 module connects to MIPI connector J5. The NXP part number
+for the OV5640 module that connects to the SabreSD board is H120729.
 
  The following example configures a direct conversion pipeline to capture
  from the OV5640, transmitting on MIPI CSI-2 virtual channel 0:
author reviewer
#5 lore raw

PATCH 4/4 media: imx.rst: Provide streaming examples for imx6q-sabresd

In reply to Fabio Estevam

Improve the documentation by providing examples on how to test camera capture on imx6q-sabresd via v4l2-ctl and Gstreamer.

---
 Documentation/media/v4l-drivers/imx.rst | 42 +++++++++++++++++++++++++
 1 file changed, 42 insertions(+)
Documentation/media/v4l-drivers/imx.rst +42 -0
index 37b944f12d93..c872cc84d9fc 100644
@@ -670,6 +670,28 @@ Streaming can then begin on "ipu1_ic_prpenc capture" node. The v4l2-ctl
 tool can be used to select any supported YUV or RGB pixelformat on the
 capture device node.
 
+In order to determine what is the /dev/video node correspondent to
+"ipu1_ic_prpenc":
+
+.. code-block:: none
+
+   # dmesg | grep ipu1_ic_prpenc | grep Registered
+   ipu1_ic_prpenc: Registered ipu1_ic_prpenc capture as /dev/video1
+
+/dev/video1 is the streaming element in this case.
+
+Starting the streaming via v4l2-ctl:
+
+.. code-block:: none
+
+   v4l2-ctl --stream-mmap -d /dev/video1
+
+Starting the streaming via Gstreamer and sending the content to the display:
+
+.. code-block:: none
+
+   gst-launch-1.0 v4l2src device=/dev/video1 ! kmssink
+
 The following example configures unprocessed video capture pipeline to
 capture from the OV5640, transmitting on MIPI CSI-2 virtual channel 0:
 
@@ -690,6 +712,26 @@ Streaming can then begin on "ipu1_csi0 capture" node. The v4l2-ctl
 tool can be used to select any supported YUV or RGB pixelformat on the
 capture device node.
 
+In order to determine what is the /dev/video node correspondent to "ipu1_csi0":
+
+.. code-block:: none
+
+   # dmesg | grep ipu1_csi0 | grep Registered
+   ipu1_csi0: Registered ipu1_csi0 capture as /dev/video0
+
+/dev/video0 is the streaming element in this case.
+
+Starting the streaming via v4l2-ctl:
+
+.. code-block:: none
+
+   v4l2-ctl --stream-mmap -d /dev/video0
+
+Starting the streaming via Gstreamer and sending the content to the display:
+
+.. code-block:: none
+
+   gst-launch-1.0 v4l2src device=/dev/video0 ! kmssink
 
 Known Issues
 ------------
Signature · 1 line
2.17.1
#6 lore raw

Re: [PATCH 4/4] media: imx.rst: Provide streaming examples for imx6q-sabresd

Quoted 13 lines · Fabio Estevam

Improve the documentation by providing examples on how to test camera capture on imx6q-sabresd via v4l2-ctl and Gstreamer.

---
  Documentation/media/v4l-drivers/imx.rst | 42 +++++++++++++++++++++++++
  1 file changed, 42 insertions(+)
Documentation/media/v4l-drivers/imx.rst +0 -0
index 37b944f12d93..c872cc84d9fc 100644

Documentation/media/v4l-drivers/imx.rst Streaming can then begin on "ipu1_ic_prpenc capture" node. The v4l2-ctl · line 678

Quoted 9 lines
Documentation/media/v4l-drivers/imx.rst +6 -0
@@ -670,6 +670,28 @@ Streaming can then begin on "ipu1_ic_prpenc capture" node. The v4l2-ctl
  tool can be used to select any supported YUV or RGB pixelformat on the
  capture device node.
 
+In order to determine what is the /dev/video node correspondent to
+"ipu1_ic_prpenc":
+
+.. code-block:: none
+
+   # dmesg | grep ipu1_ic_prpenc | grep Registered

A better way is to use "media-ctl -e":

# media-ctl -e "ipu1_ic_prpenc capture"
/dev/video1

Grepping from dmesg can fail when the log fills up and pushes out the boot "Registered ..." messages.

Steve

Quoted 46 lines
Documentation/media/v4l-drivers/imx.rst +36 -0
+   ipu1_ic_prpenc: Registered ipu1_ic_prpenc capture as /dev/video1
+
+/dev/video1 is the streaming element in this case.
+
+Starting the streaming via v4l2-ctl:
+
+.. code-block:: none
+
+   v4l2-ctl --stream-mmap -d /dev/video1
+
+Starting the streaming via Gstreamer and sending the content to the display:
+
+.. code-block:: none
+
+   gst-launch-1.0 v4l2src device=/dev/video1 ! kmssink
+
  The following example configures unprocessed video capture pipeline to
  capture from the OV5640, transmitting on MIPI CSI-2 virtual channel 0:
 
@@ -690,6 +712,26 @@ Streaming can then begin on "ipu1_csi0 capture" node. The v4l2-ctl
  tool can be used to select any supported YUV or RGB pixelformat on the
  capture device node.
 
+In order to determine what is the /dev/video node correspondent to "ipu1_csi0":
+
+.. code-block:: none
+
+   # dmesg | grep ipu1_csi0 | grep Registered
+   ipu1_csi0: Registered ipu1_csi0 capture as /dev/video0
+
+/dev/video0 is the streaming element in this case.
+
+Starting the streaming via v4l2-ctl:
+
+.. code-block:: none
+
+   v4l2-ctl --stream-mmap -d /dev/video0
+
+Starting the streaming via Gstreamer and sending the content to the display:
+
+.. code-block:: none
+
+   gst-launch-1.0 v4l2src device=/dev/video0 ! kmssink
 
  Known Issues
  ------------
#7 lore raw
In reply to Fabio Estevam

Hi Fabio, thanks for the patch.

Quoted 26 lines · Fabio Estevam

The current instructions for imx6q-sabresd do not lead to functional capture on OV5640 MIPI CSI-2.

The reason for this, as explained by Steve Longerbeam, is that OV5640 by default transmits on virtual channel 0, not channel 1 as is given in the instructions.

Adapt the instructions to use virtual channel 0 so that a working camera setup can be achieved on imx6q-sabresd.

---
Hi Steve,

I don't have access to a imx6q sabrelite with camera, but I think it needs fixing as well.

  Documentation/media/v4l-drivers/imx.rst | 23 ++++++++++++-----------
  1 file changed, 12 insertions(+), 11 deletions(-)
Documentation/media/v4l-drivers/imx.rst +0 -0
index 1246573c1019..7c3afd32e456 100644

Documentation/media/v4l-drivers/imx.rst The OV5640 module connects to MIPI connector J5 (sorry I don't have the · line 648

Quoted 8 lines
Documentation/media/v4l-drivers/imx.rst +1 -4
@@ -645,25 +645,26 @@ The OV5640 module connects to MIPI connector J5 (sorry I don't have the
  compatible module part number or URL).
 
  The following example configures a direct conversion pipeline to capture
-from the OV5640, transmitting on MIPI CSI-2 virtual channel 1. $sensorfmt
-can be any format supported by the OV5640. $sensordim is the frame
-dimension part of $sensorfmt (minus the mbus pixel code). $outputfmt can
-be any format supported by the ipu1_ic_prpenc entity at its output pad:
+from the OV5640, transmitting on MIPI CSI-2 virtual channel 0:

Since this is an Image Converter pipeline, it would be nice to demonstrate scaling and color conversion in this example. How about changing the format at IC output to be ARGB8888_1X32/800x600. So change the explanation of the pipeline above, and:

Quoted 12 lines
Documentation/media/v4l-drivers/imx.rst +3 -2
  .. code-block:: none
 
     # Setup links
     media-ctl -l "'ov5640 1-003c':0 -> 'imx6-mipi-csi2':0[1]"
-   media-ctl -l "'imx6-mipi-csi2':2 -> 'ipu1_csi1':0[1]"
-   media-ctl -l "'ipu1_csi1':1 -> 'ipu1_ic_prp':0[1]"
+   media-ctl -l "'imx6-mipi-csi2':1 -> 'ipu1_csi0_mux':0[1]"
+   media-ctl -l "'ipu1_csi0_mux':2 -> 'ipu1_csi0':0[1]"
+   media-ctl -l "'ipu1_csi0':1 -> 'ipu1_ic_prp':0[1]"
     media-ctl -l "'ipu1_ic_prp':1 -> 'ipu1_ic_prpenc':0[1]"
     media-ctl -l "'ipu1_ic_prpenc':1 -> 'ipu1_ic_prpenc capture':0[1]"

Documentation/media/v4l-drivers/imx.rst The OV5640 module connects to MIPI connector J5 (sorry I don't have the · line 663

Quoted 10 lines
Documentation/media/v4l-drivers/imx.rst +4 -5
     # Configure pads
-   media-ctl -V "'ov5640 1-003c':0 [fmt:$sensorfmt field:none]"
-   media-ctl -V "'imx6-mipi-csi2':2 [fmt:$sensorfmt field:none]"
-   media-ctl -V "'ipu1_csi1':1 [fmt:AYUV32/$sensordim field:none]"
-   media-ctl -V "'ipu1_ic_prp':1 [fmt:AYUV32/$sensordim field:none]"
-   media-ctl -V "'ipu1_ic_prpenc':1 [fmt:$outputfmt field:none]"
+   media-ctl -V "'ov5640 1-003c':0 [fmt:UYVY2X8/640x480]"
+   media-ctl -V "'imx6-mipi-csi2':1 [fmt:UYVY2X8/640x480]"
+   media-ctl -V "'ipu1_csi0_mux':2 [fmt:UYVY2X8/640x480]"
+   media-ctl -V "'ipu1_csi0':0 [fmt:UYVY2X8/640x480]"

don't need this line (I pulled this from a script that was testing crop/compose on sink pad ipu1_csi0:0, but since we're not doing that here, you can remove the format setup line for ipu1_csi0:0).

Documentation/media/v4l-drivers/imx.rst The OV5640 module connects to MIPI connector J5 (sorry I don't have the · line 666

Quoted 3 lines
Documentation/media/v4l-drivers/imx.rst +3 -0
+   media-ctl -V "'ipu1_csi0':1 [fmt:AYUV32/640x480]"
+   media-ctl -V "'ipu1_ic_prp':1 [fmt:AYUV32/640x480]"
+   media-ctl -V "'ipu1_ic_prpenc':0 [fmt:AYUV32/640x480]"

don't need this line.

Documentation/media/v4l-drivers/imx.rst The OV5640 module connects to MIPI connector J5 (sorry I don't have the · line 667

Quoted 1 line
Documentation/media/v4l-drivers/imx.rst +1 -0
+   media-ctl -V "'ipu1_ic_prpenc':1 [fmt:AYUV32/640x480]"

change to:

media-ctl -V "'ipu1_ic_prpenc':1 [fmt:ARGB8888_1X32/800x600]"

Documentation/media/v4l-drivers/imx.rst The OV5640 module connects to MIPI connector J5 (sorry I don't have the · line 670

Quoted 3 lines
Documentation/media/v4l-drivers/imx.rst +0 -0
  Streaming can then begin on "ipu1_ic_prpenc capture" node. The v4l2-ctl
  tool can be used to select any supported YUV or RGB pixelformat on the

"...The v4l2-ctl tool can be used to select any supported RGB pixelformat..."

Steve

author reviewer
#8 lore raw

Hi Steve,

Quoted 3 lines · Steve Longerbeam

change to:

media-ctl -V "'ipu1_ic_prpenc':1 [fmt:ARGB8888_1X32/800x600]"

If I make this change I get:

# v4l2-ctl --stream-mmap -d /dev/video1
[  501.143012] ipu1_ic_prpenc: capture format not valid
                VIDIOC_STREAMON returned -1 (Invalid argument)

Going back to the original example:

# media-ctl -V "'ipu1_ic_prpenc':1 [fmt:AYUV32/640x480]"

# v4l2-ctl --stream-mmap -d /dev/video1
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.03 fps

Changing only the resolution:

# media-ctl -V "'ipu1_ic_prpenc':1 [fmt:AYUV32/800x600]"

# v4l2-ctl --stream-mmap -d /dev/video1
[  658.358246] ipu1_ic_prpenc: capture format not valid
                VIDIOC_STREAMON returned -1 (Invalid argument)

Any ideas?

These are the lines I am currently using as per your feedback:

# Setup links
media-ctl -l "'ov5640 1-003c':0 -> 'imx6-mipi-csi2':0[1]" media-ctl -l "'imx6-mipi-csi2':1 -> 'ipu1_csi0_mux':0[1]" media-ctl -l "'ipu1_csi0_mux':2 -> 'ipu1_csi0':0[1]" media-ctl -l "'ipu1_csi0':1 -> 'ipu1_ic_prp':0[1]" media-ctl -l "'ipu1_ic_prp':1 -> 'ipu1_ic_prpenc':0[1]" media-ctl -l "'ipu1_ic_prpenc':1 -> 'ipu1_ic_prpenc capture':0[1]" # Configure pads
media-ctl -V "'ov5640 1-003c':0 [fmt:UYVY2X8/640x480]" media-ctl -V "'imx6-mipi-csi2':1 [fmt:UYVY2X8/640x480]" media-ctl -V "'ipu1_csi0_mux':2 [fmt:UYVY2X8/640x480]" media-ctl -V "'ipu1_csi0':1 [fmt:AYUV32/640x480]" media-ctl -V "'ipu1_ic_prp':1 [fmt:AYUV32/640x480]" media-ctl -V "'ipu1_ic_prpenc':1 [fmt:ARGB8888_1X32/800x600]"

Thanks,

Fabio Estevam

# Setup links
media-ctl -l "'ov5640 1-003c':0 -> 'imx6-mipi-csi2':0[1]" media-ctl -l "'imx6-mipi-csi2':1 -> 'ipu1_csi0_mux':0[1]" media-ctl -l "'ipu1_csi0_mux':2 -> 'ipu1_csi0':0[1]" media-ctl -l "'ipu1_csi0':1 -> 'ipu1_ic_prp':0[1]" media-ctl -l "'ipu1_ic_prp':1 -> 'ipu1_ic_prpenc':0[1]" media-ctl -l "'ipu1_ic_prpenc':1 -> 'ipu1_ic_prpenc capture':0[1]" # Configure pads
media-ctl -V "'ov5640 1-003c':0 [fmt:UYVY2X8/640x480]" media-ctl -V "'imx6-mipi-csi2':1 [fmt:UYVY2X8/640x480]" media-ctl -V "'ipu1_csi0_mux':2 [fmt:UYVY2X8/640x480]" media-ctl -V "'ipu1_csi0':1 [fmt:AYUV32/640x480]" media-ctl -V "'ipu1_ic_prp':1 [fmt:AYUV32/640x480]" media-ctl -V "'ipu1_ic_prpenc':1 [fmt:ARGB8888_1X32/800x600]"

#9 lore raw
Quoted 19 lines · Fabio Estevam

Hi Steve,

Quoted 3 lines · Steve Longerbeam

change to:

media-ctl -V "'ipu1_ic_prpenc':1 [fmt:ARGB8888_1X32/800x600]"

If I make this change I get:

# v4l2-ctl --stream-mmap -d /dev/video1
[  501.143012] ipu1_ic_prpenc: capture format not valid
                 VIDIOC_STREAMON returned -1 (Invalid argument)

Going back to the original example:

# media-ctl -V "'ipu1_ic_prpenc':1 [fmt:AYUV32/640x480]"

# v4l2-ctl --stream-mmap -d /dev/video1
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.03 fps
Quoted 8 lines

Changing only the resolution:

# media-ctl -V "'ipu1_ic_prpenc':1 [fmt:AYUV32/800x600]"

# v4l2-ctl --stream-mmap -d /dev/video1
[  658.358246] ipu1_ic_prpenc: capture format not valid
                 VIDIOC_STREAMON returned -1 (Invalid argument)

Any ideas?

You have to set a format at the capture interface as the last step before streaming can start:

# v4l2-ctl -d /dev/video1 --set-fmt-video=pixelformat=RGB3

or whatever RGB format is supported. It's enough to sinmply set one parameter, such as width or height or pixelformat. The driver will propagate the reset from the source subdev ipu1_ic_prpenc.

The capture interface used to be automatically propagated from the source subdev, but now it must be done by userland.

Steve

Quoted 36 lines

These are the lines I am currently using as per your feedback:

    # Setup links
    media-ctl -l "'ov5640 1-003c':0 -> 'imx6-mipi-csi2':0[1]"
    media-ctl -l "'imx6-mipi-csi2':1 -> 'ipu1_csi0_mux':0[1]"
    media-ctl -l "'ipu1_csi0_mux':2 -> 'ipu1_csi0':0[1]"
    media-ctl -l "'ipu1_csi0':1 -> 'ipu1_ic_prp':0[1]"
    media-ctl -l "'ipu1_ic_prp':1 -> 'ipu1_ic_prpenc':0[1]"
    media-ctl -l "'ipu1_ic_prpenc':1 -> 'ipu1_ic_prpenc capture':0[1]"
    # Configure pads
    media-ctl -V "'ov5640 1-003c':0 [fmt:UYVY2X8/640x480]"
    media-ctl -V "'imx6-mipi-csi2':1 [fmt:UYVY2X8/640x480]"
    media-ctl -V "'ipu1_csi0_mux':2 [fmt:UYVY2X8/640x480]"
    media-ctl -V "'ipu1_csi0':1 [fmt:AYUV32/640x480]"
    media-ctl -V "'ipu1_ic_prp':1 [fmt:AYUV32/640x480]"
    media-ctl -V "'ipu1_ic_prpenc':1 [fmt:ARGB8888_1X32/800x600]"

Thanks,

Fabio Estevam

    # Setup links
    media-ctl -l "'ov5640 1-003c':0 -> 'imx6-mipi-csi2':0[1]"
    media-ctl -l "'imx6-mipi-csi2':1 -> 'ipu1_csi0_mux':0[1]"
    media-ctl -l "'ipu1_csi0_mux':2 -> 'ipu1_csi0':0[1]"
    media-ctl -l "'ipu1_csi0':1 -> 'ipu1_ic_prp':0[1]"
    media-ctl -l "'ipu1_ic_prp':1 -> 'ipu1_ic_prpenc':0[1]"
    media-ctl -l "'ipu1_ic_prpenc':1 -> 'ipu1_ic_prpenc capture':0[1]"
    # Configure pads
    media-ctl -V "'ov5640 1-003c':0 [fmt:UYVY2X8/640x480]"
    media-ctl -V "'imx6-mipi-csi2':1 [fmt:UYVY2X8/640x480]"
    media-ctl -V "'ipu1_csi0_mux':2 [fmt:UYVY2X8/640x480]"
    media-ctl -V "'ipu1_csi0':1 [fmt:AYUV32/640x480]"
    media-ctl -V "'ipu1_ic_prp':1 [fmt:AYUV32/640x480]"
    media-ctl -V "'ipu1_ic_prpenc':1 [fmt:ARGB8888_1X32/800x600]"
author reviewer
#10 lore raw

Hi Steve,

Quoted 4 lines · Steve Longerbeam

You have to set a format at the capture interface as the last step before streaming can start:

# v4l2-ctl -d /dev/video1 --set-fmt-video=pixelformat=RGB3

Thanks. After running this line the stream can start via --stream-mmap:

# v4l2-ctl --stream-mmap -d /dev/video1
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.03 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.03 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.03 fps

but it fails with Gstreamer:

# gst-launch-1.0 -v v4l2src device=/dev/video1 ! kmssink
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
/GstPipeline:pipeline0/GstKMSSink:kmssink0: display-width = 1024
/GstPipeline:pipeline0/GstKMSSink:kmssink0: display-height = 768
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
Internal data stream error.
Additional debug info:
../libs/gst/base/gstbasesrc.c(3072): gst_base_src_loop ():
/GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)
ERROR: pipeline doesn't want to preroll.
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...

Should I change the Gstreamer pipeline?

Thanks

author reviewer
#11 lore raw
Quoted 1 line · Fabio Estevam

Should I change the Gstreamer pipeline?

It works if I use the v4l2convert element:

gst-launch-1.0 -v v4l2src device=/dev/video1 ! v4l2convert ! kmssink

I will send it in the next patch series.

Thanks

Quoted 2 lines

Thanks

#12 lore raw
Quoted 6 lines · Fabio Estevam
Quoted 1 line · Fabio Estevam

Should I change the Gstreamer pipeline?

It works if I use the v4l2convert element:

gst-launch-1.0 -v v4l2src device=/dev/video1 ! v4l2convert ! kmssink

I think that works because the csc-scaler mem2mem driver is able to provide the frame dimensions that kmssink requires, but I don't think you need v4l2convert, I think this should work (use can-scale=false):

gst-launch-1.0 v4l2src device=/dev/video1 ! kmssink connector-id=54 name=imx-drm sync=0 can-scale=false

but use the connector-id for your display.

Steve

#13 lore raw

Hi Fabio,

So sorry!

I was testing with the patchset:

[PATCH v7 00/11] media: imx: Miscellaneous format-related cleanups

which specifically fixes a bug with enumerating non-mbus RGB formats:

[PATCH v7 01/11] media: imx: utils: fix and simplify pixel format enumeration

Without that patchset, /dev/video1 only lists:

arm:~/scripts % v4l2-ctl -d1 --list-formats
ioctl: VIDIOC_ENUM_FMT
    Index       : 0
    Type        : Video Capture
    Pixel Format: 'RGBP'
    Name        : 16-bit RGB 5-6-5

    Index       : 1
    Type        : Video Capture
    Pixel Format: 'RGB3'
    Name        : 24-bit RGB 8-8-8

    Index       : 2
    Type        : Video Capture
    Pixel Format: 'BX24'
    Name        : 32-bit XRGB 8-8-8-8

But with the bug fix:

arm:~ % v4l2-ctl -d1 --list-formats
ioctl: VIDIOC_ENUM_FMT
        Index       : 0
        Type        : Video Capture
        Pixel Format: 'RGBP'
        Name        : 16-bit RGB 5-6-5

        Index       : 1
        Type        : Video Capture
        Pixel Format: 'RGB3'
        Name        : 24-bit RGB 8-8-8

        Index       : 2
        Type        : Video Capture
        Pixel Format: 'BGR3'
        Name        : 24-bit BGR 8-8-8

        Index       : 3
        Type        : Video Capture
        Pixel Format: 'BX24'
        Name        : 32-bit XRGB 8-8-8-8

        Index       : 4
        Type        : Video Capture
        Pixel Format: 'XR24'
        Name        : 32-bit BGRX 8-8-8-8

        Index       : 5
        Type        : Video Capture
        Pixel Format: 'RX24'
        Name        : 32-bit XBGR 8-8-8-8

        Index       : 6
        Type        : Video Capture
        Pixel Format: 'XB24'
        Name        : 32-bit RGBX 8-8-8-8

And with that, kmssink is able to negotiate XR24.

With the anticipation that the patchset mentioned will be merged, can you try again with that patchset applied, and update the imx.rst doc accordingly?

Thanks,
Steve

Quoted 18 lines · Steve Longerbeam
Quoted 6 lines · Fabio Estevam
Quoted 1 line · Fabio Estevam

Should I change the Gstreamer pipeline?

It works if I use the v4l2convert element:

gst-launch-1.0 -v v4l2src device=/dev/video1 ! v4l2convert ! kmssink

I think that works because the csc-scaler mem2mem driver is able to provide the frame dimensions that kmssink requires, but I don't think you need v4l2convert, I think this should work (use can-scale=false):

gst-launch-1.0 v4l2src device=/dev/video1 ! kmssink connector-id=54 name=imx-drm sync=0 can-scale=false

but use the connector-id for your display.

Steve

#14 lore raw
In reply to Fabio Estevam
Quoted 34 lines · Fabio Estevam

Hi Steve,

Quoted 4 lines · Steve Longerbeam

You have to set a format at the capture interface as the last step before streaming can start:

# v4l2-ctl -d /dev/video1 --set-fmt-video=pixelformat=RGB3

Thanks. After running this line the stream can start via --stream-mmap:

# v4l2-ctl --stream-mmap -d /dev/video1
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.03 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.03 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.03 fps

but it fails with Gstreamer:

# gst-launch-1.0 -v v4l2src device=/dev/video1 ! kmssink
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
/GstPipeline:pipeline0/GstKMSSink:kmssink0: display-width = 1024
/GstPipeline:pipeline0/GstKMSSink:kmssink0: display-height = 768
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
Internal data stream error.
Additional debug info:
../libs/gst/base/gstbasesrc.c(3072): gst_base_src_loop ():
/GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)
ERROR: pipeline doesn't want to preroll.
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...
Quoted 1 line

Should I change the Gstreamer pipeline?

It works for me, I'm running gstreamer version:

arm:~/scripts % gst-launch-1.0 --version
gst-launch-1.0 version 1.14.5
GStreamer 1.14.5
https://launchpad.net/distros/ubuntu/+source/gstreamer1.0

Also, I'm streaming to an HDMI monitor at 1920x1080 (passing connector-id=54 to kmssink).

I can't tell from your gst output whether you are having an issue with pixelformats or frame dimensions, but anyway here is my gst pipeline:

gst-launch-1.0 v4l2src device=/dev/video1 ! kmssink connector-id=54 name=imx-drm sync=0 can-scale=false

It might also be the can-scale property, you might need to provide can-scale=false to tell kmssink that imx-drm cannot scale 800x600 to something else.

While streaming, I checked what pixelformat the pipeline finally settled on:

arm:~/scripts % v4l2-ctl -d1 -V
Format Video Capture:
    Width/Height      : 800/600
    Pixel Format      : 'XR24'
    Field             : None
    Bytes per Line    : 3200
    Size Image        : 1920000
    Colorspace        : sRGB
    Transfer Function : sRGB
    YCbCr/HSV Encoding: ITU-R 601
    Quantization      : Full Range
    Flags             :

Steve