Skip to contentSkip to conversation

Issue with ov5640 camera sensor on apalis imx6

DISCUSSION Aishwarya Kothari 6 participants 12 messages Dec 7, 2022 – Jan 12, 2023 Media (V4L/DVB)

lore
Share & tools
lore
#0 lore raw

Hi,

I was trying ov5640 camera sensor with the mainline BSP 6.0.0-rc3 kernel and it fails to works, while it works quiet well on v5.18. When I bisect, it points out to this commit 1f391df4 (media: v4l2-async: Use endpoints in __v4l2_async_nf_add_fwnode_remote()) by Laurent Pinchart and this is what it says in the logs :

[    4.291355] imx-ipuv3-csi imx-ipuv3-csi.0: Registered ipu1_csi0 capture as /dev/video0 [    4.305228] imx-ipuv3 2400000.ipu: Registered ipu1_ic_prpenc capture as /dev/video1
[    4.317974] imx-ipuv3 2400000.ipu: Registered ipu1_ic_prpvf capture as /dev/video2
[    4.331039] imx-ipuv3-csi imx-ipuv3-csi.1: Registered ipu1_csi1 capture as /dev/video3 [    4.344255] imx-ipuv3-csi imx-ipuv3-csi.4: Registered ipu2_csi0 capture as /dev/video4 [    4.356940] imx-ipuv3 2800000.ipu: Registered ipu2_ic_prpenc capture as /dev/video5
[    4.369322] imx-ipuv3 2800000.ipu: Registered ipu2_ic_prpvf capture as /dev/video6
[    4.382119] imx-ipuv3-csi imx-ipuv3-csi.5: Registered ipu2_csi1 capture as /dev/video7 [    4.395455] imx6-mipi-csi2 21dc000.mipi: Consider updating driver imx6-mipi-csi2 to match on endpoints [    4.414971] stmpe-adc stmpe-adc: DMA mask not set

So, now when I try to revert the change done by this commit, it still fails. I assume there are a lot more changes following the commit that could lead to this failure. Below are the logs. Is there a way to get it to work without reverting any changes? Or Is there something I'm missing or have to do any changes in the corresponding files to get it to work?

TDX Wayland with XWayland Upstream 6.1.0-devel-20221109+build.104 (kirkstone) apalis-imx6-10774951 ttymxc0 Apalis-iMX6_Reference-Multimedia-Image-upstream

apalis-imx6-10774951 login: root

root@apalis-imx6-10774951:~# cat ov5640.sh
media-ctl -l "'ov5640 1-003c':0 -> 'imx6-mipi-csi2':0[1]"
media-ctl -l "'imx6-mipi-csi2':2 -> 'ipu1_csi1':0[1]"
media-ctl -l "'ipu1_csi1':2 -> 'ipu1_csi1 capture':0[1]"
# Configure pads
media-ctl -V "'ov5640 1-003c':0 [fmt:UYVY2X8/1920x1080 field:none]"
media-ctl -V "'imx6-mipi-csi2':2 [fmt:UYVY2X8/1920x1080 field:none]"
media-ctl -V "'ipu1_csi1':2 [fmt:AYUV32/1920x1080 field:none]"
root@apalis-imx6-10774951:~# sh ov5640.sh
root@apalis-imx6-10774951:~# gst-launch-1.0 v4l2src device='/dev/video3' ! videoconvert ! waylandsink
[   56.031510] ov5640 1-003c: ov5640_write_reg: error: reg=3008, val=2
[   56.649032] ov5640 1-003c: ov5640_write_reg: error: reg=3008, val=2
Setting pipeline to PAUSED ...
[   57.082630] ov5640 1-003c: ov5640_write_reg: error: reg=3008, val=2
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Failed to allocate required memory.
Additional debug info:
../gst-plugins-good-1.20.3/sys/v4l2/gstv4l2src.c(759): gst_v4l2src_decide_allocation (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
Buffer pool activation failed
Execution ended after 0:00:00.119459642
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.
Setting pipeline to NULL ...
Additional debug info:
../gstreamer-1.20.3/libs/gst/base/gstbasesrc.c(3127): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)
Freeing pipeline ...
root@apalis-imx6-10774951:~# v4l2-ctl --device /dev/video3 --stream-mmap --stream-to=x.raw --stream-count=1
[ 1124.339793] ov5640 1-003c: ov5640_write_reg: error: reg=3008, val=2
            VIDIOC_STREAMON returned -1 (Broken pipe)

Kind regards,
Aishwarya

#1 lore raw

Hi Aishwarya,

(CC'ing Jacopo and Sakari)

Quoted 19 lines · Aishwarya Kothari

Hi,

I was trying ov5640 camera sensor with the mainline BSP 6.0.0-rc3 kernel and it fails to works, while it works quiet well on v5.18. When I bisect, it points out to this commit 1f391df4 (media: v4l2-async: Use endpoints in __v4l2_async_nf_add_fwnode_remote()) by Laurent Pinchart and this is what it says in the logs :

[    4.291355] imx-ipuv3-csi imx-ipuv3-csi.0: Registered ipu1_csi0 capture as /dev/video0 [    4.305228] imx-ipuv3 2400000.ipu: Registered ipu1_ic_prpenc capture as /dev/video1
[    4.317974] imx-ipuv3 2400000.ipu: Registered ipu1_ic_prpvf capture as /dev/video2
[    4.331039] imx-ipuv3-csi imx-ipuv3-csi.1: Registered ipu1_csi1 capture as /dev/video3 [    4.344255] imx-ipuv3-csi imx-ipuv3-csi.4: Registered ipu2_csi0 capture as /dev/video4 [    4.356940] imx-ipuv3 2800000.ipu: Registered ipu2_ic_prpenc capture as /dev/video5
[    4.369322] imx-ipuv3 2800000.ipu: Registered ipu2_ic_prpvf capture as /dev/video6
[    4.382119] imx-ipuv3-csi imx-ipuv3-csi.5: Registered ipu2_csi1 capture as /dev/video7 [    4.395455] imx6-mipi-csi2 21dc000.mipi: Consider updating driver imx6-mipi-csi2 to match on endpoints [    4.414971] stmpe-adc stmpe-adc: DMA mask not set

Quoted 6 lines

So, now when I try to revert the change done by this commit, it still fails. I assume there are a lot more changes following the commit that could lead to this failure. Below are the logs.
Is there a way to get it to work without reverting any changes? Or Is there something I'm missing or have to do any changes in the corresponding files to get it to work?

The commit you reference wasn't meant to break anything, so there's clearly something I've overlooked. Jacopo, have you noticed anything similar when working on the ov5640 driver ?

Quoted 5 lines

TDX Wayland with XWayland Upstream 6.1.0-devel-20221109+build.104 (kirkstone) apalis-imx6-10774951 ttymxc0 Apalis-iMX6_Reference-Multimedia-Image-upstream

apalis-imx6-10774951 login: root

Quoted 13 lines
root@apalis-imx6-10774951:~# cat ov5640.sh
media-ctl -l "'ov5640 1-003c':0 -> 'imx6-mipi-csi2':0[1]"
media-ctl -l "'imx6-mipi-csi2':2 -> 'ipu1_csi1':0[1]"
media-ctl -l "'ipu1_csi1':2 -> 'ipu1_csi1 capture':0[1]"
# Configure pads
media-ctl -V "'ov5640 1-003c':0 [fmt:UYVY2X8/1920x1080 field:none]"
media-ctl -V "'imx6-mipi-csi2':2 [fmt:UYVY2X8/1920x1080 field:none]"
media-ctl -V "'ipu1_csi1':2 [fmt:AYUV32/1920x1080 field:none]"
root@apalis-imx6-10774951:~# sh ov5640.sh
root@apalis-imx6-10774951:~# gst-launch-1.0 v4l2src device='/dev/video3' 
! videoconvert ! waylandsink
[   56.031510] ov5640 1-003c: ov5640_write_reg: error: reg=3008, val=2
[   56.649032] ov5640 1-003c: ov5640_write_reg: error: reg=3008, val=2

This surprises me. I would have expected an issue with commit 1f391df44607 ("media: v4l2-async: Use endpoints in __v4l2_async_nf_add_fwnode_remote()") to prevent the ov5640 from being detected at all. Could you double-check the bisection, to verify that commit 229fac6c44dc works and commit 1f391df44607 doesn't ? What difference does it make in the full kernel log ?

Quoted 23 lines

Setting pipeline to PAUSED ...
[   57.082630] ov5640 1-003c: ov5640_write_reg: error: reg=3008, val=2
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Failed to allocate required memory. Additional debug info:
../gst-plugins-good-1.20.3/sys/v4l2/gstv4l2src.c(759):
gst_v4l2src_decide_allocation ():
/GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
Buffer pool activation failed
Execution ended after 0:00:00.119459642
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.
Setting pipeline to NULL ...
Additional debug info:
../gstreamer-1.20.3/libs/gst/base/gstbasesrc.c(3127): gst_base_src_loop(): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)
Freeing pipeline ...

root@apalis-imx6-10774951:~# v4l2-ctl --device /dev/video3 --stream-mmap 
--stream-to=x.raw --stream-count=1
[ 1124.339793] ov5640 1-003c: ov5640_write_reg: error: reg=3008, val=2
            VIDIOC_STREAMON returned -1 (Broken pipe)
Signature · 3 lines
Regards,

Laurent Pinchart
#2 lore raw

Hello Aishwarya,

Quoted 54 lines · Laurent Pinchart

Hi Aishwarya,

(CC'ing Jacopo and Sakari)

Quoted 19 lines · Aishwarya Kothari

Hi,

I was trying ov5640 camera sensor with the mainline BSP 6.0.0-rc3 kernel and it fails to works, while it works quiet well on v5.18. When I bisect, it points out to this commit 1f391df4 (media: v4l2-async: Use endpoints in __v4l2_async_nf_add_fwnode_remote()) by Laurent Pinchart and this is what it says in the logs :

[    4.291355] imx-ipuv3-csi imx-ipuv3-csi.0: Registered ipu1_csi0 capture as /dev/video0 [    4.305228] imx-ipuv3 2400000.ipu: Registered ipu1_ic_prpenc capture as /dev/video1
[    4.317974] imx-ipuv3 2400000.ipu: Registered ipu1_ic_prpvf capture as /dev/video2
[    4.331039] imx-ipuv3-csi imx-ipuv3-csi.1: Registered ipu1_csi1 capture as /dev/video3 [    4.344255] imx-ipuv3-csi imx-ipuv3-csi.4: Registered ipu2_csi0 capture as /dev/video4 [    4.356940] imx-ipuv3 2800000.ipu: Registered ipu2_ic_prpenc capture as /dev/video5
[    4.369322] imx-ipuv3 2800000.ipu: Registered ipu2_ic_prpvf capture as /dev/video6
[    4.382119] imx-ipuv3-csi imx-ipuv3-csi.5: Registered ipu2_csi1 capture as /dev/video7 [    4.395455] imx6-mipi-csi2 21dc000.mipi: Consider updating driver imx6-mipi-csi2 to match on endpoints [    4.414971] stmpe-adc stmpe-adc: DMA mask not set

Quoted 6 lines

So, now when I try to revert the change done by this commit, it still fails. I assume there are a lot more changes following the commit that could lead to this failure. Below are the logs.
Is there a way to get it to work without reverting any changes? Or Is there something I'm missing or have to do any changes in the corresponding files to get it to work?

The commit you reference wasn't meant to break anything, so there's clearly something I've overlooked. Jacopo, have you noticed anything similar when working on the ov5640 driver ?

Quoted 5 lines

TDX Wayland with XWayland Upstream 6.1.0-devel-20221109+build.104 (kirkstone) apalis-imx6-10774951 ttymxc0 Apalis-iMX6_Reference-Multimedia-Image-upstream

apalis-imx6-10774951 login: root

Quoted 13 lines
root@apalis-imx6-10774951:~# cat ov5640.sh
media-ctl -l "'ov5640 1-003c':0 -> 'imx6-mipi-csi2':0[1]"
media-ctl -l "'imx6-mipi-csi2':2 -> 'ipu1_csi1':0[1]"
media-ctl -l "'ipu1_csi1':2 -> 'ipu1_csi1 capture':0[1]"
# Configure pads
media-ctl -V "'ov5640 1-003c':0 [fmt:UYVY2X8/1920x1080 field:none]"
media-ctl -V "'imx6-mipi-csi2':2 [fmt:UYVY2X8/1920x1080 field:none]"
media-ctl -V "'ipu1_csi1':2 [fmt:AYUV32/1920x1080 field:none]"
root@apalis-imx6-10774951:~# sh ov5640.sh
root@apalis-imx6-10774951:~# gst-launch-1.0 v4l2src device='/dev/video3'
! videoconvert ! waylandsink
[   56.031510] ov5640 1-003c: ov5640_write_reg: error: reg=3008, val=2
[   56.649032] ov5640 1-003c: ov5640_write_reg: error: reg=3008, val=2
Quoted 7 lines

This surprises me. I would have expected an issue with commit 1f391df44607 ("media: v4l2-async: Use endpoints in __v4l2_async_nf_add_fwnode_remote()") to prevent the ov5640 from being detected at all. Could you double-check the bisection, to verify that commit 229fac6c44dc works and commit 1f391df44607 doesn't ? What difference does it make in the full kernel log ?

If my understanding is correct, 1f391df44607 is unrelated as Aishwarya reports " So, now when I try to revert the change done by this commit, it still fails".

There is a long list of changes that went in with recent kernels for ov5640 but I'm afraid I'm now sure how many of them have landed on your BSP. Try to diff with a recent mainline, and then maybe bisect from there ?

Quoted 28 lines
Quoted 19 lines

Setting pipeline to PAUSED ...
[   57.082630] ov5640 1-003c: ov5640_write_reg: error: reg=3008, val=2
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Failed to allocate required memory. Additional debug info:
../gst-plugins-good-1.20.3/sys/v4l2/gstv4l2src.c(759):
gst_v4l2src_decide_allocation ():
/GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
Buffer pool activation failed
Execution ended after 0:00:00.119459642
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.
Setting pipeline to NULL ...
Additional debug info:
../gstreamer-1.20.3/libs/gst/base/gstbasesrc.c(3127): gst_base_src_loop(): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)
Freeing pipeline ...

Quoted 4 lines
root@apalis-imx6-10774951:~# v4l2-ctl --device /dev/video3 --stream-mmap
--stream-to=x.raw --stream-count=1
[ 1124.339793] ov5640 1-003c: ov5640_write_reg: error: reg=3008, val=2
            VIDIOC_STREAMON returned -1 (Broken pipe)

--
Regards,

Laurent Pinchart

#3 lore raw

Hello,

Quoted 64 lines · Jacopo Mondi

Hello Aishwarya,

Quoted 61 lines · Laurent Pinchart

Hi Aishwarya,

(CC'ing Jacopo and Sakari)

Quoted 19 lines · Aishwarya Kothari

Hi,

I was trying ov5640 camera sensor with the mainline BSP 6.0.0-rc3 kernel and it fails to works, while it works quiet well on v5.18. When I bisect, it points out to this commit 1f391df4 (media: v4l2-async: Use endpoints in __v4l2_async_nf_add_fwnode_remote()) by Laurent Pinchart and this is what it says in the logs :

[    4.291355] imx-ipuv3-csi imx-ipuv3-csi.0: Registered ipu1_csi0 capture as /dev/video0 [    4.305228] imx-ipuv3 2400000.ipu: Registered ipu1_ic_prpenc capture as /dev/video1
[    4.317974] imx-ipuv3 2400000.ipu: Registered ipu1_ic_prpvf capture as /dev/video2
[    4.331039] imx-ipuv3-csi imx-ipuv3-csi.1: Registered ipu1_csi1 capture as /dev/video3 [    4.344255] imx-ipuv3-csi imx-ipuv3-csi.4: Registered ipu2_csi0 capture as /dev/video4 [    4.356940] imx-ipuv3 2800000.ipu: Registered ipu2_ic_prpenc capture as /dev/video5
[    4.369322] imx-ipuv3 2800000.ipu: Registered ipu2_ic_prpvf capture as /dev/video6
[    4.382119] imx-ipuv3-csi imx-ipuv3-csi.5: Registered ipu2_csi1 capture as /dev/video7 [    4.395455] imx6-mipi-csi2 21dc000.mipi: Consider updating driver imx6-mipi-csi2 to match on endpoints [    4.414971] stmpe-adc stmpe-adc: DMA mask not set

Quoted 6 lines

So, now when I try to revert the change done by this commit, it still fails. I assume there are a lot more changes following the commit that could lead to this failure. Below are the logs.
Is there a way to get it to work without reverting any changes? Or Is there something I'm missing or have to do any changes in the corresponding files to get it to work?

The commit you reference wasn't meant to break anything, so there's clearly something I've overlooked. Jacopo, have you noticed anything similar when working on the ov5640 driver ?

Quoted 5 lines

TDX Wayland with XWayland Upstream 6.1.0-devel-20221109+build.104 (kirkstone) apalis-imx6-10774951 ttymxc0 Apalis-iMX6_Reference-Multimedia-Image-upstream

apalis-imx6-10774951 login: root

Quoted 13 lines
root@apalis-imx6-10774951:~# cat ov5640.sh
media-ctl -l "'ov5640 1-003c':0 -> 'imx6-mipi-csi2':0[1]"
media-ctl -l "'imx6-mipi-csi2':2 -> 'ipu1_csi1':0[1]"
media-ctl -l "'ipu1_csi1':2 -> 'ipu1_csi1 capture':0[1]"
# Configure pads
media-ctl -V "'ov5640 1-003c':0 [fmt:UYVY2X8/1920x1080 field:none]"
media-ctl -V "'imx6-mipi-csi2':2 [fmt:UYVY2X8/1920x1080 field:none]"
media-ctl -V "'ipu1_csi1':2 [fmt:AYUV32/1920x1080 field:none]"
root@apalis-imx6-10774951:~# sh ov5640.sh
root@apalis-imx6-10774951:~# gst-launch-1.0 v4l2src device='/dev/video3'
! videoconvert ! waylandsink
[   56.031510] ov5640 1-003c: ov5640_write_reg: error: reg=3008, val=2
[   56.649032] ov5640 1-003c: ov5640_write_reg: error: reg=3008, val=2

This surprises me. I would have expected an issue with commit 1f391df44607 ("media: v4l2-async: Use endpoints in __v4l2_async_nf_add_fwnode_remote()") to prevent the ov5640 from being detected at all. Could you double-check the bisection, to verify that commit 229fac6c44dc works and commit 1f391df44607 doesn't ? What difference does it make in the full kernel log ?

Yes, the commit 229fac6c44dc works. This is the first commit 1f391df44607 ("media: v4l2-async: Use endpoints in __v4l2_async_nf_add_fwnode_remote()") that fails. The ov5640 driver is not detected and the /dev/media is not present.

This is the difference in dmesg:

dmesg_1f391df44607_nt renamed from dmesg_229fac6c44dc_nt +1 -1
@@ -1,5 +1,5 @@
 Booting Linux on physical CPU 0x0
-Linux version 5.18.0-rc3-00051-g229fac6c44dc (aishwarya@aishwarya-nb) (arm-none-linux-gnueabihf-gcc (Arm GNU Toolchain 11.3.Rel1) 11.3.1 20220712, GNU ld (Arm GNU Toolchain 11.3.Rel1) 2.38.20220708) #28 SMP Wed Dec 21 11:13:19 CET 2022
+Linux version 5.18.0-rc3-00052-g1f391df44607 (aishwarya@aishwarya-nb) (arm-none-linux-gnueabihf-gcc (Arm GNU Toolchain 11.3.Rel1) 11.3.1 20220712, GNU ld (Arm GNU Toolchain 11.3.Rel1) 2.38.20220708) #29 SMP Wed Dec 21 11:21:31 CET 2022
 CPU: ARMv7 Processor [412fc09a] revision 10 (ARMv7), cr=10c5387d
 CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instruction cache
 OF: fdt: Machine model: Toradex Apalis iMX6Q/D Module on Apalis 

Evaluation Board
@@ -228,7 +228,7 @@
SPI driver ads7846 has no spi_device_id for ti,ads7845
SPI driver ads7846 has no spi_device_id for ti,ads7873
rtc-ds1307 0-0068: registered as rtc0
-rtc-ds1307 0-0068: setting system clock to 2022-12-21T10:14:44 UTC (1671617684)
+rtc-ds1307 0-0068: setting system clock to 2022-12-21T10:22:46 UTC (1671618166)
snvs_rtc 20cc000.snvs:snvs-rtc-lp: registered as rtc1
i2c_dev: i2c /dev entries driver
Bluetooth: HCI UART driver ver 2.3
@@ -269,6 +269,7 @@
imx-ipuv3 2800000.ipu: Registered ipu2_ic_prpenc capture as /dev/video5
imx-ipuv3 2800000.ipu: Registered ipu2_ic_prpvf capture as /dev/video6
imx-ipuv3-csi imx-ipuv3-csi.5: Registered ipu2_csi1 capture as /dev/video7
+imx6-mipi-csi2 21dc000.mipi: Consider updating driver imx6-mipi-csi2 to match on endpoints
sgtl5000 2-000a: sgtl5000 revision 0x11
random: crng init done
imx6q-pcie 1ffc000.pcie: Phy link never came up
@@ -318,10 +319,12 @@
usb usb1: New USB device found, idVendor=1d6b, idProduct=0002,
bcdDevice= 5.18
usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
usb usb1: Product: EHCI Host Controller
-usb usb1: Manufacturer: Linux 5.18.0-rc3-00051-g229fac6c44dc ehci_hcd
+usb usb1: Manufacturer: Linux 5.18.0-rc3-00052-g1f391df44607 ehci_hcd
usb usb1: SerialNumber: ci_hdrc.1
hub 1-0:1.0: USB hub found
hub 1-0:1.0: 1 port detected
+video-mux 20e0000.iomuxc-gpr:ipu1_csi0_mux: Consider updating driver video-mux to match on endpoints
+video-mux 20e0000.iomuxc-gpr:ipu2_csi1_mux: Consider updating driver video-mux to match on endpoints
imx_thermal 20c8000.anatop:tempmon: Extended Commercial CPU
temperature grade - max:105C critical:100C passive:95C
fsl-ssi-dai 2028000.ssi: No cache defaults, reading back from HW
fsl-ssi-dai 2028000.ssi: No cache defaults, reading back from HW
@@ -384,7 +387,7 @@
systemd[1]: Mounting Kernel Debug File System...
systemd[1]: Mounting Kernel Trace File System...
systemd[1]: Mounting Temporary Directory /tmp...
-systemd[1]: Create List of Static Device Nodes was skipped because of a failed condition check (ConditionFileNotEmpty=/lib/modules/5.18.0-rc3-00051-g229fac6c44dc/modules.devname). +systemd[1]: Create List of Static Device Nodes was skipped because of a failed condition check (ConditionFileNotEmpty=/lib/modules/5.18.0-rc3-00052-g1f391df44607/modules.devname).
systemd[1]: Starting Load Kernel Module configfs...
systemd[1]: Starting Load Kernel Module drm...
systemd[1]: Starting Load Kernel Module fuse...
@@ -413,7 +416,7 @@
systemd[1]: Starting Remount Root and Kernel File Systems...
systemd[1]: Started Journal Service.
EXT4-fs (mmcblk2p2): re-mounted. Quota mode: none.
-systemd-journald[217]: Received client request to flush runtime journal.
+systemd-journald[216]: Received client request to flush runtime journal.
evbug: Connected device: input0 (gpio-keys at gpio-keys/input0)
coda 2040000.vpu: Firmware code revision: 46076
coda 2040000.vpu: Initialized CODA960.
@@ -426,19 +429,15 @@
ov5640 1-003c: supply DOVDD not found, using dummy regulator
ov5640 1-003c: supply AVDD not found, using dummy regulator
ov5640 1-003c: supply DVDD not found, using dummy regulator
-imx-media: Registered ipu_ic_pp csc/scaler as /dev/video12
+ov5640 1-003c: Consider updating driver ov5640 to match on endpoints
fsl-ssi-dai 2028000.ssi: No cache defaults, reading back from HW
using random self ethernet address
using random host ethernet address
-usb0: HOST MAC 26:b4:a4:fa:fe:15
-usb0: MAC 8e:e0:94:47:d3:6a
+usb0: HOST MAC 26:4d:ee:18:a3:b6
+usb0: MAC ce:ea:96:16:e2:a9
Micrel KSZ9031 Gigabit PHY 2188000.ethernet-1:07: attached PHY driver
(mii_bus:phy_addr=2188000.ethernet-1:07, irq=123)
-systemd-journald[217]: Data hash table of /run/log/journal/6126bf14c3ea4128a506478d46ca2b38/system.journal has a fill level at 75.2 (2737 of 3640 items, 2097152 file size, 766 bytes per hash table item), suggesting rotation. -systemd-journald[217]: /run/log/journal/6126bf14c3ea4128a506478d46ca2b38/system.journal: Journal header limits reached or header out-of-date, rotating.
+systemd-journald[216]: Data hash table of /run/log/journal/6126bf14c3ea4128a506478d46ca2b38/system.journal has a fill level at 75.1 (2733 of 3640 items, 2097152 file size, 767 bytes per hash table item), suggesting rotation. +systemd-journald[216]: /run/log/journal/6126bf14c3ea4128a506478d46ca2b38/system.journal: Journal header limits reached or header out-of-date, rotating.
IPv6: ADDRCONF(NETDEV_CHANGE): usb0: link becomes ready
fec 2188000.ethernet eth0: Link is Up - 1Gbps/Full - flow control rx/tx
IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
-ov5640 1-003c: ov5640_write_reg: error: reg=3008, val=2
-ov5640 1-003c: ov5640_write_reg: error: reg=3008, val=2
-ov5640 1-003c: ov5640_write_reg: error: reg=3008, val=2
-cfg80211: failed to load regulatory.db

Quoted 10 lines

If my understanding is correct, 1f391df44607 is unrelated as Aishwarya reports " So, now when I try to revert the change done by this commit, it still fails".

There is a long list of changes that went in with recent kernels for ov5640 but I'm afraid I'm now sure how many of them have landed on your BSP. Try to diff with a recent mainline, and then maybe bisect from there ?

We use the the recent mainline kernel without any changes.
It stops working at 1f391df44607. With 1f391df44607 reverted, bisecting reveals that a89f14bbcfa5 ("media: ov5640: Split DVP and CSI-2 formats") introduces another regression. I'm still investigating what could be the problem. The format used below, it no longer being recognised:
media-ctl -V "'ov5640 1-003c':0 [fmt:UYVY2X8/1920x1080 field:none]"
media-ctl -V "'imx6-mipi-csi2':2 [fmt:UYVY2X8/1920x1080 field:none]"

Quoted 23 lines
Quoted 23 lines
Quoted 23 lines

Setting pipeline to PAUSED ...
[   57.082630] ov5640 1-003c: ov5640_write_reg: error: reg=3008, val=2
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Failed to allocate required memory. Additional debug info:
../gst-plugins-good-1.20.3/sys/v4l2/gstv4l2src.c(759):
gst_v4l2src_decide_allocation ():
/GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
Buffer pool activation failed
Execution ended after 0:00:00.119459642
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.
Setting pipeline to NULL ...
Additional debug info:
../gstreamer-1.20.3/libs/gst/base/gstbasesrc.c(3127): gst_base_src_loop(): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)
Freeing pipeline ...

root@apalis-imx6-10774951:~# v4l2-ctl --device /dev/video3 --stream-mmap
--stream-to=x.raw --stream-count=1
[ 1124.339793] ov5640 1-003c: ov5640_write_reg: error: reg=3008, val=2
            VIDIOC_STREAMON returned -1 (Broken pipe)
Quoted 5 lines
Quoted 4 lines
Quoted 1 line
Quoted 1 line

--

Regards,

Laurent Pinchart

I'm grateful for any hints or comments?
Regards,
Aishwarya Kothari

#4 lore raw

Hello Aishwarya

Quoted 194 lines · Aishwarya Kothari

Hello,

Quoted 64 lines · Jacopo Mondi

Hello Aishwarya,

Quoted 61 lines · Laurent Pinchart

Hi Aishwarya,

(CC'ing Jacopo and Sakari)

Quoted 19 lines · Aishwarya Kothari

Hi,

I was trying ov5640 camera sensor with the mainline BSP 6.0.0-rc3 kernel and it fails to works, while it works quiet well on v5.18. When I bisect, it points out to this commit 1f391df4 (media: v4l2-async: Use endpoints in __v4l2_async_nf_add_fwnode_remote()) by Laurent Pinchart and this is what it says in the logs :

[    4.291355] imx-ipuv3-csi imx-ipuv3-csi.0: Registered ipu1_csi0 capture as /dev/video0 [    4.305228] imx-ipuv3 2400000.ipu: Registered ipu1_ic_prpenc capture as /dev/video1
[    4.317974] imx-ipuv3 2400000.ipu: Registered ipu1_ic_prpvf capture as /dev/video2
[    4.331039] imx-ipuv3-csi imx-ipuv3-csi.1: Registered ipu1_csi1 capture as /dev/video3 [    4.344255] imx-ipuv3-csi imx-ipuv3-csi.4: Registered ipu2_csi0 capture as /dev/video4 [    4.356940] imx-ipuv3 2800000.ipu: Registered ipu2_ic_prpenc capture as /dev/video5
[    4.369322] imx-ipuv3 2800000.ipu: Registered ipu2_ic_prpvf capture as /dev/video6
[    4.382119] imx-ipuv3-csi imx-ipuv3-csi.5: Registered ipu2_csi1 capture as /dev/video7 [    4.395455] imx6-mipi-csi2 21dc000.mipi: Consider updating driver imx6-mipi-csi2 to match on endpoints [    4.414971] stmpe-adc stmpe-adc: DMA mask not set

Quoted 6 lines

So, now when I try to revert the change done by this commit, it still fails. I assume there are a lot more changes following the commit that could lead to this failure. Below are the logs.
Is there a way to get it to work without reverting any changes? Or Is there something I'm missing or have to do any changes in the corresponding files to get it to work?

The commit you reference wasn't meant to break anything, so there's clearly something I've overlooked. Jacopo, have you noticed anything similar when working on the ov5640 driver ?

Quoted 5 lines

TDX Wayland with XWayland Upstream 6.1.0-devel-20221109+build.104 (kirkstone) apalis-imx6-10774951 ttymxc0 Apalis-iMX6_Reference-Multimedia-Image-upstream

apalis-imx6-10774951 login: root

Quoted 13 lines
root@apalis-imx6-10774951:~# cat ov5640.sh
media-ctl -l "'ov5640 1-003c':0 -> 'imx6-mipi-csi2':0[1]"
media-ctl -l "'imx6-mipi-csi2':2 -> 'ipu1_csi1':0[1]"
media-ctl -l "'ipu1_csi1':2 -> 'ipu1_csi1 capture':0[1]"
# Configure pads
media-ctl -V "'ov5640 1-003c':0 [fmt:UYVY2X8/1920x1080 field:none]"
media-ctl -V "'imx6-mipi-csi2':2 [fmt:UYVY2X8/1920x1080 field:none]"
media-ctl -V "'ipu1_csi1':2 [fmt:AYUV32/1920x1080 field:none]"
root@apalis-imx6-10774951:~# sh ov5640.sh
root@apalis-imx6-10774951:~# gst-launch-1.0 v4l2src device='/dev/video3'
! videoconvert ! waylandsink
[   56.031510] ov5640 1-003c: ov5640_write_reg: error: reg=3008, val=2
[   56.649032] ov5640 1-003c: ov5640_write_reg: error: reg=3008, val=2

This surprises me. I would have expected an issue with commit 1f391df44607 ("media: v4l2-async: Use endpoints in __v4l2_async_nf_add_fwnode_remote()") to prevent the ov5640 from being detected at all. Could you double-check the bisection, to verify that commit 229fac6c44dc works and commit 1f391df44607 doesn't ? What difference does it make in the full kernel log ?

Yes, the commit 229fac6c44dc works. This is the first commit 1f391df44607 ("media: v4l2-async: Use endpoints in __v4l2_async_nf_add_fwnode_remote()") that fails. The ov5640 driver is not detected and the /dev/media is not present.

This is the difference in dmesg:

dmesg_1f391df44607_nt renamed from dmesg_229fac6c44dc_nt +0 -1
@@ -1,5 +1,5 @@
 Booting Linux on physical CPU 0x0
-Linux version 5.18.0-rc3-00051-g229fac6c44dc (aishwarya@aishwarya-nb)
 (arm-none-linux-gnueabihf-gcc (Arm GNU Toolchain 11.3.Rel1) 11.3.1 20220712,

GNU ld (Arm GNU Toolchain 11.3.Rel1) 2.38.20220708) #28 SMP Wed Dec 21 11:13:19 CET 2022
+Linux version 5.18.0-rc3-00052-g1f391df44607 (aishwarya@aishwarya-nb) (arm-none-linux-gnueabihf-gcc (Arm GNU Toolchain 11.3.Rel1) 11.3.1 20220712, GNU ld (Arm GNU Toolchain 11.3.Rel1) 2.38.20220708) #29 SMP Wed Dec 21 11:21:31 CET 2022
CPU: ARMv7 Processor [412fc09a] revision 10 (ARMv7), cr=10c5387d
CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instruction cache
OF: fdt: Machine model: Toradex Apalis iMX6Q/D Module on Apalis Evaluation
Board

dmesg_1f391df44607_nt renamed from dmesg_229fac6c44dc_nt +1 -1
@@ -228,7 +228,7 @@
 SPI driver ads7846 has no spi_device_id for ti,ads7845
 SPI driver ads7846 has no spi_device_id for ti,ads7873
 rtc-ds1307 0-0068: registered as rtc0
-rtc-ds1307 0-0068: setting system clock to 2022-12-21T10:14:44 UTC
 (1671617684)
+rtc-ds1307 0-0068: setting system clock to 2022-12-21T10:22:46 UTC
 (1671618166)
 snvs_rtc 20cc000.snvs:snvs-rtc-lp: registered as rtc1

i2c_dev: i2c /dev entries driver
Bluetooth: HCI UART driver ver 2.3

dmesg_1f391df44607_nt renamed from dmesg_229fac6c44dc_nt +1 -0
@@ -269,6 +269,7 @@
 imx-ipuv3 2800000.ipu: Registered ipu2_ic_prpenc capture as /dev/video5
 imx-ipuv3 2800000.ipu: Registered ipu2_ic_prpvf capture as /dev/video6
 imx-ipuv3-csi imx-ipuv3-csi.5: Registered ipu2_csi1 capture as /dev/video7
+imx6-mipi-csi2 21dc000.mipi: Consider updating driver imx6-mipi-csi2 to

match on endpoints
sgtl5000 2-000a: sgtl5000 revision 0x11
random: crng init done
imx6q-pcie 1ffc000.pcie: Phy link never came up

dmesg_1f391df44607_nt renamed from dmesg_229fac6c44dc_nt +0 -0
@@ -318,10 +319,12 @@
 usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice=

5.18
usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 usb usb1: Product: EHCI Host Controller

dmesg_1f391df44607_nt renamed from dmesg_229fac6c44dc_nt +2 -1
-usb usb1: Manufacturer: Linux 5.18.0-rc3-00051-g229fac6c44dc ehci_hcd
+usb usb1: Manufacturer: Linux 5.18.0-rc3-00052-g1f391df44607 ehci_hcd
 usb usb1: SerialNumber: ci_hdrc.1
 hub 1-0:1.0: USB hub found
 hub 1-0:1.0: 1 port detected
+video-mux 20e0000.iomuxc-gpr:ipu1_csi0_mux: Consider updating driver

video-mux to match on endpoints
+video-mux 20e0000.iomuxc-gpr:ipu2_csi1_mux: Consider updating driver video-mux to match on endpoints
imx_thermal 20c8000.anatop:tempmon: Extended Commercial CPU temperature
grade - max:105C critical:100C passive:95C
fsl-ssi-dai 2028000.ssi: No cache defaults, reading back from HW fsl-ssi-dai 2028000.ssi: No cache defaults, reading back from HW

dmesg_1f391df44607_nt renamed from dmesg_229fac6c44dc_nt +0 -1
@@ -384,7 +387,7 @@
 systemd[1]: Mounting Kernel Debug File System...
 systemd[1]: Mounting Kernel Trace File System...
 systemd[1]: Mounting Temporary Directory /tmp...
-systemd[1]: Create List of Static Device Nodes was skipped because of a

failed condition check (ConditionFileNotEmpty=/lib/modules/5.18.0-rc3-00051-g229fac6c44dc/modules.devname). +systemd[1]: Create List of Static Device Nodes was skipped because of a
failed condition check (ConditionFileNotEmpty=/lib/modules/5.18.0-rc3-00052-g1f391df44607/modules.devname).
systemd[1]: Starting Load Kernel Module configfs...
systemd[1]: Starting Load Kernel Module drm...
systemd[1]: Starting Load Kernel Module fuse...

dmesg_1f391df44607_nt renamed from dmesg_229fac6c44dc_nt +4 -5
@@ -413,7 +416,7 @@
 systemd[1]: Starting Remount Root and Kernel File Systems...
 systemd[1]: Started Journal Service.
 EXT4-fs (mmcblk2p2): re-mounted. Quota mode: none.
-systemd-journald[217]: Received client request to flush runtime journal.
+systemd-journald[216]: Received client request to flush runtime journal.
 evbug: Connected device: input0 (gpio-keys at gpio-keys/input0)
 coda 2040000.vpu: Firmware code revision: 46076
 coda 2040000.vpu: Initialized CODA960.
@@ -426,19 +429,15 @@
 ov5640 1-003c: supply DOVDD not found, using dummy regulator
 ov5640 1-003c: supply AVDD not found, using dummy regulator
 ov5640 1-003c: supply DVDD not found, using dummy regulator
-imx-media: Registered ipu_ic_pp csc/scaler as /dev/video12
+ov5640 1-003c: Consider updating driver ov5640 to match on endpoints
 fsl-ssi-dai 2028000.ssi: No cache defaults, reading back from HW
 using random self ethernet address
 using random host ethernet address
-usb0: HOST MAC 26:b4:a4:fa:fe:15
-usb0: MAC 8e:e0:94:47:d3:6a
+usb0: HOST MAC 26:4d:ee:18:a3:b6
+usb0: MAC ce:ea:96:16:e2:a9
 Micrel KSZ9031 Gigabit PHY 2188000.ethernet-1:07: attached PHY driver
 (mii_bus:phy_addr=2188000.ethernet-1:07, irq=123)
-systemd-journald[217]: Data hash table of

/run/log/journal/6126bf14c3ea4128a506478d46ca2b38/system.journal has a fill level at 75.2 (2737 of 3640 items, 2097152 file size, 766 bytes per hash table item), suggesting rotation.
-systemd-journald[217]:
/run/log/journal/6126bf14c3ea4128a506478d46ca2b38/system.journal: Journal header limits reached or header out-of-date, rotating. +systemd-journald[216]: Data hash table of /run/log/journal/6126bf14c3ea4128a506478d46ca2b38/system.journal has a fill level at 75.1 (2733 of 3640 items, 2097152 file size, 767 bytes per hash table item), suggesting rotation.
+systemd-journald[216]:
/run/log/journal/6126bf14c3ea4128a506478d46ca2b38/system.journal: Journal header limits reached or header out-of-date, rotating.
IPv6: ADDRCONF(NETDEV_CHANGE): usb0: link becomes ready
fec 2188000.ethernet eth0: Link is Up - 1Gbps/Full - flow control rx/tx
IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready

dmesg_1f391df44607_nt renamed from dmesg_229fac6c44dc_nt +0 -4
-ov5640 1-003c: ov5640_write_reg: error: reg=3008, val=2
-ov5640 1-003c: ov5640_write_reg: error: reg=3008, val=2
-ov5640 1-003c: ov5640_write_reg: error: reg=3008, val=2
-cfg80211: failed to load regulatory.db
Quoted 11 lines

If my understanding is correct, 1f391df44607 is unrelated as Aishwarya reports " So, now when I try to revert the change done by this commit, it still fails".

There is a long list of changes that went in with recent kernels for ov5640 but I'm afraid I'm now sure how many of them have landed on your BSP. Try to diff with a recent mainline, and then maybe bisect from there ?

Quoted 8 lines

We use the the recent mainline kernel without any changes. It stops working at 1f391df44607. With 1f391df44607 reverted, bisecting reveals that a89f14bbcfa5 ("media: ov5640: Split DVP and CSI-2 formats") introduces another regression. I'm still investigating what could be the problem. The format used below, it no longer being recognised:
media-ctl -V "'ov5640 1-003c':0 [fmt:UYVY2X8/1920x1080 field:none]" media-ctl -V "'imx6-mipi-csi2':2 [fmt:UYVY2X8/1920x1080 field:none]"

That should be easy. Just lookg at the a89f14bbcfa5 commit and you'll see that the 2X8 media bus variants are not supported anymore in MIPI mode. Just use UYVY1X16 everywhere and you should be good.

Thanks
j

Quoted 32 lines
Quoted 23 lines
Quoted 23 lines
Quoted 23 lines

Setting pipeline to PAUSED ...
[   57.082630] ov5640 1-003c: ov5640_write_reg: error: reg=3008, val=2
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Failed to allocate required memory. Additional debug info:
../gst-plugins-good-1.20.3/sys/v4l2/gstv4l2src.c(759):
gst_v4l2src_decide_allocation ():
/GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
Buffer pool activation failed
Execution ended after 0:00:00.119459642
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.
Setting pipeline to NULL ...
Additional debug info:
../gstreamer-1.20.3/libs/gst/base/gstbasesrc.c(3127): gst_base_src_loop(): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)
Freeing pipeline ...

root@apalis-imx6-10774951:~# v4l2-ctl --device /dev/video3 --stream-mmap
--stream-to=x.raw --stream-count=1
[ 1124.339793] ov5640 1-003c: ov5640_write_reg: error: reg=3008, val=2
            VIDIOC_STREAMON returned -1 (Broken pipe)
Quoted 5 lines
Quoted 4 lines
Quoted 1 line
Quoted 1 line

--

Regards,

Laurent Pinchart

I'm grateful for any hints or comments?
Regards,
Aishwarya Kothari

#5 lore raw

Hello Jacopo and Laurent,

Quoted 179 lines · Jacopo Mondi

Hello Aishwarya

Quoted 176 lines · Aishwarya Kothari

Hello,

Quoted 64 lines · Jacopo Mondi

Hello Aishwarya,

Quoted 61 lines · Laurent Pinchart

Hi Aishwarya,

(CC'ing Jacopo and Sakari)

Quoted 19 lines · Aishwarya Kothari

Hi,

I was trying ov5640 camera sensor with the mainline BSP 6.0.0-rc3 kernel and it fails to works, while it works quiet well on v5.18. When I bisect, it points out to this commit 1f391df4 (media: v4l2-async: Use endpoints in __v4l2_async_nf_add_fwnode_remote()) by Laurent Pinchart and this is what it says in the logs :

[    4.291355] imx-ipuv3-csi imx-ipuv3-csi.0: Registered ipu1_csi0 capture as /dev/video0 [    4.305228] imx-ipuv3 2400000.ipu: Registered ipu1_ic_prpenc capture as /dev/video1
[    4.317974] imx-ipuv3 2400000.ipu: Registered ipu1_ic_prpvf capture as /dev/video2
[    4.331039] imx-ipuv3-csi imx-ipuv3-csi.1: Registered ipu1_csi1 capture as /dev/video3 [    4.344255] imx-ipuv3-csi imx-ipuv3-csi.4: Registered ipu2_csi0 capture as /dev/video4 [    4.356940] imx-ipuv3 2800000.ipu: Registered ipu2_ic_prpenc capture as /dev/video5
[    4.369322] imx-ipuv3 2800000.ipu: Registered ipu2_ic_prpvf capture as /dev/video6
[    4.382119] imx-ipuv3-csi imx-ipuv3-csi.5: Registered ipu2_csi1 capture as /dev/video7 [    4.395455] imx6-mipi-csi2 21dc000.mipi: Consider updating driver imx6-mipi-csi2 to match on endpoints [    4.414971] stmpe-adc stmpe-adc: DMA mask not set

Quoted 6 lines

So, now when I try to revert the change done by this commit, it still fails. I assume there are a lot more changes following the commit that could lead to this failure. Below are the logs.
Is there a way to get it to work without reverting any changes? Or Is there something I'm missing or have to do any changes in the corresponding files to get it to work?

The commit you reference wasn't meant to break anything, so there's clearly something I've overlooked. Jacopo, have you noticed anything similar when working on the ov5640 driver ?

Quoted 5 lines

TDX Wayland with XWayland Upstream 6.1.0-devel-20221109+build.104 (kirkstone) apalis-imx6-10774951 ttymxc0 Apalis-iMX6_Reference-Multimedia-Image-upstream

apalis-imx6-10774951 login: root

Quoted 13 lines
root@apalis-imx6-10774951:~# cat ov5640.sh
media-ctl -l "'ov5640 1-003c':0 -> 'imx6-mipi-csi2':0[1]"
media-ctl -l "'imx6-mipi-csi2':2 -> 'ipu1_csi1':0[1]"
media-ctl -l "'ipu1_csi1':2 -> 'ipu1_csi1 capture':0[1]"
# Configure pads
media-ctl -V "'ov5640 1-003c':0 [fmt:UYVY2X8/1920x1080 field:none]"
media-ctl -V "'imx6-mipi-csi2':2 [fmt:UYVY2X8/1920x1080 field:none]"
media-ctl -V "'ipu1_csi1':2 [fmt:AYUV32/1920x1080 field:none]"
root@apalis-imx6-10774951:~# sh ov5640.sh
root@apalis-imx6-10774951:~# gst-launch-1.0 v4l2src device='/dev/video3'
! videoconvert ! waylandsink
[   56.031510] ov5640 1-003c: ov5640_write_reg: error: reg=3008, val=2
[   56.649032] ov5640 1-003c: ov5640_write_reg: error: reg=3008, val=2

This surprises me. I would have expected an issue with commit 1f391df44607 ("media: v4l2-async: Use endpoints in __v4l2_async_nf_add_fwnode_remote()") to prevent the ov5640 from being detected at all. Could you double-check the bisection, to verify that commit 229fac6c44dc works and commit 1f391df44607 doesn't ? What difference does it make in the full kernel log ?

Yes, the commit 229fac6c44dc works. This is the first commit 1f391df44607 ("media: v4l2-async: Use endpoints in __v4l2_async_nf_add_fwnode_remote()") that fails. The ov5640 driver is not detected and the /dev/media is not present.

This is the difference in dmesg:

dmesg_1f391df44607_nt renamed from dmesg_229fac6c44dc_nt +0 -1
@@ -1,5 +1,5 @@
  Booting Linux on physical CPU 0x0
-Linux version 5.18.0-rc3-00051-g229fac6c44dc (aishwarya@aishwarya-nb)
 (arm-none-linux-gnueabihf-gcc (Arm GNU Toolchain 11.3.Rel1) 11.3.1 20220712,

GNU ld (Arm GNU Toolchain 11.3.Rel1) 2.38.20220708) #28 SMP Wed Dec 21 11:13:19 CET 2022
+Linux version 5.18.0-rc3-00052-g1f391df44607 (aishwarya@aishwarya-nb) (arm-none-linux-gnueabihf-gcc (Arm GNU Toolchain 11.3.Rel1) 11.3.1 20220712, GNU ld (Arm GNU Toolchain 11.3.Rel1) 2.38.20220708) #29 SMP Wed Dec 21 11:21:31 CET 2022
CPU: ARMv7 Processor [412fc09a] revision 10 (ARMv7), cr=10c5387d
CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instruction cache
OF: fdt: Machine model: Toradex Apalis iMX6Q/D Module on Apalis Evaluation
Board

dmesg_1f391df44607_nt renamed from dmesg_229fac6c44dc_nt +1 -1
@@ -228,7 +228,7 @@
  SPI driver ads7846 has no spi_device_id for ti,ads7845
  SPI driver ads7846 has no spi_device_id for ti,ads7873
  rtc-ds1307 0-0068: registered as rtc0
-rtc-ds1307 0-0068: setting system clock to 2022-12-21T10:14:44 UTC
 (1671617684)
+rtc-ds1307 0-0068: setting system clock to 2022-12-21T10:22:46 UTC
 (1671618166)
  snvs_rtc 20cc000.snvs:snvs-rtc-lp: registered as rtc1

i2c_dev: i2c /dev entries driver
Bluetooth: HCI UART driver ver 2.3

dmesg_1f391df44607_nt renamed from dmesg_229fac6c44dc_nt +1 -0
@@ -269,6 +269,7 @@
  imx-ipuv3 2800000.ipu: Registered ipu2_ic_prpenc capture as /dev/video5
  imx-ipuv3 2800000.ipu: Registered ipu2_ic_prpvf capture as /dev/video6
  imx-ipuv3-csi imx-ipuv3-csi.5: Registered ipu2_csi1 capture as /dev/video7
+imx6-mipi-csi2 21dc000.mipi: Consider updating driver imx6-mipi-csi2 to

match on endpoints
sgtl5000 2-000a: sgtl5000 revision 0x11
random: crng init done
imx6q-pcie 1ffc000.pcie: Phy link never came up

dmesg_1f391df44607_nt renamed from dmesg_229fac6c44dc_nt +0 -0
@@ -318,10 +319,12 @@
  usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice=

5.18
usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 usb usb1: Product: EHCI Host Controller

dmesg_1f391df44607_nt renamed from dmesg_229fac6c44dc_nt +2 -1
-usb usb1: Manufacturer: Linux 5.18.0-rc3-00051-g229fac6c44dc ehci_hcd
+usb usb1: Manufacturer: Linux 5.18.0-rc3-00052-g1f391df44607 ehci_hcd
  usb usb1: SerialNumber: ci_hdrc.1
  hub 1-0:1.0: USB hub found
  hub 1-0:1.0: 1 port detected
+video-mux 20e0000.iomuxc-gpr:ipu1_csi0_mux: Consider updating driver

video-mux to match on endpoints
+video-mux 20e0000.iomuxc-gpr:ipu2_csi1_mux: Consider updating driver video-mux to match on endpoints
imx_thermal 20c8000.anatop:tempmon: Extended Commercial CPU temperature
grade - max:105C critical:100C passive:95C
fsl-ssi-dai 2028000.ssi: No cache defaults, reading back from HW fsl-ssi-dai 2028000.ssi: No cache defaults, reading back from HW

dmesg_1f391df44607_nt renamed from dmesg_229fac6c44dc_nt +0 -1
@@ -384,7 +387,7 @@
  systemd[1]: Mounting Kernel Debug File System...
  systemd[1]: Mounting Kernel Trace File System...
  systemd[1]: Mounting Temporary Directory /tmp...
-systemd[1]: Create List of Static Device Nodes was skipped because of a

failed condition check (ConditionFileNotEmpty=/lib/modules/5.18.0-rc3-00051-g229fac6c44dc/modules.devname). +systemd[1]: Create List of Static Device Nodes was skipped because of a
failed condition check (ConditionFileNotEmpty=/lib/modules/5.18.0-rc3-00052-g1f391df44607/modules.devname).
systemd[1]: Starting Load Kernel Module configfs...
systemd[1]: Starting Load Kernel Module drm...
systemd[1]: Starting Load Kernel Module fuse...

dmesg_1f391df44607_nt renamed from dmesg_229fac6c44dc_nt +4 -5
@@ -413,7 +416,7 @@
  systemd[1]: Starting Remount Root and Kernel File Systems...
  systemd[1]: Started Journal Service.
  EXT4-fs (mmcblk2p2): re-mounted. Quota mode: none.
-systemd-journald[217]: Received client request to flush runtime journal.
+systemd-journald[216]: Received client request to flush runtime journal.
  evbug: Connected device: input0 (gpio-keys at gpio-keys/input0)
  coda 2040000.vpu: Firmware code revision: 46076
  coda 2040000.vpu: Initialized CODA960.
@@ -426,19 +429,15 @@
  ov5640 1-003c: supply DOVDD not found, using dummy regulator
  ov5640 1-003c: supply AVDD not found, using dummy regulator
  ov5640 1-003c: supply DVDD not found, using dummy regulator
-imx-media: Registered ipu_ic_pp csc/scaler as /dev/video12
+ov5640 1-003c: Consider updating driver ov5640 to match on endpoints
  fsl-ssi-dai 2028000.ssi: No cache defaults, reading back from HW
  using random self ethernet address
  using random host ethernet address
-usb0: HOST MAC 26:b4:a4:fa:fe:15
-usb0: MAC 8e:e0:94:47:d3:6a
+usb0: HOST MAC 26:4d:ee:18:a3:b6
+usb0: MAC ce:ea:96:16:e2:a9
  Micrel KSZ9031 Gigabit PHY 2188000.ethernet-1:07: attached PHY driver
 (mii_bus:phy_addr=2188000.ethernet-1:07, irq=123)
-systemd-journald[217]: Data hash table of

/run/log/journal/6126bf14c3ea4128a506478d46ca2b38/system.journal has a fill level at 75.2 (2737 of 3640 items, 2097152 file size, 766 bytes per hash table item), suggesting rotation.
-systemd-journald[217]:
/run/log/journal/6126bf14c3ea4128a506478d46ca2b38/system.journal: Journal header limits reached or header out-of-date, rotating. +systemd-journald[216]: Data hash table of /run/log/journal/6126bf14c3ea4128a506478d46ca2b38/system.journal has a fill level at 75.1 (2733 of 3640 items, 2097152 file size, 767 bytes per hash table item), suggesting rotation.
+systemd-journald[216]:
/run/log/journal/6126bf14c3ea4128a506478d46ca2b38/system.journal: Journal header limits reached or header out-of-date, rotating.

Quoted 8 lines

dmesg_1f391df44607_nt

Quoted 8 lines

IPv6: ADDRCONF(NETDEV_CHANGE): usb0: link becomes ready
fec 2188000.ethernet eth0: Link is Up - 1Gbps/Full - flow control rx/tx
IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready

dmesg_1f391df44607_nt renamed from dmesg_229fac6c44dc_nt +0 -4
-ov5640 1-003c: ov5640_write_reg: error: reg=3008, val=2
-ov5640 1-003c: ov5640_write_reg: error: reg=3008, val=2
-ov5640 1-003c: ov5640_write_reg: error: reg=3008, val=2
-cfg80211: failed to load regulatory.db

Well, the commit 1f391df4 (media: v4l2-async: Use endpoints in __v4l2_async_nf_add_fwnode_remote()) is actually the one introducing regression. As mentioned before, /dev/media0 is not created.

Quoted 19 lines
Quoted 10 lines
Quoted 10 lines

If my understanding is correct, 1f391df44607 is unrelated as Aishwarya reports " So, now when I try to revert the change done by this commit, it still fails".

There is a long list of changes that went in with recent kernels for ov5640 but I'm afraid I'm now sure how many of them have landed on your BSP. Try to diff with a recent mainline, and then maybe bisect from there ?

Quoted 8 lines

We use the the recent mainline kernel without any changes. It stops working at 1f391df44607. With 1f391df44607 reverted, bisecting reveals that a89f14bbcfa5 ("media: ov5640: Split DVP and CSI-2 formats") introduces another regression. I'm still investigating what could be the problem. The format used below, it no longer being recognised:
media-ctl -V "'ov5640 1-003c':0 [fmt:UYVY2X8/1920x1080 field:none]" media-ctl -V "'imx6-mipi-csi2':2 [fmt:UYVY2X8/1920x1080 field:none]"

Quoted 7 lines

That should be easy. Just lookg at the a89f14bbcfa5 commit and you'll see that the 2X8 media bus variants are not supported anymore in MIPI mode. Just use UYVY1X16 everywhere and you should be good.

Thanks
j

Thanks
Reverting the commit 1f391df4 (media: v4l2-async: Use endpoints in __v4l2_async_nf_add_fwnode_remote()) and setting the format as below works fine when tested on v6.0 and v6.1

root@apalis-imx6-10774951:~# cat ov5640.sh
media-ctl -l "'ov5640 1-003c':0 -> 'imx6-mipi-csi2':0[1]"
media-ctl -l "'imx6-mipi-csi2':2 -> 'ipu1_csi1':0[1]"
media-ctl -l "'ipu1_csi1':2 -> 'ipu1_csi1 capture':0[1]"
# Configure pads
media-ctl -V "'ov5640 1-003c':0 [fmt:UYVY8_1X16/1920x1080 field:none]"
media-ctl -V "'imx6-mipi-csi2':2 [fmt:UYVY8_1X16/1920x1080 field:none]"
media-ctl -V "'ipu1_csi1':2 [fmt:UYVY8_1X16/1920x1080 field:none]"
Quoted 24 lines
Quoted 24 lines
Quoted 24 lines
Quoted 24 lines
Quoted 24 lines

Setting pipeline to PAUSED ...
[   57.082630] ov5640 1-003c: ov5640_write_reg: error: reg=3008, val=2
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Failed to allocate required memory. Additional debug info:
../gst-plugins-good-1.20.3/sys/v4l2/gstv4l2src.c(759):
gst_v4l2src_decide_allocation ():
/GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
Buffer pool activation failed
Execution ended after 0:00:00.119459642
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.
Setting pipeline to NULL ...
Additional debug info:
../gstreamer-1.20.3/libs/gst/base/gstbasesrc.c(3127): gst_base_src_loop(): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)
Freeing pipeline ...

root@apalis-imx6-10774951:~# v4l2-ctl --device /dev/video3 --stream-mmap
--stream-to=x.raw --stream-count=1
[ 1124.339793] ov5640 1-003c: ov5640_write_reg: error: reg=3008, val=2
            VIDIOC_STREAMON returned -1 (Broken pipe)
Quoted 1 line

--

Quoted 9 lines
Quoted 8 lines
Quoted 4 lines
Quoted 3 lines

Regards,

Laurent Pinchart

I'm grateful for any hints or comments?
Regards,
Aishwarya Kothari

Regards,
Aishwarya

#6 lore raw

Hello Aishwarya

Quoted 69 lines · Aishwarya Kothari
Quoted 68 lines · Jacopo Mondi
Quoted 67 lines · Aishwarya Kothari
Quoted 51 lines · Jacopo Mondi
Quoted 50 lines · Laurent Pinchart
Quoted 19 lines · Aishwarya Kothari

Hi,

I was trying ov5640 camera sensor with the mainline BSP 6.0.0-rc3 kernel and it fails to works, while it works quiet well on v5.18. When I bisect, it points out to this commit 1f391df4 (media: v4l2-async: Use endpoints in __v4l2_async_nf_add_fwnode_remote()) by Laurent Pinchart and this is what it says in the logs :

[    4.291355] imx-ipuv3-csi imx-ipuv3-csi.0: Registered ipu1_csi0 capture as /dev/video0 [    4.305228] imx-ipuv3 2400000.ipu: Registered ipu1_ic_prpenc capture as /dev/video1
[    4.317974] imx-ipuv3 2400000.ipu: Registered ipu1_ic_prpvf capture as /dev/video2
[    4.331039] imx-ipuv3-csi imx-ipuv3-csi.1: Registered ipu1_csi1 capture as /dev/video3 [    4.344255] imx-ipuv3-csi imx-ipuv3-csi.4: Registered ipu2_csi0 capture as /dev/video4 [    4.356940] imx-ipuv3 2800000.ipu: Registered ipu2_ic_prpenc capture as /dev/video5
[    4.369322] imx-ipuv3 2800000.ipu: Registered ipu2_ic_prpvf capture as /dev/video6
[    4.382119] imx-ipuv3-csi imx-ipuv3-csi.5: Registered ipu2_csi1 capture as /dev/video7 [    4.395455] imx6-mipi-csi2 21dc000.mipi: Consider updating driver imx6-mipi-csi2 to match on endpoints [    4.414971] stmpe-adc stmpe-adc: DMA mask not set

Quoted 6 lines

So, now when I try to revert the change done by this commit, it still fails. I assume there are a lot more changes following the commit that could lead to this failure. Below are the logs.
Is there a way to get it to work without reverting any changes? Or Is there something I'm missing or have to do any changes in the corresponding files to get it to work?

The commit you reference wasn't meant to break anything, so there's clearly something I've overlooked. Jacopo, have you noticed anything similar when working on the ov5640 driver ?

Quoted 18 lines

TDX Wayland with XWayland Upstream 6.1.0-devel-20221109+build.104 (kirkstone) apalis-imx6-10774951 ttymxc0 Apalis-iMX6_Reference-Multimedia-Image-upstream

apalis-imx6-10774951 login: root

root@apalis-imx6-10774951:~# cat ov5640.sh
media-ctl -l "'ov5640 1-003c':0 -> 'imx6-mipi-csi2':0[1]"
media-ctl -l "'imx6-mipi-csi2':2 -> 'ipu1_csi1':0[1]"
media-ctl -l "'ipu1_csi1':2 -> 'ipu1_csi1 capture':0[1]"
# Configure pads
media-ctl -V "'ov5640 1-003c':0 [fmt:UYVY2X8/1920x1080 field:none]"
media-ctl -V "'imx6-mipi-csi2':2 [fmt:UYVY2X8/1920x1080 field:none]"
media-ctl -V "'ipu1_csi1':2 [fmt:AYUV32/1920x1080 field:none]"
root@apalis-imx6-10774951:~# sh ov5640.sh
root@apalis-imx6-10774951:~# gst-launch-1.0 v4l2src device='/dev/video3'
! videoconvert ! waylandsink
[   56.031510] ov5640 1-003c: ov5640_write_reg: error: reg=3008, val=2
[   56.649032] ov5640 1-003c: ov5640_write_reg: error: reg=3008, val=2
Quoted 6 lines
Quoted 6 lines

This surprises me. I would have expected an issue with commit 1f391df44607 ("media: v4l2-async: Use endpoints in __v4l2_async_nf_add_fwnode_remote()") to prevent the ov5640 from being detected at all. Could you double-check the bisection, to verify that commit 229fac6c44dc works and commit 1f391df44607 doesn't ? What difference does it make in the full kernel log ?

Yes, the commit 229fac6c44dc works. This is the first commit 1f391df44607 ("media: v4l2-async: Use endpoints in __v4l2_async_nf_add_fwnode_remote()") that fails. The ov5640 driver is not detected and the /dev/media is not present.

This is the difference in dmesg:

dmesg_1f391df44607_nt renamed from dmesg_229fac6c44dc_nt +0 -0

[snip]

Quoted 22 lines
Quoted 22 lines
Quoted 22 lines
dmesg_1f391df44607_nt renamed from dmesg_229fac6c44dc_nt +4 -1
@@ -269,6 +269,7 @@
 imx-ipuv3 2800000.ipu: Registered ipu2_ic_prpenc capture as /dev/video5
 imx-ipuv3 2800000.ipu: Registered ipu2_ic_prpvf capture as /dev/video6
 imx-ipuv3-csi imx-ipuv3-csi.5: Registered ipu2_csi1 capture as /dev/video7
+imx6-mipi-csi2 21dc000.mipi: Consider updating driver imx6-mipi-csi2 to match on endpoints
 sgtl5000 2-000a: sgtl5000 revision 0x11
 random: crng init done
 imx6q-pcie 1ffc000.pcie: Phy link never came up
@@ -318,10 +319,12 @@
 usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice=5.18
 usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
 usb usb1: Product: EHCI Host Controller
-usb usb1: Manufacturer: Linux 5.18.0-rc3-00051-g229fac6c44dc ehci_hcd
+usb usb1: Manufacturer: Linux 5.18.0-rc3-00052-g1f391df44607 ehci_hcd
 usb usb1: SerialNumber: ci_hdrc.1
 hub 1-0:1.0: USB hub found
 hub 1-0:1.0: 1 port detected
+video-mux 20e0000.iomuxc-gpr:ipu1_csi0_mux: Consider updating driver video-mux to match on endpoints
+video-mux 20e0000.iomuxc-gpr:ipu2_csi1_mux: Consider updating driver video-mux to match on endpoints
 imx_thermal 20c8000.anatop:tempmon: Extended Commercial CPU temperature grade - max:105C critical:100C passive:95C
 fsl-ssi-dai 2028000.ssi: No cache defaults, reading back from HW
 fsl-ssi-dai 2028000.ssi: No cache defaults, reading back from HW

[snip]

Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
dmesg_1f391df44607_nt renamed from dmesg_229fac6c44dc_nt +1 -1
@@ -426,19 +429,15 @@
 ov5640 1-003c: supply DOVDD not found, using dummy regulator
 ov5640 1-003c: supply AVDD not found, using dummy regulator
 ov5640 1-003c: supply DVDD not found, using dummy regulator
-imx-media: Registered ipu_ic_pp csc/scaler as /dev/video12
+ov5640 1-003c: Consider updating driver ov5640 to match on endpoints
 fsl-ssi-dai 2028000.ssi: No cache defaults, reading back from HW
 using random self ethernet address
 using random host ethernet address

[snip]

Quoted 11 lines
Quoted 7 lines

dmesg_1f391df44607_nt

Quoted 7 lines

IPv6: ADDRCONF(NETDEV_CHANGE): usb0: link becomes ready
fec 2188000.ethernet eth0: Link is Up - 1Gbps/Full - flow control rx/tx
IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready

dmesg_1f391df44607_nt renamed from dmesg_229fac6c44dc_nt +0 -4
-ov5640 1-003c: ov5640_write_reg: error: reg=3008, val=2
-ov5640 1-003c: ov5640_write_reg: error: reg=3008, val=2
-ov5640 1-003c: ov5640_write_reg: error: reg=3008, val=2
-cfg80211: failed to load regulatory.db

Well, the commit 1f391df4 (media: v4l2-async: Use endpoints in __v4l2_async_nf_add_fwnode_remote()) is actually the one introducing regression. As mentioned before, /dev/media0 is not created.

I'll try to see if I can reproduce this. I don't have a board with an i.MX6 SoC, but it may be possible to reproduce the bug with different drivers.

Quoted 33 lines
Quoted 17 lines
Quoted 9 lines
Quoted 8 lines

If my understanding is correct, 1f391df44607 is unrelated as Aishwarya reports " So, now when I try to revert the change done by this commit, it still fails".

There is a long list of changes that went in with recent kernels for ov5640 but I'm afraid I'm now sure how many of them have landed on your BSP. Try to diff with a recent mainline, and then maybe bisect from there ?

Quoted 7 lines

We use the the recent mainline kernel without any changes. It stops working at 1f391df44607. With 1f391df44607 reverted, bisecting reveals that a89f14bbcfa5 ("media: ov5640: Split DVP and CSI-2 formats") introduces another regression. I'm still investigating what could be the problem. The format used below, it no longer being recognised:
media-ctl -V "'ov5640 1-003c':0 [fmt:UYVY2X8/1920x1080 field:none]" media-ctl -V "'imx6-mipi-csi2':2 [fmt:UYVY2X8/1920x1080 field:none]"

Quoted 3 lines

That should be easy. Just lookg at the a89f14bbcfa5 commit and you'll see that the 2X8 media bus variants are not supported anymore in MIPI mode. Just use UYVY1X16 everywhere and you should be good.

Thanks
Reverting the commit 1f391df4 (media: v4l2-async: Use endpoints in __v4l2_async_nf_add_fwnode_remote()) and setting the format as below works fine when tested on v6.0 and v6.1

root@apalis-imx6-10774951:~# cat ov5640.sh
media-ctl -l "'ov5640 1-003c':0 -> 'imx6-mipi-csi2':0[1]"
media-ctl -l "'imx6-mipi-csi2':2 -> 'ipu1_csi1':0[1]"
media-ctl -l "'ipu1_csi1':2 -> 'ipu1_csi1 capture':0[1]"
# Configure pads
media-ctl -V "'ov5640 1-003c':0 [fmt:UYVY8_1X16/1920x1080 field:none]"
media-ctl -V "'imx6-mipi-csi2':2 [fmt:UYVY8_1X16/1920x1080 field:none]"
media-ctl -V "'ipu1_csi1':2 [fmt:UYVY8_1X16/1920x1080 field:none]"

[snip]

Signature · 3 lines
Regards,

Laurent Pinchart
#7 lore raw
In reply to Aishwarya Kothari

Hello Aishwarya

Quoted 226 lines · Aishwarya Kothari

Hello Jacopo and Laurent,

Quoted 179 lines · Jacopo Mondi

Hello Aishwarya

Quoted 176 lines · Aishwarya Kothari

Hello,

Quoted 64 lines · Jacopo Mondi

Hello Aishwarya,

Quoted 61 lines · Laurent Pinchart

Hi Aishwarya,

(CC'ing Jacopo and Sakari)

Quoted 19 lines · Aishwarya Kothari

Hi,

I was trying ov5640 camera sensor with the mainline BSP 6.0.0-rc3 kernel and it fails to works, while it works quiet well on v5.18. When I bisect, it points out to this commit 1f391df4 (media: v4l2-async: Use endpoints in __v4l2_async_nf_add_fwnode_remote()) by Laurent Pinchart and this is what it says in the logs :

[    4.291355] imx-ipuv3-csi imx-ipuv3-csi.0: Registered ipu1_csi0 capture as /dev/video0 [    4.305228] imx-ipuv3 2400000.ipu: Registered ipu1_ic_prpenc capture as /dev/video1
[    4.317974] imx-ipuv3 2400000.ipu: Registered ipu1_ic_prpvf capture as /dev/video2
[    4.331039] imx-ipuv3-csi imx-ipuv3-csi.1: Registered ipu1_csi1 capture as /dev/video3 [    4.344255] imx-ipuv3-csi imx-ipuv3-csi.4: Registered ipu2_csi0 capture as /dev/video4 [    4.356940] imx-ipuv3 2800000.ipu: Registered ipu2_ic_prpenc capture as /dev/video5
[    4.369322] imx-ipuv3 2800000.ipu: Registered ipu2_ic_prpvf capture as /dev/video6
[    4.382119] imx-ipuv3-csi imx-ipuv3-csi.5: Registered ipu2_csi1 capture as /dev/video7 [    4.395455] imx6-mipi-csi2 21dc000.mipi: Consider updating driver imx6-mipi-csi2 to match on endpoints [    4.414971] stmpe-adc stmpe-adc: DMA mask not set

Quoted 6 lines

So, now when I try to revert the change done by this commit, it still fails. I assume there are a lot more changes following the commit that could lead to this failure. Below are the logs.
Is there a way to get it to work without reverting any changes? Or Is there something I'm missing or have to do any changes in the corresponding files to get it to work?

The commit you reference wasn't meant to break anything, so there's clearly something I've overlooked. Jacopo, have you noticed anything similar when working on the ov5640 driver ?

Quoted 5 lines

TDX Wayland with XWayland Upstream 6.1.0-devel-20221109+build.104 (kirkstone) apalis-imx6-10774951 ttymxc0 Apalis-iMX6_Reference-Multimedia-Image-upstream

apalis-imx6-10774951 login: root

Quoted 13 lines
root@apalis-imx6-10774951:~# cat ov5640.sh
media-ctl -l "'ov5640 1-003c':0 -> 'imx6-mipi-csi2':0[1]"
media-ctl -l "'imx6-mipi-csi2':2 -> 'ipu1_csi1':0[1]"
media-ctl -l "'ipu1_csi1':2 -> 'ipu1_csi1 capture':0[1]"
# Configure pads
media-ctl -V "'ov5640 1-003c':0 [fmt:UYVY2X8/1920x1080 field:none]"
media-ctl -V "'imx6-mipi-csi2':2 [fmt:UYVY2X8/1920x1080 field:none]"
media-ctl -V "'ipu1_csi1':2 [fmt:AYUV32/1920x1080 field:none]"
root@apalis-imx6-10774951:~# sh ov5640.sh
root@apalis-imx6-10774951:~# gst-launch-1.0 v4l2src device='/dev/video3'
! videoconvert ! waylandsink
[   56.031510] ov5640 1-003c: ov5640_write_reg: error: reg=3008, val=2
[   56.649032] ov5640 1-003c: ov5640_write_reg: error: reg=3008, val=2

This surprises me. I would have expected an issue with commit 1f391df44607 ("media: v4l2-async: Use endpoints in __v4l2_async_nf_add_fwnode_remote()") to prevent the ov5640 from being detected at all. Could you double-check the bisection, to verify that commit 229fac6c44dc works and commit 1f391df44607 doesn't ? What difference does it make in the full kernel log ?

Yes, the commit 229fac6c44dc works. This is the first commit 1f391df44607 ("media: v4l2-async: Use endpoints in __v4l2_async_nf_add_fwnode_remote()") that fails. The ov5640 driver is not detected and the /dev/media is not present.

This is the difference in dmesg:

dmesg_1f391df44607_nt renamed from dmesg_229fac6c44dc_nt +0 -1
@@ -1,5 +1,5 @@
  Booting Linux on physical CPU 0x0
-Linux version 5.18.0-rc3-00051-g229fac6c44dc (aishwarya@aishwarya-nb)
 (arm-none-linux-gnueabihf-gcc (Arm GNU Toolchain 11.3.Rel1) 11.3.1 20220712,

GNU ld (Arm GNU Toolchain 11.3.Rel1) 2.38.20220708) #28 SMP Wed Dec 21 11:13:19 CET 2022
+Linux version 5.18.0-rc3-00052-g1f391df44607 (aishwarya@aishwarya-nb) (arm-none-linux-gnueabihf-gcc (Arm GNU Toolchain 11.3.Rel1) 11.3.1 20220712, GNU ld (Arm GNU Toolchain 11.3.Rel1) 2.38.20220708) #29 SMP Wed Dec 21 11:21:31 CET 2022
CPU: ARMv7 Processor [412fc09a] revision 10 (ARMv7), cr=10c5387d
CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instruction cache
OF: fdt: Machine model: Toradex Apalis iMX6Q/D Module on Apalis Evaluation
Board

dmesg_1f391df44607_nt renamed from dmesg_229fac6c44dc_nt +1 -1
@@ -228,7 +228,7 @@
  SPI driver ads7846 has no spi_device_id for ti,ads7845
  SPI driver ads7846 has no spi_device_id for ti,ads7873
  rtc-ds1307 0-0068: registered as rtc0
-rtc-ds1307 0-0068: setting system clock to 2022-12-21T10:14:44 UTC
 (1671617684)
+rtc-ds1307 0-0068: setting system clock to 2022-12-21T10:22:46 UTC
 (1671618166)
  snvs_rtc 20cc000.snvs:snvs-rtc-lp: registered as rtc1

i2c_dev: i2c /dev entries driver
Bluetooth: HCI UART driver ver 2.3

dmesg_1f391df44607_nt renamed from dmesg_229fac6c44dc_nt +1 -0
@@ -269,6 +269,7 @@
  imx-ipuv3 2800000.ipu: Registered ipu2_ic_prpenc capture as /dev/video5
  imx-ipuv3 2800000.ipu: Registered ipu2_ic_prpvf capture as /dev/video6
  imx-ipuv3-csi imx-ipuv3-csi.5: Registered ipu2_csi1 capture as /dev/video7
+imx6-mipi-csi2 21dc000.mipi: Consider updating driver imx6-mipi-csi2 to

match on endpoints
sgtl5000 2-000a: sgtl5000 revision 0x11
random: crng init done
imx6q-pcie 1ffc000.pcie: Phy link never came up

dmesg_1f391df44607_nt renamed from dmesg_229fac6c44dc_nt +0 -0
@@ -318,10 +319,12 @@
  usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice=

5.18
usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 usb usb1: Product: EHCI Host Controller

dmesg_1f391df44607_nt renamed from dmesg_229fac6c44dc_nt +2 -1
-usb usb1: Manufacturer: Linux 5.18.0-rc3-00051-g229fac6c44dc ehci_hcd
+usb usb1: Manufacturer: Linux 5.18.0-rc3-00052-g1f391df44607 ehci_hcd
  usb usb1: SerialNumber: ci_hdrc.1
  hub 1-0:1.0: USB hub found
  hub 1-0:1.0: 1 port detected
+video-mux 20e0000.iomuxc-gpr:ipu1_csi0_mux: Consider updating driver

video-mux to match on endpoints
+video-mux 20e0000.iomuxc-gpr:ipu2_csi1_mux: Consider updating driver video-mux to match on endpoints
imx_thermal 20c8000.anatop:tempmon: Extended Commercial CPU temperature
grade - max:105C critical:100C passive:95C
fsl-ssi-dai 2028000.ssi: No cache defaults, reading back from HW fsl-ssi-dai 2028000.ssi: No cache defaults, reading back from HW

dmesg_1f391df44607_nt renamed from dmesg_229fac6c44dc_nt +0 -1
@@ -384,7 +387,7 @@
  systemd[1]: Mounting Kernel Debug File System...
  systemd[1]: Mounting Kernel Trace File System...
  systemd[1]: Mounting Temporary Directory /tmp...
-systemd[1]: Create List of Static Device Nodes was skipped because of a

failed condition check (ConditionFileNotEmpty=/lib/modules/5.18.0-rc3-00051-g229fac6c44dc/modules.devname). +systemd[1]: Create List of Static Device Nodes was skipped because of a
failed condition check (ConditionFileNotEmpty=/lib/modules/5.18.0-rc3-00052-g1f391df44607/modules.devname).
systemd[1]: Starting Load Kernel Module configfs...
systemd[1]: Starting Load Kernel Module drm...
systemd[1]: Starting Load Kernel Module fuse...

dmesg_1f391df44607_nt renamed from dmesg_229fac6c44dc_nt +4 -5
@@ -413,7 +416,7 @@
  systemd[1]: Starting Remount Root and Kernel File Systems...
  systemd[1]: Started Journal Service.
  EXT4-fs (mmcblk2p2): re-mounted. Quota mode: none.
-systemd-journald[217]: Received client request to flush runtime journal.
+systemd-journald[216]: Received client request to flush runtime journal.
  evbug: Connected device: input0 (gpio-keys at gpio-keys/input0)
  coda 2040000.vpu: Firmware code revision: 46076
  coda 2040000.vpu: Initialized CODA960.
@@ -426,19 +429,15 @@
  ov5640 1-003c: supply DOVDD not found, using dummy regulator
  ov5640 1-003c: supply AVDD not found, using dummy regulator
  ov5640 1-003c: supply DVDD not found, using dummy regulator
-imx-media: Registered ipu_ic_pp csc/scaler as /dev/video12
+ov5640 1-003c: Consider updating driver ov5640 to match on endpoints
  fsl-ssi-dai 2028000.ssi: No cache defaults, reading back from HW
  using random self ethernet address
  using random host ethernet address
-usb0: HOST MAC 26:b4:a4:fa:fe:15
-usb0: MAC 8e:e0:94:47:d3:6a
+usb0: HOST MAC 26:4d:ee:18:a3:b6
+usb0: MAC ce:ea:96:16:e2:a9
  Micrel KSZ9031 Gigabit PHY 2188000.ethernet-1:07: attached PHY driver
 (mii_bus:phy_addr=2188000.ethernet-1:07, irq=123)
-systemd-journald[217]: Data hash table of

/run/log/journal/6126bf14c3ea4128a506478d46ca2b38/system.journal has a fill level at 75.2 (2737 of 3640 items, 2097152 file size, 766 bytes per hash table item), suggesting rotation.
-systemd-journald[217]:
/run/log/journal/6126bf14c3ea4128a506478d46ca2b38/system.journal: Journal header limits reached or header out-of-date, rotating. +systemd-journald[216]: Data hash table of /run/log/journal/6126bf14c3ea4128a506478d46ca2b38/system.journal has a fill level at 75.1 (2733 of 3640 items, 2097152 file size, 767 bytes per hash table item), suggesting rotation.
+systemd-journald[216]:
/run/log/journal/6126bf14c3ea4128a506478d46ca2b38/system.journal: Journal header limits reached or header out-of-date, rotating.

Quoted 8 lines

dmesg_1f391df44607_nt

Quoted 8 lines

IPv6: ADDRCONF(NETDEV_CHANGE): usb0: link becomes ready
fec 2188000.ethernet eth0: Link is Up - 1Gbps/Full - flow control rx/tx
IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready

dmesg_1f391df44607_nt renamed from dmesg_229fac6c44dc_nt +0 -4
-ov5640 1-003c: ov5640_write_reg: error: reg=3008, val=2
-ov5640 1-003c: ov5640_write_reg: error: reg=3008, val=2
-ov5640 1-003c: ov5640_write_reg: error: reg=3008, val=2
-cfg80211: failed to load regulatory.db

Well, the commit 1f391df4 (media: v4l2-async: Use endpoints in __v4l2_async_nf_add_fwnode_remote()) is actually the one introducing regression. As mentioned before, /dev/media0 is not created.

Quoted 19 lines
Quoted 10 lines
Quoted 10 lines

If my understanding is correct, 1f391df44607 is unrelated as Aishwarya reports " So, now when I try to revert the change done by this commit, it still fails".

There is a long list of changes that went in with recent kernels for ov5640 but I'm afraid I'm now sure how many of them have landed on your BSP. Try to diff with a recent mainline, and then maybe bisect from there ?

Quoted 8 lines

We use the the recent mainline kernel without any changes. It stops working at 1f391df44607. With 1f391df44607 reverted, bisecting reveals that a89f14bbcfa5 ("media: ov5640: Split DVP and CSI-2 formats") introduces another regression. I'm still investigating what could be the problem. The format used below, it no longer being recognised:
media-ctl -V "'ov5640 1-003c':0 [fmt:UYVY2X8/1920x1080 field:none]" media-ctl -V "'imx6-mipi-csi2':2 [fmt:UYVY2X8/1920x1080 field:none]"

Quoted 7 lines

That should be easy. Just lookg at the a89f14bbcfa5 commit and you'll see that the 2X8 media bus variants are not supported anymore in MIPI mode. Just use UYVY1X16 everywhere and you should be good.

Thanks
j

Thanks
Reverting the commit 1f391df4 (media: v4l2-async: Use endpoints in __v4l2_async_nf_add_fwnode_remote()) and setting the format as below works fine when tested on v6.0 and v6.1

Quoted 9 lines
root@apalis-imx6-10774951:~# cat ov5640.sh
media-ctl -l "'ov5640 1-003c':0 -> 'imx6-mipi-csi2':0[1]"
media-ctl -l "'imx6-mipi-csi2':2 -> 'ipu1_csi1':0[1]"
media-ctl -l "'ipu1_csi1':2 -> 'ipu1_csi1 capture':0[1]"
# Configure pads
media-ctl -V "'ov5640 1-003c':0 [fmt:UYVY8_1X16/1920x1080 field:none]"
media-ctl -V "'imx6-mipi-csi2':2 [fmt:UYVY8_1X16/1920x1080 field:none]"
media-ctl -V "'ipu1_csi1':2 [fmt:UYVY8_1X16/1920x1080 field:none]"

I reproduced the issue by booting the most recent media tree master on an i.MX6Q board with an ov5640 sensor connected.

Looking at the list of pending async subdevices at the end of the boot sequence (thanks Laurent for letting me know such sysfs attribute existed :)

# cat /sys/kernel/debug/v4l2-async/pending_async_subdevices
ipu2_csi1_mux:
 [fwnode] dev=21dc000.mipi, node=/soc/bus@2100000/mipi@21dc000/port@4/endpoint
ipu1_csi0_mux:
 [fwnode] dev=21dc000.mipi, node=/soc/bus@2100000/mipi@21dc000/port@1/endpoint
imx6-mipi-csi2:
ipu2_csi1:
ipu2_csi0:
ipu1_csi1:
 [fwnode] dev=21dc000.mipi, node=/soc/bus@2100000/mipi@21dc000/port@2/endpoint
ipu1_csi0:
imx-media:
ov5640 2-003c:

it is clear that there are unsatisfied dependencies on subdevices that should be registered by the mipi-csi2 driver.

The mipi-csi2 DTS layout is the following
https://elixir.bootlin.com/linux/latest/source/arch/arm/boot/dts/imx6q.dtsi#L486
with 1 source port connected to the sensor and 4 sink ports with an endpoint each connected to the 2 muxes and the 2 IPU[1,2] CSI[0,1] (see "Figure 19-1. CSI2IPU gasket connectivity" of the IMX6DQRM TRM).

The CSI-2 driver registers one subdevice only
https://elixir.bootlin.com/linux/latest/source/drivers/staging/media/imx/imx6-mipi-csi2.c#L629
and with the logic implemented in v4l2_async for heterogeneous matching
https://elixir.bootlin.com/linux/latest/source/drivers/media/v4l2-core/v4l2-async.c#L79
the first driver that probes and that searches from an endpoint in the mipi-csi2 device node gets satisfied while the other 3 will be left pending. That's why you see "ipu2_csi0:" above having no pending subdevs, while "ipu1_csi1:", "ipu1_csi0_mux:" and "ipu2_csi1_mux:" have pending subdevices.

Before commit 1f391df4 (media: v4l2-async: Use endpoints in __v4l2_async_nf_add_fwnode_remote())" all the notifiers in ipu1_csi1, ipu1_csi0_mux, ipu2_csi0 and ipu2_csi1_mux pointed to the same device node "imx6-mipi-csi2". In order to register multiple notifiers with an asd that point the same device node you can see that both the imx-media-csi and video-mux implement a workaround in the form of:

		asd = v4l2_async_nf_add_fwnode_remote(&vmux->notifier, ep,
						      struct v4l2_async_subdev);

		fwnode_handle_put(ep);

		if (IS_ERR(asd)) {
			ret = PTR_ERR(asd);
			/* OK if asd already exists */
			if (ret != -EEXIST)
				return ret;
		}

https://elixir.bootlin.com/linux/latest/source/drivers/staging/media/imx/imx-media-csi.c#L1925
https://elixir.bootlin.com/linux/latest/source/drivers/media/platform/video-mux.c#L382

as registering multiple asd with on the same fwnode fails when adding the asd at:
https://elixir.bootlin.com/linux/latest/source/drivers/media/v4l2-core/v4l2-async.c#L459
(thanks again Laurent for spotting this 'workaround')

This can also be seen in the kernel logs when running with 1f391df4 reverted and debug enabled on v4l2-async:

[    3.735368] (NULL device *): subdev descriptor already listed in this or other notifiers
[    4.242167] (NULL device *): subdev descriptor already listed in this or other notifiers
[    4.245655] (NULL device *): subdev descriptor already listed in this or other notifiers

The end result is that a single notifier that points to the mipi_csi2 device node is registered instead of 4 notifiers as it happens when matching on endpoints. When the single notifier binds it registers links for all its sink pads (see the _bound callbacks in imx-media-csi and video-mux that call v4l2_create_fwnode_links() and v4l2_create_fwnode_links_to_pad() respectively) and you have a working media-device. Fragile at best as a design, but that's what we'll have to live with I'm afraid.

Now, how to fix this. I tried to think of a way to let the remote subdev decide if it has to be matched on endpoints or device node so that the decision is up to the mipi-csi2 driver. The alternative would be to manually add to the notifier an asd that points to the device node instead of using v4l2_async_nf_add_fwnode_remote(). This would be fine for the imx-mipi-csi component, as it is always paired the mipi-csi2 device so it can safely assume the remote has to be matched on device node. However the video-mux would have to be modified in the same way, and as it is a generic component we cannot make any assumption on the remote there...

Any ideas ?

Quoted 37 lines
Quoted 25 lines
Quoted 25 lines
Quoted 25 lines
Quoted 25 lines
Quoted 25 lines

Setting pipeline to PAUSED ...
[   57.082630] ov5640 1-003c: ov5640_write_reg: error: reg=3008, val=2
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Failed to allocate required memory. Additional debug info:
../gst-plugins-good-1.20.3/sys/v4l2/gstv4l2src.c(759):
gst_v4l2src_decide_allocation ():
/GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
Buffer pool activation failed
Execution ended after 0:00:00.119459642
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.
Setting pipeline to NULL ...
Additional debug info:
../gstreamer-1.20.3/libs/gst/base/gstbasesrc.c(3127): gst_base_src_loop(): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)
Freeing pipeline ...

root@apalis-imx6-10774951:~# v4l2-ctl --device /dev/video3 --stream-mmap
--stream-to=x.raw --stream-count=1
[ 1124.339793] ov5640 1-003c: ov5640_write_reg: error: reg=3008, val=2
            VIDIOC_STREAMON returned -1 (Broken pipe)
Quoted 1 line

--

Quoted 9 lines
Quoted 8 lines
Quoted 4 lines
Quoted 3 lines

Regards,

Laurent Pinchart

I'm grateful for any hints or comments?
Regards,
Aishwarya Kothari

Regards,
Aishwarya

#8 lore raw

Hi Jacopo,

Quoted 68 lines · Jacopo Mondi
Quoted 67 lines · Aishwarya Kothari
Quoted 66 lines · Jacopo Mondi
Quoted 65 lines · Aishwarya Kothari
Quoted 51 lines · Jacopo Mondi
Quoted 50 lines · Laurent Pinchart
Quoted 19 lines · Aishwarya Kothari

Hi,

I was trying ov5640 camera sensor with the mainline BSP 6.0.0-rc3 kernel and it fails to works, while it works quiet well on v5.18. When I bisect, it points out to this commit 1f391df4 (media: v4l2-async: Use endpoints in __v4l2_async_nf_add_fwnode_remote()) by Laurent Pinchart and this is what it says in the logs :

[    4.291355] imx-ipuv3-csi imx-ipuv3-csi.0: Registered ipu1_csi0 capture as /dev/video0 [    4.305228] imx-ipuv3 2400000.ipu: Registered ipu1_ic_prpenc capture as /dev/video1
[    4.317974] imx-ipuv3 2400000.ipu: Registered ipu1_ic_prpvf capture as /dev/video2
[    4.331039] imx-ipuv3-csi imx-ipuv3-csi.1: Registered ipu1_csi1 capture as /dev/video3 [    4.344255] imx-ipuv3-csi imx-ipuv3-csi.4: Registered ipu2_csi0 capture as /dev/video4 [    4.356940] imx-ipuv3 2800000.ipu: Registered ipu2_ic_prpenc capture as /dev/video5
[    4.369322] imx-ipuv3 2800000.ipu: Registered ipu2_ic_prpvf capture as /dev/video6
[    4.382119] imx-ipuv3-csi imx-ipuv3-csi.5: Registered ipu2_csi1 capture as /dev/video7 [    4.395455] imx6-mipi-csi2 21dc000.mipi: Consider updating driver imx6-mipi-csi2 to match on endpoints [    4.414971] stmpe-adc stmpe-adc: DMA mask not set

Quoted 6 lines

So, now when I try to revert the change done by this commit, it still fails. I assume there are a lot more changes following the commit that could lead to this failure. Below are the logs.
Is there a way to get it to work without reverting any changes? Or Is there something I'm missing or have to do any changes in the corresponding files to get it to work?

The commit you reference wasn't meant to break anything, so there's clearly something I've overlooked. Jacopo, have you noticed anything similar when working on the ov5640 driver ?

Quoted 18 lines

TDX Wayland with XWayland Upstream 6.1.0-devel-20221109+build.104 (kirkstone) apalis-imx6-10774951 ttymxc0 Apalis-iMX6_Reference-Multimedia-Image-upstream

apalis-imx6-10774951 login: root

root@apalis-imx6-10774951:~# cat ov5640.sh
media-ctl -l "'ov5640 1-003c':0 -> 'imx6-mipi-csi2':0[1]"
media-ctl -l "'imx6-mipi-csi2':2 -> 'ipu1_csi1':0[1]"
media-ctl -l "'ipu1_csi1':2 -> 'ipu1_csi1 capture':0[1]"
# Configure pads
media-ctl -V "'ov5640 1-003c':0 [fmt:UYVY2X8/1920x1080 field:none]"
media-ctl -V "'imx6-mipi-csi2':2 [fmt:UYVY2X8/1920x1080 field:none]"
media-ctl -V "'ipu1_csi1':2 [fmt:AYUV32/1920x1080 field:none]"
root@apalis-imx6-10774951:~# sh ov5640.sh
root@apalis-imx6-10774951:~# gst-launch-1.0 v4l2src device='/dev/video3'
! videoconvert ! waylandsink
[   56.031510] ov5640 1-003c: ov5640_write_reg: error: reg=3008, val=2
[   56.649032] ov5640 1-003c: ov5640_write_reg: error: reg=3008, val=2
Quoted 6 lines
Quoted 6 lines

This surprises me. I would have expected an issue with commit 1f391df44607 ("media: v4l2-async: Use endpoints in __v4l2_async_nf_add_fwnode_remote()") to prevent the ov5640 from being detected at all. Could you double-check the bisection, to verify that commit 229fac6c44dc works and commit 1f391df44607 doesn't ? What difference does it make in the full kernel log ?

Yes, the commit 229fac6c44dc works. This is the first commit 1f391df44607 ("media: v4l2-async: Use endpoints in __v4l2_async_nf_add_fwnode_remote()") that fails. The ov5640 driver is not detected and the /dev/media is not present.

This is the difference in dmesg:

[snip]

Quoted 134 lines
Quoted 30 lines

Well, the commit 1f391df4 (media: v4l2-async: Use endpoints in __v4l2_async_nf_add_fwnode_remote()) is actually the one introducing regression. As mentioned before, /dev/media0 is not created.

Quoted 17 lines
Quoted 9 lines
Quoted 8 lines

If my understanding is correct, 1f391df44607 is unrelated as Aishwarya reports " So, now when I try to revert the change done by this commit, it still fails".

There is a long list of changes that went in with recent kernels for ov5640 but I'm afraid I'm now sure how many of them have landed on your BSP. Try to diff with a recent mainline, and then maybe bisect from there ?

Quoted 7 lines

We use the the recent mainline kernel without any changes. It stops working at 1f391df44607. With 1f391df44607 reverted, bisecting reveals that a89f14bbcfa5 ("media: ov5640: Split DVP and CSI-2 formats") introduces another regression. I'm still investigating what could be the problem. The format used below, it no longer being recognised:
media-ctl -V "'ov5640 1-003c':0 [fmt:UYVY2X8/1920x1080 field:none]" media-ctl -V "'imx6-mipi-csi2':2 [fmt:UYVY2X8/1920x1080 field:none]"

Quoted 4 lines

That should be easy. Just lookg at the a89f14bbcfa5 commit and you'll see that the 2X8 media bus variants are not supported anymore in MIPI mode. Just use UYVY1X16 everywhere and you should be good.

Thanks
Reverting the commit 1f391df4 (media: v4l2-async: Use endpoints in __v4l2_async_nf_add_fwnode_remote()) and setting the format as below works fine when tested on v6.0 and v6.1

Quoted 8 lines
root@apalis-imx6-10774951:~# cat ov5640.sh
media-ctl -l "'ov5640 1-003c':0 -> 'imx6-mipi-csi2':0[1]"
media-ctl -l "'imx6-mipi-csi2':2 -> 'ipu1_csi1':0[1]"
media-ctl -l "'ipu1_csi1':2 -> 'ipu1_csi1 capture':0[1]"
# Configure pads
media-ctl -V "'ov5640 1-003c':0 [fmt:UYVY8_1X16/1920x1080 field:none]"
media-ctl -V "'imx6-mipi-csi2':2 [fmt:UYVY8_1X16/1920x1080 field:none]"
media-ctl -V "'ipu1_csi1':2 [fmt:UYVY8_1X16/1920x1080 field:none]"

I reproduced the issue by booting the most recent media tree master on an i.MX6Q board with an ov5640 sensor connected.

Looking at the list of pending async subdevices at the end of the boot sequence (thanks Laurent for letting me know such sysfs attribute existed :)

# cat /sys/kernel/debug/v4l2-async/pending_async_subdevices
ipu2_csi1_mux:
 [fwnode] dev=21dc000.mipi, node=/soc/bus@2100000/mipi@21dc000/port@4/endpoint
ipu1_csi0_mux:
 [fwnode] dev=21dc000.mipi, node=/soc/bus@2100000/mipi@21dc000/port@1/endpoint
imx6-mipi-csi2:
ipu2_csi1:
ipu2_csi0:
ipu1_csi1:
 [fwnode] dev=21dc000.mipi, node=/soc/bus@2100000/mipi@21dc000/port@2/endpoint
ipu1_csi0:
imx-media:
ov5640 2-003c:

it is clear that there are unsatisfied dependencies on subdevices that should be registered by the mipi-csi2 driver.

The mipi-csi2 DTS layout is the following
https://elixir.bootlin.com/linux/latest/source/arch/arm/boot/dts/imx6q.dtsi#L486
with 1 source port connected to the sensor and 4 sink ports with an endpoint each connected to the 2 muxes and the 2 IPU[1,2] CSI[0,1] (see "Figure 19-1. CSI2IPU gasket connectivity" of the IMX6DQRM TRM).

The CSI-2 driver registers one subdevice only
https://elixir.bootlin.com/linux/latest/source/drivers/staging/media/imx/imx6-mipi-csi2.c#L629
and with the logic implemented in v4l2_async for heterogeneous matching
https://elixir.bootlin.com/linux/latest/source/drivers/media/v4l2-core/v4l2-async.c#L79
the first driver that probes and that searches from an endpoint in the mipi-csi2 device node gets satisfied while the other 3 will be left pending. That's why you see "ipu2_csi0:" above having no pending subdevs, while "ipu1_csi1:", "ipu1_csi0_mux:" and "ipu2_csi1_mux:" have pending subdevices.

Before commit 1f391df4 (media: v4l2-async: Use endpoints in __v4l2_async_nf_add_fwnode_remote())" all the notifiers in ipu1_csi1, ipu1_csi0_mux, ipu2_csi0 and ipu2_csi1_mux pointed to the same device node "imx6-mipi-csi2". In order to register multiple notifiers with an asd that point the same device node you can see that both the imx-media-csi and video-mux implement a workaround in the form of:

		asd = v4l2_async_nf_add_fwnode_remote(&vmux->notifier, ep,
						      struct v4l2_async_subdev);

		fwnode_handle_put(ep);

		if (IS_ERR(asd)) {
			ret = PTR_ERR(asd);
			/* OK if asd already exists */
			if (ret != -EEXIST)
				return ret;
		}

https://elixir.bootlin.com/linux/latest/source/drivers/staging/media/imx/imx-media-csi.c#L1925
https://elixir.bootlin.com/linux/latest/source/drivers/media/platform/video-mux.c#L382

as registering multiple asd with on the same fwnode fails when adding the asd at:
https://elixir.bootlin.com/linux/latest/source/drivers/media/v4l2-core/v4l2-async.c#L459
(thanks again Laurent for spotting this 'workaround')

This can also be seen in the kernel logs when running with 1f391df4 reverted and debug enabled on v4l2-async:

[    3.735368] (NULL device *): subdev descriptor already listed in this or other notifiers
[    4.242167] (NULL device *): subdev descriptor already listed in this or other notifiers
[    4.245655] (NULL device *): subdev descriptor already listed in this or other notifiers

The end result is that a single notifier that points to the mipi_csi2 device node is registered instead of 4 notifiers as it happens when matching on endpoints. When the single notifier binds it registers links for all its sink pads (see the _bound callbacks in imx-media-csi and video-mux that call v4l2_create_fwnode_links() and v4l2_create_fwnode_links_to_pad() respectively) and you have a working media-device. Fragile at best as a design, but that's what we'll have to live with I'm afraid.

Now, how to fix this. I tried to think of a way to let the remote subdev decide if it has to be matched on endpoints or device node so that the decision is up to the mipi-csi2 driver. The alternative would be to manually add to the notifier an asd that points to the device node instead of using v4l2_async_nf_add_fwnode_remote(). This would be fine for the imx-mipi-csi component, as it is always paired the mipi-csi2 device so it can safely assume the remote has to be matched on device node. However the video-mux would have to be modified in the same way, and as it is a generic component we cannot make any assumption on the remote there...

Quoted 1 line

Any ideas ?

Not any easy one :-S

The video-mux messes it up, as it's the only component not part of the IPU, so we can't modify it with any hardcoded assumption regarding device vs. endpoint matching or introduce any hack specific to this situation. It really worked by chance before.

One option would be to add support for registering a subdev to the v4l2-async framework with multiple endpoint nodes, but I'm scared it would require lots of changes, and lots of work. I haven't looked at what unexpected consequences this would cause. Sakari, what do you think ?

Another option would be to replace the video-mux by an ad-hoc component specific to the IPU, but I don't think that's possible without DT changes. It seems that a standard video-mux driver was a bad idea after all :-S

Quoted 23 lines
Quoted 23 lines
Quoted 23 lines
Quoted 23 lines
Quoted 23 lines
Quoted 23 lines
Quoted 23 lines

Setting pipeline to PAUSED ...
[   57.082630] ov5640 1-003c: ov5640_write_reg: error: reg=3008, val=2
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Failed to allocate required memory. Additional debug info:
../gst-plugins-good-1.20.3/sys/v4l2/gstv4l2src.c(759):
gst_v4l2src_decide_allocation ():
/GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
Buffer pool activation failed
Execution ended after 0:00:00.119459642
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.
Setting pipeline to NULL ...
Additional debug info:
../gstreamer-1.20.3/libs/gst/base/gstbasesrc.c(3127): gst_base_src_loop(): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)
Freeing pipeline ...

root@apalis-imx6-10774951:~# v4l2-ctl --device /dev/video3 --stream-mmap
--stream-to=x.raw --stream-count=1
[ 1124.339793] ov5640 1-003c: ov5640_write_reg: error: reg=3008, val=2
            VIDIOC_STREAMON returned -1 (Broken pipe)
Signature · 3 lines
Regards,

Laurent Pinchart
#9 lore raw
Quoted 110 lines · Laurent Pinchart
Quoted 109 lines · Jacopo Mondi
Quoted 4 lines · Aishwarya Kothari

Thanks
Reverting the commit 1f391df4 (media: v4l2-async: Use endpoints in __v4l2_async_nf_add_fwnode_remote()) and setting the format as below works fine when tested on v6.0 and v6.1

Quoted 8 lines
root@apalis-imx6-10774951:~# cat ov5640.sh
media-ctl -l "'ov5640 1-003c':0 -> 'imx6-mipi-csi2':0[1]"
media-ctl -l "'imx6-mipi-csi2':2 -> 'ipu1_csi1':0[1]"
media-ctl -l "'ipu1_csi1':2 -> 'ipu1_csi1 capture':0[1]"
# Configure pads
media-ctl -V "'ov5640 1-003c':0 [fmt:UYVY8_1X16/1920x1080 field:none]"
media-ctl -V "'imx6-mipi-csi2':2 [fmt:UYVY8_1X16/1920x1080 field:none]"
media-ctl -V "'ipu1_csi1':2 [fmt:UYVY8_1X16/1920x1080 field:none]"

I reproduced the issue by booting the most recent media tree master on an i.MX6Q board with an ov5640 sensor connected.

Looking at the list of pending async subdevices at the end of the boot sequence (thanks Laurent for letting me know such sysfs attribute existed :)

# cat /sys/kernel/debug/v4l2-async/pending_async_subdevices
ipu2_csi1_mux:
 [fwnode] dev=21dc000.mipi, node=/soc/bus@2100000/mipi@21dc000/port@4/endpoint
ipu1_csi0_mux:
 [fwnode] dev=21dc000.mipi, node=/soc/bus@2100000/mipi@21dc000/port@1/endpoint
imx6-mipi-csi2:
ipu2_csi1:
ipu2_csi0:
ipu1_csi1:
 [fwnode] dev=21dc000.mipi, node=/soc/bus@2100000/mipi@21dc000/port@2/endpoint
ipu1_csi0:
imx-media:
ov5640 2-003c:

it is clear that there are unsatisfied dependencies on subdevices that should be registered by the mipi-csi2 driver.

The mipi-csi2 DTS layout is the following
https://elixir.bootlin.com/linux/latest/source/arch/arm/boot/dts/imx6q.dtsi#L486
with 1 source port connected to the sensor and 4 sink ports with an endpoint each connected to the 2 muxes and the 2 IPU[1,2] CSI[0,1] (see "Figure 19-1. CSI2IPU gasket connectivity" of the IMX6DQRM TRM).

The CSI-2 driver registers one subdevice only
https://elixir.bootlin.com/linux/latest/source/drivers/staging/media/imx/imx6-mipi-csi2.c#L629
and with the logic implemented in v4l2_async for heterogeneous matching
https://elixir.bootlin.com/linux/latest/source/drivers/media/v4l2-core/v4l2-async.c#L79
the first driver that probes and that searches from an endpoint in the mipi-csi2 device node gets satisfied while the other 3 will be left pending. That's why you see "ipu2_csi0:" above having no pending subdevs, while "ipu1_csi1:", "ipu1_csi0_mux:" and "ipu2_csi1_mux:" have pending subdevices.

Before commit 1f391df4 (media: v4l2-async: Use endpoints in __v4l2_async_nf_add_fwnode_remote())" all the notifiers in ipu1_csi1, ipu1_csi0_mux, ipu2_csi0 and ipu2_csi1_mux pointed to the same device node "imx6-mipi-csi2". In order to register multiple notifiers with an asd that point the same device node you can see that both the imx-media-csi and video-mux implement a workaround in the form of:

		asd = v4l2_async_nf_add_fwnode_remote(&vmux->notifier, ep,
						      struct v4l2_async_subdev);

		fwnode_handle_put(ep);

		if (IS_ERR(asd)) {
			ret = PTR_ERR(asd);
			/* OK if asd already exists */
			if (ret != -EEXIST)
				return ret;
		}

https://elixir.bootlin.com/linux/latest/source/drivers/staging/media/imx/imx-media-csi.c#L1925
https://elixir.bootlin.com/linux/latest/source/drivers/media/platform/video-mux.c#L382

as registering multiple asd with on the same fwnode fails when adding the asd at:
https://elixir.bootlin.com/linux/latest/source/drivers/media/v4l2-core/v4l2-async.c#L459
(thanks again Laurent for spotting this 'workaround')

This can also be seen in the kernel logs when running with 1f391df4 reverted and debug enabled on v4l2-async:

[    3.735368] (NULL device *): subdev descriptor already listed in this or other notifiers
[    4.242167] (NULL device *): subdev descriptor already listed in this or other notifiers
[    4.245655] (NULL device *): subdev descriptor already listed in this or other notifiers

The end result is that a single notifier that points to the mipi_csi2 device node is registered instead of 4 notifiers as it happens when matching on endpoints. When the single notifier binds it registers links for all its sink pads (see the _bound callbacks in imx-media-csi and video-mux that call v4l2_create_fwnode_links() and v4l2_create_fwnode_links_to_pad() respectively) and you have a working media-device. Fragile at best as a design, but that's what we'll have to live with I'm afraid.

Now, how to fix this. I tried to think of a way to let the remote subdev decide if it has to be matched on endpoints or device node so that the decision is up to the mipi-csi2 driver. The alternative would be to manually add to the notifier an asd that points to the device node instead of using v4l2_async_nf_add_fwnode_remote(). This would be fine for the imx-mipi-csi component, as it is always paired the mipi-csi2 device so it can safely assume the remote has to be matched on device node. However the video-mux would have to be modified in the same way, and as it is a generic component we cannot make any assumption on the remote there...

Quoted 3 lines
Quoted 1 line

Any ideas ?

Not any easy one :-S

Wouldn't be the case that we should revert that change till we have a proper solution in place given that is not going to be an easy fix? In the end no matter the reason it was working before, this is just a regression.

Francesco

#10 lore raw
Quoted 111 lines · Francesco Dolcini
Quoted 110 lines · Laurent Pinchart
Quoted 109 lines · Jacopo Mondi
Quoted 4 lines · Aishwarya Kothari

Thanks
Reverting the commit 1f391df4 (media: v4l2-async: Use endpoints in __v4l2_async_nf_add_fwnode_remote()) and setting the format as below works fine when tested on v6.0 and v6.1

Quoted 8 lines
root@apalis-imx6-10774951:~# cat ov5640.sh
media-ctl -l "'ov5640 1-003c':0 -> 'imx6-mipi-csi2':0[1]"
media-ctl -l "'imx6-mipi-csi2':2 -> 'ipu1_csi1':0[1]"
media-ctl -l "'ipu1_csi1':2 -> 'ipu1_csi1 capture':0[1]"
# Configure pads
media-ctl -V "'ov5640 1-003c':0 [fmt:UYVY8_1X16/1920x1080 field:none]"
media-ctl -V "'imx6-mipi-csi2':2 [fmt:UYVY8_1X16/1920x1080 field:none]"
media-ctl -V "'ipu1_csi1':2 [fmt:UYVY8_1X16/1920x1080 field:none]"

I reproduced the issue by booting the most recent media tree master on an i.MX6Q board with an ov5640 sensor connected.

Looking at the list of pending async subdevices at the end of the boot sequence (thanks Laurent for letting me know such sysfs attribute existed :)

# cat /sys/kernel/debug/v4l2-async/pending_async_subdevices
ipu2_csi1_mux:
 [fwnode] dev=21dc000.mipi, node=/soc/bus@2100000/mipi@21dc000/port@4/endpoint
ipu1_csi0_mux:
 [fwnode] dev=21dc000.mipi, node=/soc/bus@2100000/mipi@21dc000/port@1/endpoint
imx6-mipi-csi2:
ipu2_csi1:
ipu2_csi0:
ipu1_csi1:
 [fwnode] dev=21dc000.mipi, node=/soc/bus@2100000/mipi@21dc000/port@2/endpoint
ipu1_csi0:
imx-media:
ov5640 2-003c:

it is clear that there are unsatisfied dependencies on subdevices that should be registered by the mipi-csi2 driver.

The mipi-csi2 DTS layout is the following
https://elixir.bootlin.com/linux/latest/source/arch/arm/boot/dts/imx6q.dtsi#L486
with 1 source port connected to the sensor and 4 sink ports with an endpoint each connected to the 2 muxes and the 2 IPU[1,2] CSI[0,1] (see "Figure 19-1. CSI2IPU gasket connectivity" of the IMX6DQRM TRM).

The CSI-2 driver registers one subdevice only
https://elixir.bootlin.com/linux/latest/source/drivers/staging/media/imx/imx6-mipi-csi2.c#L629
and with the logic implemented in v4l2_async for heterogeneous matching
https://elixir.bootlin.com/linux/latest/source/drivers/media/v4l2-core/v4l2-async.c#L79
the first driver that probes and that searches from an endpoint in the mipi-csi2 device node gets satisfied while the other 3 will be left pending. That's why you see "ipu2_csi0:" above having no pending subdevs, while "ipu1_csi1:", "ipu1_csi0_mux:" and "ipu2_csi1_mux:" have pending subdevices.

Before commit 1f391df4 (media: v4l2-async: Use endpoints in __v4l2_async_nf_add_fwnode_remote())" all the notifiers in ipu1_csi1, ipu1_csi0_mux, ipu2_csi0 and ipu2_csi1_mux pointed to the same device node "imx6-mipi-csi2". In order to register multiple notifiers with an asd that point the same device node you can see that both the imx-media-csi and video-mux implement a workaround in the form of:

		asd = v4l2_async_nf_add_fwnode_remote(&vmux->notifier, ep,
						      struct v4l2_async_subdev);

		fwnode_handle_put(ep);

		if (IS_ERR(asd)) {
			ret = PTR_ERR(asd);
			/* OK if asd already exists */
			if (ret != -EEXIST)
				return ret;
		}

https://elixir.bootlin.com/linux/latest/source/drivers/staging/media/imx/imx-media-csi.c#L1925
https://elixir.bootlin.com/linux/latest/source/drivers/media/platform/video-mux.c#L382

as registering multiple asd with on the same fwnode fails when adding the asd at:
https://elixir.bootlin.com/linux/latest/source/drivers/media/v4l2-core/v4l2-async.c#L459
(thanks again Laurent for spotting this 'workaround')

This can also be seen in the kernel logs when running with 1f391df4 reverted and debug enabled on v4l2-async:

[    3.735368] (NULL device *): subdev descriptor already listed in this or other notifiers
[    4.242167] (NULL device *): subdev descriptor already listed in this or other notifiers
[    4.245655] (NULL device *): subdev descriptor already listed in this or other notifiers

The end result is that a single notifier that points to the mipi_csi2 device node is registered instead of 4 notifiers as it happens when matching on endpoints. When the single notifier binds it registers links for all its sink pads (see the _bound callbacks in imx-media-csi and video-mux that call v4l2_create_fwnode_links() and v4l2_create_fwnode_links_to_pad() respectively) and you have a working media-device. Fragile at best as a design, but that's what we'll have to live with I'm afraid.

Now, how to fix this. I tried to think of a way to let the remote subdev decide if it has to be matched on endpoints or device node so that the decision is up to the mipi-csi2 driver. The alternative would be to manually add to the notifier an asd that points to the device node instead of using v4l2_async_nf_add_fwnode_remote(). This would be fine for the imx-mipi-csi component, as it is always paired the mipi-csi2 device so it can safely assume the remote has to be matched on device node. However the video-mux would have to be modified in the same way, and as it is a generic component we cannot make any assumption on the remote there...

Quoted 8 lines
Quoted 3 lines
Quoted 1 line

Any ideas ?

Not any easy one :-S

Wouldn't be the case that we should revert that change till we have a proper solution in place given that is not going to be an easy fix? In the end no matter the reason it was working before, this is just a regression.

I would have agreed if there wasn't a high risk that will cause regressions on other platforms that now depend on this change :-( We're thus trading one regression for another one, which may require biting the bullet and handling this properly.

Signature · 3 lines
Regards,

Laurent Pinchart
#11 lore raw
In reply to Francesco Dolcini

Hi Francesco,

Quoted 111 lines · Francesco Dolcini
Quoted 110 lines · Laurent Pinchart
Quoted 109 lines · Jacopo Mondi
Quoted 4 lines · Aishwarya Kothari

Thanks
Reverting the commit 1f391df4 (media: v4l2-async: Use endpoints in __v4l2_async_nf_add_fwnode_remote()) and setting the format as below works fine when tested on v6.0 and v6.1

Quoted 8 lines
root@apalis-imx6-10774951:~# cat ov5640.sh
media-ctl -l "'ov5640 1-003c':0 -> 'imx6-mipi-csi2':0[1]"
media-ctl -l "'imx6-mipi-csi2':2 -> 'ipu1_csi1':0[1]"
media-ctl -l "'ipu1_csi1':2 -> 'ipu1_csi1 capture':0[1]"
# Configure pads
media-ctl -V "'ov5640 1-003c':0 [fmt:UYVY8_1X16/1920x1080 field:none]"
media-ctl -V "'imx6-mipi-csi2':2 [fmt:UYVY8_1X16/1920x1080 field:none]"
media-ctl -V "'ipu1_csi1':2 [fmt:UYVY8_1X16/1920x1080 field:none]"

I reproduced the issue by booting the most recent media tree master on an i.MX6Q board with an ov5640 sensor connected.

Looking at the list of pending async subdevices at the end of the boot sequence (thanks Laurent for letting me know such sysfs attribute existed :)

# cat /sys/kernel/debug/v4l2-async/pending_async_subdevices
ipu2_csi1_mux:
 [fwnode] dev=21dc000.mipi, node=/soc/bus@2100000/mipi@21dc000/port@4/endpoint
ipu1_csi0_mux:
 [fwnode] dev=21dc000.mipi, node=/soc/bus@2100000/mipi@21dc000/port@1/endpoint
imx6-mipi-csi2:
ipu2_csi1:
ipu2_csi0:
ipu1_csi1:
 [fwnode] dev=21dc000.mipi, node=/soc/bus@2100000/mipi@21dc000/port@2/endpoint
ipu1_csi0:
imx-media:
ov5640 2-003c:

it is clear that there are unsatisfied dependencies on subdevices that should be registered by the mipi-csi2 driver.

The mipi-csi2 DTS layout is the following
https://elixir.bootlin.com/linux/latest/source/arch/arm/boot/dts/imx6q.dtsi#L486
with 1 source port connected to the sensor and 4 sink ports with an endpoint each connected to the 2 muxes and the 2 IPU[1,2] CSI[0,1] (see "Figure 19-1. CSI2IPU gasket connectivity" of the IMX6DQRM TRM).

The CSI-2 driver registers one subdevice only
https://elixir.bootlin.com/linux/latest/source/drivers/staging/media/imx/imx6-mipi-csi2.c#L629
and with the logic implemented in v4l2_async for heterogeneous matching
https://elixir.bootlin.com/linux/latest/source/drivers/media/v4l2-core/v4l2-async.c#L79
the first driver that probes and that searches from an endpoint in the mipi-csi2 device node gets satisfied while the other 3 will be left pending. That's why you see "ipu2_csi0:" above having no pending subdevs, while "ipu1_csi1:", "ipu1_csi0_mux:" and "ipu2_csi1_mux:" have pending subdevices.

Before commit 1f391df4 (media: v4l2-async: Use endpoints in __v4l2_async_nf_add_fwnode_remote())" all the notifiers in ipu1_csi1, ipu1_csi0_mux, ipu2_csi0 and ipu2_csi1_mux pointed to the same device node "imx6-mipi-csi2". In order to register multiple notifiers with an asd that point the same device node you can see that both the imx-media-csi and video-mux implement a workaround in the form of:

		asd = v4l2_async_nf_add_fwnode_remote(&vmux->notifier, ep,
						      struct v4l2_async_subdev);

		fwnode_handle_put(ep);

		if (IS_ERR(asd)) {
			ret = PTR_ERR(asd);
			/* OK if asd already exists */
			if (ret != -EEXIST)
				return ret;
		}

https://elixir.bootlin.com/linux/latest/source/drivers/staging/media/imx/imx-media-csi.c#L1925
https://elixir.bootlin.com/linux/latest/source/drivers/media/platform/video-mux.c#L382

as registering multiple asd with on the same fwnode fails when adding the asd at:
https://elixir.bootlin.com/linux/latest/source/drivers/media/v4l2-core/v4l2-async.c#L459
(thanks again Laurent for spotting this 'workaround')

This can also be seen in the kernel logs when running with 1f391df4 reverted and debug enabled on v4l2-async:

[    3.735368] (NULL device *): subdev descriptor already listed in this or other notifiers
[    4.242167] (NULL device *): subdev descriptor already listed in this or other notifiers
[    4.245655] (NULL device *): subdev descriptor already listed in this or other notifiers

The end result is that a single notifier that points to the mipi_csi2 device node is registered instead of 4 notifiers as it happens when matching on endpoints. When the single notifier binds it registers links for all its sink pads (see the _bound callbacks in imx-media-csi and video-mux that call v4l2_create_fwnode_links() and v4l2_create_fwnode_links_to_pad() respectively) and you have a working media-device. Fragile at best as a design, but that's what we'll have to live with I'm afraid.

Now, how to fix this. I tried to think of a way to let the remote subdev decide if it has to be matched on endpoints or device node so that the decision is up to the mipi-csi2 driver. The alternative would be to manually add to the notifier an asd that points to the device node instead of using v4l2_async_nf_add_fwnode_remote(). This would be fine for the imx-mipi-csi component, as it is always paired the mipi-csi2 device so it can safely assume the remote has to be matched on device node. However the video-mux would have to be modified in the same way, and as it is a generic component we cannot make any assumption on the remote there...

Quoted 8 lines
Quoted 3 lines
Quoted 1 line

Any ideas ?

Not any easy one :-S

Wouldn't be the case that we should revert that change till we have a proper solution in place given that is not going to be an easy fix? In the end no matter the reason it was working before, this is just a regression.

V4L2 async has never supported

1. multiple links from an async sub-device or

2. cases where a sub-notifier is a child of more than one parent notifier.

Device dependencies that exist in this hardware would seem to require adding support for these in V4L2 async.

Signature · 3 lines
Kind regards,

Sakari Ailus