RockPro65 RK3399 chipset, ISP and IMX214 camera
Hi everyone,
+ Ricardo and Heiko in CC as the driver originators and rockchip pros...
I'm reaching out to you based on a discussion with Sebastian Fricke, who was working on the OV13850 driver for the v5 kernel. I tried getting the IMX214 finally to work on the RockPro64 from Pine64, which only works on Android so far and I need to get that done on Mainline Linux (I did not find anyone who managed that and reported about it). However, I'm also totally stuck.
The RockPro64 runs Dietpi, which is essentially Armbian + a few tweaks.
I'm using the Armbian 5.15.18 kernel, so mainline, with a custom device tree, which in the first place powers the MIPI ports. I suspect it is a bad idea to have one pinctrl as a placeholder for 4 converters, however, I'm not too deep into proxying in the devicetree, so here's the current status:
The camera is connected to the first MIPI port. The kernel boot logs look ok to me (except for the cyclic dependency issue, but I think that does not matter much).
Without the camera configured in the device tree, it shows up as 0x0c device on the #1 I2C bus, which is a bit suspicious to me given the addresses in the documentation and the info given in the kernel documentation.
However, I essentially followed the description according to this guide to set up the RKISP:
https://linuxtv.org/downloads/v4l-dvb-apis/admin-guide/rkisp1.html
using this:
It looks like this (see also png attached).
However, the IMX214 driver does not complain until that point and seems to do right. I expected that at least something happens, however it does not. The last command does this:
VIDIOC_STREAMON returned -1 (No such device or address) and this is the kernel output
[1509.435228] imx214 1-000c: write_table error: -6 [1509.435868] imx214 1-000c: could not sent common table -6
There is no more info given, even if I do some "echo 0x3F > /sys/class/video4linux/v4l-subdev0/dev_debug" to the subdevs.
Here's the IMX214 documentation btw. that I got through a detour from CSDN.
https://www.dropbox.com/sh/5d3mp2akr3kmu7t/AADaAsSxZu2kVSIfEceStwuoa?dl=0
I'm not entirely sure, but there is something wrong somewhere and I can't find out if it is with the driver, the RKISP or anything else. Here's what "v4l2-ctl -d /dev/v4l-subdev3 --all" gives - not sure but shouldn't it show supported resolutions or something?
Any help would be appreciated.
Best
Clemens
Hi Clemens,
Quoting Clemens Arth (2022-02-23 18:36:28)
Quoted 11 lines
Hi everyone,
+ Ricardo and Heiko in CC as the driver originators and rockchip pros...
I'm reaching out to you based on a discussion with Sebastian Fricke, who was working on the OV13850 driver for the v5 kernel. I tried getting the IMX214 finally to work on the RockPro64 from Pine64, which only works on Android so far and I need to get that done on Mainline Linux (I did not find anyone who managed that and reported about it). However, I'm also totally stuck.
Quoted 8 lines
The RockPro64 runs Dietpi, which is essentially Armbian + a few tweaks. I'm using the Armbian 5.15.18 kernel, so mainline, with a custom device tree, which in the first place powers the MIPI ports. I suspect it is a bad idea to have one pinctrl as a placeholder for 4 converters, however, I'm not too deep into proxying in the devicetree, so here's the current status:
Your regulators are all listed as fixed-regulators. Are you sure there's nothing else to turn on ? I expect this was from another fragement for the same platform? So I hope it's consistent.
Can you validate that the enable-gpios definition is to the correct GPIO to enable the camera ?
Quoted 28 lines
The camera is connected to the first MIPI port. The kernel boot logs look ok to me (except for the cyclic dependency issue, but I think that does not matter much).
Without the camera configured in the device tree, it shows up as
0x0cdevice on the #1 I2C bus, which is a bit suspicious to me given the addresses in the documentation and the info given in the kernel documentation.However, I essentially followed the description according to this guide to set up the RKISP:
https://linuxtv.org/downloads/v4l-dvb-apis/admin-guide/rkisp1.html
using this:
It looks like this (see also png attached).
However, the IMX214 driver does not complain until that point and seems to do right. I expected that at least something happens, however it does not. The last command does this:
Quoted 5 lines
VIDIOC_STREAMONreturned -1 (No such device or address) and this is the kernel output[1509.435228] imx214 1-000c: write_table error: -6 [1509.435868] imx214 1-000c: could not sent common table -6
-6 is ENXIO 6 No such device or address. So I expect the device isn't responding to the I2C controller.
What shows up with i2c-detect -r -y 1 ?
Quoted 5 lines
There is no more info given, even if I do some "echo
0x3F>/sys/class/video4linux/v4l-subdev0/dev_debug" to the subdevs.Here's the IMX214 documentation btw. that I got through a detour from CSDN.
Quoted 8 lines
https://www.dropbox.com/sh/5d3mp2akr3kmu7t/AADaAsSxZu2kVSIfEceStwuoa?dl=0
I'm not entirely sure, but there is something wrong somewhere and I can't find out if it is with the driver, the RKISP or anything else. Here's what "v4l2-ctl -d
/dev/v4l-subdev3--all" gives - not sure but shouldn't it show supported resolutions or something?
You might find it useful to check what is missing to support libcamera with this sensor, then you can use cam/qcam to test it too. The RKISP1 pipeline handler in libcamera will handle all the media controller configuration, and identifying the available formats for you, but we haven't had an IMX214 sensor added yet, so you might need to add a mapping to the src/libcamera/camera_sensor_properties.cpp sensor database, and the driver is missing at least V4L2_CID_HBLANK and V4L2_CID_VBLANK that are required for libcamera. So it might not be as straightforward as I'd like, but it would be helpful I expect.
But ... I think your issues are more likely underlying hardware or DT issues, as the device sounds like it's not responding on the i2c address.
Sometimes I2C devices have a configurable address, can you check if this really is the correct I2C address for your camera?
Quoted 1 line
Any help would be appreciated.
Regards
Kieran
Quoted 3 lines
Best
Clemens
A message that is not in the archive (sent off-list or not yet received): CAPuf0ENRRjMafZUOXS45PJxQrpcK_tdCRREoHn43t54pSbVhDg@mail.gmail.com
Am Donnerstag, 24. Februar 2022, 16:10:06 CET schrieb Clemens Arth:
Quoted 41 lines
Hi Kieran,
Thx, I’m on my mobile now so I hope copy pasting works… apologies for typos…
Kieran Bingham <[email protected]> schrieb am Do. 24. Feb. 2022 um 13:49:
Quoted 21 lines
Hi Clemens,
Quoting Clemens Arth (2022-02-23 18:36:28)
Quoted 18 lines
Hi everyone,
+ Ricardo and Heiko in CC as the driver originators and rockchip pros...
I'm reaching out to you based on a discussion with Sebastian Fricke, who was working on the OV13850 driver for the v5 kernel. I tried getting the IMX214 finally to work on the RockPro64 from Pine64, which only works on Android so far and I need to get that done on Mainline Linux (I did not find anyone who managed that and reported about it). However, I'm also totally stuck.
The RockPro64 runs Dietpi, which is essentially Armbian + a few tweaks. I'm using the Armbian 5.15.18 kernel, so mainline, with a custom device tree, which in the first place powers the MIPI ports. I suspect it is a bad idea to have one pinctrl as a placeholder for 4 converters, however, I'm not too deep into proxying in the devicetree, so here's the current status:
Quoted 6 lines
Quoted 1 line
Your regulators are all listed as fixed-regulators. Are you sure there's nothing else to turn on ? I expect this was from another fragement for the same platform? So I hope it's consistent.
The schematics are here, from which I took the regulator and gpio config. The regulators all seem to be fixed ones. There is just one pin that pulls up all the regulators (
DVP_PWR).https://files.pine64.org/doc/rockpro64/rockpro64_v21-SCH.pdf
Quoted 7 lines
The IMX214 driver has only one pin, the
enable_pin, but it is somewhat different from the IMX219 for example. Looking at both driver's code I believe what is thereset_pinwith the IMX219 is theenable_pinwith the IMX214, but I am not sure about that. There is no MIPI reset on the RockPro64 afaik. Therefore I think it needs to be wired to theDVP_PDN0_Hpin, but other drivers define that one specifically and it apparently does something different.
For the pins also check the direction (active_low / active_high). I remember having fun somewhere when changing between the vendor kernel and mainline.
Quoted 88 lines
Quoted 34 lines
Can you validate that the enable-gpios definition is to the correct GPIO to enable the camera ?
Quoted 31 lines
The camera is connected to the first MIPI port. The kernel boot logs look ok to me (except for the cyclic dependency issue, but I think that does not matter much).
Without the camera configured in the device tree, it shows up as
0x0cdevice on the #1 I2C bus, which is a bit suspicious to me given the addresses in the documentation and the info given in the kernel documentation.However, I essentially followed the description according to this guide to set up the RKISP:
https://linuxtv.org/downloads/v4l-dvb-apis/admin-guide/rkisp1.html
using this:
It looks like this (see also png attached).
However, the IMX214 driver does not complain until that point and seems to do right. I expected that at least something happens, however it does not. The last command does this:
VIDIOC_STREAMONreturned -1 (No such device or address) and this is the kernel outputQuoted 8 lines
Quoted 2 lines
[1509.435228] imx214 1-000c: write_table error: -6 [1509.435868] imx214 1-000c: could not sent common table -6-6 is ENXIO 6 No such device or address. So I expect the device isn't responding to the I2C controller.
What shows up with i2c-detect -r -y 1 ?
From the top of my head, it shows 1c on the
0x0caddress iirc, but only if I remove the IMX from the DT. Otherwise the driver takes over and it shows UU. I removed the camera physically and it was gone on i2cdetect, so I suspect that it indeed is the camera. From the driver and the documentation I need to configure it 4-lane, as it is hardcoded in the driver (compared to the application notes for registers for the ImX214).Quoted 17 lines
Quoted 5 lines
There is no more info given, even if I do some "echo
0x3F>/sys/class/video4linux/v4l-subdev0/dev_debug" to the subdevs.Here's the IMX214 documentation btw. that I got through a detour from
CSDN.
Quoted 2 lines
https://www.dropbox.com/sh/5d3mp2akr3kmu7t/AADaAsSxZu2kVSIfEceStwuoa?dl=0
Quoted 7 lines
I'm not entirely sure, but there is something wrong somewhere and I can't find out if it is with the driver, the RKISP or anything else. Here's what "v4l2-ctl -d
/dev/v4l-subdev3--all" gives - not sure but shouldn't it show supported resolutions or something?Quoted 9 lines
You might find it useful to check what is missing to support libcamera with this sensor, then you can use cam/qcam to test it too. The RKISP1 pipeline handler in libcamera will handle all the media controller configuration, and identifying the available formats for you, but we haven't had an IMX214 sensor added yet, so you might need to add a mapping to the src/libcamera/camera_sensor_properties.cpp sensor database, and the driver is missing at least
V4L2_CID_HBLANKandV4L2_CID_VBLANKthat are required for libcamera. So it might not be as straightforward as I'd like, but it would be helpful I expect.I tried that at an earlier stage, to no avail unfortunately. But I will try again as soon as I get back to my desk.
Quoted 6 lines
But ... I think your issues are more likely underlying hardware or DT issues, as the device sounds like it's not responding on the i2c address.
Quoted 6 lines
Quoted 4 lines
Sometimes I2C devices have a configurable address, can you check if this really is the correct I2C address for your camera?
That’s one of the issues. Ricardo wrote about iirc
0x10and0x1a, but theapp note says something entirely different (forgot), and my camera appears to be on yet another address…
not specific to cameras, but in the past I had i2c devices that set the address depending on the state of a gpio during powerup - which was floating in my old case, producing random settings ;-) .
Heiko
Hi
I think Heiko and Kieran have already given you a lot of clues.
I would recommend to start looking at i2c communication with i2c
tools: i2cget, i2set, i2cdetect.... before trying any video operation.
Your life will be much easier if you get your hands into a logic analyser like this one https://www.saleae.com/
Regarding the i2c address, bear in mind that vendors and Linux might use different nomenclature (7 bits to 8 bits).
Good luck!
Quoted 159 lines · Heiko Stuebner
Am Donnerstag, 24. Februar 2022, 16:10:06 CET schrieb Clemens Arth:
Quoted 41 lines
Hi Kieran,
Thx, I’m on my mobile now so I hope copy pasting works… apologies for typos…
Kieran Bingham <[email protected]> schrieb am Do. 24. Feb. 2022 um 13:49:
Quoted 21 lines
Hi Clemens,
Quoting Clemens Arth (2022-02-23 18:36:28)
Quoted 18 lines
Hi everyone,
+ Ricardo and Heiko in CC as the driver originators and rockchip pros...
I'm reaching out to you based on a discussion with Sebastian Fricke, who was working on the OV13850 driver for the v5 kernel. I tried getting the IMX214 finally to work on the RockPro64 from Pine64, which only works on Android so far and I need to get that done on Mainline Linux (I did not find anyone who managed that and reported about it). However, I'm also totally stuck.
The RockPro64 runs Dietpi, which is essentially Armbian + a few tweaks. I'm using the Armbian 5.15.18 kernel, so mainline, with a custom device tree, which in the first place powers the MIPI ports. I suspect it is a bad idea to have one pinctrl as a placeholder for 4 converters, however, I'm not too deep into proxying in the devicetree, so here's the current status:
Quoted 6 lines
Quoted 1 line
Your regulators are all listed as fixed-regulators. Are you sure there's nothing else to turn on ? I expect this was from another fragement for the same platform? So I hope it's consistent.
The schematics are here, from which I took the regulator and gpio config. The regulators all seem to be fixed ones. There is just one pin that pulls up all the regulators (
DVP_PWR).https://files.pine64.org/doc/rockpro64/rockpro64_v21-SCH.pdf
Quoted 7 lines
The IMX214 driver has only one pin, the
enable_pin, but it is somewhat different from the IMX219 for example. Looking at both driver's code I believe what is thereset_pinwith the IMX219 is theenable_pinwith the IMX214, but I am not sure about that. There is no MIPI reset on the RockPro64 afaik. Therefore I think it needs to be wired to theDVP_PDN0_Hpin, but other drivers define that one specifically and it apparently does something different.For the pins also check the direction (
active_low/active_high). I remember having fun somewhere when changing between the vendor kernel and mainline.Quoted 88 lines
Quoted 34 lines
Can you validate that the enable-gpios definition is to the correct GPIO to enable the camera ?
Quoted 31 lines
The camera is connected to the first MIPI port. The kernel boot logs look ok to me (except for the cyclic dependency issue, but I think that does not matter much).
Without the camera configured in the device tree, it shows up as
0x0cdevice on the #1 I2C bus, which is a bit suspicious to me given the addresses in the documentation and the info given in the kernel documentation.However, I essentially followed the description according to this guide to set up the RKISP:
https://linuxtv.org/downloads/v4l-dvb-apis/admin-guide/rkisp1.html
using this:
It looks like this (see also png attached).
However, the IMX214 driver does not complain until that point and seems to do right. I expected that at least something happens, however it does not. The last command does this:
VIDIOC_STREAMONreturned -1 (No such device or address) and this is the kernel outputQuoted 8 lines
Quoted 2 lines
[1509.435228] imx214 1-000c: write_table error: -6 [1509.435868] imx214 1-000c: could not sent common table -6-6 is ENXIO 6 No such device or address. So I expect the device isn't responding to the I2C controller.
What shows up with i2c-detect -r -y 1 ?
From the top of my head, it shows 1c on the
0x0caddress iirc, but only if I remove the IMX from the DT. Otherwise the driver takes over and it shows UU. I removed the camera physically and it was gone on i2cdetect, so I suspect that it indeed is the camera. From the driver and the documentation I need to configure it 4-lane, as it is hardcoded in the driver (compared to the application notes for registers for the ImX214).Quoted 17 lines
Quoted 5 lines
There is no more info given, even if I do some "echo
0x3F>/sys/class/video4linux/v4l-subdev0/dev_debug" to the subdevs.Here's the IMX214 documentation btw. that I got through a detour from
CSDN.
Quoted 2 lines
https://www.dropbox.com/sh/5d3mp2akr3kmu7t/AADaAsSxZu2kVSIfEceStwuoa?dl=0
Quoted 7 lines
I'm not entirely sure, but there is something wrong somewhere and I can't find out if it is with the driver, the RKISP or anything else. Here's what "v4l2-ctl -d
/dev/v4l-subdev3--all" gives - not sure but shouldn't it show supported resolutions or something?Quoted 9 lines
You might find it useful to check what is missing to support libcamera with this sensor, then you can use cam/qcam to test it too. The RKISP1 pipeline handler in libcamera will handle all the media controller configuration, and identifying the available formats for you, but we haven't had an IMX214 sensor added yet, so you might need to add a mapping to the src/libcamera/camera_sensor_properties.cpp sensor database, and the driver is missing at least
V4L2_CID_HBLANKandV4L2_CID_VBLANKthat are required for libcamera. So it might not be as straightforward as I'd like, but it would be helpful I expect.I tried that at an earlier stage, to no avail unfortunately. But I will try again as soon as I get back to my desk.
Quoted 6 lines
But ... I think your issues are more likely underlying hardware or DT issues, as the device sounds like it's not responding on the i2c address.
Quoted 6 lines
Quoted 4 lines
Sometimes I2C devices have a configurable address, can you check if this really is the correct I2C address for your camera?
That’s one of the issues. Ricardo wrote about iirc
0x10and0x1a, but theapp note says something entirely different (forgot), and my camera appears to be on yet another address…
not specific to cameras, but in the past I had i2c devices that set the address depending on the state of a gpio during powerup - which was floating in my old case, producing random settings ;-) .
Heiko
Signature · 1 line
Ricardo Ribalda
Hi,
so I moved on with some tests and trying to fix the driver and the current libcamera version. The driver has still wrong registers I guess, but I can fix them later. Here's the output of v4l-ctl
which looks kind of ok to me. However what does absolutely make no sense to me is the output of the gst-launcher, which also makes lc_compliance go totally crazy at some point:
Although I dived through a large number of drivers, I did not get how to set the available formats. While Ricardo included 2 formats, there are obviously 6 or so according to the documentation, however, how are those defined such that they show up correctly there? It's clear to me that the available formats should be defined first, before moving on to getting something out from the device, right? Is this a part of the libcamera part or the driver?
I had to add quite a bunch of controls to the driver, and without checking the i2c communication in detail, there are at least no errors when I do
v4l2-ctl -d /dev/v4l-subdev3 --set-ctrl "test_pattern=2"
So I suspect that this actually works as expected...
Best
Clemens
Am 25.02.22 um 08:37 schrieb Ricardo Ribalda Delgado:
Quoted 177 lines
Hi
I think Heiko and Kieran have already given you a lot of clues.
I would recommend to start looking at i2c communication with i2c
tools: i2cget, i2set, i2cdetect.... before trying any video operation.Your life will be much easier if you get your hands into a logic analyser like this one https://www.saleae.com/
Regarding the i2c address, bear in mind that vendors and Linux might use different nomenclature (7 bits to 8 bits).
Good luck!
Quoted 159 lines · Heiko Stuebner
Am Donnerstag, 24. Februar 2022, 16:10:06 CET schrieb Clemens Arth:
Quoted 41 lines
Hi Kieran,
Thx, I’m on my mobile now so I hope copy pasting works… apologies for typos…
Kieran Bingham <[email protected]> schrieb am Do. 24. Feb. 2022 um 13:49:
Quoted 21 lines
Hi Clemens,
Quoting Clemens Arth (2022-02-23 18:36:28)
Quoted 18 lines
Hi everyone,
+ Ricardo and Heiko in CC as the driver originators and rockchip pros...
I'm reaching out to you based on a discussion with Sebastian Fricke, who was working on the OV13850 driver for the v5 kernel. I tried getting the IMX214 finally to work on the RockPro64 from Pine64, which only works on Android so far and I need to get that done on Mainline Linux (I did not find anyone who managed that and reported about it). However, I'm also totally stuck.
The RockPro64 runs Dietpi, which is essentially Armbian + a few tweaks. I'm using the Armbian 5.15.18 kernel, so mainline, with a custom device tree, which in the first place powers the MIPI ports. I suspect it is a bad idea to have one pinctrl as a placeholder for 4 converters, however, I'm not too deep into proxying in the devicetree, so here's the current status:
Quoted 6 lines
Quoted 1 line
Your regulators are all listed as fixed-regulators. Are you sure there's nothing else to turn on ? I expect this was from another fragement for the same platform? So I hope it's consistent.
The schematics are here, from which I took the regulator and gpio config. The regulators all seem to be fixed ones. There is just one pin that pulls up all the regulators (
DVP_PWR).https://files.pine64.org/doc/rockpro64/rockpro64_v21-SCH.pdf
Quoted 7 lines
The IMX214 driver has only one pin, the
enable_pin, but it is somewhat different from the IMX219 for example. Looking at both driver's code I believe what is thereset_pinwith the IMX219 is theenable_pinwith the IMX214, but I am not sure about that. There is no MIPI reset on the RockPro64 afaik. Therefore I think it needs to be wired to theDVP_PDN0_Hpin, but other drivers define that one specifically and it apparently does something different.For the pins also check the direction (
active_low/active_high). I remember having fun somewhere when changing between the vendor kernel and mainline.Quoted 88 lines
Quoted 34 lines
Can you validate that the enable-gpios definition is to the correct GPIO to enable the camera ?
Quoted 31 lines
The camera is connected to the first MIPI port. The kernel boot logs look ok to me (except for the cyclic dependency issue, but I think that does not matter much).
Without the camera configured in the device tree, it shows up as
0x0cdevice on the #1 I2C bus, which is a bit suspicious to me given the addresses in the documentation and the info given in the kernel documentation.However, I essentially followed the description according to this guide to set up the RKISP:
https://linuxtv.org/downloads/v4l-dvb-apis/admin-guide/rkisp1.html
using this:
It looks like this (see also png attached).
However, the IMX214 driver does not complain until that point and seems to do right. I expected that at least something happens, however it does not. The last command does this:
VIDIOC_STREAMONreturned -1 (No such device or address) and this is the kernel outputQuoted 8 lines
Quoted 2 lines
[1509.435228] imx214 1-000c: write_table error: -6 [1509.435868] imx214 1-000c: could not sent common table -6-6 is ENXIO 6 No such device or address. So I expect the device isn't responding to the I2C controller.
What shows up with i2c-detect -r -y 1 ?
From the top of my head, it shows 1c on the
0x0caddress iirc, but only if
I remove the IMX from the DT. Otherwise the driver takes over and it shows UU. I removed the camera physically and it was gone on i2cdetect, so I suspect that it indeed is the camera. From the driver and the documentation I need to configure it 4-lane, as it is hardcoded in the driver (compared to the application notes for registers for the ImX214).Quoted 17 lines
Quoted 5 lines
There is no more info given, even if I do some "echo
0x3F>/sys/class/video4linux/v4l-subdev0/dev_debug" to the subdevs.Here's the IMX214 documentation btw. that I got through a detour from
CSDN.
Quoted 2 lines
https://www.dropbox.com/sh/5d3mp2akr3kmu7t/AADaAsSxZu2kVSIfEceStwuoa?dl=0
Quoted 7 lines
I'm not entirely sure, but there is something wrong somewhere and I can't find out if it is with the driver, the RKISP or anything else. Here's what "v4l2-ctl -d
/dev/v4l-subdev3--all" gives - not sure but shouldn't it show supported resolutions or something?Quoted 9 lines
You might find it useful to check what is missing to support libcamera with this sensor, then you can use cam/qcam to test it too. The RKISP1 pipeline handler in libcamera will handle all the media controller configuration, and identifying the available formats for you, but we haven't had an IMX214 sensor added yet, so you might need to add a mapping to the src/libcamera/camera_sensor_properties.cpp sensor database, and the driver is missing at least
V4L2_CID_HBLANKandV4L2_CID_VBLANKthat are required for libcamera. So it might not be as straightforward as I'd like, but it would be helpful I expect.I tried that at an earlier stage, to no avail unfortunately. But I will try again as soon as I get back to my desk.
Quoted 6 lines
But ... I think your issues are more likely underlying hardware or DT issues, as the device sounds like it's not responding on the i2c address.
Quoted 6 lines
Quoted 4 lines
Sometimes I2C devices have a configurable address, can you check if this really is the correct I2C address for your camera?
That’s one of the issues. Ricardo wrote about iirc
0x10and0x1a, but theapp note says something entirely different (forgot), and my camera appears to be on yet another address…
not specific to cameras, but in the past I had i2c devices that set the address depending on the state of a gpio during powerup - which was floating in my old case, producing random settings ;-) .
Heiko
Quoted 15 lines · Clemens Arth
Hi,
so I moved on with some tests and trying to fix the driver and the current libcamera version. The driver has still wrong registers I guess, but I can fix them later. Here's the output of v4l-ctl
which looks kind of ok to me. However what does absolutely make no sense to me is the output of the gst-launcher, which also makes
lc_compliancego totally crazy at some point:
Quoted 7 lines
Although I dived through a large number of drivers, I did not get how to set the available formats. While Ricardo included 2 formats, there are obviously 6 or so according to the documentation, however, how are those defined such that they show up correctly there? It's clear to me that the available formats should be defined first, before moving on to getting something out from the device, right? Is this a part of the libcamera part or the driver?
I had limited access to the doc, so most of the register setting come from comparing with other sensors and from 3rd party drivers (nvidia kernel)
Also my isp only supported the 2 formats that I added support to.
Quoted 192 lines
I had to add quite a bunch of controls to the driver, and without checking the i2c communication in detail, there are at least no errors when I do
v4l2-ctl -d
/dev/v4l-subdev3--set-ctrl "test_pattern=2"So I suspect that this actually works as expected...
Best
ClemensAm 25.02.22 um 08:37 schrieb Ricardo Ribalda Delgado:
Quoted 177 lines
Hi
I think Heiko and Kieran have already given you a lot of clues.
I would recommend to start looking at i2c communication with i2c
tools: i2cget, i2set, i2cdetect.... before trying any video operation.Your life will be much easier if you get your hands into a logic analyser like this one https://www.saleae.com/
Regarding the i2c address, bear in mind that vendors and Linux might use different nomenclature (7 bits to 8 bits).
Good luck!
Quoted 159 lines · Heiko Stuebner
Am Donnerstag, 24. Februar 2022, 16:10:06 CET schrieb Clemens Arth:
Quoted 41 lines
Hi Kieran,
Thx, I’m on my mobile now so I hope copy pasting works… apologies for typos…
Kieran Bingham <[email protected]> schrieb am Do. 24. Feb. 2022 um 13:49:
Quoted 21 lines
Hi Clemens,
Quoting Clemens Arth (2022-02-23 18:36:28)
Quoted 18 lines
Hi everyone,
+ Ricardo and Heiko in CC as the driver originators and rockchip pros...
I'm reaching out to you based on a discussion with Sebastian Fricke, who was working on the OV13850 driver for the v5 kernel. I tried getting the IMX214 finally to work on the RockPro64 from Pine64, which only works on Android so far and I need to get that done on Mainline Linux (I did not find anyone who managed that and reported about it). However, I'm also totally stuck.
The RockPro64 runs Dietpi, which is essentially Armbian + a few tweaks. I'm using the Armbian 5.15.18 kernel, so mainline, with a custom device tree, which in the first place powers the MIPI ports. I suspect it is a bad idea to have one pinctrl as a placeholder for 4 converters, however, I'm not too deep into proxying in the devicetree, so here's the current status:
Quoted 6 lines
Quoted 1 line
Your regulators are all listed as fixed-regulators. Are you sure there's nothing else to turn on ? I expect this was from another fragement for the same platform? So I hope it's consistent.
The schematics are here, from which I took the regulator and gpio config. The regulators all seem to be fixed ones. There is just one pin that pulls up all the regulators (
DVP_PWR).https://files.pine64.org/doc/rockpro64/rockpro64_v21-SCH.pdf
Quoted 7 lines
The IMX214 driver has only one pin, the
enable_pin, but it is somewhat different from the IMX219 for example. Looking at both driver's code I believe what is thereset_pinwith the IMX219 is theenable_pinwith the IMX214, but I am not sure about that. There is no MIPI reset on the RockPro64 afaik. Therefore I think it needs to be wired to theDVP_PDN0_Hpin, but other drivers define that one specifically and it apparently does something different.For the pins also check the direction (
active_low/active_high). I remember having fun somewhere when changing between the vendor kernel and mainline.Quoted 88 lines
Quoted 34 lines
Can you validate that the enable-gpios definition is to the correct GPIO to enable the camera ?
Quoted 31 lines
The camera is connected to the first MIPI port. The kernel boot logs look ok to me (except for the cyclic dependency issue, but I think that does not matter much).
Without the camera configured in the device tree, it shows up as
0x0cdevice on the #1 I2C bus, which is a bit suspicious to me given the addresses in the documentation and the info given in the kernel documentation.However, I essentially followed the description according to this guide to set up the RKISP:
https://linuxtv.org/downloads/v4l-dvb-apis/admin-guide/rkisp1.html
using this:
It looks like this (see also png attached).
However, the IMX214 driver does not complain until that point and seems to do right. I expected that at least something happens, however it does not. The last command does this:
VIDIOC_STREAMONreturned -1 (No such device or address) and this is the kernel outputQuoted 8 lines
Quoted 2 lines
[1509.435228] imx214 1-000c: write_table error: -6 [1509.435868] imx214 1-000c: could not sent common table -6-6 is ENXIO 6 No such device or address. So I expect the device isn't responding to the I2C controller.
What shows up with i2c-detect -r -y 1 ?
From the top of my head, it shows 1c on the
0x0caddress iirc, but only if
I remove the IMX from the DT. Otherwise the driver takes over and it shows UU. I removed the camera physically and it was gone on i2cdetect, so I suspect that it indeed is the camera. From the driver and the documentation I need to configure it 4-lane, as it is hardcoded in the driver (compared to the application notes for registers for the ImX214).Quoted 17 lines
Quoted 5 lines
There is no more info given, even if I do some "echo
0x3F>/sys/class/video4linux/v4l-subdev0/dev_debug" to the subdevs.Here's the IMX214 documentation btw. that I got through a detour from
CSDN.
Quoted 2 lines
https://www.dropbox.com/sh/5d3mp2akr3kmu7t/AADaAsSxZu2kVSIfEceStwuoa?dl=0
Quoted 7 lines
I'm not entirely sure, but there is something wrong somewhere and I can't find out if it is with the driver, the RKISP or anything else. Here's what "v4l2-ctl -d
/dev/v4l-subdev3--all" gives - not sure but shouldn't it show supported resolutions or something?Quoted 9 lines
You might find it useful to check what is missing to support libcamera with this sensor, then you can use cam/qcam to test it too. The RKISP1 pipeline handler in libcamera will handle all the media controller configuration, and identifying the available formats for you, but we haven't had an IMX214 sensor added yet, so you might need to add a mapping to the src/libcamera/camera_sensor_properties.cpp sensor database, and the driver is missing at least
V4L2_CID_HBLANKandV4L2_CID_VBLANKthat are required for libcamera. So it might not be as straightforward as I'd like, but it would be helpful I expect.I tried that at an earlier stage, to no avail unfortunately. But I will try again as soon as I get back to my desk.
Quoted 6 lines
But ... I think your issues are more likely underlying hardware or DT issues, as the device sounds like it's not responding on the i2c address.
Quoted 6 lines
Quoted 4 lines
Sometimes I2C devices have a configurable address, can you check if this really is the correct I2C address for your camera?
That’s one of the issues. Ricardo wrote about iirc
0x10and0x1a, but theapp note says something entirely different (forgot), and my camera appears to be on yet another address…
not specific to cameras, but in the past I had i2c devices that set the address depending on the state of a gpio during powerup - which was floating in my old case, producing random settings ;-) .
Heiko
Signature · 1 line
Ricardo Ribalda
Am 28.02.22 um 09:15 schrieb Ricardo Ribalda Delgado:
Quoted 24 lines · Clemens Arth
Quoted 15 lines
Hi,
so I moved on with some tests and trying to fix the driver and the current libcamera version. The driver has still wrong registers I guess, but I can fix them later. Here's the output of v4l-ctl
which looks kind of ok to me. However what does absolutely make no sense to me is the output of the gst-launcher, which also makes
lc_compliancego totally crazy at some point:Quoted 7 lines
Although I dived through a large number of drivers, I did not get how to set the available formats. While Ricardo included 2 formats, there are obviously 6 or so according to the documentation, however, how are those defined such that they show up correctly there? It's clear to me that the available formats should be defined first, before moving on to getting something out from the device, right? Is this a part of the libcamera part or the driver?
Quoted 6 lines
I had limited access to the doc, so most of the register setting come from comparing with other sensors and from 3rd party drivers (nvidia kernel)
Also my isp only supported the 2 formats that I added support to.
Yes, I know - no offense, just an observation, seems that you got a lot of the registers right anyway :-)
I need to dig into the i2c stuff indeed... I was playing around with it a lot yesterday, but the documentation of the rockpro64 stuff is overall real crap imho and I really wonder how to expect that the community will fix things without being a hardware/software/kernel pro and wasting tons of time for essentially nothing...
there is something weird going on on the I2C bus and 100kHz seems also the only frequency that gives at least something. If I manually play around with setting gpios there are 2 more interfaces popping up for a second or so on 0x19 and 0x1a, which might indeed be the camera interfaces, but they vanish immediately, like being powered off or something. However, the 400kHz does not give any devices whatsoever -
the i2cget and i2cset tools seem to work, but the 0x0c device does not give me any meaningful output with respect to the documented registers... and the 16-bit addressing is a pain... so I probably really need to do as you suggested and get some debugging interface...
Quoted 194 lines
Quoted 192 lines
I had to add quite a bunch of controls to the driver, and without checking the i2c communication in detail, there are at least no errors when I do
v4l2-ctl -d
/dev/v4l-subdev3--set-ctrl "test_pattern=2"So I suspect that this actually works as expected...
Best
ClemensAm 25.02.22 um 08:37 schrieb Ricardo Ribalda Delgado:
Quoted 177 lines
Hi
I think Heiko and Kieran have already given you a lot of clues.
I would recommend to start looking at i2c communication with i2c
tools: i2cget, i2set, i2cdetect.... before trying any video operation.Your life will be much easier if you get your hands into a logic analyser like this one https://www.saleae.com/
Regarding the i2c address, bear in mind that vendors and Linux might use different nomenclature (7 bits to 8 bits).
Good luck!
Quoted 159 lines · Heiko Stuebner
Am Donnerstag, 24. Februar 2022, 16:10:06 CET schrieb Clemens Arth:
Quoted 41 lines
Hi Kieran,
Thx, I’m on my mobile now so I hope copy pasting works… apologies for typos…
Kieran Bingham <[email protected]> schrieb am Do. 24. Feb. 2022 um 13:49:
Quoted 21 lines
Hi Clemens,
Quoting Clemens Arth (2022-02-23 18:36:28)
Quoted 18 lines
Hi everyone,
+ Ricardo and Heiko in CC as the driver originators and rockchip pros...
I'm reaching out to you based on a discussion with Sebastian Fricke, who was working on the OV13850 driver for the v5 kernel. I tried getting the IMX214 finally to work on the RockPro64 from Pine64, which only works on Android so far and I need to get that done on Mainline Linux (I did not find anyone who managed that and reported about it). However, I'm also totally stuck.
The RockPro64 runs Dietpi, which is essentially Armbian + a few tweaks. I'm using the Armbian 5.15.18 kernel, so mainline, with a custom device tree, which in the first place powers the MIPI ports. I suspect it is a bad idea to have one pinctrl as a placeholder for 4 converters, however, I'm not too deep into proxying in the devicetree, so here's the current status:
Quoted 6 lines
Quoted 1 line
Your regulators are all listed as fixed-regulators. Are you sure there's nothing else to turn on ? I expect this was from another fragement for the same platform? So I hope it's consistent.
The schematics are here, from which I took the regulator and gpio config. The regulators all seem to be fixed ones. There is just one pin that pulls up all the regulators (
DVP_PWR).https://files.pine64.org/doc/rockpro64/rockpro64_v21-SCH.pdf
Quoted 7 lines
The IMX214 driver has only one pin, the
enable_pin, but it is somewhat different from the IMX219 for example. Looking at both driver's code I believe what is thereset_pinwith the IMX219 is theenable_pinwith the IMX214, but I am not sure about that. There is no MIPI reset on the RockPro64 afaik. Therefore I think it needs to be wired to theDVP_PDN0_Hpin, but other drivers define that one specifically and it apparently does something different.For the pins also check the direction (
active_low/active_high). I remember having fun somewhere when changing between the vendor kernel and mainline.Quoted 88 lines
Quoted 34 lines
Can you validate that the enable-gpios definition is to the correct GPIO to enable the camera ?
Quoted 31 lines
The camera is connected to the first MIPI port. The kernel boot logs look ok to me (except for the cyclic dependency issue, but I think that does not matter much).
Without the camera configured in the device tree, it shows up as
0x0cdevice on the #1 I2C bus, which is a bit suspicious to me given the addresses in the documentation and the info given in the kernel documentation.However, I essentially followed the description according to this guide to set up the RKISP:
https://linuxtv.org/downloads/v4l-dvb-apis/admin-guide/rkisp1.html
using this:
It looks like this (see also png attached).
However, the IMX214 driver does not complain until that point and seems to do right. I expected that at least something happens, however it does not. The last command does this:
VIDIOC_STREAMONreturned -1 (No such device or address) and this is the kernel outputQuoted 8 lines
Quoted 2 lines
[1509.435228] imx214 1-000c: write_table error: -6 [1509.435868] imx214 1-000c: could not sent common table -6-6 is ENXIO 6 No such device or address. So I expect the device isn't responding to the I2C controller.
What shows up with i2c-detect -r -y 1 ?
From the top of my head, it shows 1c on the
0x0caddress iirc, but only if
I remove the IMX from the DT. Otherwise the driver takes over and it shows UU. I removed the camera physically and it was gone on i2cdetect, so I suspect that it indeed is the camera. From the driver and the documentation I need to configure it 4-lane, as it is hardcoded in the driver (compared to the application notes for registers for the ImX214).Quoted 17 lines
Quoted 5 lines
There is no more info given, even if I do some "echo
0x3F>/sys/class/video4linux/v4l-subdev0/dev_debug" to the subdevs.Here's the IMX214 documentation btw. that I got through a detour from
CSDN.
Quoted 2 lines
https://www.dropbox.com/sh/5d3mp2akr3kmu7t/AADaAsSxZu2kVSIfEceStwuoa?dl=0
Quoted 7 lines
I'm not entirely sure, but there is something wrong somewhere and I can't find out if it is with the driver, the RKISP or anything else. Here's what "v4l2-ctl -d
/dev/v4l-subdev3--all" gives - not sure but shouldn't it show supported resolutions or something?Quoted 9 lines
You might find it useful to check what is missing to support libcamera with this sensor, then you can use cam/qcam to test it too. The RKISP1 pipeline handler in libcamera will handle all the media controller configuration, and identifying the available formats for you, but we haven't had an IMX214 sensor added yet, so you might need to add a mapping to the src/libcamera/camera_sensor_properties.cpp sensor database, and the driver is missing at least
V4L2_CID_HBLANKandV4L2_CID_VBLANKthat are required for libcamera. So it might not be as straightforward as I'd like, but it would be helpful I expect.I tried that at an earlier stage, to no avail unfortunately. But I will try again as soon as I get back to my desk.
Quoted 6 lines
But ... I think your issues are more likely underlying hardware or DT issues, as the device sounds like it's not responding on the i2c address.
Quoted 6 lines
Quoted 4 lines
Sometimes I2C devices have a configurable address, can you check if this really is the correct I2C address for your camera?
That’s one of the issues. Ricardo wrote about iirc
0x10and0x1a, but theapp note says something entirely different (forgot), and my camera appears to be on yet another address…
not specific to cameras, but in the past I had i2c devices that set the address depending on the state of a gpio during powerup - which was floating in my old case, producing random settings ;-) .
Heiko
Hi all,
so I have good news. With the help of one of the pinetab developers, I finally managed to get a working DT together. I did not realize I had to use another clock to drive it, but the rest of the DT was already ok.
While I played around with the IMX driver trying to improve it, I finally went back to the current mainline branch version testing it with the rkisp1 and - voila! - the large 13M res does not work, but the 1920x1080 worked out immediately, using the pure command-line config of media-ctl and video-ctl - the libcamera refuses operation due to the missing database info and the lack of certain caps in the driver.
There are a few things on my list though and I wanted to hear your thoughts before I move on now.
1) First of all, the image seems to be very badly balanced in terms of color and brightness. I assume it has to do with not properly adjusting parameters of the camera. I did not play around with it, as I have no graphical user interface on the board - it would make more sense I do that once I am able to visualize changes in real-time. @Ricardo, would you be willing to review changes I do to the IMX214 driver in the kernel based on the application guide I shared earlier? As I'm not a driver guy AT ALL, fixing things for me might break things for others, and I don't know how to verify that without having anyone else to check (I mean, do more than just code review, probably).
2) The DT overlay - it is nice that I have it now, but I had to put everything together myself - should I contribute it into mainline (or armbian), does that make sense and what is the procedure?
3) I don't have any other camera to work with, as I just ordered FPC FCC pinouts for 24-pin cams and have to rewire/solder a converter for the RockPro64. That might happen as soon as I get them from China - no Amazon or anything available right now. Still the DT seems to be the most valuable part of all of this right now.
4) The libcamera fix - I saw lc_compliance and gst-video work already after a few minor copy/paste fixes, but by pulling the driver from mainline broke it again expectedly. What's the plausible roadmap? Fixing libcamera alone resorts to fixing 10 lines of code probably, but that does not make sense without fixing the IMX214 driver in the kernel first, right?
Best
Clemens
Am 28.02.22 um 09:15 schrieb Ricardo Ribalda Delgado:
Quoted 224 lines · Clemens Arth
Quoted 15 lines
Hi,
so I moved on with some tests and trying to fix the driver and the current libcamera version. The driver has still wrong registers I guess, but I can fix them later. Here's the output of v4l-ctl
which looks kind of ok to me. However what does absolutely make no sense to me is the output of the gst-launcher, which also makes
lc_compliancego totally crazy at some point:Quoted 7 lines
Although I dived through a large number of drivers, I did not get how to set the available formats. While Ricardo included 2 formats, there are obviously 6 or so according to the documentation, however, how are those defined such that they show up correctly there? It's clear to me that the available formats should be defined first, before moving on to getting something out from the device, right? Is this a part of the libcamera part or the driver?
I had limited access to the doc, so most of the register setting come from comparing with other sensors and from 3rd party drivers (nvidia kernel)
Also my isp only supported the 2 formats that I added support to.
Quoted 192 lines
I had to add quite a bunch of controls to the driver, and without checking the i2c communication in detail, there are at least no errors when I do
v4l2-ctl -d
/dev/v4l-subdev3--set-ctrl "test_pattern=2"So I suspect that this actually works as expected...
Best
ClemensAm 25.02.22 um 08:37 schrieb Ricardo Ribalda Delgado:
Quoted 177 lines
Hi
I think Heiko and Kieran have already given you a lot of clues.
I would recommend to start looking at i2c communication with i2c
tools: i2cget, i2set, i2cdetect.... before trying any video operation.Your life will be much easier if you get your hands into a logic analyser like this one https://www.saleae.com/
Regarding the i2c address, bear in mind that vendors and Linux might use different nomenclature (7 bits to 8 bits).
Good luck!
Quoted 159 lines · Heiko Stuebner
Am Donnerstag, 24. Februar 2022, 16:10:06 CET schrieb Clemens Arth:
Quoted 41 lines
Hi Kieran,
Thx, I’m on my mobile now so I hope copy pasting works… apologies for typos…
Kieran Bingham <[email protected]> schrieb am Do. 24. Feb. 2022 um 13:49:
Quoted 21 lines
Hi Clemens,
Quoting Clemens Arth (2022-02-23 18:36:28)
Quoted 18 lines
Hi everyone,
+ Ricardo and Heiko in CC as the driver originators and rockchip pros...
I'm reaching out to you based on a discussion with Sebastian Fricke, who was working on the OV13850 driver for the v5 kernel. I tried getting the IMX214 finally to work on the RockPro64 from Pine64, which only works on Android so far and I need to get that done on Mainline Linux (I did not find anyone who managed that and reported about it). However, I'm also totally stuck.
The RockPro64 runs Dietpi, which is essentially Armbian + a few tweaks. I'm using the Armbian 5.15.18 kernel, so mainline, with a custom device tree, which in the first place powers the MIPI ports. I suspect it is a bad idea to have one pinctrl as a placeholder for 4 converters, however, I'm not too deep into proxying in the devicetree, so here's the current status:
Quoted 6 lines
Quoted 1 line
Your regulators are all listed as fixed-regulators. Are you sure there's nothing else to turn on ? I expect this was from another fragement for the same platform? So I hope it's consistent.
The schematics are here, from which I took the regulator and gpio config. The regulators all seem to be fixed ones. There is just one pin that pulls up all the regulators (
DVP_PWR).https://files.pine64.org/doc/rockpro64/rockpro64_v21-SCH.pdf
Quoted 7 lines
The IMX214 driver has only one pin, the
enable_pin, but it is somewhat different from the IMX219 for example. Looking at both driver's code I believe what is thereset_pinwith the IMX219 is theenable_pinwith the IMX214, but I am not sure about that. There is no MIPI reset on the RockPro64 afaik. Therefore I think it needs to be wired to theDVP_PDN0_Hpin, but other drivers define that one specifically and it apparently does something different.For the pins also check the direction (
active_low/active_high). I remember having fun somewhere when changing between the vendor kernel and mainline.Quoted 88 lines
Quoted 34 lines
Can you validate that the enable-gpios definition is to the correct GPIO to enable the camera ?
Quoted 31 lines
The camera is connected to the first MIPI port. The kernel boot logs look ok to me (except for the cyclic dependency issue, but I think that does not matter much).
Without the camera configured in the device tree, it shows up as
0x0cdevice on the #1 I2C bus, which is a bit suspicious to me given the addresses in the documentation and the info given in the kernel documentation.However, I essentially followed the description according to this guide to set up the RKISP:
https://linuxtv.org/downloads/v4l-dvb-apis/admin-guide/rkisp1.html
using this:
It looks like this (see also png attached).
However, the IMX214 driver does not complain until that point and seems to do right. I expected that at least something happens, however it does not. The last command does this:
VIDIOC_STREAMONreturned -1 (No such device or address) and this is the kernel outputQuoted 8 lines
Quoted 2 lines
[1509.435228] imx214 1-000c: write_table error: -6 [1509.435868] imx214 1-000c: could not sent common table -6-6 is ENXIO 6 No such device or address. So I expect the device isn't responding to the I2C controller.
What shows up with i2c-detect -r -y 1 ?
From the top of my head, it shows 1c on the
0x0caddress iirc, but only if
I remove the IMX from the DT. Otherwise the driver takes over and it shows UU. I removed the camera physically and it was gone on i2cdetect, so I suspect that it indeed is the camera. From the driver and the documentation I need to configure it 4-lane, as it is hardcoded in the driver (compared to the application notes for registers for the ImX214).Quoted 17 lines
Quoted 5 lines
There is no more info given, even if I do some "echo
0x3F>/sys/class/video4linux/v4l-subdev0/dev_debug" to the subdevs.Here's the IMX214 documentation btw. that I got through a detour from
CSDN.
Quoted 2 lines
https://www.dropbox.com/sh/5d3mp2akr3kmu7t/AADaAsSxZu2kVSIfEceStwuoa?dl=0
Quoted 7 lines
I'm not entirely sure, but there is something wrong somewhere and I can't find out if it is with the driver, the RKISP or anything else. Here's what "v4l2-ctl -d
/dev/v4l-subdev3--all" gives - not sure but shouldn't it show supported resolutions or something?Quoted 9 lines
You might find it useful to check what is missing to support libcamera with this sensor, then you can use cam/qcam to test it too. The RKISP1 pipeline handler in libcamera will handle all the media controller configuration, and identifying the available formats for you, but we haven't had an IMX214 sensor added yet, so you might need to add a mapping to the src/libcamera/camera_sensor_properties.cpp sensor database, and the driver is missing at least
V4L2_CID_HBLANKandV4L2_CID_VBLANKthat are required for libcamera. So it might not be as straightforward as I'd like, but it would be helpful I expect.I tried that at an earlier stage, to no avail unfortunately. But I will try again as soon as I get back to my desk.
Quoted 6 lines
But ... I think your issues are more likely underlying hardware or DT issues, as the device sounds like it's not responding on the i2c address.
Quoted 6 lines
Quoted 4 lines
Sometimes I2C devices have a configurable address, can you check if this really is the correct I2C address for your camera?
That’s one of the issues. Ricardo wrote about iirc
0x10and0x1a, but theapp note says something entirely different (forgot), and my camera appears to be on yet another address…
not specific to cameras, but in the past I had i2c devices that set the address depending on the state of a gpio during powerup - which was floating in my old case, producing random settings ;-) .
Heiko
Hey Clemens,
Quoted 3 lines · Clemens Arth
Hi all,
so I have good news. With the help of one of the pinetab developers, I finally managed to get a working DT together. I did not realize I had to use another clock to drive it, but the rest of the DT was already ok.
That is good to hear, congratz.
Quoted 2 lines
While I played around with the IMX driver trying to improve it, I finally went back to the current mainline branch version testing it with the rkisp1 and - voila! - the large 13M res does not work, but
Oh that sounds very familiar I worked on a fix for this for a while as I had the same problem with OV13850:
https://patchwork.kernel.org/project/linux-media/patch/[email protected]/
I haven't finished it yet, as I stopped working on the OV13850. If you want to you can finish the patch, the remaining problem was that result should be centered automatically.
I remember that one issue with the high resolution format were the very low FPS (15).
Quoted 1 line
the 1920x1080 worked out immediately, using the pure command-line config of media-ctl and video-ctl - the libcamera refuses operation due to the missing database info and the lack of certain caps in the driver.
Can you add your log to the mail?
Quoted 6 lines
There are a few things on my list though and I wanted to hear your thoughts before I move on now.
1) First of all, the image seems to be very badly balanced in terms of color and brightness. I assume it has to do with not properly adjusting parameters of the camera. I did not play around with it, as I have no graphical user interface on the board - it would make more sense I do that once I am able to visualize changes in real-time. @Ricardo, would you be willing to review changes I do to the IMX214
driver in the kernel based on the application guide I shared earlier? As I'm not a driver guy AT ALL, fixing things for me might break things for others, and I don't know how to verify that without having anyone else to check (I mean, do more than just code review, probably).
Could you provide a link to a frame via imgur or something similar?
Quoted 6 lines
2) The DT overlay - it is nice that I have it now, but I had to put everything together myself - should I contribute it into mainline (or armbian), does that make sense and what is the procedure?
3) I don't have any other camera to work with, as I just ordered FPC FCC pinouts for 24-pin cams and have to rewire/solder a converter for the RockPro64. That might happen as soon as I get them from China - no Amazon or anything available right now. Still the DT seems to be the most valuable part of all of this right now.
4) The libcamera fix - I saw
lc_complianceand gst-video work already after a few minor copy/paste fixes, but by pulling the driver from mainline broke it again expectedly. What's the plausible roadmap? Fixing libcamera alone resorts to fixing 10 lines of code probably, but that does not make sense without fixing the IMX214 driver in the kernel first, right?
This is also a bit to vague, please provide logs that describe your issues. What did you fix, can you maybe provide a diff/patch?
Quoted 3 lines
Best
Clemens
Greetings,
Sebastian
Hi Clemens,
Quoting Clemens Arth (2022-03-02 08:43:28)
Quoted 16 lines
Hi all,
so I have good news. With the help of one of the pinetab developers, I finally managed to get a working DT together. I did not realize I had to use another clock to drive it, but the rest of the DT was already ok.
While I played around with the IMX driver trying to improve it, I finally went back to the current mainline branch version testing it with the rkisp1 and - voila! - the large 13M res does not work, but the 1920x1080 worked out immediately, using the pure command-line config of media-ctl and video-ctl - the libcamera refuses operation due to the missing database info and the lack of certain caps in the driver.
There are a few things on my list though and I wanted to hear your thoughts before I move on now.
Quoted 10 lines
1) First of all, the image seems to be very badly balanced in terms of color and brightness. I assume it has to do with not properly adjusting parameters of the camera. I did not play around with it, as I have no graphical user interface on the board - it would make more sense I do that once I am able to visualize changes in real-time. @Ricardo, would you be willing to review changes I do to the IMX214 driver in the kernel based on the application guide I shared earlier? As I'm not a driver guy AT ALL, fixing things for me might break things for others, and I don't know how to verify that without having anyone else to check (I mean, do more than just code review, probably).
This is because the algorithms to handle color and brightness are software algorithms that have to be run to calculate the correct parameters (in real time, unless you have a fixed lighting environment).
This is handled by the libcamera IPA component, and the existing 3a algorithms for the RKISP that we have started to develop. However, these are not all the way there yet, and there is active development that you will probably want to apply locally if they don't get merged to libcamera before you test again.
Mostly this series I believe:
https://patchwork.libcamera.org/project/libcamera/list/?series=2937
Furthermore, as you've seen libcamera needs a few controls to be available in the driver, and a camera sensor helper to help it correctly adapt the results of the calculation to the register settings to apply.
We can help you with those on the libcamera mailing list.
https://lists.libcamera.org/listinfo/libcamera-devel
Quoted 3 lines
2) The DT overlay - it is nice that I have it now, but I had to put everything together myself - should I contribute it into mainline (or armbian), does that make sense and what is the procedure?
If the camera is removable, then it would probably need to be an overlay perhaps. That's a discussion for Heiko to continue I think.
Quoted 5 lines
3) I don't have any other camera to work with, as I just ordered FPC FCC pinouts for 24-pin cams and have to rewire/solder a converter for the RockPro64. That might happen as soon as I get them from China - no Amazon or anything available right now. Still the DT seems to be the most valuable part of all of this right now.
Even if it ends up not being something that can be merged to the kernel, please do share it on a public mailing list so that it can be indexed by search engines for the next person who tries to do this to discover your work.
You could post it as a patch marked [PATCH FOR TESTING ONLY] or such.
Quoted 6 lines
4) The libcamera fix - I saw
lc_complianceand gst-video work already after a few minor copy/paste fixes, but by pulling the driver from mainline broke it again expectedly. What's the plausible roadmap? Fixing libcamera alone resorts to fixing 10 lines of code probably, but that does not make sense without fixing the IMX214 driver in the kernel first, right?
I'd like to see code to know what to reference for that. But yes, if you have fixes to support this cameara with libcamera, please submit to the libcamera mailing list. If there is an upstream kernel driver we will work towards merging them. Certainly fixing the IMX214 driver in the kernel is required, but we can work in parallel, you don't have to wait.
--
Kieran.
Quoted 231 lines
Best
ClemensAm 28.02.22 um 09:15 schrieb Ricardo Ribalda Delgado:
Quoted 224 lines · Clemens Arth
Quoted 15 lines
Hi,
so I moved on with some tests and trying to fix the driver and the current libcamera version. The driver has still wrong registers I guess, but I can fix them later. Here's the output of v4l-ctl
which looks kind of ok to me. However what does absolutely make no sense to me is the output of the gst-launcher, which also makes
lc_compliancego totally crazy at some point:Quoted 7 lines
Although I dived through a large number of drivers, I did not get how to set the available formats. While Ricardo included 2 formats, there are obviously 6 or so according to the documentation, however, how are those defined such that they show up correctly there? It's clear to me that the available formats should be defined first, before moving on to getting something out from the device, right? Is this a part of the libcamera part or the driver?
I had limited access to the doc, so most of the register setting come from comparing with other sensors and from 3rd party drivers (nvidia kernel)
Also my isp only supported the 2 formats that I added support to.
Quoted 192 lines
I had to add quite a bunch of controls to the driver, and without checking the i2c communication in detail, there are at least no errors when I do
v4l2-ctl -d
/dev/v4l-subdev3--set-ctrl "test_pattern=2"So I suspect that this actually works as expected...
Best
ClemensAm 25.02.22 um 08:37 schrieb Ricardo Ribalda Delgado:
Quoted 177 lines
Hi
I think Heiko and Kieran have already given you a lot of clues.
I would recommend to start looking at i2c communication with i2c
tools: i2cget, i2set, i2cdetect.... before trying any video operation.Your life will be much easier if you get your hands into a logic analyser like this one https://www.saleae.com/
Regarding the i2c address, bear in mind that vendors and Linux might use different nomenclature (7 bits to 8 bits).
Good luck!
Quoted 159 lines · Heiko Stuebner
Am Donnerstag, 24. Februar 2022, 16:10:06 CET schrieb Clemens Arth:
Quoted 41 lines
Hi Kieran,
Thx, I’m on my mobile now so I hope copy pasting works… apologies for typos…
Kieran Bingham <[email protected]> schrieb am Do. 24. Feb. 2022 um 13:49:
Quoted 21 lines
Hi Clemens,
Quoting Clemens Arth (2022-02-23 18:36:28)
Quoted 18 lines
Hi everyone,
+ Ricardo and Heiko in CC as the driver originators and rockchip pros...
I'm reaching out to you based on a discussion with Sebastian Fricke, who was working on the OV13850 driver for the v5 kernel. I tried getting the IMX214 finally to work on the RockPro64 from Pine64, which only works on Android so far and I need to get that done on Mainline Linux (I did not find anyone who managed that and reported about it). However, I'm also totally stuck.
The RockPro64 runs Dietpi, which is essentially Armbian + a few tweaks. I'm using the Armbian 5.15.18 kernel, so mainline, with a custom device tree, which in the first place powers the MIPI ports. I suspect it is a bad idea to have one pinctrl as a placeholder for 4 converters, however, I'm not too deep into proxying in the devicetree, so here's the current status:
Quoted 6 lines
Quoted 1 line
Your regulators are all listed as fixed-regulators. Are you sure there's nothing else to turn on ? I expect this was from another fragement for the same platform? So I hope it's consistent.
The schematics are here, from which I took the regulator and gpio config. The regulators all seem to be fixed ones. There is just one pin that pulls up all the regulators (
DVP_PWR).https://files.pine64.org/doc/rockpro64/rockpro64_v21-SCH.pdf
Quoted 7 lines
The IMX214 driver has only one pin, the
enable_pin, but it is somewhat different from the IMX219 for example. Looking at both driver's code I believe what is thereset_pinwith the IMX219 is theenable_pinwith the IMX214, but I am not sure about that. There is no MIPI reset on the RockPro64 afaik. Therefore I think it needs to be wired to theDVP_PDN0_Hpin, but other drivers define that one specifically and it apparently does something different.For the pins also check the direction (
active_low/active_high). I remember having fun somewhere when changing between the vendor kernel and mainline.Quoted 88 lines
Quoted 34 lines
Can you validate that the enable-gpios definition is to the correct GPIO to enable the camera ?
Quoted 31 lines
The camera is connected to the first MIPI port. The kernel boot logs look ok to me (except for the cyclic dependency issue, but I think that does not matter much).
Without the camera configured in the device tree, it shows up as
0x0cdevice on the #1 I2C bus, which is a bit suspicious to me given the addresses in the documentation and the info given in the kernel documentation.However, I essentially followed the description according to this guide to set up the RKISP:
https://linuxtv.org/downloads/v4l-dvb-apis/admin-guide/rkisp1.html
using this:
It looks like this (see also png attached).
However, the IMX214 driver does not complain until that point and seems to do right. I expected that at least something happens, however it does not. The last command does this:
VIDIOC_STREAMONreturned -1 (No such device or address) and this is the kernel outputQuoted 8 lines
Quoted 2 lines
[1509.435228] imx214 1-000c: write_table error: -6 [1509.435868] imx214 1-000c: could not sent common table -6-6 is ENXIO 6 No such device or address. So I expect the device isn't responding to the I2C controller.
What shows up with i2c-detect -r -y 1 ?
From the top of my head, it shows 1c on the
0x0caddress iirc, but only if
I remove the IMX from the DT. Otherwise the driver takes over and it shows UU. I removed the camera physically and it was gone on i2cdetect, so I suspect that it indeed is the camera. From the driver and the documentation I need to configure it 4-lane, as it is hardcoded in the driver (compared to the application notes for registers for the ImX214).Quoted 17 lines
Quoted 5 lines
There is no more info given, even if I do some "echo
0x3F>/sys/class/video4linux/v4l-subdev0/dev_debug" to the subdevs.Here's the IMX214 documentation btw. that I got through a detour from
CSDN.
Quoted 2 lines
https://www.dropbox.com/sh/5d3mp2akr3kmu7t/AADaAsSxZu2kVSIfEceStwuoa?dl=0
Quoted 7 lines
I'm not entirely sure, but there is something wrong somewhere and I can't find out if it is with the driver, the RKISP or anything else. Here's what "v4l2-ctl -d
/dev/v4l-subdev3--all" gives - not sure but shouldn't it show supported resolutions or something?Quoted 9 lines
You might find it useful to check what is missing to support libcamera with this sensor, then you can use cam/qcam to test it too. The RKISP1 pipeline handler in libcamera will handle all the media controller configuration, and identifying the available formats for you, but we haven't had an IMX214 sensor added yet, so you might need to add a mapping to the src/libcamera/camera_sensor_properties.cpp sensor database, and the driver is missing at least
V4L2_CID_HBLANKandV4L2_CID_VBLANKthat are required for libcamera. So it might not be as straightforward as I'd like, but it would be helpful I expect.I tried that at an earlier stage, to no avail unfortunately. But I will try again as soon as I get back to my desk.
Quoted 6 lines
But ... I think your issues are more likely underlying hardware or DT issues, as the device sounds like it's not responding on the i2c address.
Quoted 6 lines
Quoted 4 lines
Sometimes I2C devices have a configurable address, can you check if this really is the correct I2C address for your camera?
That’s one of the issues. Ricardo wrote about iirc
0x10and0x1a, but theapp note says something entirely different (forgot), and my camera appears to be on yet another address…
not specific to cameras, but in the past I had i2c devices that set the address depending on the state of a gpio during powerup - which was floating in my old case, producing random settings ;-) .
Heiko