Add support for the Sony Exmor-RS IMX300 camera sensor
Subsystems (3)
v3 cover Add support for the Sony Exmor-RS IMX300 camera sensor
From: AngeloGioacchino Del Regno <[email protected]>
This patch series adds support for the IMX300 camera sensor, (one of the) first Exmor-RS Stacked CMOS sensor(s), with support for both of the supported aspect ratios (4:3 and 16:9).
This driver came out from reverse engineering of so called "userspace drivers" from Sony Xperia smartphones.
I tried to document all of my findings and giving a sense to the registers as much as possible, but that was only partially possible and resembles some names from the IMX219 public datasheet, even though the addresses are basically completely different.
This camera sensor driver was tested with all the resolutions declared in it on two phones: Sony Xperia XA2 and XA2 Ultra, on a SDM630 SoC (camss patches for this SoC will come in a later series) and is working great.
- Changes in v3:
- Removed unneeded fallthrough statements
- Fixed double mode initialization at probe time
- Fixed typo in the dt-binding description (8->25MPixels)
- Fixed dt-binding data-lanes description, added to required properties
- Changes in v2:
- Changed dt-binding name and fixed a misconception about lane operation (sensor supports 2/4-Lane, driver supports 4-Lane only)
- Now using lowercase names for regulator supplies
- Fixed redefinition of clock-noncontinuous property
- Added informations about constraints on data bus frequencies
- Fixed MAINTAINERS: removed git tree
AngeloGioacchino Del Regno (2):
media: i2c: Add driver for the Sony Exmor-RS IMX300 camera sensor
media: dt-bindings: media: i2c: Add IMX300 CMOS sensor binding
.../bindings/media/i2c/sony,imx300.yaml | 112 + MAINTAINERS | 7 + drivers/media/i2c/Kconfig | 13 + drivers/media/i2c/Makefile | 1 + drivers/media/i2c/imx300.c | 3081 +++++++++++++++++ 5 files changed, 3214 insertions(+) create mode 100644 Documentation/devicetree/bindings/media/i2c/sony,imx300.yaml create mode 100644 drivers/media/i2c/imx300.c
Signature · 1 line
2.29.2
PATCH v3 1/2 media: i2c: Add driver for the Sony Exmor-RS IMX300 camera sensor
From: AngeloGioacchino Del Regno <[email protected]>
This is a custom multi-aspect 25MegaPixels sensor from Sony, found in many Sony Xperia smartphones from various eras.
The camera assembly for this sensor usually (at least in Xperia phones) has a lens that does not cover the entire sensor area, which means that the real corners are blind and that, in many lighting conditions, some more pixels in the corners are very getting obscured (as no decent amount of light can get in)... so, the maximum resolution that can produce a good image is:
- In 4:3 aspect ratio, 5520x4160 (23.0MP)
- In 16:9 aspect ratio, 5984x3392 (20.3MP).
This sensor supports high frame rates (>=60FPS) when in binning mode and both RAW8 and RAW10 output modes.
In this version of the driver, support has been provided for the following resolutions:
W x H SZ MAX_FPS BINNING
- 5520x4160 23.0MP 23 No - 5984x3392 20.3MP 26 No - 2992x1696 3.8MP 60 Yes - 1424x800 1.2MP 120 Yes
Note 1: The "standard" camera assy for IMX300 also contains an actuator (to focus the image), but this driver only manages the actual image sensor.
Note 2: The command tables for this sensor were reverse engineered from a downstream "userspace driver" that has been released in various versions on various Xperia smartphones. Register layout seems to be only vaguely similar to IMX219, which has a public datasheet from where some names for the figured out registers were taken and added to the driver:
these names are probably not the right ones, but they surely represent the intended thing.
--- drivers/media/i2c/Kconfig | 13 + drivers/media/i2c/Makefile | 1 + drivers/media/i2c/imx300.c | 3081 ++++++++++++++++++++++++++++++++++++ 3 files changed, 3095 insertions(+) create mode 100644 drivers/media/i2c/imx300.c
drivers/media/i2c/Kconfig +13 -0
@@ -801,6 +801,19 @@ config VIDEO_IMX290 To compile this driver as a module, choose M here: the module will be called imx290. +config VIDEO_IMX300 + tristate "Sony IMX300 Exmor RS sensor support" + depends on I2C && VIDEO_V4L2 + select MEDIA_CONTROLLER + select VIDEO_V4L2_SUBDEV_API + select V4L2_FWNODE + help + This is a Video4Linux2 sensor driver for the Sony + IMX300 Exmor RS multi-aspect sensor. + + To compile this driver as a module, choose M here: the + module will be called imx300. + config VIDEO_IMX319 tristate "Sony IMX319 sensor support" depends on I2C && VIDEO_V4L2
drivers/media/i2c/Makefile +1 -0
@@ -117,6 +117,7 @@ obj-$(CONFIG_VIDEO_IMX219) += imx219.o obj-$(CONFIG_VIDEO_IMX258) += imx258.o obj-$(CONFIG_VIDEO_IMX274) += imx274.o obj-$(CONFIG_VIDEO_IMX290) += imx290.o +obj-$(CONFIG_VIDEO_IMX300) += imx300.o obj-$(CONFIG_VIDEO_IMX319) += imx319.o obj-$(CONFIG_VIDEO_IMX355) += imx355.o obj-$(CONFIG_VIDEO_MAX9286) += max9286.o
drivers/media/i2c/imx300.c new file +3081 -0
2,974 lines not on this page · raw message
The rest of this diff is not shown. View the raw message.
Signature · 1 line
2.29.2
Part of the diff is not shown here. View the raw message.
Re: [PATCH v3 1/2] media: i2c: Add driver for the Sony Exmor-RS IMX300 camera sensor
Hello,
Quoted 37 lines · On Fri, Nov 27, 2020 at 11:30:46PM +0100, [email protected] wrote:
From: AngeloGioacchino Del Regno <[email protected]>
This is a custom multi-aspect 25MegaPixels sensor from Sony, found in many Sony Xperia smartphones from various eras.
The camera assembly for this sensor usually (at least in Xperia phones) has a lens that does not cover the entire sensor area, which means that the real corners are blind and that, in many lighting conditions, some more pixels in the corners are very getting obscured (as no decent amount of light can get in)... so, the maximum resolution that can produce a good image is:
- In 4:3 aspect ratio, 5520x4160 (23.0MP)
- In 16:9 aspect ratio, 5984x3392 (20.3MP).This sensor supports high frame rates (>=60FPS) when in binning mode and both RAW8 and RAW10 output modes.
In this version of the driver, support has been provided for the following resolutions:
W x H SZMAX_FPSBINNING- 5520x4160 23.0MP 23 No - 5984x3392 20.3MP 26 No - 2992x1696 3.8MP 60 Yes - 1424x800 1.2MP 120 YesNote 1: The "standard" camera assy for IMX300 also contains an actuator (to focus the image), but this driver only manages the actual image sensor.
Note 2: The command tables for this sensor were reverse engineered from a downstream "userspace driver" that has been released in various versions on various Xperia smartphones. Register layout seems to be only vaguely similar to IMX219, which has a public datasheet from where some names for the figured out registers were taken and added to the driver:
these names are probably not the right ones, but they surely represent the intended thing.
Quoted 1 line
Signed-off-by AngeloGioacchino Del Regno <[email protected]>
Just a few drive-by comments, as I'm looking at selection targets for other imx sensors and I've noticed this one on the list
Quoted 7 lines
--- drivers/media/i2c/Kconfig | 13 + drivers/media/i2c/Makefile | 1 + drivers/media/i2c/imx300.c | 3081 ++++++++++++++++++++++++++++++++++++ 3 files changed, 3095 insertions(+) create mode 100644 drivers/media/i2c/imx300.c
[snip]
Quoted 4 lines
+/*
+ * ** IMX300 native and active pixel array size **
+ *
+ * Being this a multi-aspect sensor, the following native W/H apply, but
Quoted 8 lines
+ * beware: the module assembly usually has a (round) lens that is shadowing
+ * or covering the corners of the (square) image sensor, so the maximum
+ * output resolution must be lower than the maximum sensor resolution
+ * otherwise we get something like a view from a porthole... :)
+ *
+ * For 4:3 aspect ratio, max is: 5984x4140 (25MP)
+ * For 16:9 aspect ratio, max is: 5984x3392 (20.3MP)
+ */
These are the sizes
drivers/media/i2c/imx300.c line 189
Quoted 6 lines
drivers/media/i2c/imx300.c new file +6
-0+#define IMX300_NATIVE_WIDTH 5980U +#define IMX300_NATIVE_HEIGHT 4140U +#define IMX300_PIXEL_ARRAY_LEFT 0U +#define IMX300_PIXEL_ARRAY_TOP 0U +#define IMX300_PIXEL_ARRAY_WIDTH 5520U +#define IMX300_PIXEL_ARRAY_HEIGHT 4160U
And here is how they are applied to selection targets
Quoted 18 lines
drivers/media/i2c/imx300.c new file +18
-0+case V4L2_SEL_TGT_CROP : { +struct imx300 *imx300 =to_imx300 (sd); + +mutex_lock (&imx300->mutex); + sel->r = *__imx300_get_pad_crop (imx300, cfg, sel->pad, + sel->which); +mutex_unlock (&imx300->mutex); + +return 0 ; + } + +case V4L2_SEL_TGT_NATIVE_SIZE : + sel->r.top =0 ; + sel->r.left =0 ; + sel->r.width =IMX300_NATIVE_WIDTH ; + sel->r.height =IMX300_NATIVE_HEIGHT ; + +return 0 ;
drivers/media/i2c/imx300.c line 2456
Quoted 10 lines
drivers/media/i2c/imx300.c new file +10
-0+ +case V4L2_SEL_TGT_CROP_DEFAULT : + sel->r.top =IMX300_PIXEL_ARRAY_TOP ; + sel->r.left =IMX300_PIXEL_ARRAY_LEFT ; + sel->r.width =IMX300_PIXEL_ARRAY_WIDTH ; + sel->r.height =IMX300_PIXEL_ARRAY_HEIGHT ; + +return 0 ; + } +
1) CROP_DEFAULT should be contained in NATIVE_SIZE (actually all targets are contained in NATIVE_SIZE, and are defined relatively to it). This means that IMX300_PIXEL_ARRAY_HEIGHT > IMX300_NATIVE_HEIGHT is probably wrong.
2) You need to specify CROP_BOUNDS too. If the driver does not support reading optically black pixels using the selection API, it should be identical to CROP_DEFAULT (v4l2-compliance should report that, see
https://git.linuxtv.org/media_tree.git/commit/?id=1ed36ecd1459b653cced8929bfb37dba94b64c5d
3) CROP_DEFAULT (and BOUNDS) should report the sensor readable active area. You have one mode where the TGT_CROP rectangle width is bigger than the CROP_DEFAULT (and BOUNDS) rectangle width (5984 > 5520). I think 5984 should probably be made the CROP_DEFAULT (and BOUNDS) width then, as the 5984 mode's width implies the 464 pixels exceeding 5520 are readable and valid for image capture.
It's not easy to get these right with a datasheet sometimes, without one it quickly becomes a matter of guessing. But even if the values are computed as 'best effort' we should try to respect the relationships between the different selection targets.
Thanks
j
PATCH v3 2/2 media: dt-bindings: media: i2c: Add IMX300 CMOS sensor binding
In reply to [email protected]From: AngeloGioacchino Del Regno <[email protected]>
Add YAML device tree binding for IMX300 CMOS image sensor, and the relevant MAINTAINERS entries.
--- .../bindings/media/i2c/sony,imx300.yaml | 112 ++++++++++++++++++ MAINTAINERS | 7 ++ 2 files changed, 119 insertions(+) create mode 100644 Documentation/devicetree/bindings/media/i2c/sony,imx300.yaml
Documentation/devicetree/bindings/media/i2c/sony,imx300.yaml new file +112 -0
@@ -0,0 +1,112 @@ +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) +%YAML 1.2 +--- +$id: http://devicetree.org/schemas/media/i2c/sony,imx300.yaml# +$schema: http://devicetree.org/meta-schemas/core.yaml# + +title: Sony 1/2.3-Inch 25Mpixel Stacked CMOS Digital Image Sensor + +maintainers: + - AngeloGioacchino Del Regno <[email protected]> + +description: |- + The Sony IMX300 is a 1/2.3-inch Stacked CMOS (Exmor-RS) digital image + sensor with a pixel size of 1.08um and an active array size of + 5948H x 4140V. It is programmable through I2C interface at address 0x10. + Image data is sent through MIPI CSI-2, which is configured as either 2 or + 4 data lanes. + +properties: + compatible: + const: sony,imx300 + + reg: + maxItems: 1 + + clocks: + maxItems: 1 + + vdig-supply: + description: + Digital I/O voltage supply, 1.15-1.20 volts + + vana-supply: + description: + Analog voltage supply, 2.2 volts + + vddl-supply: + description: + Digital core voltage supply, 1.8 volts + + reset-gpios: + description: |- + Reference to the GPIO connected to the xclr pin, if any. + Must be released (set high) after all supplies are applied. + + # See ../video-interfaces.txt for more details + port: + type: object + properties: + endpoint: + type: object + + properties: + data-lanes: + description: |- + The driver only supports four-lane operation. + items: + - const: 0 + - const: 1 + - const: 2 + - const: 3 + + clock-noncontinuous: true + + link-frequencies: + $ref: /schemas/types.yaml#/definitions/uint64-array + description: + Allowed data bus frequencies. The driver currently needs + to switch between 780000000 and 480000000 Hz in order to + guarantee functionality of all modes. + + required: + - data-lanes + - link-frequencies + +required: + - compatible + - reg + - clocks + - vana-supply + - vdig-supply + - vddl-supply + - port + +additionalProperties: false + +examples: + - | + i2c0 { + #address-cells = <1>; + #size-cells = <0>; + + imx300: sensor@10 { + compatible = "sony,imx300"; + reg = <0x10>; + clocks = <&imx300_xclk>; + vana-supply = <&imx300_vana>; /* 2.2v */ + vdig-supply = <&imx300_vdig>; /* 1.2v */ + vddl-supply = <&imx300_vddl>; /* 1.8v */ + + port { + imx300_0: endpoint { + remote-endpoint = <&csi1_ep>; + data-lanes = <0 1 2 3>; + clock-noncontinuous; + link-frequencies = /bits/ 64 <780000000 480000000>; + }; + }; + }; + }; + +...
MAINTAINERS +7 -0
@@ -16279,6 +16279,13 @@ T: git git://linuxtv.org/media_tree.git F: Documentation/devicetree/bindings/media/i2c/imx290.txt F: drivers/media/i2c/imx290.c +SONY IMX300 SENSOR DRIVER +M: AngeloGioacchino Del Regno <[email protected]> +L: [email protected] +S: Maintained +F: Documentation/devicetree/bindings/media/i2c/sony,imx300.yaml +F: drivers/media/i2c/imx300.c + SONY IMX319 SENSOR DRIVER M: Bingbu Cao <[email protected]> L: [email protected]
Signature · 1 line
2.29.2
Re: [PATCH v3 2/2] media: dt-bindings: media: i2c: Add IMX300 CMOS sensor binding
Quoted 5 lines · On Fri, 27 Nov 2020 23:30:47 +0100, [email protected] wrote:
From: AngeloGioacchino Del Regno <[email protected]>
Add YAML device tree binding for IMX300 CMOS image sensor, and the relevant
MAINTAINERSentries.
Quoted 7 lines
Signed-off-by AngeloGioacchino Del Regno <[email protected]>--- .../bindings/media/i2c/sony,imx300.yaml | 112 ++++++++++++++++++ MAINTAINERS | 7 ++ 2 files changed, 119 insertions(+) create mode 100644 Documentation/devicetree/bindings/media/i2c/sony,imx300.yaml