Add support for the Sony IMX681 camera sensor

The Sony IMX681 is the user-facing camera on the Microsoft Surface Pro 11 for Business (Intel Lunar Lake, IPU7), enumerated as ACPI device SONY0681. Without a driver the camera does not appear at all - not as a degraded device, not at all.

  1/3  dt-bindings: media: Add Sony IMX681            (mine)
  2/3  media: i2c: Add Sony IMX681 sensor driver      (Andre Gilerson's)
  3/3  media: ipu-bridge: Add Sony IMX681             (mine)

The driver is Andre's work, reverse-engineered from I2C traces taken under Windows. I am carrying the submission, not the code: his Signed-off-by is on 2/3 with mine beneath it as the person passing it on.

Changes in v4
=============

All three are Krzysztof Kozlowski's, from his review of v3 on 11 September.

- MAINTAINERS: the T: line is gone. He is right that neither Andre nor I have commit rights to that tree, and the line was ours rather than his - the MAINTAINERS hunk went in with v2.

- dev_err() followed by a return is now dev_err_probe(). He flagged one site, at the CCI init. The same shape was in twelve others on the probe path, so all thirteen are converted rather than only the one he quoted. Seven further sites are left alone, in imx681_start_streaming(), imx681_stop_streaming() and imx681_power_on(): dev_err_probe() is wrong outside probe, and power_on() is reached from runtime-PM resume as well as from probe, so it is not a probe path even though probe calls it.

- The clock-rate dev_dbg is gone. clk_summary says it.

Nothing else changed. No behaviour changes except that the converted sites now return the error through dev_err_probe() instead of beside it.

Trailers, and a correction I owe
================================

Krzysztof's Reviewed-by is on 1/3, and on 1/3 only.

German Pablo Lindo's Tested-by is on 2/3 and never on 1/3, per Krzysztof's "Just remember that Tested-by does not apply here".

The correction, in the open because the error was: the v3 cover letter called the tester by a surname I had taken from his email address rather than from him. It was wrong. His name is German Pablo Lindo, and the Tested-by below is his own wording.

Andre, and why this goes out before your ack
============================================

Krzysztof asked that Andre ack the binding, since it names him as maintainer. That is properly his and he is away until 28 September, so this version does not have it.

It goes now anyway, for a reason that is about the tags rather than about
impatience: patchwork.linuxtv.org populates its tags field for nobody on this instance - not for our two patches and not for four accepted ones I checked against today - so a maintainer applying from patchwork picks up neither trailer.
They exist in the lore archive and in these commit messages, and nowhere else. A v4 is the only thing that carries them.

Andre - the ack is yours whenever you are back, and nothing here needs it in order to be read or reviewed.

Not addressed by this version: nobody has commented on 2/3 or 3/3 yet, and the mirroring question raised against v3 is still open with Sakari, Dan and Hans. If any of that lands, it lands in v5.

Checked
=======

checkpatch --strict, sparse and W=1 are all clean on 2/3 and 3/3. 1/3's only checkpatch warning asks whether MAINTAINERS needs updating, which 2/3 does.

Built and run on the machine rather than only compiled: the driver probes, ipu-bridge binds it at 3844x2640 over two lanes at 969.6 MHz per lane, and captured frames render as a coherent image rather than as noise.

Limitations
===========

error paths The regulator get succeeds here because INT3472 hands out
dummies for dvdd and dovdd, so the fatal path is reasoned rather than exercised.

IVSC None of the IVSC HIDs is present on this machine, so

                    sensor->csi_dev is NULL and ipu-bridge never enters those
                    paths.

one machine One Surface Pro 11 and one sensor sample. German has the
same HID on another machine but reaches IPU7 through the staging driver, which is a second path rather than a second test of this one.

one mode 3844x2640 SGRBG10 at 969.6 MHz per lane is the only
configuration the driver has, and the only one measured.

no sanitisers Not built with KASAN or lockdep. That is a kernel rebuild
here rather than a problem, so ask if it would help.

Based on media/next at aedd77ea8168 ("media: qcom: camss: use fwnode_graph_for_each_endpoint_scoped() to simplify code"). That branch rewinds - v3's declared base is no longer in it either - so if this does not apply where you are, say so and I will rebase rather than guess.

German reported this HID as a bug on this list on 3 September and has had no reply since; he is on Cc here.

https://lore.kernel.org/linux-media/[email protected]/

2.54.0 (Apple Git-157)

The Sony IMX681 is a CMOS image sensor used as the user-facing camera on several Microsoft Surface devices. It operates from analog 2.8 V, digital 1.05 V and interface 1.8 V supplies, is programmable over I2C, and outputs 10-bit Bayer data over a two-lane MIPI CSI-2 D-PHY interface.

---
 .../bindings/media/i2c/sony,imx681.yaml       | 107 ++++++++++++++++++
 1 file changed, 107 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/media/i2c/sony,imx681.yaml

new file mode 100644
index 0000000000..6d2e4ddcee
@@ -0,0 +1,107 @@
+# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/media/i2c/sony,imx681.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Sony IMX681 CMOS Image Sensor
+
+maintainers:
+  - Andre Gilerson <[email protected]>
+
+description:
+  The Sony IMX681 is a CMOS active pixel image sensor with a square pixel
+  array, used as the user-facing camera on several Microsoft Surface devices.
+  It operates from analog 2.8 V, digital 1.05 V and interface 1.8 V supplies,
+  is programmable over I2C, and outputs 10-bit Bayer data over a two-lane
+  MIPI CSI-2 D-PHY interface.
+
+allOf:
+  - $ref: /schemas/media/video-interface-devices.yaml#
+
+properties:
+  compatible:
+    const: sony,imx681
+
+  reg:
+    maxItems: 1
+
+  clocks:
+    description: Input clock (19.2 MHz)
+    maxItems: 1
+
+  avdd-supply:
+    description: Analog power supply (2.8 V)
+
+  dvdd-supply:
+    description: Digital power supply (1.05 V)
+
+  dovdd-supply:
+    description: Interface power supply (1.8 V)
+
+  reset-gpios:
+    description: Sensor reset (XCLR) GPIO, active low
+    maxItems: 1
+
+  port:
+    $ref: /schemas/graph.yaml#/$defs/port-base
+    unevaluatedProperties: false
+
+    properties:
+      endpoint:
+        $ref: /schemas/media/video-interfaces.yaml#
+        unevaluatedProperties: false
+
+        properties:
+          data-lanes:
+            items:
+              - const: 1
+              - const: 2
+
+        required:
+          - data-lanes
+          - link-frequencies
+
+    required:
+      - endpoint
+
+required:
+  - compatible
+  - reg
+  - clocks
+  - avdd-supply
+  - dvdd-supply
+  - dovdd-supply
+  - port
+
+unevaluatedProperties: false
+
+examples:
+  - |
+    #include <dt-bindings/gpio/gpio.h>
+
+    i2c {
+        #address-cells = <1>;
+        #size-cells = <0>;
+
+        camera-sensor@10 {
+            compatible = "sony,imx681";
+            reg = <0x10>;
+            clocks = <&clock_cam>;
+            avdd-supply = <&vcc2v8_cam>;
+            dvdd-supply = <&vcc1v05_cam>;
+            dovdd-supply = <&vcc1v8_cam>;
+            reset-gpios = <&gpio 4 GPIO_ACTIVE_LOW>;
+            orientation = <0>;
+            rotation = <0>;
+
+            port {
+                imx681_ep: endpoint {
+                    data-lanes = <1 2>;
+                    link-frequencies = /bits/ 64 <969600000>;
+                    remote-endpoint = <&csi_in>;
+                };
+            };
+        };
+    };
+...

2.54.0 (Apple Git-157)

From: Andre Gilerson <[email protected]>

The Sony IMX681 is the user-facing camera on the Microsoft Surface Pro 11 for Business (Intel Lunar Lake, IPU7), where it is enumerated as ACPI device SONY0681. Without a driver the camera does not appear at all.

There is no public documentation for this sensor. The initialisation sequence and the mode registers were recovered from I2C traces taken under Windows, and the driver does not pretend otherwise: imx681_init_regs[] is 21 register writes whose individual meaning is not known.

What is not from the traces is derived and written down. The link frequency comes from the PLL configuration visible in the same traces - 19.2 MHz EXCK, PLL2_MUL 303, PLL2_PRE_DIV 3, giving 1939.2 MHz on the bus and therefore 969.6 MHz per lane - and the pixel rate follows from that, the lane count and the bit depth. The gain law and the black level were measured against the sensor rather than taken from the traces; the numbers are in the cover letter.

The driver uses the streams API, v4l2-cci for register access, the subdev state API and runtime PM, and validates the endpoint's lane count and link frequency against what the firmware describes.

---
 MAINTAINERS                |   7 +
 drivers/media/i2c/Kconfig  |  10 +
 drivers/media/i2c/Makefile |   1 +
 drivers/media/i2c/imx681.c | 874 +++++++++++++++++++++++++++++++++++++
 4 files changed, 892 insertions(+)
 create mode 100644 drivers/media/i2c/imx681.c

index 4cc4a2dc6d..4479f96d0d 100644
@@ -25609,6 +25609,13 @@ S:	Maintained
 F:	Documentation/devicetree/bindings/media/i2c/sony,imx678.yaml
 F:	drivers/media/i2c/imx678.c
 
+SONY IMX681 SENSOR DRIVER
+M:	Andre Gilerson <[email protected]>
+L:	[email protected]
+S:	Maintained
+F:	Documentation/devicetree/bindings/media/i2c/sony,imx681.yaml
+F:	drivers/media/i2c/imx681.c
+
 SONY MEMORYSTICK SUBSYSTEM
 M:	Maxim Levitsky <[email protected]>
 M:	Alex Dubov <[email protected]>

index 4d99464791..c759a2d239 100644
@@ -321,6 +321,16 @@ config VIDEO_IMX678
 	  To compile this driver as a module, choose M here: the
 	  module will be called imx678.
 
+config VIDEO_IMX681
+	tristate "Sony IMX681 sensor support"
+	select V4L2_CCI_I2C
+	help
+	  This is a Video4Linux2 sensor driver for the Sony
+	  IMX681 camera.
+
+	  To compile this driver as a module, choose M here: the
+	  module will be called imx681.
+
 config VIDEO_MAX9271_LIB
 	tristate
 

index fd1cb25718..98bcecf0c4 100644
@@ -64,6 +64,7 @@ obj-$(CONFIG_VIDEO_IMX412) += imx412.o
 obj-$(CONFIG_VIDEO_IMX415) += imx415.o
 obj-$(CONFIG_VIDEO_IMX678) += imx678.o
 obj-$(CONFIG_VIDEO_IMX471) += imx471.o
+obj-$(CONFIG_VIDEO_IMX681) += imx681.o
 obj-$(CONFIG_VIDEO_IR_I2C) += ir-kbd-i2c.o
 obj-$(CONFIG_VIDEO_ISL7998X) += isl7998x.o
 obj-$(CONFIG_VIDEO_IT6625) += it6625.o

2.54.0 (Apple Git-157)

The Microsoft Surface Pro 11 for Business (Intel Lunar Lake, IPU7) carries a Sony IMX681 as its user-facing sensor, enumerated as ACPI device SONY0681. Add it so ipu_bridge_connect_sensors() builds its half of the graph; without the entry the sensor is never connected, whatever driver is present.

One link frequency, 969.6 MHz, on two CSI-2 lanes.

---
 drivers/media/pci/intel/ipu-bridge.c | 2 ++
 1 file changed, 2 insertions(+)

index 77257311d1..a1a4023f81 100644
@@ -116,6 +116,8 @@ static const struct ipu_sensor_config ipu_supported_sensors[] = {
 	IPU_SENSOR_CONFIG("OVTI5693", 1, 419200000),
 	/* Omnivision OV8856 */
 	IPU_SENSOR_CONFIG("OVTI8856", 3, 180000000, 360000000, 720000000),
+	/* Sony IMX681 */
+	IPU_SENSOR_CONFIG("SONY0681", 1, 969600000),
 	/* Sony IMX471 */
 	IPU_SENSOR_CONFIG("SONY471A", 1, 200000000),
 	/* Sony IMX471 (found on Lenovo X1 Carbon G14) */

2.54.0 (Apple Git-157)