Skip to contentSkip to conversation

media: i2c: Add ov9734 image sensor driver

PATCH In review Bingbu Cao 5 participants 8 messages Oct 29, 2020 – Nov 19, 2020

Subsystems (3)
lore
Share & tools
lore

Bingbu Cao

author
#0 lore raw

PATCH 1/1 media: i2c: Add ov9734 image sensor driver

From: Tianshu Qiu <[email protected]>

Add a v4l2 sub-device driver for the OminiVision ov9734 image sensor which can deliver maximum 720p image frames at 30 fps. This driver also add vertical blanking, exposure, test pattern, digital and analog gain control for the image sensor.

---
 MAINTAINERS                |    8 +
 drivers/media/i2c/Kconfig  |   14 +
 drivers/media/i2c/Makefile |    1 +
 drivers/media/i2c/ov9734.c | 1018 ++++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 1041 insertions(+)
 create mode 100644 drivers/media/i2c/ov9734.c
MAINTAINERS +8 -0
index 7a12633747c8..dba67504ef8c 100644
@@ -12857,6 +12857,14 @@ T:	git git://linuxtv.org/media_tree.git
 F:	Documentation/devicetree/bindings/media/i2c/ov9650.txt
 F:	drivers/media/i2c/ov9650.c
 
+OMNIVISION OV9734 SENSOR DRIVER
+M:	Tianshu Qiu <[email protected]>
+R:	Bingbu Cao <[email protected]>
+L:	[email protected]
+S:	Maintained
+T:	git git://linuxtv.org/media_tree.git
+F:	drivers/media/i2c/ov9734.c
+
 ONENAND FLASH DRIVER
 M:	Kyungmin Park <[email protected]>
 L:	[email protected]
drivers/media/i2c/Kconfig +14 -0
index 878f66ef2719..f4f4072bccc7 100644
@@ -1050,6 +1050,20 @@ config VIDEO_OV9650
 	  This is a V4L2 sensor driver for the Omnivision
 	  OV9650 and OV9652 camera sensors.
 
+config VIDEO_OV9734
+	tristate "OmniVision OV9734 sensor support"
+	depends on VIDEO_V4L2 && I2C
+	depends on ACPI || COMPILE_TEST
+	select MEDIA_CONTROLLER
+	select VIDEO_V4L2_SUBDEV_API
+	select V4L2_FWNODE
+	help
+	  This is a Video4Linux2 sensor driver for the OmniVision
+	  OV9734 camera.
+
+	  To compile this driver as a module, choose M here: the
+	  module's name is ov9734.
+
 config VIDEO_OV13858
 	tristate "OmniVision OV13858 sensor support"
 	depends on I2C && VIDEO_V4L2
drivers/media/i2c/Makefile +1 -0
index f0a77473979d..b448506ce503 100644
@@ -83,6 +83,7 @@ obj-$(CONFIG_VIDEO_OV7740) += ov7740.o
 obj-$(CONFIG_VIDEO_OV8856) += ov8856.o
 obj-$(CONFIG_VIDEO_OV9640) += ov9640.o
 obj-$(CONFIG_VIDEO_OV9650) += ov9650.o
+obj-$(CONFIG_VIDEO_OV9734) += ov9734.o
 obj-$(CONFIG_VIDEO_OV13858) += ov13858.o
 obj-$(CONFIG_VIDEO_MT9M001) += mt9m001.o
 obj-$(CONFIG_VIDEO_MT9M032) += mt9m032.o
drivers/media/i2c/ov9734.c new file +1018 -0

1,018 lines not on this page · raw message

Signature · 1 line
2.7.4
Reviewed-by on v1 1/1
#1 lore raw

On (20/10/29 10:59), Bingbu Cao wrote:
[..]

Looks good to me,

Several comments below.

Quoted 8 lines
drivers/media/i2c/ov9734.c new file +8 -0
+static int ov9734_set_ctrl(struct v4l2_ctrl *ctrl)
+{
+	struct ov9734 *ov9734 = container_of(ctrl->handler,
+					     struct ov9734, ctrl_handler);
+	struct i2c_client *client = v4l2_get_subdevdata(&ov9734->sd);
+	s64 exposure_max;
+	int ret = 0;
+

drivers/media/i2c/ov9734.c line 473

Quoted 10 lines
drivers/media/i2c/ov9734.c new file +10 -0
+	/* Propagate change of current control to all related controls */
+	if (ctrl->id == V4L2_CID_VBLANK) {
+		/* Update max exposure while meeting expected vblanking */
+		exposure_max = ov9734->cur_mode->height + ctrl->val -
+			OV9734_EXPOSURE_MAX_MARGIN;
+		__v4l2_ctrl_modify_range(ov9734->exposure,
+					 ov9734->exposure->minimum,
+					 exposure_max, ov9734->exposure->step,
+					 exposure_max);
+	}

Should this be done after pm_runtime_get_if_in_use()?

drivers/media/i2c/ov9734.c line 478

Quoted 4 lines
drivers/media/i2c/ov9734.c new file +4 -0
+	/* V4L2 controls values will be applied only when power is already up */
+	if (!pm_runtime_get_if_in_use(&client->dev))
+		return 0;
+

Here.

[..]

Quoted 18 lines
drivers/media/i2c/ov9734.c new file +18 -0
+static int ov9734_set_stream(struct v4l2_subdev *sd, int enable)
+{
+	struct ov9734 *ov9734 = to_ov9734(sd);
+	struct i2c_client *client = v4l2_get_subdevdata(sd);
+	int ret = 0;
+
+	if (ov9734->streaming == enable)
+		return 0;
+
+	mutex_lock(&ov9734->mutex);
+	if (enable) {
+		ret = pm_runtime_get_sync(&client->dev);
+		if (ret < 0) {
+			pm_runtime_put_noidle(&client->dev);
+			mutex_unlock(&ov9734->mutex);
+			return ret;
+		}
+

drivers/media/i2c/ov9734.c line 661

Quoted 10 lines
drivers/media/i2c/ov9734.c new file +10 -0
+		ret = ov9734_start_streaming(ov9734);
+		if (ret) {
+			enable = 0;
+			ov9734_stop_streaming(ov9734);
+			pm_runtime_put(&client->dev);
+		}
+	} else {
+		ov9734_stop_streaming(ov9734);
+		pm_runtime_put(&client->dev);
+	}

I assume that we will never see erroneous ->s_stream(0) calls or
->s_stream(0) after unsuccessful ->s_stream(1). Otherwise we will
do extra pm_runtime_put(), not matched by pm_runtime_get().

	-ss
#2 lore raw
Quoted 24 lines · Sergey Senozhatsky
Quoted 24 lines
drivers/media/i2c/ov9734.c new file +24 -0
+static int ov9734_set_stream(struct v4l2_subdev *sd, int enable)
+{
+	struct ov9734 *ov9734 = to_ov9734(sd);
+	struct i2c_client *client = v4l2_get_subdevdata(sd);
+	int ret = 0;
+
+	if (ov9734->streaming == enable)
+		return 0;
+
+	mutex_lock(&ov9734->mutex);
+	if (enable) {
+		ret = pm_runtime_get_sync(&client->dev);
+		if (ret < 0) {
+			pm_runtime_put_noidle(&client->dev);
+			mutex_unlock(&ov9734->mutex);
+			return ret;
+		}
+
+		ret = ov9734_start_streaming(ov9734);
+		if (ret) {
+			enable = 0;
+			ov9734_stop_streaming(ov9734);
+			pm_runtime_put(&client->dev);
+		}
Quoted 8 lines

drivers/media/i2c/ov9734.c line 661

Quoted 4 lines
drivers/media/i2c/ov9734.c new file +4 -0
+	} else {
+		ov9734_stop_streaming(ov9734);
+		pm_runtime_put(&client->dev);
+	}

I assume that we will never see erroneous ->s_stream(0) calls or
->s_stream(0) after unsuccessful ->s_stream(1). Otherwise we will
do extra pm_runtime_put(), not matched by pm_runtime_get().

Oh, no. There is (unprotected) if (ov9734->streaming) check at the very top, so we are probably fine.

	-ss
#3 lore raw
Quoted 35 lines · Sergey Senozhatsky
Quoted 24 lines · Sergey Senozhatsky
Quoted 24 lines
drivers/media/i2c/ov9734.c new file +24 -0
+static int ov9734_set_stream(struct v4l2_subdev *sd, int enable)
+{
+   struct ov9734 *ov9734 = to_ov9734(sd);
+   struct i2c_client *client = v4l2_get_subdevdata(sd);
+   int ret = 0;
+
+   if (ov9734->streaming == enable)
+           return 0;
+
+   mutex_lock(&ov9734->mutex);
+   if (enable) {
+           ret = pm_runtime_get_sync(&client->dev);
+           if (ret < 0) {
+                   pm_runtime_put_noidle(&client->dev);
+                   mutex_unlock(&ov9734->mutex);
+                   return ret;
+           }
+
+           ret = ov9734_start_streaming(ov9734);
+           if (ret) {
+                   enable = 0;
+                   ov9734_stop_streaming(ov9734);
+                   pm_runtime_put(&client->dev);
+           }
Quoted 8 lines

drivers/media/i2c/ov9734.c line 661

Quoted 4 lines
drivers/media/i2c/ov9734.c new file +4 -0
+   } else {
+           ov9734_stop_streaming(ov9734);
+           pm_runtime_put(&client->dev);
+   }

I assume that we will never see erroneous ->s_stream(0) calls or
->s_stream(0) after unsuccessful ->s_stream(1). Otherwise we will
do extra pm_runtime_put(), not matched by pm_runtime_get().

Quoted 2 lines

Oh, no. There is (unprotected) if (ov9734->streaming) check at the very top, so we are probably fine.

Hmm, should it be protected?

maintainer
#4 lore raw

Hi Tomasz,

Quoted 37 lines · Tomasz Figa
Quoted 35 lines · Sergey Senozhatsky
Quoted 32 lines · Sergey Senozhatsky
Quoted 18 lines
drivers/media/i2c/ov9734.c new file +18 -0
+static int ov9734_set_stream(struct v4l2_subdev *sd, int enable)
+{
+   struct ov9734 *ov9734 = to_ov9734(sd);
+   struct i2c_client *client = v4l2_get_subdevdata(sd);
+   int ret = 0;
+
+   if (ov9734->streaming == enable)
+           return 0;
+
+   mutex_lock(&ov9734->mutex);
+   if (enable) {
+           ret = pm_runtime_get_sync(&client->dev);
+           if (ret < 0) {
+                   pm_runtime_put_noidle(&client->dev);
+                   mutex_unlock(&ov9734->mutex);
+                   return ret;
+           }
+

drivers/media/i2c/ov9734.c line 661

Quoted 10 lines
drivers/media/i2c/ov9734.c new file +10 -0
+           ret = ov9734_start_streaming(ov9734);
+           if (ret) {
+                   enable = 0;
+                   ov9734_stop_streaming(ov9734);
+                   pm_runtime_put(&client->dev);
+           }
+   } else {
+           ov9734_stop_streaming(ov9734);
+           pm_runtime_put(&client->dev);
+   }

I assume that we will never see erroneous ->s_stream(0) calls or
->s_stream(0) after unsuccessful ->s_stream(1). Otherwise we will
do extra pm_runtime_put(), not matched by pm_runtime_get().

Quoted 4 lines
Quoted 2 lines

Oh, no. There is (unprotected) if (ov9734->streaming) check at the very top, so we are probably fine.

Hmm, should it be protected?

In principle, yes.

The patch is already in a pull request I've sent to Mauro, so any changes should be on top.

Signature · 3 lines
Kind regards,

Sakari Ailus
#5 lore raw

Tomasz and Sakari, thanks.

Quoted 43 lines · Sakari Ailus

Hi Tomasz,

Quoted 40 lines · Tomasz Figa
Quoted 37 lines · Sergey Senozhatsky
Quoted 24 lines · Sergey Senozhatsky
Quoted 24 lines
drivers/media/i2c/ov9734.c new file +24 -0
+static int ov9734_set_stream(struct v4l2_subdev *sd, int enable)
+{
+   struct ov9734 *ov9734 = to_ov9734(sd);
+   struct i2c_client *client = v4l2_get_subdevdata(sd);
+   int ret = 0;
+
+   if (ov9734->streaming == enable)
+           return 0;
+
+   mutex_lock(&ov9734->mutex);
+   if (enable) {
+           ret = pm_runtime_get_sync(&client->dev);
+           if (ret < 0) {
+                   pm_runtime_put_noidle(&client->dev);
+                   mutex_unlock(&ov9734->mutex);
+                   return ret;
+           }
+
+           ret = ov9734_start_streaming(ov9734);
+           if (ret) {
+                   enable = 0;
+                   ov9734_stop_streaming(ov9734);
+                   pm_runtime_put(&client->dev);
+           }
Quoted 8 lines

drivers/media/i2c/ov9734.c line 661

Quoted 4 lines
drivers/media/i2c/ov9734.c new file +4 -0
+   } else {
+           ov9734_stop_streaming(ov9734);
+           pm_runtime_put(&client->dev);
+   }

I assume that we will never see erroneous ->s_stream(0) calls or
->s_stream(0) after unsuccessful ->s_stream(1). Otherwise we will
do extra pm_runtime_put(), not matched by pm_runtime_get().

Oh, no. There is (unprotected) if (ov9734->streaming) check at the very top, so we are probably fine.

Quoted 6 lines
Quoted 1 line

Hmm, should it be protected?

In principle, yes.

The patch is already in a pull request I've sent to Mauro, so any changes should be on top.

Will submit a patch based on the top.

Quoted 1 line
Signature · 2 lines
Best regards,
Bingbu Cao
#6 lore raw
In reply to Sergey Senozhatsky

Sergey, thanks for review.

Quoted 20 lines · Sergey Senozhatsky

On (20/10/29 10:59), Bingbu Cao wrote:
[..]

Looks good to me,

Several comments below.

Quoted 12 lines
drivers/media/i2c/ov9734.c new file +12 -0
+static int ov9734_set_ctrl(struct v4l2_ctrl *ctrl)
+{
+	struct ov9734 *ov9734 = container_of(ctrl->handler,
+					     struct ov9734, ctrl_handler);
+	struct i2c_client *client = v4l2_get_subdevdata(&ov9734->sd);
+	s64 exposure_max;
+	int ret = 0;
+
+	/* Propagate change of current control to all related controls */
+	if (ctrl->id == V4L2_CID_VBLANK) {
+		/* Update max exposure while meeting expected vblanking */
+		exposure_max = ov9734->cur_mode->height + ctrl->val -
Quoted 8 lines

drivers/media/i2c/ov9734.c line 473

Quoted 6 lines
drivers/media/i2c/ov9734.c new file +6 -0
+			OV9734_EXPOSURE_MAX_MARGIN;
+		__v4l2_ctrl_modify_range(ov9734->exposure,
+					 ov9734->exposure->minimum,
+					 exposure_max, ov9734->exposure->step,
+					 exposure_max);
+	}

Should this be done after pm_runtime_get_if_in_use()?

My inaccurate understanding - as v4l2 see that this control was set by sub-device without error, so it will record the value from the user as the new value, so update the exposure range to allow user to set the exposure as well even it did not take affect.

Sakari, do you have any comments about this?

Quoted 45 lines

drivers/media/i2c/ov9734.c line 478

Quoted 5 lines
drivers/media/i2c/ov9734.c new file +4 -0
+	/* V4L2 controls values will be applied only when power is already up */
+	if (!pm_runtime_get_if_in_use(&client->dev))
+		return 0;
+

Here.

[..]

Quoted 18 lines
drivers/media/i2c/ov9734.c new file +18 -0
+static int ov9734_set_stream(struct v4l2_subdev *sd, int enable)
+{
+	struct ov9734 *ov9734 = to_ov9734(sd);
+	struct i2c_client *client = v4l2_get_subdevdata(sd);
+	int ret = 0;
+
+	if (ov9734->streaming == enable)
+		return 0;
+
+	mutex_lock(&ov9734->mutex);
+	if (enable) {
+		ret = pm_runtime_get_sync(&client->dev);
+		if (ret < 0) {
+			pm_runtime_put_noidle(&client->dev);
+			mutex_unlock(&ov9734->mutex);
+			return ret;
+		}
+

drivers/media/i2c/ov9734.c line 661

Quoted 10 lines
drivers/media/i2c/ov9734.c new file +10 -0
+		ret = ov9734_start_streaming(ov9734);
+		if (ret) {
+			enable = 0;
+			ov9734_stop_streaming(ov9734);
+			pm_runtime_put(&client->dev);
+		}
+	} else {
+		ov9734_stop_streaming(ov9734);
+		pm_runtime_put(&client->dev);
+	}

I assume that we will never see erroneous ->s_stream(0) calls or
->s_stream(0) after unsuccessful ->s_stream(1). Otherwise we will
do extra pm_runtime_put(), not matched by pm_runtime_get().

	-ss
Signature · 2 lines
Best regards,
Bingbu Cao
maintainer
#7 lore raw

Hi Bingbu, Sergey,

Quoted 32 lines · Bingbu Cao

Sergey, thanks for review.

Quoted 20 lines · Sergey Senozhatsky

On (20/10/29 10:59), Bingbu Cao wrote:
[..]

Looks good to me,

Several comments below.

Quoted 12 lines
drivers/media/i2c/ov9734.c new file +12 -0
+static int ov9734_set_ctrl(struct v4l2_ctrl *ctrl)
+{
+	struct ov9734 *ov9734 = container_of(ctrl->handler,
+					     struct ov9734, ctrl_handler);
+	struct i2c_client *client = v4l2_get_subdevdata(&ov9734->sd);
+	s64 exposure_max;
+	int ret = 0;
+
+	/* Propagate change of current control to all related controls */
+	if (ctrl->id == V4L2_CID_VBLANK) {
+		/* Update max exposure while meeting expected vblanking */
+		exposure_max = ov9734->cur_mode->height + ctrl->val -
Quoted 8 lines

drivers/media/i2c/ov9734.c line 473

Quoted 6 lines
drivers/media/i2c/ov9734.c new file +6 -0
+			OV9734_EXPOSURE_MAX_MARGIN;
+		__v4l2_ctrl_modify_range(ov9734->exposure,
+					 ov9734->exposure->minimum,
+					 exposure_max, ov9734->exposure->step,
+					 exposure_max);
+	}

Should this be done after pm_runtime_get_if_in_use()?

Quoted 5 lines

My inaccurate understanding - as v4l2 see that this control was set by sub-device without error, so it will record the value from the user as the new value, so update the exposure range to allow user to set the exposure as well even it did not take affect.

Sakari, do you have any comments about this?

Just changing the range does not require powering on the device. So this is as intended AFAIU.

Signature · 3 lines
Regards,

Sakari Ailus