Skip to contentSkip to conversation

V4L2 sub-device active state helper, CCS fixes

PATCH In review Sakari Ailus 3 participants 12 messages Oct 9, 2023 – Oct 14, 2023

Subsystems (8)
lore
Share & tools
lore

Sakari Ailus

author maintainer
#0 lore raw

v2 cover V4L2 sub-device active state helper, CCS fixes

Hi folks,

This set includes a helper for working with V4L2 sub-device active state as well as a fix for the CCS driver sub-device state patch.

since v1:

- There were other drivers using __v4l2_subdev_state_alloc(). Ouch. Change those users as well. Note that this function should not be used in drivers, hence API niceness is not a high priority.

Sakari Ailus (4):
media: v4l: subdev: Set sub-device active state earlier
media: v4l: subdev: Add a helper to tell if a sub-device state is
active
media: ccs: Rework initialising sub-device state
media: ccs: Fix a (harmless) lockdep warning

 drivers/media/i2c/ccs/ccs-core.c              | 64 ++++++++++++-------
 .../platform/renesas/rcar-vin/rcar-v4l2.c     |  3 +-
 .../media/platform/renesas/vsp1/vsp1_entity.c |  3 +-
 drivers/media/v4l2-core/v4l2-subdev.c         | 14 ++--
 drivers/staging/media/tegra-video/vi.c        |  2 +-
 include/media/v4l2-subdev.h                   | 11 +++-
 6 files changed, 66 insertions(+), 31 deletions(-)
Signature · 1 line
2.39.2
author maintainer
#1 lore raw

PATCH v2 1/4 media: v4l: subdev: Set sub-device active state earlier

Set sub-device active state earlier in order to have it available to the sub-device driver when driver's init_cfg operation is called.

---
 .../media/platform/renesas/rcar-vin/rcar-v4l2.c    |  3 ++-
 drivers/media/platform/renesas/vsp1/vsp1_entity.c  |  3 ++-
 drivers/media/v4l2-core/v4l2-subdev.c              | 14 +++++++++-----
 drivers/staging/media/tegra-video/vi.c             |  2 +-
 include/media/v4l2-subdev.h                        |  3 ++-
 5 files changed, 16 insertions(+), 9 deletions(-)
drivers/media/platform/renesas/rcar-vin/rcar-v4l2.c +2 -1
index 073f70c6ac68..26c2470dc289 100644
@@ -266,7 +266,8 @@ static int rvin_try_format(struct rvin_dev *vin, u32 which,
 	 * FIXME: Drop this call, drivers are not supposed to use
 	 * __v4l2_subdev_state_alloc().
 	 */
-	sd_state = __v4l2_subdev_state_alloc(sd, "rvin:state->lock", &key);
+	sd_state = __v4l2_subdev_state_alloc(sd, "rvin:state->lock", &key,
+					     false);
 	if (IS_ERR(sd_state))
 		return PTR_ERR(sd_state);
 
drivers/media/platform/renesas/vsp1/vsp1_entity.c +2 -1
index c31f05a80bb5..ab0f47a5fd96 100644
@@ -680,7 +680,8 @@ int vsp1_entity_init(struct vsp1_device *vsp1, struct vsp1_entity *entity,
 	 * __v4l2_subdev_state_alloc().
 	 */
 	entity->config = __v4l2_subdev_state_alloc(&entity->subdev,
-						   "vsp1:config->lock", &key);
+						   "vsp1:config->lock", &key,
+						   false);
 	if (IS_ERR(entity->config)) {
 		media_entity_cleanup(&entity->subdev.entity);
 		return PTR_ERR(entity->config);
drivers/media/v4l2-core/v4l2-subdev.c +9 -5
index d295a4e87b66..cd590951c393 100644
@@ -53,7 +53,7 @@ static int subdev_fh_init(struct v4l2_subdev_fh *fh, struct v4l2_subdev *sd)
 	struct v4l2_subdev_state *state;
 	static struct lock_class_key key;
 
-	state = __v4l2_subdev_state_alloc(sd, "fh->state->lock", &key);
+	state = __v4l2_subdev_state_alloc(sd, "fh->state->lock", &key, false);
 	if (IS_ERR(state))
 		return PTR_ERR(state);
 
@@ -1426,7 +1426,7 @@ EXPORT_SYMBOL_GPL(v4l2_subdev_has_pad_interdep);
 
 struct v4l2_subdev_state *
 __v4l2_subdev_state_alloc(struct v4l2_subdev *sd, const char *lock_name,
-			  struct lock_class_key *lock_key)
+			  struct lock_class_key *lock_key, bool active)
 {
 	struct v4l2_subdev_state *state;
 	int ret;
@@ -1451,6 +1451,9 @@ __v4l2_subdev_state_alloc(struct v4l2_subdev *sd, const char *lock_name,
 		}
 	}
 
+	if (active)
+		sd->active_state = state;
+
 	/*
 	 * There can be no race at this point, but we lock the state anyway to
 	 * satisfy lockdep checks.
@@ -1470,6 +1473,9 @@ __v4l2_subdev_state_alloc(struct v4l2_subdev *sd, const char *lock_name,
 
 	kfree(state);
 
+	if (active)
+		sd->active_state = NULL;
+
 	return ERR_PTR(ret);
 }
 EXPORT_SYMBOL_GPL(__v4l2_subdev_state_alloc);
@@ -1493,12 +1499,10 @@ int __v4l2_subdev_init_finalize(struct v4l2_subdev *sd, const char *name,
 {
 	struct v4l2_subdev_state *state;
 
-	state = __v4l2_subdev_state_alloc(sd, name, key);
+	state = __v4l2_subdev_state_alloc(sd, name, key, true);
 	if (IS_ERR(state))
 		return PTR_ERR(state);
 
-	sd->active_state = state;
-
 	return 0;
 }
 EXPORT_SYMBOL_GPL(__v4l2_subdev_init_finalize);
drivers/staging/media/tegra-video/vi.c +1 -1
index e98b3010520e..733fccb157fe 100644
@@ -450,7 +450,7 @@ static int __tegra_channel_try_format(struct tegra_vi_channel *chan,
 	 * __v4l2_subdev_state_alloc().
 	 */
 	sd_state = __v4l2_subdev_state_alloc(subdev, "tegra:state->lock",
-					     &key);
+					     &key, false);
 	if (IS_ERR(sd_state))
 		return PTR_ERR(sd_state);
 	/*
include/media/v4l2-subdev.h +2 -1
index c1f90c1223a7..ea05b70389b3 100644
@@ -1346,7 +1346,8 @@ bool v4l2_subdev_has_pad_interdep(struct media_entity *entity,
  */
 struct v4l2_subdev_state *__v4l2_subdev_state_alloc(struct v4l2_subdev *sd,
 						    const char *lock_name,
-						    struct lock_class_key *key);
+						    struct lock_class_key *key,
+						    bool active);
 
 /**
  * __v4l2_subdev_state_free - free a v4l2_subdev_state
Signature · 1 line
2.39.2
#2 lore raw

Re: [PATCH v2 1/4] media: v4l: subdev: Set sub-device active state earlier

kernel test robot noticed the following build warnings:

Hi Sakari,

kernel test robot noticed the following build warnings:

[auto build test WARNING on linuxtv-media-stage/master] [also build test WARNING on linus/master v6.6-rc5 next-20231013] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Sakari-Ailus/media-v4l-subdev-Set-sub-device-active-state-earlier/20231010-061025
base:   https://git.linuxtv.org/media_stage.git master
patch link:    https://lore.kernel.org/r/20231009220906.221303-2-sakari.ailus%40linux.intel.com
patch subject: [PATCH v2 1/4] media: v4l: subdev: Set sub-device active state earlier
reproduce: (https://download.01.org/0day-ci/archive/20231014/[email protected]/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/

All warnings (new ones prefixed by >>):

>> ./include/media/v4l2-subdev.h:1350: warning: Function parameter or member 'active' not described in '__v4l2_subdev_state_alloc'

vim +1350 ./include/media/v4l2-subdev.h

1317
1318 /**
1319 * v4l2_subdev_has_pad_interdep - MC has_pad_interdep implementation for subdevs
1320 *
1321 * @entity: pointer to &struct media_entity
1322 * @pad0: pad number for the first pad
1323 * @pad1: pad number for the second pad
1324 *
1325 * This function is an implementation of the
1326 * media_entity_operations.has_pad_interdep operation for subdevs that
1327 * implement the multiplexed streams API (as indicated by the
1328 * V4L2_SUBDEV_FL_STREAMS subdev flag).
1329 *
1330 * It considers two pads interdependent if there is an active route between pad0
1331 * and pad1.
1332 */
1333 bool v4l2_subdev_has_pad_interdep(struct media_entity *entity,
1334 unsigned int pad0, unsigned int pad1);
1335
1336 /**
1337 * __v4l2_subdev_state_alloc - allocate v4l2_subdev_state
1338 *
1339 * @sd: pointer to &struct v4l2_subdev for which the state is being allocated.
1340 * @lock_name: name of the state lock
1341 * @key: lock_class_key for the lock
1342 *
1343 * Must call __v4l2_subdev_state_free() when state is no longer needed.
1344 *
1345 * Not to be called directly by the drivers.
1346 */
1347 struct v4l2_subdev_state *__v4l2_subdev_state_alloc(struct v4l2_subdev *sd,

  1348							    const char *lock_name,
  1349							    struct lock_class_key *key,
> 1350							    bool active);

1351

Signature · 2 lines
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
author maintainer
#3 lore raw

PATCH v2 2/4 media: v4l: subdev: Add a helper to tell if a sub-device state is active

In reply to Sakari Ailus

Add a helper called v4l2_subdev_state_whence() to return enum v4l2_subdev_format_whence to tell whether a state the driver is dealing with it active or not.

---
 include/media/v4l2-subdev.h | 8 ++++++++
 1 file changed, 8 insertions(+)
include/media/v4l2-subdev.h +8 -0
index ea05b70389b3..ff211b0f07ff 100644
@@ -1645,6 +1645,14 @@ v4l2_subdev_state_get_opposite_stream_format(struct v4l2_subdev_state *state,
 u64 v4l2_subdev_state_xlate_streams(const struct v4l2_subdev_state *state,
 				    u32 pad0, u32 pad1, u64 *streams);
 
+static inline enum v4l2_subdev_format_whence
+v4l2_subdev_state_whence(struct v4l2_subdev *sd,
+			 struct v4l2_subdev_state *state)
+{
+	return sd->active_state == state ?
+		V4L2_SUBDEV_FORMAT_ACTIVE : V4L2_SUBDEV_FORMAT_TRY;
+}
+
 /**
  * enum v4l2_subdev_routing_restriction - Subdevice internal routing restrictions
  *
Signature · 1 line
2.39.2
author maintainer
#4 lore raw

PATCH v2 3/4 media: ccs: Rework initialising sub-device state

In reply to Sakari Ailus

Initialise sub-device state in init_cfg callback using ccs_propagate() to the extent it covers of the initialisation. This fixes a bug where the driver configuration was incorrectly initialised.

Fixes 4b05b1baae3e ("media: ccs: Use sub-device active state")
---
 drivers/media/i2c/ccs/ccs-core.c | 47 ++++++++++++++++++++------------
 1 file changed, 30 insertions(+), 17 deletions(-)
drivers/media/i2c/ccs/ccs-core.c +30 -17
index 2abfd5932e02..6589b0d84e06 100644
@@ -2075,6 +2075,7 @@ static void ccs_propagate(struct v4l2_subdev *subdev,
 	struct ccs_sensor *sensor = to_ccs_sensor(subdev);
 	struct ccs_subdev *ssd = to_ccs_subdev(subdev);
 	struct v4l2_rect *comp, *crops[CCS_PADS];
+	struct v4l2_mbus_framefmt *fmt;
 
 	ccs_get_crop_compose(subdev, sd_state, crops, &comp);
 
@@ -2096,6 +2097,9 @@ static void ccs_propagate(struct v4l2_subdev *subdev,
 		fallthrough;
 	case V4L2_SEL_TGT_COMPOSE:
 		*crops[CCS_PAD_SRC] = *comp;
+		fmt = v4l2_subdev_get_pad_format(subdev, sd_state, CCS_PAD_SRC);
+		fmt->width = comp->width;
+		fmt->height = comp->height;
 		if (which == V4L2_SUBDEV_FORMAT_ACTIVE && ssd == sensor->src)
 			sensor->src_src = *crops[CCS_PAD_SRC];
 		break;
@@ -3003,31 +3007,40 @@ static int ccs_init_cfg(struct v4l2_subdev *sd,
 {
 	struct ccs_subdev *ssd = to_ccs_subdev(sd);
 	struct ccs_sensor *sensor = ssd->sensor;
-	unsigned int i;
+	unsigned int pad = ssd == sensor->pixel_array ?
+		CCS_PA_PAD_SRC : CCS_PAD_SINK;
+	struct v4l2_mbus_framefmt *fmt =
+		v4l2_subdev_get_pad_format(sd, sd_state, pad);
+	struct v4l2_rect *crop =
+		v4l2_subdev_get_pad_crop(sd, sd_state, pad);
 
 	mutex_lock(&sensor->mutex);
 
-	for (i = 0; i < ssd->npads; i++) {
-		struct v4l2_mbus_framefmt *fmt =
-			v4l2_subdev_get_pad_format(sd, sd_state, i);
-		struct v4l2_rect *crop =
-			v4l2_subdev_get_pad_crop(sd, sd_state, i);
-		struct v4l2_rect *comp;
-
-		ccs_get_native_size(ssd, crop);
+	ccs_get_native_size(ssd, crop);
 
-		fmt->width = crop->width;
-		fmt->height = crop->height;
-		fmt->code = sensor->internal_csi_format->code;
-		fmt->field = V4L2_FIELD_NONE;
+	fmt->width = crop->width;
+	fmt->height = crop->height;
+	fmt->code = sensor->internal_csi_format->code;
+	fmt->field = V4L2_FIELD_NONE;
 
-		if (ssd == sensor->pixel_array)
-			continue;
+	if (ssd == sensor->pixel_array) {
+		if (v4l2_subdev_state_whence(sd, sd_state) ==
+		    V4L2_SUBDEV_FORMAT_ACTIVE)
+			sensor->pa_src = *crop;
 
-		comp = v4l2_subdev_get_pad_compose(sd, sd_state, i);
-		*comp = *crop;
+		mutex_unlock(&sensor->mutex);
+		return 0;
 	}
 
+	fmt = v4l2_subdev_get_pad_format(sd, sd_state, CCS_PAD_SRC);
+	fmt->code = ssd == sensor->src ?
+		sensor->csi_format->code : sensor->internal_csi_format->code;
+	fmt->field = V4L2_FIELD_NONE;
+
+	ccs_propagate(sd, sd_state,
+		      v4l2_subdev_state_whence(sd, sd_state),
+		      V4L2_SEL_TGT_CROP);
+
 	mutex_unlock(&sensor->mutex);
 
 	return 0;
Signature · 1 line
2.39.2
author maintainer
#5 lore raw

PATCH v2 4/4 media: ccs: Fix a (harmless) lockdep warning

In reply to Sakari Ailus

The v4l2_subdev_init_finalize() is a macro that creates an unique lockdep key and name. As the CCS driver initialises all three of its sub-devices using the same call site, this creates a lockdep warning. Address it.

Fixes 4b05b1baae3e ("media: ccs: Use sub-device active state")
---
 drivers/media/i2c/ccs/ccs-core.c | 17 ++++++++++++-----
 1 file changed, 12 insertions(+), 5 deletions(-)
drivers/media/i2c/ccs/ccs-core.c +12 -5
index 6589b0d84e06..2b3b712d60b5 100644
@@ -2955,7 +2955,9 @@ static void ccs_cleanup(struct ccs_sensor *sensor)
 
 static int ccs_init_subdev(struct ccs_sensor *sensor,
 			   struct ccs_subdev *ssd, const char *name,
-			   unsigned short num_pads, u32 function)
+			   unsigned short num_pads, u32 function,
+			   const char *lock_name,
+			   struct lock_class_key *lock_key)
 {
 	struct i2c_client *client = v4l2_get_subdevdata(&sensor->src->sd);
 	int rval;
@@ -2993,7 +2995,7 @@ static int ccs_init_subdev(struct ccs_sensor *sensor,
 		return rval;
 	}
 
-	rval = v4l2_subdev_init_finalize(&ssd->sd);
+	rval = __v4l2_subdev_init_finalize(&ssd->sd, lock_name, lock_key);
 	if (rval) {
 		media_entity_cleanup(&ssd->sd.entity);
 		return rval;
@@ -3208,6 +3210,8 @@ static int ccs_firmware_name(struct i2c_client *client,
 
 static int ccs_probe(struct i2c_client *client)
 {
+	static struct lock_class_key pixel_array_lock_key, binner_lock_key,
+		scaler_lock_key;
 	const struct ccs_device *ccsdev = device_get_match_data(&client->dev);
 	struct ccs_sensor *sensor;
 	const struct firmware *fw;
@@ -3491,15 +3495,18 @@ static int ccs_probe(struct i2c_client *client)
 	}
 
 	rval = ccs_init_subdev(sensor, sensor->scaler, " scaler", 2,
-			       MEDIA_ENT_F_PROC_VIDEO_SCALER);
+			       MEDIA_ENT_F_PROC_VIDEO_SCALER,
+			       "ccs scaler mutex", &scaler_lock_key);
 	if (rval)
 		goto out_cleanup;
 	rval = ccs_init_subdev(sensor, sensor->binner, " binner", 2,
-			       MEDIA_ENT_F_PROC_VIDEO_SCALER);
+			       MEDIA_ENT_F_PROC_VIDEO_SCALER,
+			       "ccs binner mutex", &binner_lock_key);
 	if (rval)
 		goto out_cleanup;
 	rval = ccs_init_subdev(sensor, sensor->pixel_array, " pixel_array", 1,
-			       MEDIA_ENT_F_CAM_SENSOR);
+			       MEDIA_ENT_F_CAM_SENSOR, "ccs pixel array mutex",
+			       &pixel_array_lock_key);
 	if (rval)
 		goto out_cleanup;
 
Signature · 1 line
2.39.2
#6 lore raw
In reply to Sakari Ailus
Quoted 18 lines · Sakari Ailus

Hi folks,

This set includes a helper for working with V4L2 sub-device active state as well as a fix for the CCS driver sub-device state patch.

since v1:

- There were other drivers using __v4l2_subdev_state_alloc(). Ouch. Change those users as well. Note that this function should not be used in drivers, hence API niceness is not a high priority.

Sakari Ailus (4):
media: v4l: subdev: Set sub-device active state earlier
media: v4l: subdev: Add a helper to tell if a sub-device state is
active
media: ccs: Rework initialising sub-device state
media: ccs: Fix a (harmless) lockdep warning

Quoted 8 lines
  drivers/media/i2c/ccs/ccs-core.c              | 64 ++++++++++++-------
  .../platform/renesas/rcar-vin/rcar-v4l2.c     |  3 +-
  .../media/platform/renesas/vsp1/vsp1_entity.c |  3 +-
  drivers/media/v4l2-core/v4l2-subdev.c         | 14 ++--
  drivers/staging/media/tegra-video/vi.c        |  2 +-
  include/media/v4l2-subdev.h                   | 11 +++-
  6 files changed, 66 insertions(+), 31 deletions(-)

I'm not familiar with the CCS driver, and you don't explain much why you are doing this, so it's a bit unclear to me. But I don't like it.

The idea with the subdev state was that the driver doesn't need to know whether it's active-state or try-state. It should behave the same way in both cases. This series goes against that.

Can you explain a bit what the issue is and what you are doing in this series?

Tomi

author maintainer
#7 lore raw

Moi,

Quoted 31 lines · Tomi Valkeinen
Quoted 18 lines · Sakari Ailus

Hi folks,

This set includes a helper for working with V4L2 sub-device active state as well as a fix for the CCS driver sub-device state patch.

since v1:

- There were other drivers using __v4l2_subdev_state_alloc(). Ouch. Change those users as well. Note that this function should not be used in drivers, hence API niceness is not a high priority.

Sakari Ailus (4):
media: v4l: subdev: Set sub-device active state earlier
media: v4l: subdev: Add a helper to tell if a sub-device state is
active
media: ccs: Rework initialising sub-device state
media: ccs: Fix a (harmless) lockdep warning

Quoted 8 lines
  drivers/media/i2c/ccs/ccs-core.c              | 64 ++++++++++++-------
  .../platform/renesas/rcar-vin/rcar-v4l2.c     |  3 +-
  .../media/platform/renesas/vsp1/vsp1_entity.c |  3 +-
  drivers/media/v4l2-core/v4l2-subdev.c         | 14 ++--
  drivers/staging/media/tegra-video/vi.c        |  2 +-
  include/media/v4l2-subdev.h                   | 11 +++-
  6 files changed, 66 insertions(+), 31 deletions(-)

I'm not familiar with the CCS driver, and you don't explain much why you are doing this, so it's a bit unclear to me. But I don't like it.

Quoted 6 lines

The idea with the subdev state was that the driver doesn't need to know whether it's active-state or try-state. It should behave the same way in both cases. This series goes against that.

Can you explain a bit what the issue is and what you are doing in this series?

The driver maintains internal state and that needs to be updated when the configuration (including what's in sub-device state) changes. Generally the driver knows (as for the whence field) which state it's dealing with but that is not the case for init_cfg.

Alternatively I could split the internal workings of the driver into active and try states but I prefer to improve the framework and make the driver simpler.

Deducing the internal configuration solely based on sub-device state is not really feasible.

Signature · 3 lines
Terveisin,

Sakari Ailus
#8 lore raw
Quoted 38 lines · Sakari Ailus

Moi,

Quoted 35 lines · Tomi Valkeinen
Quoted 18 lines · Sakari Ailus

Hi folks,

This set includes a helper for working with V4L2 sub-device active state as well as a fix for the CCS driver sub-device state patch.

since v1:

- There were other drivers using __v4l2_subdev_state_alloc(). Ouch. Change those users as well. Note that this function should not be used in drivers, hence API niceness is not a high priority.

Sakari Ailus (4):
media: v4l: subdev: Set sub-device active state earlier
media: v4l: subdev: Add a helper to tell if a sub-device state is
active
media: ccs: Rework initialising sub-device state
media: ccs: Fix a (harmless) lockdep warning

Quoted 8 lines
   drivers/media/i2c/ccs/ccs-core.c              | 64 ++++++++++++-------
   .../platform/renesas/rcar-vin/rcar-v4l2.c     |  3 +-
   .../media/platform/renesas/vsp1/vsp1_entity.c |  3 +-
   drivers/media/v4l2-core/v4l2-subdev.c         | 14 ++--
   drivers/staging/media/tegra-video/vi.c        |  2 +-
   include/media/v4l2-subdev.h                   | 11 +++-
   6 files changed, 66 insertions(+), 31 deletions(-)

I'm not familiar with the CCS driver, and you don't explain much why you are doing this, so it's a bit unclear to me. But I don't like it.

The idea with the subdev state was that the driver doesn't need to know whether it's active-state or try-state. It should behave the same way in both cases. This series goes against that.

Quoted 7 lines
Quoted 2 lines

Can you explain a bit what the issue is and what you are doing in this series?

The driver maintains internal state and that needs to be updated when the configuration (including what's in sub-device state) changes. Generally the driver knows (as for the whence field) which state it's dealing with but that is not the case for init_cfg.

So you need to set the subdev state to sd->active_state earlier (in patch 1) so that the driver can use v4l2_subdev_state_whence()? You don't really explain that in the patch descriptions.

In other words, if init_cfg() were to get a whence-parameter, this wouldn't be needed? (I don't want to do that, just trying to understand what's going on).

Quoted 3 lines

Alternatively I could split the internal workings of the driver into active and try states but I prefer to improve the framework and make the driver simpler.

I agree with the goal, but I don't think this is improving the framework.

If we decide that we need to know if a state is an active-state or a try-state, I think it's better to add a whence-field into the state itself. But I'd rather not.

Quoted 2 lines

Deducing the internal configuration solely based on sub-device state is not really feasible.

Am I correct in that what you really need is a way to sub-class the subdev state, so that you can have all that internal state in the subdev state, and thus all the code in the driver can work in "whence-agnostic" manner? That's something we've been thinking about for a long time.

But the driver needs to be fixed now, not at some point in the future, of course. If the sub-classing would solve the issue (i.e. we have a plan in mind), I'd rather just hack this in the driver, instead of extending the framework, which might easily lead to other drivers going the wrong way too.

How about a private flag, set before calling v4l2_subdev_init_finalize(), and unset after the call. ccs_init_cfg() can look at that flag an if it's set, it's initializing an active state.

Tomi

author maintainer
#9 lore raw

Moi,

Quoted 47 lines · Tomi Valkeinen
Quoted 38 lines · Sakari Ailus

Moi,

Quoted 35 lines · Tomi Valkeinen
Quoted 18 lines · Sakari Ailus

Hi folks,

This set includes a helper for working with V4L2 sub-device active state as well as a fix for the CCS driver sub-device state patch.

since v1:

- There were other drivers using __v4l2_subdev_state_alloc(). Ouch. Change those users as well. Note that this function should not be used in drivers, hence API niceness is not a high priority.

Sakari Ailus (4):
media: v4l: subdev: Set sub-device active state earlier
media: v4l: subdev: Add a helper to tell if a sub-device state is
active
media: ccs: Rework initialising sub-device state
media: ccs: Fix a (harmless) lockdep warning

Quoted 8 lines
   drivers/media/i2c/ccs/ccs-core.c              | 64 ++++++++++++-------
   .../platform/renesas/rcar-vin/rcar-v4l2.c     |  3 +-
   .../media/platform/renesas/vsp1/vsp1_entity.c |  3 +-
   drivers/media/v4l2-core/v4l2-subdev.c         | 14 ++--
   drivers/staging/media/tegra-video/vi.c        |  2 +-
   include/media/v4l2-subdev.h                   | 11 +++-
   6 files changed, 66 insertions(+), 31 deletions(-)

I'm not familiar with the CCS driver, and you don't explain much why you are doing this, so it's a bit unclear to me. But I don't like it.

The idea with the subdev state was that the driver doesn't need to know whether it's active-state or try-state. It should behave the same way in both cases. This series goes against that.

Quoted 7 lines
Quoted 2 lines

Can you explain a bit what the issue is and what you are doing in this series?

The driver maintains internal state and that needs to be updated when the configuration (including what's in sub-device state) changes. Generally the driver knows (as for the whence field) which state it's dealing with but that is not the case for init_cfg.

Quoted 7 lines

So you need to set the subdev state to sd->active_state earlier (in patch 1) so that the driver can use v4l2_subdev_state_whence()? You don't really explain that in the patch descriptions.

In other words, if init_cfg() were to get a whence-parameter, this wouldn't be needed? (I don't want to do that, just trying to understand what's going on).

Yes, that's correct.

Quoted 10 lines
Quoted 4 lines

Alternatively I could split the internal workings of the driver into active and try states but I prefer to improve the framework and make the driver simpler.

I agree with the goal, but I don't think this is improving the framework.

If we decide that we need to know if a state is an active-state or a try-state, I think it's better to add a whence-field into the state itself. But I'd rather not.

In all other callbacks the whence is known but not in init_cfg.

Comparing with sd->active_state does this pretty cleanly, without a need to change APIs.

Quoted 8 lines
Quoted 3 lines

Deducing the internal configuration solely based on sub-device state is not really feasible.

Am I correct in that what you really need is a way to sub-class the subdev state, so that you can have all that internal state in the subdev state, and thus all the code in the driver can work in "whence-agnostic" manner? That's something we've been thinking about for a long time.

In the case of the CCS driver it's not necessary to store these parameters for try states: they are driver internal only.

Quoted 9 lines

But the driver needs to be fixed now, not at some point in the future, of course. If the sub-classing would solve the issue (i.e. we have a plan in mind), I'd rather just hack this in the driver, instead of extending the framework, which might easily lead to other drivers going the wrong way too.

How about a private flag, set before calling v4l2_subdev_init_finalize(), and unset after the call. ccs_init_cfg() can look at that flag an if it's set, it's initializing an active state.

I dislike using driver specific flags in generic APIs.

While trivial sensor drivers have no use for such functionality, I would be surprised if there would not be other similar cases.

Signature · 3 lines
Terveisin,

Sakari Ailus
#10 lore raw
Quoted 54 lines · Sakari Ailus

Moi,

Quoted 51 lines · Tomi Valkeinen
Quoted 38 lines · Sakari Ailus

Moi,

Quoted 35 lines · Tomi Valkeinen
Quoted 18 lines · Sakari Ailus

Hi folks,

This set includes a helper for working with V4L2 sub-device active state as well as a fix for the CCS driver sub-device state patch.

since v1:

- There were other drivers using __v4l2_subdev_state_alloc(). Ouch. Change those users as well. Note that this function should not be used in drivers, hence API niceness is not a high priority.

Sakari Ailus (4):
media: v4l: subdev: Set sub-device active state earlier
media: v4l: subdev: Add a helper to tell if a sub-device state is
active
media: ccs: Rework initialising sub-device state
media: ccs: Fix a (harmless) lockdep warning

Quoted 8 lines

drivers/media/i2c/ccs/ccs-core.c | 64 ++++++++++++-------
.../platform/renesas/rcar-vin/rcar-v4l2.c | 3 +-
.../media/platform/renesas/vsp1/vsp1_entity.c | 3 +-
drivers/media/v4l2-core/v4l2-subdev.c | 14 ++--
drivers/staging/media/tegra-video/vi.c | 2 +-
include/media/v4l2-subdev.h | 11 +++-
6 files changed, 66 insertions(+), 31 deletions(-)

I'm not familiar with the CCS driver, and you don't explain much why you are doing this, so it's a bit unclear to me. But I don't like it.

The idea with the subdev state was that the driver doesn't need to know whether it's active-state or try-state. It should behave the same way in both cases. This series goes against that.

Quoted 7 lines
Quoted 2 lines

Can you explain a bit what the issue is and what you are doing in this series?

The driver maintains internal state and that needs to be updated when the configuration (including what's in sub-device state) changes. Generally the driver knows (as for the whence field) which state it's dealing with but that is not the case for init_cfg.

So you need to set the subdev state to sd->active_state earlier (in patch 1) so that the driver can use v4l2_subdev_state_whence()? You don't really explain that in the patch descriptions.

Quoted 5 lines
Quoted 3 lines

In other words, if init_cfg() were to get a whence-parameter, this wouldn't be needed? (I don't want to do that, just trying to understand what's going on).

Yes, that's correct.

Ok. So the driver keeps some private state separately from the subdev state, and keeps track of only the active state details, and thus it needs to know the whence in init_cfg so that it can fill in those details only when dealing with active state.

Quoted 11 lines
Quoted 9 lines
Quoted 3 lines

Alternatively I could split the internal workings of the driver into active and try states but I prefer to improve the framework and make the driver simpler.

I agree with the goal, but I don't think this is improving the framework.

If we decide that we need to know if a state is an active-state or a try-state, I think it's better to add a whence-field into the state itself. But I'd rather not.

In all other callbacks the whence is known but not in init_cfg.

Yes, those should be removed =).

Well, that may not work, but at least we should consider if a specific callback should ever care about active/try, and if not, remove the whence.

Quoted 7 lines

Comparing with sd->active_state does this pretty cleanly, without a need to change APIs.

Quoted 4 lines
Quoted 3 lines

Deducing the internal configuration solely based on sub-device state is not really feasible.

Quoted 7 lines
Quoted 4 lines

Am I correct in that what you really need is a way to sub-class the subdev state, so that you can have all that internal state in the subdev state, and thus all the code in the driver can work in "whence-agnostic" manner? That's something we've been thinking about for a long time.

In the case of the CCS driver it's not necessary to store these parameters for try states: they are driver internal only.

That's fine, the sub-classed state can only have driver internal data (in addition to the standard data).

I think a follow-up question here is: why do you need to keep track of state that's only needed for active state? What is that state?

Quoted 10 lines
Quoted 8 lines

But the driver needs to be fixed now, not at some point in the future, of course. If the sub-classing would solve the issue (i.e. we have a plan in mind), I'd rather just hack this in the driver, instead of extending the framework, which might easily lead to other drivers going the wrong way too.

How about a private flag, set before calling v4l2_subdev_init_finalize(), and unset after the call. ccs_init_cfg() can look at that flag an if it's set, it's initializing an active state.

I dislike using driver specific flags in generic APIs.

I meant in the driver private state. Add a field to the driver's private data struct, and use that. Then it's wholly inside the driver, and needs just a few lines of hack-code, easy to remove later.

Quoted 2 lines

While trivial sensor drivers have no use for such functionality, I would be surprised if there would not be other similar cases.

Can you open this up a bit? Why do drivers need to know if an operation is about active or try state?

Answering my own question:

In some cases it's implicit: when starting the streaming, it's always active state. Similarly in, e.g., get_frame_desc.

Some operations might allow changing HW settings while streaming is enabled. I guess set_fmt might allow changing some specific things even while streaming is enabled. And if so, the callback needs to know if this is for active state or not.

But why does a driver need to know the whence in init_cfg? With sub-classed state, init_cfg should work fine for both active-state and try-state.

Tomi

author maintainer
#11 lore raw

Moi,

Quoted 59 lines · Tomi Valkeinen
Quoted 58 lines · Sakari Ailus

Moi,

Quoted 54 lines · Tomi Valkeinen
Quoted 38 lines · Sakari Ailus

Moi,

Quoted 35 lines · Tomi Valkeinen
Quoted 18 lines · Sakari Ailus

Hi folks,

This set includes a helper for working with V4L2 sub-device active state as well as a fix for the CCS driver sub-device state patch.

since v1:

- There were other drivers using __v4l2_subdev_state_alloc(). Ouch. Change those users as well. Note that this function should not be used in drivers, hence API niceness is not a high priority.

Sakari Ailus (4):
media: v4l: subdev: Set sub-device active state earlier
media: v4l: subdev: Add a helper to tell if a sub-device state is
active
media: ccs: Rework initialising sub-device state
media: ccs: Fix a (harmless) lockdep warning

Quoted 8 lines

drivers/media/i2c/ccs/ccs-core.c | 64 ++++++++++++-------
.../platform/renesas/rcar-vin/rcar-v4l2.c | 3 +-
.../media/platform/renesas/vsp1/vsp1_entity.c | 3 +-
drivers/media/v4l2-core/v4l2-subdev.c | 14 ++--
drivers/staging/media/tegra-video/vi.c | 2 +-
include/media/v4l2-subdev.h | 11 +++-
6 files changed, 66 insertions(+), 31 deletions(-)

I'm not familiar with the CCS driver, and you don't explain much why you are doing this, so it's a bit unclear to me. But I don't like it.

The idea with the subdev state was that the driver doesn't need to know whether it's active-state or try-state. It should behave the same way in both cases. This series goes against that.

Quoted 7 lines
Quoted 2 lines

Can you explain a bit what the issue is and what you are doing in this series?

The driver maintains internal state and that needs to be updated when the configuration (including what's in sub-device state) changes. Generally the driver knows (as for the whence field) which state it's dealing with but that is not the case for init_cfg.

So you need to set the subdev state to sd->active_state earlier (in patch 1) so that the driver can use v4l2_subdev_state_whence()? You don't really explain that in the patch descriptions.

In other words, if init_cfg() were to get a whence-parameter, this wouldn't be needed? (I don't want to do that, just trying to understand what's going on).

Quoted 6 lines
Quoted 1 line

Yes, that's correct.

Ok. So the driver keeps some private state separately from the subdev state, and keeps track of only the active state details, and thus it needs to know the whence in init_cfg so that it can fill in those details only when dealing with active state.

Precisely so.

Quoted 30 lines
Quoted 5 lines
Quoted 5 lines
Quoted 4 lines

Alternatively I could split the internal workings of the driver into active and try states but I prefer to improve the framework and make the driver simpler.

Quoted 7 lines
Quoted 5 lines

I agree with the goal, but I don't think this is improving the framework.

If we decide that we need to know if a state is an active-state or a try-state, I think it's better to add a whence-field into the state itself. But I'd rather not.

In all other callbacks the whence is known but not in init_cfg.

Yes, those should be removed =).

Well, that may not work, but at least we should consider if a specific callback should ever care about active/try, and if not, remove the whence.

Quoted 12 lines

Comparing with sd->active_state does this pretty cleanly, without a need to change APIs.

Quoted 8 lines
Quoted 3 lines

Deducing the internal configuration solely based on sub-device state is not really feasible.

Am I correct in that what you really need is a way to sub-class the subdev state, so that you can have all that internal state in the subdev state, and thus all the code in the driver can work in "whence-agnostic" manner? That's something we've been thinking about for a long time.

Quoted 8 lines
Quoted 2 lines

In the case of the CCS driver it's not necessary to store these parameters for try states: they are driver internal only.

That's fine, the sub-classed state can only have driver internal data (in addition to the standard data).

I think a follow-up question here is: why do you need to keep track of state that's only needed for active state? What is that state?

Sensor register values effectively. These are programmed just before streaming starts. There's scaling factor, whether scaling is horizontal-only, both vertical and horizontal or if there's no scaling. Similarly for binning.

Quoted 27 lines
Quoted 11 lines
Quoted 9 lines

But the driver needs to be fixed now, not at some point in the future, of course. If the sub-classing would solve the issue (i.e. we have a plan in mind), I'd rather just hack this in the driver, instead of extending the framework, which might easily lead to other drivers going the wrong way too.

How about a private flag, set before calling v4l2_subdev_init_finalize(), and unset after the call. ccs_init_cfg() can look at that flag an if it's set, it's initializing an active state.

I dislike using driver specific flags in generic APIs.

I meant in the driver private state. Add a field to the driver's private data struct, and use that. Then it's wholly inside the driver, and needs just a few lines of hack-code, easy to remove later.

Quoted 2 lines

While trivial sensor drivers have no use for such functionality, I would be surprised if there would not be other similar cases.

Can you open this up a bit? Why do drivers need to know if an operation is about active or try state?

Answering my own question:

In some cases it's implicit: when starting the streaming, it's always active state. Similarly in, e.g., get_frame_desc.

Quoted 7 lines

Some operations might allow changing HW settings while streaming is enabled. I guess set_fmt might allow changing some specific things even while streaming is enabled. And if so, the callback needs to know if this is for active state or not.

But why does a driver need to know the whence in init_cfg? With sub-classed state, init_cfg should work fine for both active-state and try-state.

See above.

While init_cfg does not (obviously) enable binning or scaling, it still deals with the same configuration parameters.

Signature · 3 lines
Regards,

Sakari Ailus