Collapse vimc into single monolithic driver

PATCH Superseded Shuah Khan 4 participants 27 messages Aug 9, 2019 – Aug 13, 2019 Media (V4L/DVB)

Share & tools

Shuah Khan

author

2019-08-09 21:45 UTC #0 lore raw

cover Collapse vimc into single monolithic driver

vimc uses Component API to split the driver into functional components. The real hardware resembles a monolith structure than component and component structure added a level of complexity making it hard to maintain without adding any real benefit.

The sensor is one vimc component that would makes sense to be a separate module to closely align with the real hardware. It would be easier to collapse vimc into single monolithic driver first and then split the sensor off as a separate module.

This patch series emoves the component API and makes minimal changes to the code base preserving the functional division of the code structure. Preserving the functional structure allows us to split the sensor off as a separate module in the future.

Major design elements in this change are:
- Use existing struct vimc_ent_config and struct vimc_pipeline_config to drive the initialization of the functional components.
- Make vimc_ent_config global by moving it to vimc.h
- Add two new hooks add and rm to initialize and register, unregister and free subdevs.
- All component API is now gone and bind and unbind hooks are modified to do "add" and "rm" with minimal changes to just add and rm subdevs.
- vimc-core's bind and unbind are now register and unregister.
- vimc-core invokes "add" hooks from its vimc_register_devices(). The "add" hooks remain the same and register subdevs. They don't create platform devices of their own and use vimc's pdev.dev as their reference device. The "add" hooks save their vimc_ent_device(s) in the corresponding vimc_ent_config.
- vimc-core invokes "rm" hooks from its unregister to unregister subdevs and cleanup.
- vimc-core invokes "add" and "rm" hooks with pointer to struct vimc_device and the corresponding struct vimc_ent_config pointer.

The following configure and stream test works on all devices.

    media-ctl -d platform:vimc -V '"Sensor A":0[fmt:SBGGR8_1X8/640x480]'
    media-ctl -d platform:vimc -V '"Debayer A":0[fmt:SBGGR8_1X8/640x480]'
    media-ctl -d platform:vimc -V '"Sensor B":0[fmt:SBGGR8_1X8/640x480]'
    media-ctl -d platform:vimc -V '"Debayer B":0[fmt:SBGGR8_1X8/640x480]'
    
    v4l2-ctl -z platform:vimc -d "RGB/YUV Capture" -v width=1920,height=1440
    v4l2-ctl -z platform:vimc -d "Raw Capture 0" -v pixelformat=BA81
    v4l2-ctl -z platform:vimc -d "Raw Capture 1" -v pixelformat=BA81
    
    v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video1
    v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video2
    v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video3

The third patch in the series fixes a general protection fault found when rmmod is done while stream is active.

vimc_print_dot (--print-dot) topology after this change:

digraph board {
	rankdir=TB
	n00000001 [label="{{} | Sensor A\n/dev/v4l-subdev0 | {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green]
	n00000001:port0 -> n00000005:port0 [style=bold]
	n00000001:port0 -> n0000000b [style=bold]
	n00000003 [label="{{} | Sensor B\n/dev/v4l-subdev1 | {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green]
	n00000003:port0 -> n00000008:port0 [style=bold]
	n00000003:port0 -> n0000000f [style=bold]
	n00000005 [label="{{<port0> 0} | Debayer A\n/dev/v4l-subdev2 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
	n00000005:port1 -> n00000015:port0
	n00000008 [label="{{<port0> 0} | Debayer B\n/dev/v4l-subdev3 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
	n00000008:port1 -> n00000015:port0 [style=dashed]
	n0000000b [label="Raw Capture 0\n/dev/video1", shape=box, style=filled, fillcolor=yellow]
	n0000000f [label="Raw Capture 1\n/dev/video2", shape=box, style=filled, fillcolor=yellow]
	n00000013 [label="{{} | RGB/YUV Input\n/dev/v4l-subdev4 | {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green]
	n00000013:port0 -> n00000015:port0 [style=dashed]
	n00000015 [label="{{<port0> 0} | Scaler\n/dev/v4l-subdev5 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
	n00000015:port1 -> n00000018 [style=bold]
	n00000018 [label="RGB/YUV Capture\n/dev/video3", shape=box, style=filled, fillcolor=yellow]
}

Shuah Khan (3):
media: vimc: move private defines to a common header
media: vimc: Collapse component structure into a single monolithic
driver
media: vimc: Fix gpf in rmmod path when stream is active

 drivers/media/platform/vimc/Makefile       |   7 +-
 drivers/media/platform/vimc/vimc-capture.c |  96 ++----------
 drivers/media/platform/vimc/vimc-common.c  |   2 +-
 drivers/media/platform/vimc/vimc-core.c    | 174 +++++++--------------
 drivers/media/platform/vimc/vimc-debayer.c |  84 ++--------
 drivers/media/platform/vimc/vimc-scaler.c  |  83 ++--------
 drivers/media/platform/vimc/vimc-sensor.c  |  82 ++--------
 drivers/media/platform/vimc/vimc.h         | 121 ++++++++++++++
 8 files changed, 230 insertions(+), 419 deletions(-)
 create mode 100644 drivers/media/platform/vimc/vimc.h

Signature · 1 line

2.17.1

Shuah Khan

author

2019-08-09 21:45 UTC #1 lore raw

PATCH 1/3 media: vimc: move private defines to a common header

In preparation for collapsing the component driver structure into a monolith, move private device structure defines to a new common header file.

Signed-off-by Shuah Khan <[email protected]>

---
 drivers/media/platform/vimc/vimc-capture.c |  21 +----
 drivers/media/platform/vimc/vimc-core.c    |  18 +---
 drivers/media/platform/vimc/vimc-debayer.c |  16 +---
 drivers/media/platform/vimc/vimc-scaler.c  |  15 +--
 drivers/media/platform/vimc/vimc-sensor.c  |  13 +--
 drivers/media/platform/vimc/vimc.h         | 102 +++++++++++++++++++++
 6 files changed, 107 insertions(+), 78 deletions(-)
 create mode 100644 drivers/media/platform/vimc/vimc.h

drivers/media/platform/vimc/vimc-capture.c +1 ~~-20~~

index 664855708fdf..c52fc5d97c2d 100644
@@ -13,6 +13,7 @@
 #include <media/videobuf2-core.h>
 #include <media/videobuf2-vmalloc.h>
 
+#include "vimc.h"
 #include "vimc-common.h"
 #include "vimc-streamer.h"

@@ -44,26 +45,6 @@ static const u32 vimc_cap_supported_pixfmt[] = {
 	V4L2_PIX_FMT_SRGGB12,
 };
 
-struct vimc_cap_device {
-	struct vimc_ent_device ved;
-	struct video_device vdev;
-	struct device *dev;
-	struct v4l2_pix_format format;
-	struct vb2_queue queue;
-	struct list_head buf_list;
-	/*
-	 * NOTE: in a real driver, a spin lock must be used to access the
-	 * queue because the frames are generated from a hardware interruption
-	 * and the isr is not allowed to sleep.
-	 * Even if it is not necessary a spinlock in the vimc driver, we
-	 * use it here as a code reference
-	 */
-	spinlock_t qlock;
-	struct mutex lock;
-	u32 sequence;
-	struct vimc_stream stream;
-};
-
 static const struct v4l2_pix_format fmt_default = {
 	.width = 640,
 	.height = 480,

drivers/media/platform/vimc/vimc-core.c +1 ~~-17~~

index 571c55aa0e16..c9b351472272 100644
@@ -12,6 +12,7 @@
 #include <media/media-device.h>
 #include <media/v4l2-device.h>
 
+#include "vimc.h"
 #include "vimc-common.h"
 
 #define VIMC_MDEV_MODEL_NAME "VIMC MDEV"

@@ -24,23 +25,6 @@
 	.flags = link_flags,					\
 }
 
-struct vimc_device {
-	/* The platform device */
-	struct platform_device pdev;
-
-	/* The pipeline configuration */
-	const struct vimc_pipeline_config *pipe_cfg;
-
-	/* The Associated media_device parent */
-	struct media_device mdev;
-
-	/* Internal v4l2 parent device*/
-	struct v4l2_device v4l2_dev;
-
-	/* Subdevices */
-	struct platform_device **subdevs;
-};
-
 /* Structure which describes individual configuration for each entity */
 struct vimc_ent_config {
 	const char *name;

drivers/media/platform/vimc/vimc-debayer.c +1 ~~-15~~

index 00598fbf3cba..750752bb173c 100644
@@ -13,6 +13,7 @@
 #include <linux/v4l2-mediabus.h>
 #include <media/v4l2-subdev.h>
 
+#include "vimc.h"
 #include "vimc-common.h"
 
 #define VIMC_DEB_DRV_NAME "vimc-debayer"

@@ -44,21 +45,6 @@ struct vimc_deb_pix_map {
 	enum vimc_deb_rgb_colors order[2][2];
 };
 
-struct vimc_deb_device {
-	struct vimc_ent_device ved;
-	struct v4l2_subdev sd;
-	struct device *dev;
-	/* The active format */
-	struct v4l2_mbus_framefmt sink_fmt;
-	u32 src_code;
-	void (*set_rgb_src)(struct vimc_deb_device *vdeb, unsigned int lin,
-			    unsigned int col, unsigned int rgb[3]);
-	/* Values calculated when the stream starts */
-	u8 *src_frame;
-	const struct vimc_deb_pix_map *sink_pix_map;
-	unsigned int sink_bpp;
-};
-
 static const struct v4l2_mbus_framefmt sink_fmt_default = {
 	.width = 640,
 	.height = 480,

drivers/media/platform/vimc/vimc-scaler.c +1 ~~-14~~

index c7123a45c55b..fe99b9102ada 100644
@@ -13,6 +13,7 @@
 #include <linux/v4l2-mediabus.h>
 #include <media/v4l2-subdev.h>
 
+#include "vimc.h"
 #include "vimc-common.h"
 
 #define VIMC_SCA_DRV_NAME "vimc-scaler"

@@ -31,20 +32,6 @@ static const u32 vimc_sca_supported_pixfmt[] = {
 	V4L2_PIX_FMT_ARGB32,
 };
 
-struct vimc_sca_device {
-	struct vimc_ent_device ved;
-	struct v4l2_subdev sd;
-	struct device *dev;
-	/* NOTE: the source fmt is the same as the sink
-	 * with the width and hight multiplied by mult
-	 */
-	struct v4l2_mbus_framefmt sink_fmt;
-	/* Values calculated when the stream starts */
-	u8 *src_frame;
-	unsigned int src_line_size;
-	unsigned int bpp;
-};
-
 static const struct v4l2_mbus_framefmt sink_fmt_default = {
 	.width = 640,
 	.height = 480,

drivers/media/platform/vimc/vimc-sensor.c +1 ~~-12~~

index 51359472eef2..6c57b1e262f9 100644
@@ -16,22 +16,11 @@
 #include <media/v4l2-subdev.h>
 #include <media/tpg/v4l2-tpg.h>
 
+#include "vimc.h"
 #include "vimc-common.h"
 
 #define VIMC_SEN_DRV_NAME "vimc-sensor"
 
-struct vimc_sen_device {
-	struct vimc_ent_device ved;
-	struct v4l2_subdev sd;
-	struct device *dev;
-	struct tpg_data tpg;
-	struct task_struct *kthread_sen;
-	u8 *frame;
-	/* The active format */
-	struct v4l2_mbus_framefmt mbus_format;
-	struct v4l2_ctrl_handler hdl;
-};
-
 static const struct v4l2_mbus_framefmt fmt_default = {
 	.width = 640,
 	.height = 480,

drivers/media/platform/vimc/vimc.h new file +102 -0

102 lines not on this page · raw message

Signature · 1 line

2.17.1

Helen Koike

2019-08-10 03:15 UTC #2 lore raw

Re: [PATCH 1/3] media: vimc: move private defines to a common header

Hi Shuah,

Thanks for the patch.

drivers/media/platform/vimc/vimc.h

Quoted 190 lines · Shuah Khan

In preparation for collapsing the component driver structure into a monolith, move private device structure defines to a new common header file.

Signed-off-by Shuah Khan <[email protected]>

---
 drivers/media/platform/vimc/vimc-capture.c |  21 +----
 drivers/media/platform/vimc/vimc-core.c    |  18 +---
 drivers/media/platform/vimc/vimc-debayer.c |  16 +---
 drivers/media/platform/vimc/vimc-scaler.c  |  15 +--
 drivers/media/platform/vimc/vimc-sensor.c  |  13 +--
 drivers/media/platform/vimc/vimc.h         | 102 +++++++++++++++++++++
 6 files changed, 107 insertions(+), 78 deletions(-)
 create mode 100644 drivers/media/platform/vimc/vimc.h

drivers/media/platform/vimc/vimc-capture.c +1 ~~-20~~

index 664855708fdf..c52fc5d97c2d 100644
@@ -13,6 +13,7 @@
 #include <media/videobuf2-core.h>
 #include <media/videobuf2-vmalloc.h>
 
+#include "vimc.h"
 #include "vimc-common.h"
 #include "vimc-streamer.h"

@@ -44,26 +45,6 @@ static const u32 vimc_cap_supported_pixfmt[] = {
 	V4L2_PIX_FMT_SRGGB12,
 };
 
-struct vimc_cap_device {
-	struct vimc_ent_device ved;
-	struct video_device vdev;
-	struct device *dev;
-	struct v4l2_pix_format format;
-	struct vb2_queue queue;
-	struct list_head buf_list;
-	/*
-	 * NOTE: in a real driver, a spin lock must be used to access the
-	 * queue because the frames are generated from a hardware interruption
-	 * and the isr is not allowed to sleep.
-	 * Even if it is not necessary a spinlock in the vimc driver, we
-	 * use it here as a code reference
-	 */
-	spinlock_t qlock;
-	struct mutex lock;
-	u32 sequence;
-	struct vimc_stream stream;
-};
-
 static const struct v4l2_pix_format fmt_default = {
 	.width = 640,
 	.height = 480,

drivers/media/platform/vimc/vimc-core.c +1 ~~-17~~

index 571c55aa0e16..c9b351472272 100644
@@ -12,6 +12,7 @@
 #include <media/media-device.h>
 #include <media/v4l2-device.h>
 
+#include "vimc.h"
 #include "vimc-common.h"
 
 #define VIMC_MDEV_MODEL_NAME "VIMC MDEV"

@@ -24,23 +25,6 @@
 	.flags = link_flags,					\
 }
 
-struct vimc_device {
-	/* The platform device */
-	struct platform_device pdev;
-
-	/* The pipeline configuration */
-	const struct vimc_pipeline_config *pipe_cfg;
-
-	/* The Associated media_device parent */
-	struct media_device mdev;
-
-	/* Internal v4l2 parent device*/
-	struct v4l2_device v4l2_dev;
-
-	/* Subdevices */
-	struct platform_device **subdevs;
-};
-
 /* Structure which describes individual configuration for each entity */
 struct vimc_ent_config {
 	const char *name;

drivers/media/platform/vimc/vimc-debayer.c +1 ~~-15~~

index 00598fbf3cba..750752bb173c 100644
@@ -13,6 +13,7 @@
 #include <linux/v4l2-mediabus.h>
 #include <media/v4l2-subdev.h>
 
+#include "vimc.h"
 #include "vimc-common.h"
 
 #define VIMC_DEB_DRV_NAME "vimc-debayer"

@@ -44,21 +45,6 @@ struct vimc_deb_pix_map {
 	enum vimc_deb_rgb_colors order[2][2];
 };
 
-struct vimc_deb_device {
-	struct vimc_ent_device ved;
-	struct v4l2_subdev sd;
-	struct device *dev;
-	/* The active format */
-	struct v4l2_mbus_framefmt sink_fmt;
-	u32 src_code;
-	void (*set_rgb_src)(struct vimc_deb_device *vdeb, unsigned int lin,
-			    unsigned int col, unsigned int rgb[3]);
-	/* Values calculated when the stream starts */
-	u8 *src_frame;
-	const struct vimc_deb_pix_map *sink_pix_map;
-	unsigned int sink_bpp;
-};
-
 static const struct v4l2_mbus_framefmt sink_fmt_default = {
 	.width = 640,
 	.height = 480,

drivers/media/platform/vimc/vimc-scaler.c +1 ~~-14~~

index c7123a45c55b..fe99b9102ada 100644
@@ -13,6 +13,7 @@
 #include <linux/v4l2-mediabus.h>
 #include <media/v4l2-subdev.h>
 
+#include "vimc.h"
 #include "vimc-common.h"
 
 #define VIMC_SCA_DRV_NAME "vimc-scaler"

@@ -31,20 +32,6 @@ static const u32 vimc_sca_supported_pixfmt[] = {
 	V4L2_PIX_FMT_ARGB32,
 };
 
-struct vimc_sca_device {
-	struct vimc_ent_device ved;
-	struct v4l2_subdev sd;
-	struct device *dev;
-	/* NOTE: the source fmt is the same as the sink
-	 * with the width and hight multiplied by mult
-	 */
-	struct v4l2_mbus_framefmt sink_fmt;
-	/* Values calculated when the stream starts */
-	u8 *src_frame;
-	unsigned int src_line_size;
-	unsigned int bpp;
-};
-
 static const struct v4l2_mbus_framefmt sink_fmt_default = {
 	.width = 640,
 	.height = 480,

drivers/media/platform/vimc/vimc-sensor.c +1 ~~-12~~

index 51359472eef2..6c57b1e262f9 100644
@@ -16,22 +16,11 @@
 #include <media/v4l2-subdev.h>
 #include <media/tpg/v4l2-tpg.h>
 
+#include "vimc.h"
 #include "vimc-common.h"
 
 #define VIMC_SEN_DRV_NAME "vimc-sensor"
 
-struct vimc_sen_device {
-	struct vimc_ent_device ved;
-	struct v4l2_subdev sd;
-	struct device *dev;
-	struct tpg_data tpg;
-	struct task_struct *kthread_sen;
-	u8 *frame;
-	/* The active format */
-	struct v4l2_mbus_framefmt mbus_format;
-	struct v4l2_ctrl_handler hdl;
-};
-
 static const struct v4l2_mbus_framefmt fmt_default = {
 	.width = 640,
 	.height = 480,

drivers/media/platform/vimc/vimc.h new file +0 -0

new file mode 100644
index 000000000000..a5adebdda941

I was wondering if instead of creating a vimc.h, if we could move to vimc-common.h,just because it is not clear to me the difference between the two now, what do you think?

Regards,
Helen

Quoted 104 lines

drivers/media/platform/vimc/vimc.h new file +102 -0

@@ -0,0 +1,102 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ *
+ * Copyright (C) 2019 Shuah Khan <[email protected]>
+ *
+ */
+
+/*
+ * This file defines vimc driver device and sub-device structures.
+ */
+
+#ifndef _VIMC_H_
+#define _VIMC_H_
+
+#include <linux/platform_device.h>
+#include <media/media-device.h>
+#include <media/v4l2-device.h>
+#include <media/videobuf2-core.h>
+#include <media/tpg/v4l2-tpg.h>
+#include <media/v4l2-ctrls.h>
+
+#include "vimc-common.h"
+
+struct vimc_cap_device {
+	struct vimc_ent_device ved;
+	struct video_device vdev;
+	struct device *dev;
+	struct v4l2_pix_format format;
+	struct vb2_queue queue;
+	struct list_head buf_list;
+	/*
+	 * NOTE: in a real driver, a spin lock must be used to access the
+	 * queue because the frames are generated from a hardware interruption
+	 * and the isr is not allowed to sleep.
+	 * Even if it is not necessary a spinlock in the vimc driver, we
+	 * use it here as a code reference
+	 */
+	spinlock_t qlock;
+	struct mutex lock;
+	u32 sequence;
+	struct vimc_stream stream;
+};
+
+struct vimc_sca_device {
+	struct vimc_ent_device ved;
+	struct v4l2_subdev sd;
+	struct device *dev;
+	/* NOTE: the source fmt is the same as the sink
+	 * with the width and hight multiplied by mult
+	 */
+	struct v4l2_mbus_framefmt sink_fmt;
+	/* Values calculated when the stream starts */
+	u8 *src_frame;
+	unsigned int src_line_size;
+	unsigned int bpp;
+};
+
+struct vimc_deb_device {
+	struct vimc_ent_device ved;
+	struct v4l2_subdev sd;
+	struct device *dev;
+	/* The active format */
+	struct v4l2_mbus_framefmt sink_fmt;
+	u32 src_code;
+	void (*set_rgb_src)(struct vimc_deb_device *vdeb, unsigned int lin,
+			    unsigned int col, unsigned int rgb[3]);
+	/* Values calculated when the stream starts */
+	u8 *src_frame;
+	const struct vimc_deb_pix_map *sink_pix_map;
+	unsigned int sink_bpp;
+};
+
+struct vimc_sen_device {
+	struct vimc_ent_device ved;
+	struct v4l2_subdev sd;
+	struct device *dev;
+	struct tpg_data tpg;
+	struct task_struct *kthread_sen;
+	u8 *frame;
+	/* The active format */
+	struct v4l2_mbus_framefmt mbus_format;
+	struct v4l2_ctrl_handler hdl;
+};
+
+struct vimc_device {
+	/* The platform device */
+	struct platform_device pdev;
+
+	/* The pipeline configuration */
+	const struct vimc_pipeline_config *pipe_cfg;
+
+	/* The Associated media_device parent */
+	struct media_device mdev;
+
+	/* Internal v4l2 parent device*/
+	struct v4l2_device v4l2_dev;
+
+	/* Subdevices */
+	struct platform_device **subdevs;
+};
+
+#endif

Shuah Khan

author

2019-08-12 14:03 UTC #3 lore raw

Re: [PATCH 1/3] media: vimc: move private defines to a common header

Hi Helen,

Quoted 191 lines · Helen Koike

Hi Shuah,

Thanks for the patch.

Quoted 186 lines · Shuah Khan

In preparation for collapsing the component driver structure into a monolith, move private device structure defines to a new common header file.

Signed-off-by Shuah Khan <[email protected]>

---
  drivers/media/platform/vimc/vimc-capture.c |  21 +----
  drivers/media/platform/vimc/vimc-core.c    |  18 +---
  drivers/media/platform/vimc/vimc-debayer.c |  16 +---
  drivers/media/platform/vimc/vimc-scaler.c  |  15 +--
  drivers/media/platform/vimc/vimc-sensor.c  |  13 +--
  drivers/media/platform/vimc/vimc.h         | 102 +++++++++++++++++++++
  6 files changed, 107 insertions(+), 78 deletions(-)

create mode 100644 drivers/media/platform/vimc/vimc.h

drivers/media/platform/vimc/vimc-capture.c +1 ~~-20~~

index 664855708fdf..c52fc5d97c2d 100644
@@ -13,6 +13,7 @@
  #include <media/videobuf2-core.h>
  #include <media/videobuf2-vmalloc.h>
 
+#include "vimc.h"
  #include "vimc-common.h"
  #include "vimc-streamer.h"

@@ -44,26 +45,6 @@ static const u32 vimc_cap_supported_pixfmt[] = {
  	V4L2_PIX_FMT_SRGGB12,
  };
 
-struct vimc_cap_device {
-	struct vimc_ent_device ved;
-	struct video_device vdev;
-	struct device *dev;
-	struct v4l2_pix_format format;
-	struct vb2_queue queue;
-	struct list_head buf_list;
-	/*
-	 * NOTE: in a real driver, a spin lock must be used to access the
-	 * queue because the frames are generated from a hardware interruption
-	 * and the isr is not allowed to sleep.
-	 * Even if it is not necessary a spinlock in the vimc driver, we
-	 * use it here as a code reference
-	 */
-	spinlock_t qlock;
-	struct mutex lock;
-	u32 sequence;
-	struct vimc_stream stream;
-};
-
  static const struct v4l2_pix_format fmt_default = {
  	.width = 640,
  	.height = 480,

drivers/media/platform/vimc/vimc-core.c +1 ~~-17~~

index 571c55aa0e16..c9b351472272 100644
@@ -12,6 +12,7 @@
  #include <media/media-device.h>
  #include <media/v4l2-device.h>
 
+#include "vimc.h"
  #include "vimc-common.h"
 
  #define VIMC_MDEV_MODEL_NAME "VIMC MDEV"

@@ -24,23 +25,6 @@
  	.flags = link_flags,					\
  }
 
-struct vimc_device {
-	/* The platform device */
-	struct platform_device pdev;
-
-	/* The pipeline configuration */
-	const struct vimc_pipeline_config *pipe_cfg;
-
-	/* The Associated media_device parent */
-	struct media_device mdev;
-
-	/* Internal v4l2 parent device*/
-	struct v4l2_device v4l2_dev;
-
-	/* Subdevices */
-	struct platform_device **subdevs;
-};
-
  /* Structure which describes individual configuration for each entity */
  struct vimc_ent_config {
  	const char *name;

drivers/media/platform/vimc/vimc-debayer.c +1 ~~-15~~

index 00598fbf3cba..750752bb173c 100644
@@ -13,6 +13,7 @@
  #include <linux/v4l2-mediabus.h>
  #include <media/v4l2-subdev.h>
 
+#include "vimc.h"
  #include "vimc-common.h"
 
  #define VIMC_DEB_DRV_NAME "vimc-debayer"

@@ -44,21 +45,6 @@ struct vimc_deb_pix_map {
  	enum vimc_deb_rgb_colors order[2][2];
  };
 
-struct vimc_deb_device {
-	struct vimc_ent_device ved;
-	struct v4l2_subdev sd;
-	struct device *dev;
-	/* The active format */
-	struct v4l2_mbus_framefmt sink_fmt;
-	u32 src_code;
-	void (*set_rgb_src)(struct vimc_deb_device *vdeb, unsigned int lin,
-			    unsigned int col, unsigned int rgb[3]);
-	/* Values calculated when the stream starts */
-	u8 *src_frame;
-	const struct vimc_deb_pix_map *sink_pix_map;
-	unsigned int sink_bpp;
-};
-
  static const struct v4l2_mbus_framefmt sink_fmt_default = {
  	.width = 640,
  	.height = 480,

drivers/media/platform/vimc/vimc-scaler.c +1 ~~-14~~

index c7123a45c55b..fe99b9102ada 100644
@@ -13,6 +13,7 @@
  #include <linux/v4l2-mediabus.h>
  #include <media/v4l2-subdev.h>
 
+#include "vimc.h"
  #include "vimc-common.h"
 
  #define VIMC_SCA_DRV_NAME "vimc-scaler"

@@ -31,20 +32,6 @@ static const u32 vimc_sca_supported_pixfmt[] = {
  	V4L2_PIX_FMT_ARGB32,
  };
 
-struct vimc_sca_device {
-	struct vimc_ent_device ved;
-	struct v4l2_subdev sd;
-	struct device *dev;
-	/* NOTE: the source fmt is the same as the sink
-	 * with the width and hight multiplied by mult
-	 */
-	struct v4l2_mbus_framefmt sink_fmt;
-	/* Values calculated when the stream starts */
-	u8 *src_frame;
-	unsigned int src_line_size;
-	unsigned int bpp;
-};
-
  static const struct v4l2_mbus_framefmt sink_fmt_default = {
  	.width = 640,
  	.height = 480,

drivers/media/platform/vimc/vimc-sensor.c +1 ~~-12~~

index 51359472eef2..6c57b1e262f9 100644
@@ -16,22 +16,11 @@
  #include <media/v4l2-subdev.h>
  #include <media/tpg/v4l2-tpg.h>
 
+#include "vimc.h"
  #include "vimc-common.h"
 
  #define VIMC_SEN_DRV_NAME "vimc-sensor"
 
-struct vimc_sen_device {
-	struct vimc_ent_device ved;
-	struct v4l2_subdev sd;
-	struct device *dev;
-	struct tpg_data tpg;
-	struct task_struct *kthread_sen;
-	u8 *frame;
-	/* The active format */
-	struct v4l2_mbus_framefmt mbus_format;
-	struct v4l2_ctrl_handler hdl;
-};
-
  static const struct v4l2_mbus_framefmt fmt_default = {
  	.width = 640,
  	.height = 480,

drivers/media/platform/vimc/vimc.h +0 -0

Quoted 9 lines

drivers/media/platform/vimc/vimc.h
Quoted 4 lines
new file mode 100644
index 000000000000..a5adebdda941
drivers/media/platform/vimc/vimc.h new file +0 -0
I was wondering if instead of creating a vimc.h, if we could move to vimc-common.h,just because it is not clear to me the difference between the two now, what do you think?

My thinking is that vimc-common.h is common for all the subdevs and putting vimc-core defines and structures it shares it with the subdev files can be in a separate file.

It is more of design choice to keep structures and defined organized. Originally I was thinking all the subdev device structires need to be global, and my patch set I sent out as such doesn't need that. I just overlooked that when I sent the patches out.

This reduces the number of things that need to be common, I don't really have any strong reasons for either choice of adding common defines to vimc-common.h vs vimc.h - maybe with a slight tilt towards liking vimc.h

thanks,
-- Shuah

Laurent Pinchart

2019-08-10 14:14 UTC #4 lore raw

Re: [PATCH 1/3] media: vimc: move private defines to a common header

In reply to Shuah Khan

Hi Shuah,

Thank you for the patch.

Quoted 3 lines · Shuah Khan

In preparation for collapsing the component driver structure into a monolith, move private device structure defines to a new common header file.

Apart from the vimc_device structure, this doesn't seem to be needed. I'd rather keep each structure private to the .c file that handles it, and only share vimc_device globally.

Quoted 289 lines

Signed-off-by Shuah Khan <[email protected]>

---
 drivers/media/platform/vimc/vimc-capture.c |  21 +----
 drivers/media/platform/vimc/vimc-core.c    |  18 +---
 drivers/media/platform/vimc/vimc-debayer.c |  16 +---
 drivers/media/platform/vimc/vimc-scaler.c  |  15 +--
 drivers/media/platform/vimc/vimc-sensor.c  |  13 +--
 drivers/media/platform/vimc/vimc.h         | 102 +++++++++++++++++++++
 6 files changed, 107 insertions(+), 78 deletions(-)
 create mode 100644 drivers/media/platform/vimc/vimc.h

drivers/media/platform/vimc/vimc-capture.c +1 ~~-20~~

index 664855708fdf..c52fc5d97c2d 100644
@@ -13,6 +13,7 @@
 #include <media/videobuf2-core.h>
 #include <media/videobuf2-vmalloc.h>
 
+#include "vimc.h"
 #include "vimc-common.h"
 #include "vimc-streamer.h"

@@ -44,26 +45,6 @@ static const u32 vimc_cap_supported_pixfmt[] = {
 	V4L2_PIX_FMT_SRGGB12,
 };
 
-struct vimc_cap_device {
-	struct vimc_ent_device ved;
-	struct video_device vdev;
-	struct device *dev;
-	struct v4l2_pix_format format;
-	struct vb2_queue queue;
-	struct list_head buf_list;
-	/*
-	 * NOTE: in a real driver, a spin lock must be used to access the
-	 * queue because the frames are generated from a hardware interruption
-	 * and the isr is not allowed to sleep.
-	 * Even if it is not necessary a spinlock in the vimc driver, we
-	 * use it here as a code reference
-	 */
-	spinlock_t qlock;
-	struct mutex lock;
-	u32 sequence;
-	struct vimc_stream stream;
-};
-
 static const struct v4l2_pix_format fmt_default = {
 	.width = 640,
 	.height = 480,

drivers/media/platform/vimc/vimc-core.c +1 ~~-17~~

index 571c55aa0e16..c9b351472272 100644
@@ -12,6 +12,7 @@
 #include <media/media-device.h>
 #include <media/v4l2-device.h>
 
+#include "vimc.h"
 #include "vimc-common.h"
 
 #define VIMC_MDEV_MODEL_NAME "VIMC MDEV"

@@ -24,23 +25,6 @@
 	.flags = link_flags,					\
 }
 
-struct vimc_device {
-	/* The platform device */
-	struct platform_device pdev;
-
-	/* The pipeline configuration */
-	const struct vimc_pipeline_config *pipe_cfg;
-
-	/* The Associated media_device parent */
-	struct media_device mdev;
-
-	/* Internal v4l2 parent device*/
-	struct v4l2_device v4l2_dev;
-
-	/* Subdevices */
-	struct platform_device **subdevs;
-};
-
 /* Structure which describes individual configuration for each entity */
 struct vimc_ent_config {
 	const char *name;

drivers/media/platform/vimc/vimc-debayer.c +1 ~~-15~~

index 00598fbf3cba..750752bb173c 100644
@@ -13,6 +13,7 @@
 #include <linux/v4l2-mediabus.h>
 #include <media/v4l2-subdev.h>
 
+#include "vimc.h"
 #include "vimc-common.h"
 
 #define VIMC_DEB_DRV_NAME "vimc-debayer"

@@ -44,21 +45,6 @@ struct vimc_deb_pix_map {
 	enum vimc_deb_rgb_colors order[2][2];
 };
 
-struct vimc_deb_device {
-	struct vimc_ent_device ved;
-	struct v4l2_subdev sd;
-	struct device *dev;
-	/* The active format */
-	struct v4l2_mbus_framefmt sink_fmt;
-	u32 src_code;
-	void (*set_rgb_src)(struct vimc_deb_device *vdeb, unsigned int lin,
-			    unsigned int col, unsigned int rgb[3]);
-	/* Values calculated when the stream starts */
-	u8 *src_frame;
-	const struct vimc_deb_pix_map *sink_pix_map;
-	unsigned int sink_bpp;
-};
-
 static const struct v4l2_mbus_framefmt sink_fmt_default = {
 	.width = 640,
 	.height = 480,

drivers/media/platform/vimc/vimc-scaler.c +1 ~~-14~~

index c7123a45c55b..fe99b9102ada 100644
@@ -13,6 +13,7 @@
 #include <linux/v4l2-mediabus.h>
 #include <media/v4l2-subdev.h>
 
+#include "vimc.h"
 #include "vimc-common.h"
 
 #define VIMC_SCA_DRV_NAME "vimc-scaler"

@@ -31,20 +32,6 @@ static const u32 vimc_sca_supported_pixfmt[] = {
 	V4L2_PIX_FMT_ARGB32,
 };
 
-struct vimc_sca_device {
-	struct vimc_ent_device ved;
-	struct v4l2_subdev sd;
-	struct device *dev;
-	/* NOTE: the source fmt is the same as the sink
-	 * with the width and hight multiplied by mult
-	 */
-	struct v4l2_mbus_framefmt sink_fmt;
-	/* Values calculated when the stream starts */
-	u8 *src_frame;
-	unsigned int src_line_size;
-	unsigned int bpp;
-};
-
 static const struct v4l2_mbus_framefmt sink_fmt_default = {
 	.width = 640,
 	.height = 480,

drivers/media/platform/vimc/vimc-sensor.c +1 ~~-12~~

index 51359472eef2..6c57b1e262f9 100644
@@ -16,22 +16,11 @@
 #include <media/v4l2-subdev.h>
 #include <media/tpg/v4l2-tpg.h>
 
+#include "vimc.h"
 #include "vimc-common.h"
 
 #define VIMC_SEN_DRV_NAME "vimc-sensor"
 
-struct vimc_sen_device {
-	struct vimc_ent_device ved;
-	struct v4l2_subdev sd;
-	struct device *dev;
-	struct tpg_data tpg;
-	struct task_struct *kthread_sen;
-	u8 *frame;
-	/* The active format */
-	struct v4l2_mbus_framefmt mbus_format;
-	struct v4l2_ctrl_handler hdl;
-};
-
 static const struct v4l2_mbus_framefmt fmt_default = {
 	.width = 640,
 	.height = 480,

drivers/media/platform/vimc/vimc.h new file +102 -0

new file mode 100644
index 000000000000..a5adebdda941
@@ -0,0 +1,102 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ *
+ * Copyright (C) 2019 Shuah Khan <[email protected]>
+ *
+ */
+
+/*
+ * This file defines vimc driver device and sub-device structures.
+ */
+
+#ifndef _VIMC_H_
+#define _VIMC_H_
+
+#include <linux/platform_device.h>
+#include <media/media-device.h>
+#include <media/v4l2-device.h>
+#include <media/videobuf2-core.h>
+#include <media/tpg/v4l2-tpg.h>
+#include <media/v4l2-ctrls.h>
+
+#include "vimc-common.h"
+
+struct vimc_cap_device {
+	struct vimc_ent_device ved;
+	struct video_device vdev;
+	struct device *dev;
+	struct v4l2_pix_format format;
+	struct vb2_queue queue;
+	struct list_head buf_list;
+	/*
+	 * NOTE: in a real driver, a spin lock must be used to access the
+	 * queue because the frames are generated from a hardware interruption
+	 * and the isr is not allowed to sleep.
+	 * Even if it is not necessary a spinlock in the vimc driver, we
+	 * use it here as a code reference
+	 */
+	spinlock_t qlock;
+	struct mutex lock;
+	u32 sequence;
+	struct vimc_stream stream;
+};
+
+struct vimc_sca_device {
+	struct vimc_ent_device ved;
+	struct v4l2_subdev sd;
+	struct device *dev;
+	/* NOTE: the source fmt is the same as the sink
+	 * with the width and hight multiplied by mult
+	 */
+	struct v4l2_mbus_framefmt sink_fmt;
+	/* Values calculated when the stream starts */
+	u8 *src_frame;
+	unsigned int src_line_size;
+	unsigned int bpp;
+};
+
+struct vimc_deb_device {
+	struct vimc_ent_device ved;
+	struct v4l2_subdev sd;
+	struct device *dev;
+	/* The active format */
+	struct v4l2_mbus_framefmt sink_fmt;
+	u32 src_code;
+	void (*set_rgb_src)(struct vimc_deb_device *vdeb, unsigned int lin,
+			    unsigned int col, unsigned int rgb[3]);
+	/* Values calculated when the stream starts */
+	u8 *src_frame;
+	const struct vimc_deb_pix_map *sink_pix_map;
+	unsigned int sink_bpp;
+};
+
+struct vimc_sen_device {
+	struct vimc_ent_device ved;
+	struct v4l2_subdev sd;
+	struct device *dev;
+	struct tpg_data tpg;
+	struct task_struct *kthread_sen;
+	u8 *frame;
+	/* The active format */
+	struct v4l2_mbus_framefmt mbus_format;
+	struct v4l2_ctrl_handler hdl;
+};
+
+struct vimc_device {
+	/* The platform device */
+	struct platform_device pdev;
+
+	/* The pipeline configuration */
+	const struct vimc_pipeline_config *pipe_cfg;
+
+	/* The Associated media_device parent */
+	struct media_device mdev;
+
+	/* Internal v4l2 parent device*/
+	struct v4l2_device v4l2_dev;
+
+	/* Subdevices */
+	struct platform_device **subdevs;
+};
+
+#endif

Signature · 3 lines

Regards,

Laurent Pinchart

Shuah Khan

author

2019-08-12 14:19 UTC #5 lore raw

Re: [PATCH 1/3] media: vimc: move private defines to a common header

Hi Laurent,

Quoted 4 lines · Laurent Pinchart

Hi Shuah,
Thank you for the patch.

Quoted 9 lines

Quoted 3 lines · Shuah Khan
In preparation for collapsing the component driver structure into a monolith, move private device structure defines to a new common header file.
Apart from the vimc_device structure, this doesn't seem to be needed. I'd rather keep each structure private to the .c file that handles it, and only share vimc_device globally.

Right. I initially thought that I needed these global. Once I completed the patches without needing these as global, I overlooked updating the patches.

I will take care of that. Any thoughts on vimc.h vs. adding vimc_device struct to existing vimc-common.h

As I explained to Helen in response to her comment about:

"My thinking is that vimc-common.h is common for all the subdevs and putting vimc-core defines and structures it shares it with the subdev files can be in a separate file.

Thanks all for a quick review and testing. I will work on v2 with your comments. I want to make sure topology either looks the same as what is in media master. I think it is, but I want to double check.

thanks,
-- Shuah

Laurent Pinchart

2019-08-12 14:24 UTC #6 lore raw

Re: [PATCH 1/3] media: vimc: move private defines to a common header

Hi Shua,

Quoted 23 lines · Shuah Khan

Quoted 8 lines · Laurent Pinchart
Quoted 3 lines · Shuah Khan
In preparation for collapsing the component driver structure into a monolith, move private device structure defines to a new common header file.
Apart from the vimc_device structure, this doesn't seem to be needed. I'd rather keep each structure private to the .c file that handles it, and only share vimc_device globally.
Right. I initially thought that I needed these global. Once I completed the patches without needing these as global, I overlooked updating the patches.
I will take care of that. Any thoughts on vimc.h vs. adding vimc_device struct to existing vimc-common.h
As I explained to Helen in response to her comment about:
"My thinking is that vimc-common.h is common for all the subdevs and putting vimc-core defines and structures it shares it with the subdev files can be in a separate file.

Quoted 8 lines

It is more of design choice to keep structures and defined organized. Originally I was thinking all the subdev device structires need to be global, and my patch set I sent out as such doesn't need that. I just overlooked that when I sent the patches out.
This reduces the number of things that need to be common, I don't really have any strong reasons for either choice of adding common defines to vimc-common.h vs vimc.h - maybe with a slight tilt towards vimc.h"

The vimc_device structure fits nicely in vimc-common.h in my opinion, as it's used by every component. I don't care much either way.

Quoted 3 lines

Thanks all for a quick review and testing. I will work on v2 with your comments. I want to make sure topology either looks the same as what is in media master. I think it is, but I want to double check.

Signature · 3 lines

Regards,

Laurent Pinchart

Shuah Khan

author

2019-08-12 14:27 UTC #7 lore raw

Re: [PATCH 1/3] media: vimc: move private defines to a common header

Quoted 31 lines · Laurent Pinchart

Hi Shua,
Quoted 28 lines · Shuah Khan
Quoted 8 lines · Laurent Pinchart
Quoted 3 lines · Shuah Khan
In preparation for collapsing the component driver structure into a monolith, move private device structure defines to a new common header file.
Apart from the vimc_device structure, this doesn't seem to be needed. I'd rather keep each structure private to the .c file that handles it, and only share vimc_device globally.
Right. I initially thought that I needed these global. Once I completed the patches without needing these as global, I overlooked updating the patches.
I will take care of that. Any thoughts on vimc.h vs. adding vimc_device struct to existing vimc-common.h
As I explained to Helen in response to her comment about:
"My thinking is that vimc-common.h is common for all the subdevs and putting vimc-core defines and structures it shares it with the subdev files can be in a separate file.
It is more of design choice to keep structures and defined organized. Originally I was thinking all the subdev device structires need to be global, and my patch set I sent out as such doesn't need that. I just overlooked that when I sent the patches out.

Quoted 7 lines

Quoted 3 lines
This reduces the number of things that need to be common, I don't really have any strong reasons for either choice of adding common defines to vimc-common.h vs vimc.h - maybe with a slight tilt towards vimc.h"
The vimc_device structure fits nicely in vimc-common.h in my opinion, as it's used by every component. I don't care much either way.

Sounds good to me.

thanks,
-- Shuah

Shuah Khan

author

2019-08-09 21:45 UTC #8 lore raw

PATCH 2/3 media: vimc: Collapse component structure into a single monolithic driver

In reply to Shuah Khan

Collapse it into a single monolithic driver removing the Component API. This patch removes the component API and makes minimal changes to the code base preserving the functional division of the code structure. Preserving the functional structure allows us to split the sensor off as a separate module in the future.

The following configure and stream test works on all devices.

media-ctl -d platform:vimc -V '"Sensor A":0[fmt:SBGGR8_1X8/640x480]' media-ctl -d platform:vimc -V '"Debayer A":0[fmt:SBGGR8_1X8/640x480]' media-ctl -d platform:vimc -V '"Sensor B":0[fmt:SBGGR8_1X8/640x480]' media-ctl -d platform:vimc -V '"Debayer B":0[fmt:SBGGR8_1X8/640x480]'

v4l2-ctl -z platform:vimc -d "RGB/YUV Capture" -v width=1920,height=1440 v4l2-ctl -z platform:vimc -d "Raw Capture 0" -v pixelformat=BA81 v4l2-ctl -z platform:vimc -d "Raw Capture 1" -v pixelformat=BA81

v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video1 v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video2 v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video3

Signed-off-by Shuah Khan <[email protected]>

---
 drivers/media/platform/vimc/Makefile       |   7 +-
 drivers/media/platform/vimc/vimc-capture.c |  75 ++--------
 drivers/media/platform/vimc/vimc-core.c    | 156 ++++++++-------------
 drivers/media/platform/vimc/vimc-debayer.c |  68 ++-------
 drivers/media/platform/vimc/vimc-scaler.c  |  68 ++-------
 drivers/media/platform/vimc/vimc-sensor.c  |  69 ++-------
 drivers/media/platform/vimc/vimc.h         |  25 +++-
 7 files changed, 125 insertions(+), 343 deletions(-)

drivers/media/platform/vimc/Makefile +4 -3

index 96d06f030c31..a53b2b532e9f 100644
@@ -1,5 +1,6 @@
 # SPDX-License-Identifier: GPL-2.0
-vimc-y := vimc-core.o vimc-common.o vimc-streamer.o
+vimc-y := vimc-core.o vimc-common.o vimc-streamer.o vimc-capture.o \
+		vimc-debayer.o vimc-scaler.o vimc-sensor.o
+
+obj-$(CONFIG_VIDEO_VIMC) += vimc.o
 
-obj-$(CONFIG_VIDEO_VIMC) += vimc.o vimc-capture.o vimc-debayer.o \
-                vimc-scaler.o vimc-sensor.o

drivers/media/platform/vimc/vimc-capture.c +13 ~~-62~~

137 lines not on this page · raw message

drivers/media/platform/vimc/vimc-core.c +55 ~~-101~~

302 lines not on this page · raw message

drivers/media/platform/vimc/vimc-debayer.c +10 ~~-58~~

111 lines not on this page · raw message

drivers/media/platform/vimc/vimc-scaler.c +11 ~~-57~~

110 lines not on this page · raw message

drivers/media/platform/vimc/vimc-sensor.c +10 ~~-59~~

116 lines not on this page · raw message

drivers/media/platform/vimc/vimc.h +22 -3

index a5adebdda941..cd527c6fe6fc 100644
@@ -92,11 +92,30 @@ struct vimc_device {
 	/* The Associated media_device parent */
 	struct media_device mdev;
 
-	/* Internal v4l2 parent device*/
+	/* Internal v4l2 parent device */
 	struct v4l2_device v4l2_dev;
+};
 
-	/* Subdevices */
-	struct platform_device **subdevs;
+/* Structure which describes individual configuration for each entity */
+struct vimc_ent_config {
+	const char *name;
+	const char *drv;
+	struct vimc_ent_device *ved;
+	int (*add)(struct vimc_device *vimc, struct vimc_ent_config *vent);
+	void (*rm)(struct vimc_device *vimc, struct vimc_ent_config *vent);
 };
 
+/* prototypes for vimc_ent_config add and rm hooks */
+int vimc_cap_add(struct vimc_device *vimc, struct vimc_ent_config *vent);
+void vimc_cap_rm(struct vimc_device *vimc, struct vimc_ent_config *vent);
+
+int vimc_deb_add(struct vimc_device *vimc, struct vimc_ent_config *vent);
+void vimc_deb_rm(struct vimc_device *vimc, struct vimc_ent_config *vent);
+
+int vimc_sca_add(struct vimc_device *vimc, struct vimc_ent_config *vent);
+void vimc_sca_rm(struct vimc_device *vimc, struct vimc_ent_config *vent);
+
+int vimc_sen_add(struct vimc_device *vimc, struct vimc_ent_config *vent);
+void vimc_sen_rm(struct vimc_device *vimc, struct vimc_ent_config *vent);
+
 #endif

Signature · 1 line

2.17.1

Helen Koike

2019-08-10 04:12 UTC #9 lore raw

Re: [PATCH 2/3] media: vimc: Collapse component structure into a single monolithic driver

Hi Shuah,

Thanks for the patch, just some small comments.

Quoted 323 lines · Shuah Khan

The following configure and stream test works on all devices.

v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video1 v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video2 v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video3

Signed-off-by Shuah Khan <[email protected]>

---
 drivers/media/platform/vimc/Makefile       |   7 +-
 drivers/media/platform/vimc/vimc-capture.c |  75 ++--------
 drivers/media/platform/vimc/vimc-core.c    | 156 ++++++++-------------
 drivers/media/platform/vimc/vimc-debayer.c |  68 ++-------
 drivers/media/platform/vimc/vimc-scaler.c  |  68 ++-------
 drivers/media/platform/vimc/vimc-sensor.c  |  69 ++-------
 drivers/media/platform/vimc/vimc.h         |  25 +++-
 7 files changed, 125 insertions(+), 343 deletions(-)

drivers/media/platform/vimc/Makefile +4 -3

index 96d06f030c31..a53b2b532e9f 100644
@@ -1,5 +1,6 @@
 # SPDX-License-Identifier: GPL-2.0
-vimc-y := vimc-core.o vimc-common.o vimc-streamer.o
+vimc-y := vimc-core.o vimc-common.o vimc-streamer.o vimc-capture.o \
+		vimc-debayer.o vimc-scaler.o vimc-sensor.o
+
+obj-$(CONFIG_VIDEO_VIMC) += vimc.o
 
-obj-$(CONFIG_VIDEO_VIMC) += vimc.o vimc-capture.o vimc-debayer.o \
-                vimc-scaler.o vimc-sensor.o

drivers/media/platform/vimc/vimc-capture.c +13 ~~-62~~

index c52fc5d97c2d..b7b2d3c3d4f8 100644
@@ -5,10 +5,6 @@
  * Copyright (C) 2015-2017 Helen Koike <[email protected]>
  */
 
-#include <linux/component.h>
-#include <linux/module.h>
-#include <linux/mod_devicetable.h>
-#include <linux/platform_device.h>
 #include <media/v4l2-ioctl.h>
 #include <media/videobuf2-core.h>
 #include <media/videobuf2-vmalloc.h>

@@ -17,8 +13,6 @@
 #include "vimc-common.h"
 #include "vimc-streamer.h"
 
-#define VIMC_CAP_DRV_NAME "vimc-capture"
-
 static const u32 vimc_cap_supported_pixfmt[] = {
 	V4L2_PIX_FMT_BGR24,
 	V4L2_PIX_FMT_RGB24,

@@ -348,11 +342,11 @@ static void vimc_cap_release(struct video_device *vdev)
 	kfree(vcap);
 }
 
-static void vimc_cap_comp_unbind(struct device *comp, struct device *master,
-				 void *master_data)
+void vimc_cap_rm(struct vimc_device *vimc, struct vimc_ent_config *vent)
 {
-	struct vimc_ent_device *ved = dev_get_drvdata(comp);
-	struct vimc_cap_device *vcap = container_of(ved, struct vimc_cap_device,
+	struct vimc_ent_device *ved = vent->ved;
+	struct vimc_cap_device *vcap = container_of(ved,
+						    struct vimc_cap_device,
 						    ved);
 
 	vb2_queue_release(&vcap->queue);

@@ -399,11 +393,9 @@ static void *vimc_cap_process_frame(struct vimc_ent_device *ved,
 	return NULL;
 }
 
-static int vimc_cap_comp_bind(struct device *comp, struct device *master,
-			      void *master_data)
+int vimc_cap_add(struct vimc_device *vimc, struct vimc_ent_config *vent)
 {
-	struct v4l2_device *v4l2_dev = master_data;
-	struct vimc_platform_data *pdata = comp->platform_data;
+	struct v4l2_device *v4l2_dev = &vimc->v4l2_dev;
 	struct vimc_cap_device *vcap;
 	struct video_device *vdev;
 	struct vb2_queue *q;

@@ -423,7 +415,7 @@ static int vimc_cap_comp_bind(struct device *comp, struct device *master,
 	}
 
 	/* Initialize the media entity */
-	vcap->vdev.entity.name = pdata->entity_name;
+	vcap->vdev.entity.name = vent->name;
 	vcap->vdev.entity.function = MEDIA_ENT_F_IO_V4L;
 	ret = media_entity_pads_init(&vcap->vdev.entity,
 				     1, vcap->ved.pads);

@@ -447,8 +439,8 @@ static int vimc_cap_comp_bind(struct device *comp, struct device *master,
 
 	ret = vb2_queue_init(q);
 	if (ret) {
-		dev_err(comp, "%s: vb2 queue init failed (err=%d)\n",
-			pdata->entity_name, ret);
+		dev_err(&vimc->pdev.dev, "%s: vb2 queue init failed (err=%d)\n",
+			vent->name, ret);
 		goto err_clean_m_ent;
 	}

@@ -465,8 +457,7 @@ static int vimc_cap_comp_bind(struct device *comp, struct device *master,
 	vcap->ved.ent = &vcap->vdev.entity;
 	vcap->ved.process_frame = vimc_cap_process_frame;
 	vcap->ved.vdev_get_format = vimc_cap_get_format;
-	dev_set_drvdata(comp, &vcap->ved);
-	vcap->dev = comp;
+	vcap->dev = &vimc->pdev.dev;
 
 	/* Initialize the video_device struct */
 	vdev = &vcap->vdev;

@@ -479,17 +470,18 @@ static int vimc_cap_comp_bind(struct device *comp, struct device *master,
 	vdev->queue = q;
 	vdev->v4l2_dev = v4l2_dev;
 	vdev->vfl_dir = VFL_DIR_RX;
-	strscpy(vdev->name, pdata->entity_name, sizeof(vdev->name));
+	strscpy(vdev->name, vent->name, sizeof(vdev->name));
 	video_set_drvdata(vdev, &vcap->ved);
 
 	/* Register the video_device with the v4l2 and the media framework */
 	ret = video_register_device(vdev, VFL_TYPE_GRABBER, -1);
 	if (ret) {
-		dev_err(comp, "%s: video register failed (err=%d)\n",
+		dev_err(&vimc->pdev.dev, "%s: video register failed (err=%d)\n",
 			vcap->vdev.name, ret);
 		goto err_release_queue;
 	}
 
+	vent->ved = &vcap->ved;
 	return 0;
 
 err_release_queue:

@@ -503,44 +495,3 @@ static int vimc_cap_comp_bind(struct device *comp, struct device *master,
 
 	return ret;
 }
-
-static const struct component_ops vimc_cap_comp_ops = {
-	.bind = vimc_cap_comp_bind,
-	.unbind = vimc_cap_comp_unbind,
-};
-
-static int vimc_cap_probe(struct platform_device *pdev)
-{
-	return component_add(&pdev->dev, &vimc_cap_comp_ops);
-}
-
-static int vimc_cap_remove(struct platform_device *pdev)
-{
-	component_del(&pdev->dev, &vimc_cap_comp_ops);
-
-	return 0;
-}
-
-static const struct platform_device_id vimc_cap_driver_ids[] = {
-	{
-		.name           = VIMC_CAP_DRV_NAME,
-	},
-	{ }
-};
-
-static struct platform_driver vimc_cap_pdrv = {
-	.probe		= vimc_cap_probe,
-	.remove		= vimc_cap_remove,
-	.id_table	= vimc_cap_driver_ids,
-	.driver		= {
-		.name	= VIMC_CAP_DRV_NAME,
-	},
-};
-
-module_platform_driver(vimc_cap_pdrv);
-
-MODULE_DEVICE_TABLE(platform, vimc_cap_driver_ids);
-
-MODULE_DESCRIPTION("Virtual Media Controller Driver (VIMC) Capture");
-MODULE_AUTHOR("Helen Mae Koike Fornazier <[email protected]>");
-MODULE_LICENSE("GPL");

drivers/media/platform/vimc/vimc-core.c +20 -9

index c9b351472272..3bd89cf38a90 100644
@@ -5,7 +5,6 @@
  * Copyright (C) 2015-2017 Helen Koike <[email protected]>
  */
 
-#include <linux/component.h>
 #include <linux/init.h>
 #include <linux/module.h>
 #include <linux/platform_device.h>

@@ -25,12 +24,6 @@
 	.flags = link_flags,					\
 }
 
-/* Structure which describes individual configuration for each entity */
-struct vimc_ent_config {
-	const char *name;
-	const char *drv;
-};
-
 /* Structure which describes links between entities */
 struct vimc_ent_link {
 	unsigned int src_ent;

@@ -42,7 +35,7 @@ struct vimc_ent_link {
 
 /* Structure which describes the whole topology */
 struct vimc_pipeline_config {
-	const struct vimc_ent_config *ents;
+	struct vimc_ent_config *ents;
 	size_t num_ents;
 	const struct vimc_ent_link *links;
 	size_t num_links;

@@ -52,43 +45,61 @@ struct vimc_pipeline_config {
  * Topology Configuration
  */
 
-static const struct vimc_ent_config ent_config[] = {
+static struct vimc_ent_config ent_config[] = {
 	{
 		.name = "Sensor A",
 		.drv = "vimc-sensor",
+		.add = vimc_sen_add,
+		.rm = vimc_sen_rm,
 	},
 	{
 		.name = "Sensor B",
 		.drv = "vimc-sensor",
+		.add = vimc_sen_add,
+		.rm = vimc_sen_rm,
 	},
 	{
 		.name = "Debayer A",
 		.drv = "vimc-debayer",
+		.add = vimc_deb_add,
+		.rm = vimc_deb_rm,
 	},
 	{
 		.name = "Debayer B",
 		.drv = "vimc-debayer",
+		.add = vimc_deb_add,
+		.rm = vimc_deb_rm,
 	},
 	{
 		.name = "Raw Capture 0",
 		.drv = "vimc-capture",
+		.add = vimc_cap_add,
+		.rm = vimc_cap_rm,
 	},
 	{
 		.name = "Raw Capture 1",
 		.drv = "vimc-capture",
+		.add = vimc_cap_add,
+		.rm = vimc_cap_rm,
 	},
 	{
 		.name = "RGB/YUV Input",
 		/* TODO: change this to vimc-input when it is implemented */
 		.drv = "vimc-sensor",
+		.add = vimc_sen_add,
+		.rm = vimc_sen_rm,
 	},
 	{
 		.name = "Scaler",
 		.drv = "vimc-scaler",
+		.add = vimc_sca_add,
+		.rm = vimc_sca_rm,
 	},
 	{
 		.name = "RGB/YUV Capture",
 		.drv = "vimc-capture",
+		.add = vimc_cap_add,
+		.rm = vimc_cap_rm,
 	},
 };

drivers/media/platform/vimc/vimc-core.c static const struct vimc_ent_link ent_links[] = { · line 125

Quoted 5 lines

drivers/media/platform/vimc/vimc-core.c +1 -1

@@ -111,7 +122,7 @@ static const struct vimc_ent_link ent_links[] = {
 	VIMC_ENT_LINK(7, 1, 8, 0, MEDIA_LNK_FL_ENABLED | MEDIA_LNK_FL_IMMUTABLE),
 };
 
-static const struct vimc_pipeline_config pipe_cfg = {
+static struct vimc_pipeline_config pipe_cfg = {

Any special reason why the const was dropped here?

Quoted 22 lines

drivers/media/platform/vimc/vimc-core.c +3 -6

 	.ents		= ent_config,
 	.num_ents	= ARRAY_SIZE(ent_config),
 	.links		= ent_links,

@@ -128,14 +139,11 @@ static int vimc_create_links(struct vimc_device *vimc)
 	/* Initialize the links between entities */
 	for (i = 0; i < vimc->pipe_cfg->num_links; i++) {
 		const struct vimc_ent_link *link = &vimc->pipe_cfg->links[i];
-		/*
-		 * TODO: Check another way of retrieving ved struct without
-		 * relying on platform_get_drvdata
-		 */
+
 		struct vimc_ent_device *ved_src =
-			platform_get_drvdata(vimc->subdevs[link->src_ent]);
+			vimc->pipe_cfg->ents[link->src_ent].ved;
 		struct vimc_ent_device *ved_sink =
-			platform_get_drvdata(vimc->subdevs[link->sink_ent]);
+			vimc->pipe_cfg->ents[link->sink_ent].ved;
 
 		ret = media_create_pad_link(ved_src->ent, link->src_pad,
 					    ved_sink->ent, link->sink_pad,

drivers/media/platform/vimc/vimc-core.c static int vimc_create_links(struct vimc_device *vimc) · line 160

Quoted 10 lines

drivers/media/platform/vimc/vimc-core.c +2 -4

@@ -147,13 +155,28 @@ static int vimc_create_links(struct vimc_device *vimc)
 	return 0;
 }
 
-static int vimc_comp_bind(struct device *master)
+static int vimc_add_subdevs(struct vimc_device *vimc)
 {
-	struct vimc_device *vimc = container_of(to_platform_device(master),
-						struct vimc_device, pdev);
-	int ret;
+	int i;

unsigned int

drivers/media/platform/vimc/vimc-core.c static int vimc_create_links(struct vimc_device *vimc) · line 161

Quoted 1 line

drivers/media/platform/vimc/vimc-core.c +1 -0
+	int ret = 0;

no need to init to 0, but not a problem either.

Quoted 558 lines

drivers/media/platform/vimc/vimc-core.c +28 ~~-81~~

+
+	for (i = 0; i < vimc->pipe_cfg->num_ents; i++) {
+		dev_dbg(&vimc->pdev.dev, "new entity for %s\n",
+			vimc->pipe_cfg->ents[i].name);
+		ret = vimc->pipe_cfg->ents[i].add(vimc,
+				&vimc->pipe_cfg->ents[i]);
+		if (ret) {
+			dev_err(&vimc->pdev.dev, "add new entity for %s\n",
+				vimc->pipe_cfg->ents[i].name);
+			return ret;
+		}
+	}
+	return 0;
+}
 
-	dev_dbg(master, "bind");
+static int vimc_register_devices(struct vimc_device *vimc)
+{
+	int ret;
 
 	/* Register the v4l2 struct */
 	ret = v4l2_device_register(vimc->mdev.dev, &vimc->v4l2_dev);

@@ -163,22 +186,20 @@ static int vimc_comp_bind(struct device *master)
 		return ret;
 	}
 
-	/* Bind subdevices */
-	ret = component_bind_all(master, &vimc->v4l2_dev);
-	if (ret)
-		goto err_v4l2_unregister;
+	/* Invoke enitity config hooks to initialize and register subdevs */
+	vimc_add_subdevs(vimc);
 
 	/* Initialize links */
 	ret = vimc_create_links(vimc);
 	if (ret)
-		goto err_comp_unbind_all;
+		goto err_v4l2_unregister;
 
 	/* Register the media device */
 	ret = media_device_register(&vimc->mdev);
 	if (ret) {
 		dev_err(vimc->mdev.dev,
 			"media device register failed (err=%d)\n", ret);
-		goto err_comp_unbind_all;
+		goto err_v4l2_unregister;
 	}
 
 	/* Expose all subdev's nodes*/

@@ -195,98 +216,36 @@ static int vimc_comp_bind(struct device *master)
 err_mdev_unregister:
 	media_device_unregister(&vimc->mdev);
 	media_device_cleanup(&vimc->mdev);
-err_comp_unbind_all:
-	component_unbind_all(master, NULL);
 err_v4l2_unregister:
 	v4l2_device_unregister(&vimc->v4l2_dev);
 
 	return ret;
 }
 
-static void vimc_comp_unbind(struct device *master)
+static void vimc_unregister(struct vimc_device *vimc)
 {
-	struct vimc_device *vimc = container_of(to_platform_device(master),
-						struct vimc_device, pdev);
-
-	dev_dbg(master, "unbind");
-
 	media_device_unregister(&vimc->mdev);
 	media_device_cleanup(&vimc->mdev);
-	component_unbind_all(master, NULL);
 	v4l2_device_unregister(&vimc->v4l2_dev);
 }
 
-static int vimc_comp_compare(struct device *comp, void *data)
-{
-	return comp == data;
-}
-
-static struct component_match *vimc_add_subdevs(struct vimc_device *vimc)
-{
-	struct component_match *match = NULL;
-	struct vimc_platform_data pdata;
-	int i;
-
-	for (i = 0; i < vimc->pipe_cfg->num_ents; i++) {
-		dev_dbg(&vimc->pdev.dev, "new pdev for %s\n",
-			vimc->pipe_cfg->ents[i].drv);
-
-		strscpy(pdata.entity_name, vimc->pipe_cfg->ents[i].name,
-			sizeof(pdata.entity_name));
-
-		vimc->subdevs[i] = platform_device_register_data(&vimc->pdev.dev,
-						vimc->pipe_cfg->ents[i].drv,
-						PLATFORM_DEVID_AUTO,
-						&pdata,
-						sizeof(pdata));
-		if (IS_ERR(vimc->subdevs[i])) {
-			match = ERR_CAST(vimc->subdevs[i]);
-			while (--i >= 0)
-				platform_device_unregister(vimc->subdevs[i]);
-
-			return match;
-		}
-
-		component_match_add(&vimc->pdev.dev, &match, vimc_comp_compare,
-				    &vimc->subdevs[i]->dev);
-	}
-
-	return match;
-}
-
 static void vimc_rm_subdevs(struct vimc_device *vimc)
 {
 	unsigned int i;
 
 	for (i = 0; i < vimc->pipe_cfg->num_ents; i++)
-		platform_device_unregister(vimc->subdevs[i]);
+		vimc->pipe_cfg->ents[i].rm(vimc, &vimc->pipe_cfg->ents[i]);
 }
 
-static const struct component_master_ops vimc_comp_ops = {
-	.bind = vimc_comp_bind,
-	.unbind = vimc_comp_unbind,
-};
-
 static int vimc_probe(struct platform_device *pdev)
 {
 	struct vimc_device *vimc = container_of(pdev, struct vimc_device, pdev);
-	struct component_match *match = NULL;
-	int ret;
+	int ret = 0;
 
 	dev_dbg(&pdev->dev, "probe");
 
 	memset(&vimc->mdev, 0, sizeof(vimc->mdev));
 
-	/* Create platform_device for each entity in the topology*/
-	vimc->subdevs = devm_kcalloc(&vimc->pdev.dev, vimc->pipe_cfg->num_ents,
-				     sizeof(*vimc->subdevs), GFP_KERNEL);
-	if (!vimc->subdevs)
-		return -ENOMEM;
-
-	match = vimc_add_subdevs(vimc);
-	if (IS_ERR(match))
-		return PTR_ERR(match);
-
 	/* Link the media device within the v4l2_device */
 	vimc->v4l2_dev.mdev = &vimc->mdev;

@@ -298,16 +257,11 @@ static int vimc_probe(struct platform_device *pdev)
 	vimc->mdev.dev = &pdev->dev;
 	media_device_init(&vimc->mdev);
 
-	/* Add self to the component system */
-	ret = component_master_add_with_match(&pdev->dev, &vimc_comp_ops,
-					      match);
-	if (ret) {
+	ret = vimc_register_devices(vimc);
+	if (ret)
 		media_device_cleanup(&vimc->mdev);
-		vimc_rm_subdevs(vimc);
-		return ret;
-	}
 
-	return 0;
+	return ret;
 }
 
 static int vimc_remove(struct platform_device *pdev)

@@ -316,8 +270,8 @@ static int vimc_remove(struct platform_device *pdev)
 
 	dev_dbg(&pdev->dev, "remove");
 
-	component_master_del(&pdev->dev, &vimc_comp_ops);
 	vimc_rm_subdevs(vimc);
+	vimc_unregister(vimc);
 
 	return 0;
 }

drivers/media/platform/vimc/vimc-debayer.c +10 ~~-58~~

index 750752bb173c..b58e0946ac2a 100644
@@ -5,10 +5,7 @@
  * Copyright (C) 2015-2017 Helen Koike <[email protected]>
  */
 
-#include <linux/component.h>
 #include <linux/module.h>
-#include <linux/mod_devicetable.h>
-#include <linux/platform_device.h>
 #include <linux/vmalloc.h>
 #include <linux/v4l2-mediabus.h>
 #include <media/v4l2-subdev.h>

@@ -16,9 +13,7 @@
 #include "vimc.h"
 #include "vimc-common.h"
 
-#define VIMC_DEB_DRV_NAME "vimc-debayer"
-/* This module only supports transforming a bayer format
- * to V4L2_PIX_FMT_RGB24
+/* Supports transforming a bayer format to V4L2_PIX_FMT_RGB24
  */
 #define VIMC_DEB_SRC_PIXFMT V4L2_PIX_FMT_RGB24
 #define VIMC_DEB_SRC_MBUS_FMT_DEFAULT MEDIA_BUS_FMT_RGB888_1X24

@@ -499,21 +494,19 @@ static const struct v4l2_subdev_internal_ops vimc_deb_int_ops = {
 	.release = vimc_deb_release,
 };
 
-static void vimc_deb_comp_unbind(struct device *comp, struct device *master,
-				 void *master_data)
+void vimc_deb_rm(struct vimc_device *vimc, struct vimc_ent_config *vent)
 {
-	struct vimc_ent_device *ved = dev_get_drvdata(comp);
-	struct vimc_deb_device *vdeb = container_of(ved, struct vimc_deb_device,
+	struct vimc_ent_device *ved = vent->ved;
+	struct vimc_deb_device *vdeb = container_of(ved,
+						    struct vimc_deb_device,
 						    ved);
 
 	vimc_ent_sd_unregister(ved, &vdeb->sd);
 }
 
-static int vimc_deb_comp_bind(struct device *comp, struct device *master,
-			      void *master_data)
+int vimc_deb_add(struct vimc_device *vimc, struct vimc_ent_config *vent)
 {
-	struct v4l2_device *v4l2_dev = master_data;
-	struct vimc_platform_data *pdata = comp->platform_data;
+	struct v4l2_device *v4l2_dev = &vimc->v4l2_dev;
 	struct vimc_deb_device *vdeb;
 	int ret;

@@ -524,7 +517,7 @@ static int vimc_deb_comp_bind(struct device *comp, struct device *master,
 
 	/* Initialize ved and sd */
 	ret = vimc_ent_sd_register(&vdeb->ved, &vdeb->sd, v4l2_dev,
-				   pdata->entity_name,
+				   vent->name,
 				   MEDIA_ENT_F_PROC_VIDEO_PIXEL_ENC_CONV, 2,
 				   (const unsigned long[2]) {MEDIA_PAD_FL_SINK,
 				   MEDIA_PAD_FL_SOURCE},

@@ -535,8 +528,7 @@ static int vimc_deb_comp_bind(struct device *comp, struct device *master,
 	}
 
 	vdeb->ved.process_frame = vimc_deb_process_frame;
-	dev_set_drvdata(comp, &vdeb->ved);
-	vdeb->dev = comp;
+	vdeb->dev = &vimc->pdev.dev;
 
 	/* Initialize the frame format */
 	vdeb->sink_fmt = sink_fmt_default;

@@ -549,46 +541,6 @@ static int vimc_deb_comp_bind(struct device *comp, struct device *master,
 	 */
 	vdeb->set_rgb_src = vimc_deb_set_rgb_pix_rgb24;
 
+	vent->ved = &vdeb->ved;
 	return 0;
 }
-
-static const struct component_ops vimc_deb_comp_ops = {
-	.bind = vimc_deb_comp_bind,
-	.unbind = vimc_deb_comp_unbind,
-};
-
-static int vimc_deb_probe(struct platform_device *pdev)
-{
-	return component_add(&pdev->dev, &vimc_deb_comp_ops);
-}
-
-static int vimc_deb_remove(struct platform_device *pdev)
-{
-	component_del(&pdev->dev, &vimc_deb_comp_ops);
-
-	return 0;
-}
-
-static const struct platform_device_id vimc_deb_driver_ids[] = {
-	{
-		.name           = VIMC_DEB_DRV_NAME,
-	},
-	{ }
-};
-
-static struct platform_driver vimc_deb_pdrv = {
-	.probe		= vimc_deb_probe,
-	.remove		= vimc_deb_remove,
-	.id_table	= vimc_deb_driver_ids,
-	.driver		= {
-		.name	= VIMC_DEB_DRV_NAME,
-	},
-};
-
-module_platform_driver(vimc_deb_pdrv);
-
-MODULE_DEVICE_TABLE(platform, vimc_deb_driver_ids);
-
-MODULE_DESCRIPTION("Virtual Media Controller Driver (VIMC) Debayer");
-MODULE_AUTHOR("Helen Mae Koike Fornazier <[email protected]>");
-MODULE_LICENSE("GPL");

drivers/media/platform/vimc/vimc-scaler.c +11 ~~-57~~

index fe99b9102ada..7cd478fee3ae 100644
@@ -5,10 +5,7 @@
  * Copyright (C) 2015-2017 Helen Koike <[email protected]>
  */
 
-#include <linux/component.h>
 #include <linux/module.h>
-#include <linux/mod_devicetable.h>
-#include <linux/platform_device.h>
 #include <linux/vmalloc.h>
 #include <linux/v4l2-mediabus.h>
 #include <media/v4l2-subdev.h>

@@ -16,8 +13,6 @@
 #include "vimc.h"
 #include "vimc-common.h"
 
-#define VIMC_SCA_DRV_NAME "vimc-scaler"
-
 static unsigned int sca_mult = 3;
 module_param(sca_mult, uint, 0000);
 MODULE_PARM_DESC(sca_mult, " the image size multiplier");

@@ -331,22 +326,21 @@ static const struct v4l2_subdev_internal_ops vimc_sca_int_ops = {
 	.release = vimc_sca_release,
 };
 
-static void vimc_sca_comp_unbind(struct device *comp, struct device *master,
-				 void *master_data)
+void vimc_sca_rm(struct vimc_device *vimc, struct vimc_ent_config *vent)
 {
-	struct vimc_ent_device *ved = dev_get_drvdata(comp);
-	struct vimc_sca_device *vsca = container_of(ved, struct vimc_sca_device,
+	struct vimc_ent_device *ved = vent->ved;
+	struct vimc_sca_device *vsca = container_of(ved,
+						    struct vimc_sca_device,
 						    ved);
 
+
 	vimc_ent_sd_unregister(ved, &vsca->sd);
 }
 
 
-static int vimc_sca_comp_bind(struct device *comp, struct device *master,
-			      void *master_data)
+int vimc_sca_add(struct vimc_device *vimc, struct vimc_ent_config *vent)
 {
-	struct v4l2_device *v4l2_dev = master_data;
-	struct vimc_platform_data *pdata = comp->platform_data;
+	struct v4l2_device *v4l2_dev = &vimc->v4l2_dev;
 	struct vimc_sca_device *vsca;
 	int ret;

@@ -357,7 +351,7 @@ static int vimc_sca_comp_bind(struct device *comp, struct device *master,
 
 	/* Initialize ved and sd */
 	ret = vimc_ent_sd_register(&vsca->ved, &vsca->sd, v4l2_dev,
-				   pdata->entity_name,
+				   vent->name,
 				   MEDIA_ENT_F_PROC_VIDEO_SCALER, 2,
 				   (const unsigned long[2]) {MEDIA_PAD_FL_SINK,
 				   MEDIA_PAD_FL_SOURCE},

@@ -368,52 +362,12 @@ static int vimc_sca_comp_bind(struct device *comp, struct device *master,
 	}
 
 	vsca->ved.process_frame = vimc_sca_process_frame;
-	dev_set_drvdata(comp, &vsca->ved);
-	vsca->dev = comp;
+	vsca->dev = &vimc->pdev.dev;
+
+	vent->ved = &vsca->ved;
 
 	/* Initialize the frame format */
 	vsca->sink_fmt = sink_fmt_default;
 
 	return 0;
 }
-
-static const struct component_ops vimc_sca_comp_ops = {
-	.bind = vimc_sca_comp_bind,
-	.unbind = vimc_sca_comp_unbind,
-};
-
-static int vimc_sca_probe(struct platform_device *pdev)
-{
-	return component_add(&pdev->dev, &vimc_sca_comp_ops);
-}
-
-static int vimc_sca_remove(struct platform_device *pdev)
-{
-	component_del(&pdev->dev, &vimc_sca_comp_ops);
-
-	return 0;
-}
-
-static const struct platform_device_id vimc_sca_driver_ids[] = {
-	{
-		.name           = VIMC_SCA_DRV_NAME,
-	},
-	{ }
-};
-
-static struct platform_driver vimc_sca_pdrv = {
-	.probe		= vimc_sca_probe,
-	.remove		= vimc_sca_remove,
-	.id_table	= vimc_sca_driver_ids,
-	.driver		= {
-		.name	= VIMC_SCA_DRV_NAME,
-	},
-};
-
-module_platform_driver(vimc_sca_pdrv);
-
-MODULE_DEVICE_TABLE(platform, vimc_sca_driver_ids);
-
-MODULE_DESCRIPTION("Virtual Media Controller Driver (VIMC) Scaler");
-MODULE_AUTHOR("Helen Mae Koike Fornazier <[email protected]>");
-MODULE_LICENSE("GPL");

drivers/media/platform/vimc/vimc-sensor.c +10 ~~-59~~

index 6c57b1e262f9..bd237e030a1b 100644
@@ -5,10 +5,6 @@
  * Copyright (C) 2015-2017 Helen Koike <[email protected]>
  */
 
-#include <linux/component.h>
-#include <linux/module.h>
-#include <linux/mod_devicetable.h>
-#include <linux/platform_device.h>
 #include <linux/v4l2-mediabus.h>
 #include <linux/vmalloc.h>
 #include <media/v4l2-ctrls.h>

@@ -19,8 +15,6 @@
 #include "vimc.h"
 #include "vimc-common.h"
 
-#define VIMC_SEN_DRV_NAME "vimc-sensor"
-
 static const struct v4l2_mbus_framefmt fmt_default = {
 	.width = 640,
 	.height = 480,

@@ -268,12 +262,12 @@ static const struct v4l2_subdev_internal_ops vimc_sen_int_ops = {
 	.release = vimc_sen_release,
 };
 
-static void vimc_sen_comp_unbind(struct device *comp, struct device *master,
-				 void *master_data)
+void vimc_sen_rm(struct vimc_device *vimc, struct vimc_ent_config *vent)
 {
-	struct vimc_ent_device *ved = dev_get_drvdata(comp);
-	struct vimc_sen_device *vsen =
-				container_of(ved, struct vimc_sen_device, ved);
+	struct vimc_ent_device *ved = vent->ved;
+	struct vimc_sen_device *vsen = container_of(ved,
+						    struct vimc_sen_device,
+						    ved);
 
 	vimc_ent_sd_unregister(ved, &vsen->sd);
 }

@@ -295,11 +289,9 @@ static const struct v4l2_ctrl_config vimc_sen_ctrl_test_pattern = {
 	.qmenu = tpg_pattern_strings,
 };
 
-static int vimc_sen_comp_bind(struct device *comp, struct device *master,
-			      void *master_data)
+int vimc_sen_add(struct vimc_device *vimc, struct vimc_ent_config *vent)
 {
-	struct v4l2_device *v4l2_dev = master_data;
-	struct vimc_platform_data *pdata = comp->platform_data;
+	struct v4l2_device *v4l2_dev = &vimc->v4l2_dev;
 	struct vimc_sen_device *vsen;
 	int ret;

@@ -332,7 +324,7 @@ static int vimc_sen_comp_bind(struct device *comp, struct device *master,
 
 	/* Initialize ved and sd */
 	ret = vimc_ent_sd_register(&vsen->ved, &vsen->sd, v4l2_dev,
-				   pdata->entity_name,
+				   vent->name,
 				   MEDIA_ENT_F_CAM_SENSOR, 1,
 				   (const unsigned long[1]) {MEDIA_PAD_FL_SOURCE},
 				   &vimc_sen_int_ops, &vimc_sen_ops);

@@ -340,8 +332,7 @@ static int vimc_sen_comp_bind(struct device *comp, struct device *master,
 		goto err_free_hdl;
 
 	vsen->ved.process_frame = vimc_sen_process_frame;
-	dev_set_drvdata(comp, &vsen->ved);
-	vsen->dev = comp;
+	vsen->dev = &vimc->pdev.dev;
 
 	/* Initialize the frame format */
 	vsen->mbus_format = fmt_default;

@@ -353,6 +344,7 @@ static int vimc_sen_comp_bind(struct device *comp, struct device *master,
 	if (ret)
 		goto err_unregister_ent_sd;
 
+	vent->ved = &vsen->ved;
 	return 0;
 
 err_unregister_ent_sd:

@@ -364,44 +356,3 @@ static int vimc_sen_comp_bind(struct device *comp, struct device *master,
 
 	return ret;
 }
-
-static const struct component_ops vimc_sen_comp_ops = {
-	.bind = vimc_sen_comp_bind,
-	.unbind = vimc_sen_comp_unbind,
-};
-
-static int vimc_sen_probe(struct platform_device *pdev)
-{
-	return component_add(&pdev->dev, &vimc_sen_comp_ops);
-}
-
-static int vimc_sen_remove(struct platform_device *pdev)
-{
-	component_del(&pdev->dev, &vimc_sen_comp_ops);
-
-	return 0;
-}
-
-static const struct platform_device_id vimc_sen_driver_ids[] = {
-	{
-		.name           = VIMC_SEN_DRV_NAME,
-	},
-	{ }
-};
-
-static struct platform_driver vimc_sen_pdrv = {
-	.probe		= vimc_sen_probe,
-	.remove		= vimc_sen_remove,
-	.id_table	= vimc_sen_driver_ids,
-	.driver		= {
-		.name	= VIMC_SEN_DRV_NAME,
-	},
-};
-
-module_platform_driver(vimc_sen_pdrv);
-
-MODULE_DEVICE_TABLE(platform, vimc_sen_driver_ids);
-
-MODULE_DESCRIPTION("Virtual Media Controller Driver (VIMC) Sensor");
-MODULE_AUTHOR("Helen Mae Koike Fornazier <[email protected]>");
-MODULE_LICENSE("GPL");

drivers/media/platform/vimc/vimc.h +0 -1

index a5adebdda941..cd527c6fe6fc 100644
@@ -92,11 +92,30 @@ struct vimc_device {
 	/* The Associated media_device parent */
 	struct media_device mdev;
 
-	/* Internal v4l2 parent device*/

drivers/media/platform/vimc/vimc.h struct vimc_device { · line 102

Quoted 10 lines

drivers/media/platform/vimc/vimc.h +6 -2

+	/* Internal v4l2 parent device */
 	struct v4l2_device v4l2_dev;
+};
 
-	/* Subdevices */
-	struct platform_device **subdevs;
+/* Structure which describes individual configuration for each entity */
+struct vimc_ent_config {
+	const char *name;
+	const char *drv;

I was tempted in suggesting to remove this field as it is not being used anymore, but it will be usefull for the configfs patch.

Thanks
Helen

Quoted 20 lines

drivers/media/platform/vimc/vimc.h +16 -0

+	struct vimc_ent_device *ved;
+	int (*add)(struct vimc_device *vimc, struct vimc_ent_config *vent);
+	void (*rm)(struct vimc_device *vimc, struct vimc_ent_config *vent);
 };
 
+/* prototypes for vimc_ent_config add and rm hooks */
+int vimc_cap_add(struct vimc_device *vimc, struct vimc_ent_config *vent);
+void vimc_cap_rm(struct vimc_device *vimc, struct vimc_ent_config *vent);
+
+int vimc_deb_add(struct vimc_device *vimc, struct vimc_ent_config *vent);
+void vimc_deb_rm(struct vimc_device *vimc, struct vimc_ent_config *vent);
+
+int vimc_sca_add(struct vimc_device *vimc, struct vimc_ent_config *vent);
+void vimc_sca_rm(struct vimc_device *vimc, struct vimc_ent_config *vent);
+
+int vimc_sen_add(struct vimc_device *vimc, struct vimc_ent_config *vent);
+void vimc_sen_rm(struct vimc_device *vimc, struct vimc_ent_config *vent);
+
 #endif

Shuah Khan

author

2019-08-12 14:12 UTC #10 lore raw

Re: [PATCH 2/3] media: vimc: Collapse component structure into a single monolithic driver

Quoted 327 lines · Helen Koike

Hi Shuah,

Thanks for the patch, just some small comments.

Quoted 322 lines · Shuah Khan

The following configure and stream test works on all devices.

v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video1 v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video2 v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video3

Signed-off-by Shuah Khan <[email protected]>

---
  drivers/media/platform/vimc/Makefile       |   7 +-
  drivers/media/platform/vimc/vimc-capture.c |  75 ++--------
  drivers/media/platform/vimc/vimc-core.c    | 156 ++++++++-------------
  drivers/media/platform/vimc/vimc-debayer.c |  68 ++-------
  drivers/media/platform/vimc/vimc-scaler.c  |  68 ++-------
  drivers/media/platform/vimc/vimc-sensor.c  |  69 ++-------
  drivers/media/platform/vimc/vimc.h         |  25 +++-
  7 files changed, 125 insertions(+), 343 deletions(-)

drivers/media/platform/vimc/Makefile +4 -3

index 96d06f030c31..a53b2b532e9f 100644
@@ -1,5 +1,6 @@
  # SPDX-License-Identifier: GPL-2.0
-vimc-y := vimc-core.o vimc-common.o vimc-streamer.o
+vimc-y := vimc-core.o vimc-common.o vimc-streamer.o vimc-capture.o \
+		vimc-debayer.o vimc-scaler.o vimc-sensor.o
+
+obj-$(CONFIG_VIDEO_VIMC) += vimc.o
 
-obj-$(CONFIG_VIDEO_VIMC) += vimc.o vimc-capture.o vimc-debayer.o \
-                vimc-scaler.o vimc-sensor.o

drivers/media/platform/vimc/vimc-capture.c +13 ~~-62~~

index c52fc5d97c2d..b7b2d3c3d4f8 100644
@@ -5,10 +5,6 @@
   * Copyright (C) 2015-2017 Helen Koike <[email protected]>
   */
 
-#include <linux/component.h>
-#include <linux/module.h>
-#include <linux/mod_devicetable.h>
-#include <linux/platform_device.h>
  #include <media/v4l2-ioctl.h>
  #include <media/videobuf2-core.h>
  #include <media/videobuf2-vmalloc.h>

@@ -17,8 +13,6 @@
  #include "vimc-common.h"
  #include "vimc-streamer.h"
 
-#define VIMC_CAP_DRV_NAME "vimc-capture"
-
  static const u32 vimc_cap_supported_pixfmt[] = {
  	V4L2_PIX_FMT_BGR24,
  	V4L2_PIX_FMT_RGB24,

@@ -348,11 +342,11 @@ static void vimc_cap_release(struct video_device *vdev)
  	kfree(vcap);
  }
 
-static void vimc_cap_comp_unbind(struct device *comp, struct device *master,
-				 void *master_data)
+void vimc_cap_rm(struct vimc_device *vimc, struct vimc_ent_config *vent)
  {
-	struct vimc_ent_device *ved = dev_get_drvdata(comp);
-	struct vimc_cap_device *vcap = container_of(ved, struct vimc_cap_device,
+	struct vimc_ent_device *ved = vent->ved;
+	struct vimc_cap_device *vcap = container_of(ved,
+						    struct vimc_cap_device,
  						    ved);
 
  	vb2_queue_release(&vcap->queue);

@@ -399,11 +393,9 @@ static void *vimc_cap_process_frame(struct vimc_ent_device *ved,
  	return NULL;
  }
 
-static int vimc_cap_comp_bind(struct device *comp, struct device *master,
-			      void *master_data)
+int vimc_cap_add(struct vimc_device *vimc, struct vimc_ent_config *vent)
  {
-	struct v4l2_device *v4l2_dev = master_data;
-	struct vimc_platform_data *pdata = comp->platform_data;
+	struct v4l2_device *v4l2_dev = &vimc->v4l2_dev;
  	struct vimc_cap_device *vcap;
  	struct video_device *vdev;
  	struct vb2_queue *q;

@@ -423,7 +415,7 @@ static int vimc_cap_comp_bind(struct device *comp, struct device *master,
  	}
 
  	/* Initialize the media entity */
-	vcap->vdev.entity.name = pdata->entity_name;
+	vcap->vdev.entity.name = vent->name;
  	vcap->vdev.entity.function = MEDIA_ENT_F_IO_V4L;
  	ret = media_entity_pads_init(&vcap->vdev.entity,
  				     1, vcap->ved.pads);

@@ -447,8 +439,8 @@ static int vimc_cap_comp_bind(struct device *comp, struct device *master,
 
  	ret = vb2_queue_init(q);
  	if (ret) {
-		dev_err(comp, "%s: vb2 queue init failed (err=%d)\n",
-			pdata->entity_name, ret);
+		dev_err(&vimc->pdev.dev, "%s: vb2 queue init failed (err=%d)\n",
+			vent->name, ret);
  		goto err_clean_m_ent;
  	}

@@ -465,8 +457,7 @@ static int vimc_cap_comp_bind(struct device *comp, struct device *master,
  	vcap->ved.ent = &vcap->vdev.entity;
  	vcap->ved.process_frame = vimc_cap_process_frame;
  	vcap->ved.vdev_get_format = vimc_cap_get_format;
-	dev_set_drvdata(comp, &vcap->ved);
-	vcap->dev = comp;
+	vcap->dev = &vimc->pdev.dev;
 
  	/* Initialize the video_device struct */
  	vdev = &vcap->vdev;

@@ -479,17 +470,18 @@ static int vimc_cap_comp_bind(struct device *comp, struct device *master,
  	vdev->queue = q;
  	vdev->v4l2_dev = v4l2_dev;
  	vdev->vfl_dir = VFL_DIR_RX;
-	strscpy(vdev->name, pdata->entity_name, sizeof(vdev->name));
+	strscpy(vdev->name, vent->name, sizeof(vdev->name));
  	video_set_drvdata(vdev, &vcap->ved);
 
  	/* Register the video_device with the v4l2 and the media framework */
  	ret = video_register_device(vdev, VFL_TYPE_GRABBER, -1);
  	if (ret) {
-		dev_err(comp, "%s: video register failed (err=%d)\n",
+		dev_err(&vimc->pdev.dev, "%s: video register failed (err=%d)\n",
  			vcap->vdev.name, ret);
  		goto err_release_queue;
  	}
 
+	vent->ved = &vcap->ved;
  	return 0;
 
  err_release_queue:

@@ -503,44 +495,3 @@ static int vimc_cap_comp_bind(struct device *comp, struct device *master,
 
  	return ret;
  }
-
-static const struct component_ops vimc_cap_comp_ops = {
-	.bind = vimc_cap_comp_bind,
-	.unbind = vimc_cap_comp_unbind,
-};
-
-static int vimc_cap_probe(struct platform_device *pdev)
-{
-	return component_add(&pdev->dev, &vimc_cap_comp_ops);
-}
-
-static int vimc_cap_remove(struct platform_device *pdev)
-{
-	component_del(&pdev->dev, &vimc_cap_comp_ops);
-
-	return 0;
-}
-
-static const struct platform_device_id vimc_cap_driver_ids[] = {
-	{
-		.name           = VIMC_CAP_DRV_NAME,
-	},
-	{ }
-};
-
-static struct platform_driver vimc_cap_pdrv = {
-	.probe		= vimc_cap_probe,
-	.remove		= vimc_cap_remove,
-	.id_table	= vimc_cap_driver_ids,
-	.driver		= {
-		.name	= VIMC_CAP_DRV_NAME,
-	},
-};
-
-module_platform_driver(vimc_cap_pdrv);
-
-MODULE_DEVICE_TABLE(platform, vimc_cap_driver_ids);
-
-MODULE_DESCRIPTION("Virtual Media Controller Driver (VIMC) Capture");
-MODULE_AUTHOR("Helen Mae Koike Fornazier <[email protected]>");
-MODULE_LICENSE("GPL");

drivers/media/platform/vimc/vimc-core.c +20 -9

index c9b351472272..3bd89cf38a90 100644
@@ -5,7 +5,6 @@
   * Copyright (C) 2015-2017 Helen Koike <[email protected]>
   */
 
-#include <linux/component.h>
  #include <linux/init.h>
  #include <linux/module.h>
  #include <linux/platform_device.h>

@@ -25,12 +24,6 @@
  	.flags = link_flags,					\
  }
 
-/* Structure which describes individual configuration for each entity */
-struct vimc_ent_config {
-	const char *name;
-	const char *drv;
-};
-
  /* Structure which describes links between entities */
  struct vimc_ent_link {
  	unsigned int src_ent;

@@ -42,7 +35,7 @@ struct vimc_ent_link {
 
  /* Structure which describes the whole topology */
  struct vimc_pipeline_config {
-	const struct vimc_ent_config *ents;
+	struct vimc_ent_config *ents;
  	size_t num_ents;
  	const struct vimc_ent_link *links;
  	size_t num_links;

@@ -52,43 +45,61 @@ struct vimc_pipeline_config {
   * Topology Configuration
   */
 
-static const struct vimc_ent_config ent_config[] = {
+static struct vimc_ent_config ent_config[] = {
  	{
  		.name = "Sensor A",
  		.drv = "vimc-sensor",
+		.add = vimc_sen_add,
+		.rm = vimc_sen_rm,
  	},
  	{
  		.name = "Sensor B",
  		.drv = "vimc-sensor",
+		.add = vimc_sen_add,
+		.rm = vimc_sen_rm,
  	},
  	{
  		.name = "Debayer A",
  		.drv = "vimc-debayer",
+		.add = vimc_deb_add,
+		.rm = vimc_deb_rm,
  	},
  	{
  		.name = "Debayer B",
  		.drv = "vimc-debayer",
+		.add = vimc_deb_add,
+		.rm = vimc_deb_rm,
  	},
  	{
  		.name = "Raw Capture 0",
  		.drv = "vimc-capture",
+		.add = vimc_cap_add,
+		.rm = vimc_cap_rm,
  	},
  	{
  		.name = "Raw Capture 1",
  		.drv = "vimc-capture",
+		.add = vimc_cap_add,
+		.rm = vimc_cap_rm,
  	},
  	{
  		.name = "RGB/YUV Input",
  		/* TODO: change this to vimc-input when it is implemented */
  		.drv = "vimc-sensor",
+		.add = vimc_sen_add,
+		.rm = vimc_sen_rm,
  	},
  	{
  		.name = "Scaler",
  		.drv = "vimc-scaler",
+		.add = vimc_sca_add,
+		.rm = vimc_sca_rm,
  	},
  	{
  		.name = "RGB/YUV Capture",
  		.drv = "vimc-capture",
+		.add = vimc_cap_add,
+		.rm = vimc_cap_rm,
  	},
  };

Quoted 9 lines

drivers/media/platform/vimc/vimc-core.c static const struct vimc_ent_link ent_links[] = { · line 125
Quoted 6 lines
drivers/media/platform/vimc/vimc-core.c +1 -1
@@ -111,7 +122,7 @@ static const struct vimc_ent_link ent_links[] = {
  	VIMC_ENT_LINK(7, 1, 8, 0, MEDIA_LNK_FL_ENABLED | MEDIA_LNK_FL_IMMUTABLE),
  };
 
-static const struct vimc_pipeline_config pipe_cfg = {
+static struct vimc_pipeline_config pipe_cfg = {
Any special reason why the const was dropped here?

Since this gets passed in .add and .rm hooks to be updated by subdevs to save their struct vimc_ent_device for use by vimc-core, this needs to writable. Without removing const, we will see warns.

Quoted 601 lines

Quoted 22 lines

drivers/media/platform/vimc/vimc-core.c +3 -6

  	.ents		= ent_config,
  	.num_ents	= ARRAY_SIZE(ent_config),
  	.links		= ent_links,

@@ -128,14 +139,11 @@ static int vimc_create_links(struct vimc_device *vimc)
  	/* Initialize the links between entities */
  	for (i = 0; i < vimc->pipe_cfg->num_links; i++) {
  		const struct vimc_ent_link *link = &vimc->pipe_cfg->links[i];
-		/*
-		 * TODO: Check another way of retrieving ved struct without
-		 * relying on platform_get_drvdata
-		 */
+
  		struct vimc_ent_device *ved_src =
-			platform_get_drvdata(vimc->subdevs[link->src_ent]);
+			vimc->pipe_cfg->ents[link->src_ent].ved;
  		struct vimc_ent_device *ved_sink =
-			platform_get_drvdata(vimc->subdevs[link->sink_ent]);
+			vimc->pipe_cfg->ents[link->sink_ent].ved;
 
  		ret = media_create_pad_link(ved_src->ent, link->src_pad,
  					    ved_sink->ent, link->sink_pad,

drivers/media/platform/vimc/vimc-core.c static int vimc_create_links(struct vimc_device *vimc) · line 160

Quoted 10 lines

drivers/media/platform/vimc/vimc-core.c +2 -4

@@ -147,13 +155,28 @@ static int vimc_create_links(struct vimc_device *vimc)
  	return 0;
  }
 
-static int vimc_comp_bind(struct device *master)
+static int vimc_add_subdevs(struct vimc_device *vimc)
  {
-	struct vimc_device *vimc = container_of(to_platform_device(master),
-						struct vimc_device, pdev);
-	int ret;
+	int i;

unsigned int

drivers/media/platform/vimc/vimc-core.c static int vimc_create_links(struct vimc_device *vimc) · line 161

Quoted 1 line

drivers/media/platform/vimc/vimc-core.c +1 -0
+	int ret = 0;

no need to init to 0, but not a problem either.

Quoted 562 lines

drivers/media/platform/vimc/vimc-core.c +28 ~~-81~~

+
+	for (i = 0; i < vimc->pipe_cfg->num_ents; i++) {
+		dev_dbg(&vimc->pdev.dev, "new entity for %s\n",
+			vimc->pipe_cfg->ents[i].name);
+		ret = vimc->pipe_cfg->ents[i].add(vimc,
+				&vimc->pipe_cfg->ents[i]);
+		if (ret) {
+			dev_err(&vimc->pdev.dev, "add new entity for %s\n",
+				vimc->pipe_cfg->ents[i].name);
+			return ret;
+		}
+	}
+	return 0;
+}
 
-	dev_dbg(master, "bind");
+static int vimc_register_devices(struct vimc_device *vimc)
+{
+	int ret;
 
  	/* Register the v4l2 struct */
  	ret = v4l2_device_register(vimc->mdev.dev, &vimc->v4l2_dev);

@@ -163,22 +186,20 @@ static int vimc_comp_bind(struct device *master)
  		return ret;
  	}
 
-	/* Bind subdevices */
-	ret = component_bind_all(master, &vimc->v4l2_dev);
-	if (ret)
-		goto err_v4l2_unregister;
+	/* Invoke enitity config hooks to initialize and register subdevs */
+	vimc_add_subdevs(vimc);
 
  	/* Initialize links */
  	ret = vimc_create_links(vimc);
  	if (ret)
-		goto err_comp_unbind_all;
+		goto err_v4l2_unregister;
 
  	/* Register the media device */
  	ret = media_device_register(&vimc->mdev);
  	if (ret) {
  		dev_err(vimc->mdev.dev,
  			"media device register failed (err=%d)\n", ret);
-		goto err_comp_unbind_all;
+		goto err_v4l2_unregister;
  	}
 
  	/* Expose all subdev's nodes*/

@@ -195,98 +216,36 @@ static int vimc_comp_bind(struct device *master)
  err_mdev_unregister:
  	media_device_unregister(&vimc->mdev);
  	media_device_cleanup(&vimc->mdev);
-err_comp_unbind_all:
-	component_unbind_all(master, NULL);
  err_v4l2_unregister:
  	v4l2_device_unregister(&vimc->v4l2_dev);
 
  	return ret;
  }
 
-static void vimc_comp_unbind(struct device *master)
+static void vimc_unregister(struct vimc_device *vimc)
  {
-	struct vimc_device *vimc = container_of(to_platform_device(master),
-						struct vimc_device, pdev);
-
-	dev_dbg(master, "unbind");
-
  	media_device_unregister(&vimc->mdev);
  	media_device_cleanup(&vimc->mdev);
-	component_unbind_all(master, NULL);
  	v4l2_device_unregister(&vimc->v4l2_dev);
  }
 
-static int vimc_comp_compare(struct device *comp, void *data)
-{
-	return comp == data;
-}
-
-static struct component_match *vimc_add_subdevs(struct vimc_device *vimc)
-{
-	struct component_match *match = NULL;
-	struct vimc_platform_data pdata;
-	int i;
-
-	for (i = 0; i < vimc->pipe_cfg->num_ents; i++) {
-		dev_dbg(&vimc->pdev.dev, "new pdev for %s\n",
-			vimc->pipe_cfg->ents[i].drv);
-
-		strscpy(pdata.entity_name, vimc->pipe_cfg->ents[i].name,
-			sizeof(pdata.entity_name));
-
-		vimc->subdevs[i] = platform_device_register_data(&vimc->pdev.dev,
-						vimc->pipe_cfg->ents[i].drv,
-						PLATFORM_DEVID_AUTO,
-						&pdata,
-						sizeof(pdata));
-		if (IS_ERR(vimc->subdevs[i])) {
-			match = ERR_CAST(vimc->subdevs[i]);
-			while (--i >= 0)
-				platform_device_unregister(vimc->subdevs[i]);
-
-			return match;
-		}
-
-		component_match_add(&vimc->pdev.dev, &match, vimc_comp_compare,
-				    &vimc->subdevs[i]->dev);
-	}
-
-	return match;
-}
-
  static void vimc_rm_subdevs(struct vimc_device *vimc)
  {
  	unsigned int i;
 
  	for (i = 0; i < vimc->pipe_cfg->num_ents; i++)
-		platform_device_unregister(vimc->subdevs[i]);
+		vimc->pipe_cfg->ents[i].rm(vimc, &vimc->pipe_cfg->ents[i]);
  }
 
-static const struct component_master_ops vimc_comp_ops = {
-	.bind = vimc_comp_bind,
-	.unbind = vimc_comp_unbind,
-};
-
  static int vimc_probe(struct platform_device *pdev)
  {
  	struct vimc_device *vimc = container_of(pdev, struct vimc_device, pdev);
-	struct component_match *match = NULL;
-	int ret;
+	int ret = 0;
 
  	dev_dbg(&pdev->dev, "probe");
 
  	memset(&vimc->mdev, 0, sizeof(vimc->mdev));
 
-	/* Create platform_device for each entity in the topology*/
-	vimc->subdevs = devm_kcalloc(&vimc->pdev.dev, vimc->pipe_cfg->num_ents,
-				     sizeof(*vimc->subdevs), GFP_KERNEL);
-	if (!vimc->subdevs)
-		return -ENOMEM;
-
-	match = vimc_add_subdevs(vimc);
-	if (IS_ERR(match))
-		return PTR_ERR(match);
-
  	/* Link the media device within the v4l2_device */
  	vimc->v4l2_dev.mdev = &vimc->mdev;

@@ -298,16 +257,11 @@ static int vimc_probe(struct platform_device *pdev)
  	vimc->mdev.dev = &pdev->dev;
  	media_device_init(&vimc->mdev);
 
-	/* Add self to the component system */
-	ret = component_master_add_with_match(&pdev->dev, &vimc_comp_ops,
-					      match);
-	if (ret) {
+	ret = vimc_register_devices(vimc);
+	if (ret)
  		media_device_cleanup(&vimc->mdev);
-		vimc_rm_subdevs(vimc);
-		return ret;
-	}
 
-	return 0;
+	return ret;
  }
 
  static int vimc_remove(struct platform_device *pdev)

@@ -316,8 +270,8 @@ static int vimc_remove(struct platform_device *pdev)
 
  	dev_dbg(&pdev->dev, "remove");
 
-	component_master_del(&pdev->dev, &vimc_comp_ops);
  	vimc_rm_subdevs(vimc);
+	vimc_unregister(vimc);
 
  	return 0;
  }

drivers/media/platform/vimc/vimc-debayer.c +10 ~~-58~~

index 750752bb173c..b58e0946ac2a 100644
@@ -5,10 +5,7 @@
   * Copyright (C) 2015-2017 Helen Koike <[email protected]>
   */
 
-#include <linux/component.h>
  #include <linux/module.h>
-#include <linux/mod_devicetable.h>
-#include <linux/platform_device.h>
  #include <linux/vmalloc.h>
  #include <linux/v4l2-mediabus.h>
  #include <media/v4l2-subdev.h>

@@ -16,9 +13,7 @@
  #include "vimc.h"
  #include "vimc-common.h"
 
-#define VIMC_DEB_DRV_NAME "vimc-debayer"
-/* This module only supports transforming a bayer format
- * to V4L2_PIX_FMT_RGB24
+/* Supports transforming a bayer format to V4L2_PIX_FMT_RGB24
   */
  #define VIMC_DEB_SRC_PIXFMT V4L2_PIX_FMT_RGB24
  #define VIMC_DEB_SRC_MBUS_FMT_DEFAULT MEDIA_BUS_FMT_RGB888_1X24

@@ -499,21 +494,19 @@ static const struct v4l2_subdev_internal_ops vimc_deb_int_ops = {
  	.release = vimc_deb_release,
  };
 
-static void vimc_deb_comp_unbind(struct device *comp, struct device *master,
-				 void *master_data)
+void vimc_deb_rm(struct vimc_device *vimc, struct vimc_ent_config *vent)
  {
-	struct vimc_ent_device *ved = dev_get_drvdata(comp);
-	struct vimc_deb_device *vdeb = container_of(ved, struct vimc_deb_device,
+	struct vimc_ent_device *ved = vent->ved;
+	struct vimc_deb_device *vdeb = container_of(ved,
+						    struct vimc_deb_device,
  						    ved);
 
  	vimc_ent_sd_unregister(ved, &vdeb->sd);
  }
 
-static int vimc_deb_comp_bind(struct device *comp, struct device *master,
-			      void *master_data)
+int vimc_deb_add(struct vimc_device *vimc, struct vimc_ent_config *vent)
  {
-	struct v4l2_device *v4l2_dev = master_data;
-	struct vimc_platform_data *pdata = comp->platform_data;
+	struct v4l2_device *v4l2_dev = &vimc->v4l2_dev;
  	struct vimc_deb_device *vdeb;
  	int ret;

@@ -524,7 +517,7 @@ static int vimc_deb_comp_bind(struct device *comp, struct device *master,
 
  	/* Initialize ved and sd */
  	ret = vimc_ent_sd_register(&vdeb->ved, &vdeb->sd, v4l2_dev,
-				   pdata->entity_name,
+				   vent->name,
  				   MEDIA_ENT_F_PROC_VIDEO_PIXEL_ENC_CONV, 2,
  				   (const unsigned long[2]) {MEDIA_PAD_FL_SINK,
  				   MEDIA_PAD_FL_SOURCE},

@@ -535,8 +528,7 @@ static int vimc_deb_comp_bind(struct device *comp, struct device *master,
  	}
 
  	vdeb->ved.process_frame = vimc_deb_process_frame;
-	dev_set_drvdata(comp, &vdeb->ved);
-	vdeb->dev = comp;
+	vdeb->dev = &vimc->pdev.dev;
 
  	/* Initialize the frame format */
  	vdeb->sink_fmt = sink_fmt_default;

@@ -549,46 +541,6 @@ static int vimc_deb_comp_bind(struct device *comp, struct device *master,
  	 */
  	vdeb->set_rgb_src = vimc_deb_set_rgb_pix_rgb24;
 
+	vent->ved = &vdeb->ved;
  	return 0;
  }
-
-static const struct component_ops vimc_deb_comp_ops = {
-	.bind = vimc_deb_comp_bind,
-	.unbind = vimc_deb_comp_unbind,
-};
-
-static int vimc_deb_probe(struct platform_device *pdev)
-{
-	return component_add(&pdev->dev, &vimc_deb_comp_ops);
-}
-
-static int vimc_deb_remove(struct platform_device *pdev)
-{
-	component_del(&pdev->dev, &vimc_deb_comp_ops);
-
-	return 0;
-}
-
-static const struct platform_device_id vimc_deb_driver_ids[] = {
-	{
-		.name           = VIMC_DEB_DRV_NAME,
-	},
-	{ }
-};
-
-static struct platform_driver vimc_deb_pdrv = {
-	.probe		= vimc_deb_probe,
-	.remove		= vimc_deb_remove,
-	.id_table	= vimc_deb_driver_ids,
-	.driver		= {
-		.name	= VIMC_DEB_DRV_NAME,
-	},
-};
-
-module_platform_driver(vimc_deb_pdrv);
-
-MODULE_DEVICE_TABLE(platform, vimc_deb_driver_ids);
-
-MODULE_DESCRIPTION("Virtual Media Controller Driver (VIMC) Debayer");
-MODULE_AUTHOR("Helen Mae Koike Fornazier <[email protected]>");
-MODULE_LICENSE("GPL");

drivers/media/platform/vimc/vimc-scaler.c +11 ~~-57~~

index fe99b9102ada..7cd478fee3ae 100644
@@ -5,10 +5,7 @@
   * Copyright (C) 2015-2017 Helen Koike <[email protected]>
   */
 
-#include <linux/component.h>
  #include <linux/module.h>
-#include <linux/mod_devicetable.h>
-#include <linux/platform_device.h>
  #include <linux/vmalloc.h>
  #include <linux/v4l2-mediabus.h>
  #include <media/v4l2-subdev.h>

@@ -16,8 +13,6 @@
  #include "vimc.h"
  #include "vimc-common.h"
 
-#define VIMC_SCA_DRV_NAME "vimc-scaler"
-
  static unsigned int sca_mult = 3;
  module_param(sca_mult, uint, 0000);
  MODULE_PARM_DESC(sca_mult, " the image size multiplier");

@@ -331,22 +326,21 @@ static const struct v4l2_subdev_internal_ops vimc_sca_int_ops = {
  	.release = vimc_sca_release,
  };
 
-static void vimc_sca_comp_unbind(struct device *comp, struct device *master,
-				 void *master_data)
+void vimc_sca_rm(struct vimc_device *vimc, struct vimc_ent_config *vent)
  {
-	struct vimc_ent_device *ved = dev_get_drvdata(comp);
-	struct vimc_sca_device *vsca = container_of(ved, struct vimc_sca_device,
+	struct vimc_ent_device *ved = vent->ved;
+	struct vimc_sca_device *vsca = container_of(ved,
+						    struct vimc_sca_device,
  						    ved);
 
+
  	vimc_ent_sd_unregister(ved, &vsca->sd);
  }
 
 
-static int vimc_sca_comp_bind(struct device *comp, struct device *master,
-			      void *master_data)
+int vimc_sca_add(struct vimc_device *vimc, struct vimc_ent_config *vent)
  {
-	struct v4l2_device *v4l2_dev = master_data;
-	struct vimc_platform_data *pdata = comp->platform_data;
+	struct v4l2_device *v4l2_dev = &vimc->v4l2_dev;
  	struct vimc_sca_device *vsca;
  	int ret;

@@ -357,7 +351,7 @@ static int vimc_sca_comp_bind(struct device *comp, struct device *master,
 
  	/* Initialize ved and sd */
  	ret = vimc_ent_sd_register(&vsca->ved, &vsca->sd, v4l2_dev,
-				   pdata->entity_name,
+				   vent->name,
  				   MEDIA_ENT_F_PROC_VIDEO_SCALER, 2,
  				   (const unsigned long[2]) {MEDIA_PAD_FL_SINK,
  				   MEDIA_PAD_FL_SOURCE},

@@ -368,52 +362,12 @@ static int vimc_sca_comp_bind(struct device *comp, struct device *master,
  	}
 
  	vsca->ved.process_frame = vimc_sca_process_frame;
-	dev_set_drvdata(comp, &vsca->ved);
-	vsca->dev = comp;
+	vsca->dev = &vimc->pdev.dev;
+
+	vent->ved = &vsca->ved;
 
  	/* Initialize the frame format */
  	vsca->sink_fmt = sink_fmt_default;
 
  	return 0;
  }
-
-static const struct component_ops vimc_sca_comp_ops = {
-	.bind = vimc_sca_comp_bind,
-	.unbind = vimc_sca_comp_unbind,
-};
-
-static int vimc_sca_probe(struct platform_device *pdev)
-{
-	return component_add(&pdev->dev, &vimc_sca_comp_ops);
-}
-
-static int vimc_sca_remove(struct platform_device *pdev)
-{
-	component_del(&pdev->dev, &vimc_sca_comp_ops);
-
-	return 0;
-}
-
-static const struct platform_device_id vimc_sca_driver_ids[] = {
-	{
-		.name           = VIMC_SCA_DRV_NAME,
-	},
-	{ }
-};
-
-static struct platform_driver vimc_sca_pdrv = {
-	.probe		= vimc_sca_probe,
-	.remove		= vimc_sca_remove,
-	.id_table	= vimc_sca_driver_ids,
-	.driver		= {
-		.name	= VIMC_SCA_DRV_NAME,
-	},
-};
-
-module_platform_driver(vimc_sca_pdrv);
-
-MODULE_DEVICE_TABLE(platform, vimc_sca_driver_ids);
-
-MODULE_DESCRIPTION("Virtual Media Controller Driver (VIMC) Scaler");
-MODULE_AUTHOR("Helen Mae Koike Fornazier <[email protected]>");
-MODULE_LICENSE("GPL");

drivers/media/platform/vimc/vimc-sensor.c +10 ~~-59~~

index 6c57b1e262f9..bd237e030a1b 100644
@@ -5,10 +5,6 @@
   * Copyright (C) 2015-2017 Helen Koike <[email protected]>
   */
 
-#include <linux/component.h>
-#include <linux/module.h>
-#include <linux/mod_devicetable.h>
-#include <linux/platform_device.h>
  #include <linux/v4l2-mediabus.h>
  #include <linux/vmalloc.h>
  #include <media/v4l2-ctrls.h>

@@ -19,8 +15,6 @@
  #include "vimc.h"
  #include "vimc-common.h"
 
-#define VIMC_SEN_DRV_NAME "vimc-sensor"
-
  static const struct v4l2_mbus_framefmt fmt_default = {
  	.width = 640,
  	.height = 480,

@@ -268,12 +262,12 @@ static const struct v4l2_subdev_internal_ops vimc_sen_int_ops = {
  	.release = vimc_sen_release,
  };
 
-static void vimc_sen_comp_unbind(struct device *comp, struct device *master,
-				 void *master_data)
+void vimc_sen_rm(struct vimc_device *vimc, struct vimc_ent_config *vent)
  {
-	struct vimc_ent_device *ved = dev_get_drvdata(comp);
-	struct vimc_sen_device *vsen =
-				container_of(ved, struct vimc_sen_device, ved);
+	struct vimc_ent_device *ved = vent->ved;
+	struct vimc_sen_device *vsen = container_of(ved,
+						    struct vimc_sen_device,
+						    ved);
 
  	vimc_ent_sd_unregister(ved, &vsen->sd);
  }

@@ -295,11 +289,9 @@ static const struct v4l2_ctrl_config vimc_sen_ctrl_test_pattern = {
  	.qmenu = tpg_pattern_strings,
  };
 
-static int vimc_sen_comp_bind(struct device *comp, struct device *master,
-			      void *master_data)
+int vimc_sen_add(struct vimc_device *vimc, struct vimc_ent_config *vent)
  {
-	struct v4l2_device *v4l2_dev = master_data;
-	struct vimc_platform_data *pdata = comp->platform_data;
+	struct v4l2_device *v4l2_dev = &vimc->v4l2_dev;
  	struct vimc_sen_device *vsen;
  	int ret;

@@ -332,7 +324,7 @@ static int vimc_sen_comp_bind(struct device *comp, struct device *master,
 
  	/* Initialize ved and sd */
  	ret = vimc_ent_sd_register(&vsen->ved, &vsen->sd, v4l2_dev,
-				   pdata->entity_name,
+				   vent->name,
  				   MEDIA_ENT_F_CAM_SENSOR, 1,
  				   (const unsigned long[1]) {MEDIA_PAD_FL_SOURCE},
  				   &vimc_sen_int_ops, &vimc_sen_ops);

@@ -340,8 +332,7 @@ static int vimc_sen_comp_bind(struct device *comp, struct device *master,
  		goto err_free_hdl;
 
  	vsen->ved.process_frame = vimc_sen_process_frame;
-	dev_set_drvdata(comp, &vsen->ved);
-	vsen->dev = comp;
+	vsen->dev = &vimc->pdev.dev;
 
  	/* Initialize the frame format */
  	vsen->mbus_format = fmt_default;

@@ -353,6 +344,7 @@ static int vimc_sen_comp_bind(struct device *comp, struct device *master,
  	if (ret)
  		goto err_unregister_ent_sd;
 
+	vent->ved = &vsen->ved;
  	return 0;
 
  err_unregister_ent_sd:

@@ -364,44 +356,3 @@ static int vimc_sen_comp_bind(struct device *comp, struct device *master,
 
  	return ret;
  }
-
-static const struct component_ops vimc_sen_comp_ops = {
-	.bind = vimc_sen_comp_bind,
-	.unbind = vimc_sen_comp_unbind,
-};
-
-static int vimc_sen_probe(struct platform_device *pdev)
-{
-	return component_add(&pdev->dev, &vimc_sen_comp_ops);
-}
-
-static int vimc_sen_remove(struct platform_device *pdev)
-{
-	component_del(&pdev->dev, &vimc_sen_comp_ops);
-
-	return 0;
-}
-
-static const struct platform_device_id vimc_sen_driver_ids[] = {
-	{
-		.name           = VIMC_SEN_DRV_NAME,
-	},
-	{ }
-};
-
-static struct platform_driver vimc_sen_pdrv = {
-	.probe		= vimc_sen_probe,
-	.remove		= vimc_sen_remove,
-	.id_table	= vimc_sen_driver_ids,
-	.driver		= {
-		.name	= VIMC_SEN_DRV_NAME,
-	},
-};
-
-module_platform_driver(vimc_sen_pdrv);
-
-MODULE_DEVICE_TABLE(platform, vimc_sen_driver_ids);
-
-MODULE_DESCRIPTION("Virtual Media Controller Driver (VIMC) Sensor");
-MODULE_AUTHOR("Helen Mae Koike Fornazier <[email protected]>");
-MODULE_LICENSE("GPL");

drivers/media/platform/vimc/vimc.h +2 -1

index a5adebdda941..cd527c6fe6fc 100644
@@ -92,11 +92,30 @@ struct vimc_device {
  	/* The Associated media_device parent */
  	struct media_device mdev;
 
-	/* Internal v4l2 parent device*/
+	/* Internal v4l2 parent device */
  	struct v4l2_device v4l2_dev;
+};

Quoted 10 lines

drivers/media/platform/vimc/vimc.h struct vimc_device { · line 102
Quoted 6 lines
drivers/media/platform/vimc/vimc.h +4 -2
-	/* Subdevices */
-	struct platform_device **subdevs;
+/* Structure which describes individual configuration for each entity */
+struct vimc_ent_config {
+	const char *name;
+	const char *drv;
I was tempted in suggesting to remove this field as it is not being used anymore, but it will be usefull for the configfs patch.

.drv isn't used anymore. I can remove that. name is still used.

thanks,
-- Shuah

Shuah Khan

author

2019-08-09 21:46 UTC #11 lore raw

PATCH 3/3 media: vimc: Fix gpf in rmmod path when stream is active

In reply to Shuah Khan

If vimc module is removed while streaming is in progress, sensor subdev unregister runs into general protection fault when it tries to unregister media entities.

vimc_ent_sd_unregister() is fixed to call vimc_pads_cleanup() to release media pads after v4l2_device_unregister_subdev() is done unregistering the subdev and stop accessing media objects.

kernel: [ 4136.715839] general protection fault: 0000 [#1] SMP PTI
kernel: [ 4136.715847] CPU: 2 PID: 1972 Comm: bash Not tainted 5.3.0-rc2+ #4
kernel: [ 4136.715850] Hardware name: Dell Inc. OptiPlex 790/0HY9JP, BIOS A18 09/24/2013
kernel: [ 4136.715858] RIP: 0010:media_gobj_destroy.part.16+0x1f/0x60
kernel: [ 4136.715863] Code: ff 66 2e 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 89 fe 48 89 e5 53 48 89 fb 48 c7 c7 00 7f cf b0 e8 24 fa ff ff 48 8b 03 <48> 83 80 a0 00 00 00 01 48 8b 43 18 48 8b 53 10 48 89 42 08 48 89
kernel: [ 4136.715866] RSP: 0018:ffff9b2248fe3cb0 EFLAGS: 00010246
kernel: [ 4136.715870] RAX: bcf2bfbfa0d63c2f RBX: ffff88c3eb37e9c0 RCX: 00000000802a0018
kernel: [ 4136.715873] RDX: ffff88c3e4f6a078 RSI: ffff88c3eb37e9c0 RDI: ffffffffb0cf7f00
kernel: [ 4136.715876] RBP: ffff9b2248fe3cb8 R08: 0000000001000002 R09: ffffffffb0492b00
kernel: [ 4136.715879] R10: ffff9b2248fe3c28 R11: 0000000000000001 R12: 0000000000000038
kernel: [ 4136.715881] R13: ffffffffc09a1628 R14: ffff88c3e4f6a028 R15: fffffffffffffff2
kernel: [ 4136.715885] FS: 00007f8389647740(0000) GS:ffff88c465500000(0000) knlGS:0000000000000000
kernel: [ 4136.715888] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
kernel: [ 4136.715891] CR2: 000055d008f80fd8 CR3: 00000001996ec005 CR4: 00000000000606e0
kernel: [ 4136.715894] Call Trace:
kernel: [ 4136.715903] media_gobj_destroy+0x14/0x20
kernel: [ 4136.715908] __media_device_unregister_entity+0xb3/0xe0
kernel: [ 4136.715915] media_device_unregister_entity+0x30/0x40
kernel: [ 4136.715920] v4l2_device_unregister_subdev+0xa8/0xe0
kernel: [ 4136.715928] vimc_ent_sd_unregister+0x1e/0x30 [vimc]
kernel: [ 4136.715933] vimc_sen_rm+0x16/0x20 [vimc]
kernel: [ 4136.715938] vimc_remove+0x3e/0xa0 [vimc]
kernel: [ 4136.715945] platform_drv_remove+0x25/0x50
kernel: [ 4136.715951] device_release_driver_internal+0xe0/0x1b0
kernel: [ 4136.715956] device_driver_detach+0x14/0x20
kernel: [ 4136.715960] unbind_store+0xd1/0x130
kernel: [ 4136.715965] drv_attr_store+0x27/0x40
kernel: [ 4136.715971] sysfs_kf_write+0x48/0x60
kernel: [ 4136.715976] kernfs_fop_write+0x128/0x1b0
kernel: [ 4136.715982] __vfs_write+0x1b/0x40
kernel: [ 4136.715987] vfs_write+0xc3/0x1d0
kernel: [ 4136.715993] ksys_write+0xaa/0xe0
kernel: [ 4136.715999] __x64_sys_write+0x1a/0x20
kernel: [ 4136.716005] do_syscall_64+0x5a/0x130
kernel: [ 4136.716010] entry_SYSCALL_64_after_hwframe+0x4

Signed-off-by Shuah Khan <[email protected]>

---
 drivers/media/platform/vimc/vimc-common.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

drivers/media/platform/vimc/vimc-common.c +1 -1

index 03016f204d05..abace5e0b4d3 100644
@@ -373,7 +373,7 @@ EXPORT_SYMBOL_GPL(vimc_ent_sd_register);
 void vimc_ent_sd_unregister(struct vimc_ent_device *ved, struct v4l2_subdev *sd)
 {
 	media_entity_cleanup(ved->ent);
-	vimc_pads_cleanup(ved->pads);
 	v4l2_device_unregister_subdev(sd);
+	vimc_pads_cleanup(ved->pads);
 }
 EXPORT_SYMBOL_GPL(vimc_ent_sd_unregister);

Signature · 1 line

2.17.1

André Almeida

2019-08-09 23:53 UTC #12 lore raw

In reply to Shuah Khan

Hello Shuah,

Thanks for the patch, I did some comments below.

Quoted 49 lines · Shuah Khan

vimc uses Component API to split the driver into functional components. The real hardware resembles a monolith structure than component and component structure added a level of complexity making it hard to maintain without adding any real benefit.
The sensor is one vimc component that would makes sense to be a separate module to closely align with the real hardware. It would be easier to collapse vimc into single monolithic driver first and then split the sensor off as a separate module.
This patch series emoves the component API and makes minimal changes to the code base preserving the functional division of the code structure. Preserving the functional structure allows us to split the sensor off as a separate module in the future.
Major design elements in this change are:
- Use existing struct vimc_ent_config and struct vimc_pipeline_config to drive the initialization of the functional components.
- Make vimc_ent_config global by moving it to vimc.h
- Add two new hooks add and rm to initialize and register, unregister and free subdevs.
- All component API is now gone and bind and unbind hooks are modified to do "add" and "rm" with minimal changes to just add and rm subdevs.
- vimc-core's bind and unbind are now register and unregister.
- vimc-core invokes "add" hooks from its vimc_register_devices(). The "add" hooks remain the same and register subdevs. They don't create platform devices of their own and use vimc's pdev.dev as their reference device. The "add" hooks save their vimc_ent_device(s) in the corresponding vimc_ent_config.
- vimc-core invokes "rm" hooks from its unregister to unregister subdevs and cleanup.
- vimc-core invokes "add" and "rm" hooks with pointer to struct vimc_device and the corresponding struct vimc_ent_config pointer.
The following configure and stream test works on all devices.
    media-ctl -d platform:vimc -V '"Sensor A":0[fmt:SBGGR8_1X8/640x480]'
    media-ctl -d platform:vimc -V '"Debayer A":0[fmt:SBGGR8_1X8/640x480]'
    media-ctl -d platform:vimc -V '"Sensor B":0[fmt:SBGGR8_1X8/640x480]'
    media-ctl -d platform:vimc -V '"Debayer B":0[fmt:SBGGR8_1X8/640x480]'
    
    v4l2-ctl -z platform:vimc -d "RGB/YUV Capture" -v width=1920,height=1440
    v4l2-ctl -z platform:vimc -d "Raw Capture 0" -v pixelformat=BA81
    v4l2-ctl -z platform:vimc -d "Raw Capture 1" -v pixelformat=BA81
    
    v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video1
    v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video2
    v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video3

Quoted 2 lines

The third patch in the series fixes a general protection fault found when rmmod is done while stream is active.

I applied your patch on top of media_tree/master and I did some testing. Not sure if I did something wrong, but just adding and removing the module generated a kernel panic:

~# modprobe vimc
~# rmmod vimc

[   16.452974] stack segment: 0000 [#1] SMP PTI
[   16.453688] CPU: 0 PID: 2038 Comm: rmmod Not tainted 5.3.0-rc2+ #36
[   16.454678] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996),
BIOS 1.12.0-20181126_142135-anatol 04/01/2014
[   16.456191] RIP: 0010:kfree+0x4d/0x240

[   16.469188] Call Trace:
[   16.469666]  vimc_remove+0x35/0x90 [vimc]
[   16.470436]  platform_drv_remove+0x1f/0x40
[   16.471233]  device_release_driver_internal+0xd3/0x1b0
[   16.472184]  driver_detach+0x37/0x6b
[   16.472882]  bus_remove_driver+0x50/0xc1
[   16.473569]  vimc_exit+0xc/0xca0 [vimc]
[   16.474231]  __x64_sys_delete_module+0x18d/0x240
[   16.475036]  do_syscall_64+0x43/0x110
[   16.475656]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
[   16.476504] RIP: 0033:0x7fceb8dafa4b

[   16.484853] Modules linked in: vimc(-) videobuf2_vmalloc
videobuf2_memops v4l2_tpg videobuf2_v4l2 videobuf2_common
[   16.486187] ---[ end trace 91e5e0894e254d49 ]---
[   16.486758] RIP: 0010:kfree+0x4d/0x240

fish: “rmmod vimc” terminated by signal SIGSEGV (Address boundary error)

I just added the module after booting, no other action was made. Here is how my git log --oneline looks like:

897d708e922b media: vimc: Fix gpf in rmmod path when stream is active 2e4a5ad8ad6d media: vimc: Collapse component structure into a single monolithic driver
7c8da1687e92 media: vimc: move private defines to a common header 97299a303532 media: Remove dev_err() usage after platform_get_irq() 25a3d6bac6b9 media: adv7511/cobalt: rename driver name to adv7511-v4l2
...

Quoted 14 lines

vimc_print_dot (--print-dot) topology after this change:

digraph board {
	rankdir=TB
	n00000001 [label="{{} | Sensor A\n/dev/v4l-subdev0 | {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green]
	n00000001:port0 -> n00000005:port0 [style=bold]
	n00000001:port0 -> n0000000b [style=bold]
	n00000003 [label="{{} | Sensor B\n/dev/v4l-subdev1 | {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green]
	n00000003:port0 -> n00000008:port0 [style=bold]
	n00000003:port0 -> n0000000f [style=bold]
	n00000005 [label="{{<port0> 0} | Debayer A\n/dev/v4l-subdev2 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
	n00000005:port1 -> n00000015:port0
	n00000008 [label="{{<port0> 0} | Debayer B\n/dev/v4l-subdev3 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
	n00000008:port1 -> n00000015:port0 [style=dashed]

Quoted 8 lines

	n0000000b [label="Raw Capture 0\n/dev/video1", shape=box, style=filled, fillcolor=yellow]
	n0000000f [label="Raw Capture 1\n/dev/video2", shape=box, style=filled, fillcolor=yellow]
	n00000013 [label="{{} | RGB/YUV Input\n/dev/v4l-subdev4 | {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green]
	n00000013:port0 -> n00000015:port0 [style=dashed]
	n00000015 [label="{{<port0> 0} | Scaler\n/dev/v4l-subdev5 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
	n00000015:port1 -> n00000018 [style=bold]
	n00000018 [label="RGB/YUV Capture\n/dev/video3", shape=box, style=filled, fillcolor=yellow]
}

Since the topology changed, it would be nice to change in the documentation as well. The current dot file can be found at Documentation/media/v4l-drivers/vimc.dot and it's rendered at this
page: https://www.kernel.org/doc/html/latest/media/v4l-drivers/vimc.html

Thanks,
André

Quoted 18 lines

Shuah Khan (3):
media: vimc: move private defines to a common header
media: vimc: Collapse component structure into a single monolithic
driver
media: vimc: Fix gpf in rmmod path when stream is active

 drivers/media/platform/vimc/Makefile       |   7 +-
 drivers/media/platform/vimc/vimc-capture.c |  96 ++----------
 drivers/media/platform/vimc/vimc-common.c  |   2 +-
 drivers/media/platform/vimc/vimc-core.c    | 174 +++++++--------------
 drivers/media/platform/vimc/vimc-debayer.c |  84 ++--------
 drivers/media/platform/vimc/vimc-scaler.c  |  83 ++--------
 drivers/media/platform/vimc/vimc-sensor.c  |  82 ++--------
 drivers/media/platform/vimc/vimc.h         | 121 ++++++++++++++
 8 files changed, 230 insertions(+), 419 deletions(-)
 create mode 100644 drivers/media/platform/vimc/vimc.h

Shuah Khan

author

2019-08-10 00:17 UTC #13 lore raw

Hi Andre,

Quoted 54 lines · André Almeida

Hello Shuah,
Thanks for the patch, I did some comments below.
Quoted 49 lines · Shuah Khan
vimc uses Component API to split the driver into functional components. The real hardware resembles a monolith structure than component and component structure added a level of complexity making it hard to maintain without adding any real benefit.
The sensor is one vimc component that would makes sense to be a separate module to closely align with the real hardware. It would be easier to collapse vimc into single monolithic driver first and then split the sensor off as a separate module.
This patch series emoves the component API and makes minimal changes to the code base preserving the functional division of the code structure. Preserving the functional structure allows us to split the sensor off as a separate module in the future.
Major design elements in this change are:
- Use existing struct vimc_ent_config and struct vimc_pipeline_config to drive the initialization of the functional components.
- Make vimc_ent_config global by moving it to vimc.h
- Add two new hooks add and rm to initialize and register, unregister and free subdevs.
- All component API is now gone and bind and unbind hooks are modified to do "add" and "rm" with minimal changes to just add and rm subdevs.
- vimc-core's bind and unbind are now register and unregister.
- vimc-core invokes "add" hooks from its vimc_register_devices(). The "add" hooks remain the same and register subdevs. They don't create platform devices of their own and use vimc's pdev.dev as their reference device. The "add" hooks save their vimc_ent_device(s) in the corresponding vimc_ent_config.
- vimc-core invokes "rm" hooks from its unregister to unregister subdevs and cleanup.
- vimc-core invokes "add" and "rm" hooks with pointer to struct vimc_device and the corresponding struct vimc_ent_config pointer.
The following configure and stream test works on all devices.
     media-ctl -d platform:vimc -V '"Sensor A":0[fmt:SBGGR8_1X8/640x480]'
     media-ctl -d platform:vimc -V '"Debayer A":0[fmt:SBGGR8_1X8/640x480]'
     media-ctl -d platform:vimc -V '"Sensor B":0[fmt:SBGGR8_1X8/640x480]'
     media-ctl -d platform:vimc -V '"Debayer B":0[fmt:SBGGR8_1X8/640x480]'
     
     v4l2-ctl -z platform:vimc -d "RGB/YUV Capture" -v width=1920,height=1440
     v4l2-ctl -z platform:vimc -d "Raw Capture 0" -v pixelformat=BA81
     v4l2-ctl -z platform:vimc -d "Raw Capture 1" -v pixelformat=BA81
     
     v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video1
     v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video2
     v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video3

Quoted 6 lines

Quoted 2 lines
The third patch in the series fixes a general protection fault found when rmmod is done while stream is active.
I applied your patch on top of media_tree/master and I did some testing. Not sure if I did something wrong, but just adding and removing the module generated a kernel panic:

Thanks for testing.

Odd. I tested modprobe and rmmod both.I was working on Linux 5.3-rc2. I will apply these to media latest and work from there. I have to rebase these on top of the reverts from Lucas and Helen

Quoted 68 lines

~# modprobe vimc
~# rmmod vimc

[   16.452974] stack segment: 0000 [#1] SMP PTI
[   16.453688] CPU: 0 PID: 2038 Comm: rmmod Not tainted 5.3.0-rc2+ #36
[   16.454678] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996),
BIOS 1.12.0-20181126_142135-anatol 04/01/2014
[   16.456191] RIP: 0010:kfree+0x4d/0x240

[   16.469188] Call Trace:
[   16.469666]  vimc_remove+0x35/0x90 [vimc]
[   16.470436]  platform_drv_remove+0x1f/0x40
[   16.471233]  device_release_driver_internal+0xd3/0x1b0
[   16.472184]  driver_detach+0x37/0x6b
[   16.472882]  bus_remove_driver+0x50/0xc1
[   16.473569]  vimc_exit+0xc/0xca0 [vimc]
[   16.474231]  __x64_sys_delete_module+0x18d/0x240
[   16.475036]  do_syscall_64+0x43/0x110
[   16.475656]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
[   16.476504] RIP: 0033:0x7fceb8dafa4b

[   16.484853] Modules linked in: vimc(-) videobuf2_vmalloc
videobuf2_memops v4l2_tpg videobuf2_v4l2 videobuf2_common
[   16.486187] ---[ end trace 91e5e0894e254d49 ]---
[   16.486758] RIP: 0010:kfree+0x4d/0x240

fish: “rmmod vimc” terminated by signal SIGSEGV (Address boundary error)

I just added the module after booting, no other action was made. Here is how my git log --oneline looks like:

Quoted 14 lines

vimc_print_dot (--print-dot) topology after this change:

digraph board {
	rankdir=TB
	n00000001 [label="{{} | Sensor A\n/dev/v4l-subdev0 | {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green]
	n00000001:port0 -> n00000005:port0 [style=bold]
	n00000001:port0 -> n0000000b [style=bold]
	n00000003 [label="{{} | Sensor B\n/dev/v4l-subdev1 | {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green]
	n00000003:port0 -> n00000008:port0 [style=bold]
	n00000003:port0 -> n0000000f [style=bold]
	n00000005 [label="{{<port0> 0} | Debayer A\n/dev/v4l-subdev2 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
	n00000005:port1 -> n00000015:port0
	n00000008 [label="{{<port0> 0} | Debayer B\n/dev/v4l-subdev3 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
	n00000008:port1 -> n00000015:port0 [style=dashed]

Quoted 8 lines

	n0000000b [label="Raw Capture 0\n/dev/video1", shape=box, style=filled, fillcolor=yellow]
	n0000000f [label="Raw Capture 1\n/dev/video2", shape=box, style=filled, fillcolor=yellow]
	n00000013 [label="{{} | RGB/YUV Input\n/dev/v4l-subdev4 | {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green]
	n00000013:port0 -> n00000015:port0 [style=dashed]
	n00000015 [label="{{<port0> 0} | Scaler\n/dev/v4l-subdev5 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
	n00000015:port1 -> n00000018 [style=bold]
	n00000018 [label="RGB/YUV Capture\n/dev/video3", shape=box, style=filled, fillcolor=yellow]
}

Quoted 5 lines

Since the topology changed, it would be nice to change in the documentation as well. The current dot file can be found at Documentation/media/v4l-drivers/vimc.dot and it's rendered at this
page: https://www.kernel.org/doc/html/latest/media/v4l-drivers/vimc.html

Topology shouldn't have changed. No changes to links or pads etc. I will take a look to be sure. I agree that if topology changes document should be updated.

thanks,
-- Shuah

André Almeida

2019-08-10 00:25 UTC #14 lore raw

Quoted 72 lines · Shuah Khan

Hi Andre,
Quoted 62 lines · André Almeida
Hello Shuah,
Thanks for the patch, I did some comments below.
Quoted 57 lines · Shuah Khan
vimc uses Component API to split the driver into functional components. The real hardware resembles a monolith structure than component and component structure added a level of complexity making it hard to maintain without adding any real benefit.
     The sensor is one vimc component that would makes sense to be a separate
module to closely align with the real hardware. It would be easier to collapse vimc into single monolithic driver first and then split the sensor off as a separate module.
This patch series emoves the component API and makes minimal changes to the code base preserving the functional division of the code structure. Preserving the functional structure allows us to split the sensor off as a separate module in the future.
Major design elements in this change are:
     - Use existing struct vimc_ent_config and struct vimc_pipeline_config
       to drive the initialization of the functional components.
     - Make vimc_ent_config global by moving it to vimc.h
     - Add two new hooks add and rm to initialize and register, unregister
       and free subdevs.
     - All component API is now gone and bind and unbind hooks are modified
       to do "add" and "rm" with minimal changes to just add and rm subdevs.
     - vimc-core's bind and unbind are now register and unregister.
     - vimc-core invokes "add" hooks from its vimc_register_devices(). The "add" hooks remain the same and register subdevs. They don't create platform devices of their own and use vimc's pdev.dev as their reference device. The "add" hooks save their vimc_ent_device(s)
       in the corresponding vimc_ent_config.
     - vimc-core invokes "rm" hooks from its unregister to unregister subdevs
       and cleanup.
     - vimc-core invokes "add" and "rm" hooks with pointer to struct vimc_device
       and the corresponding struct vimc_ent_config pointer. The following configure and stream test works on all devices. media-ctl -d platform:vimc -V '"Sensor A":0[fmt:SBGGR8_1X8/640x480]'
     media-ctl -d platform:vimc -V '"Debayer A":0[fmt:SBGGR8_1X8/640x480]'
     media-ctl -d platform:vimc -V '"Sensor B":0[fmt:SBGGR8_1X8/640x480]'
     media-ctl -d platform:vimc -V '"Debayer B":0[fmt:SBGGR8_1X8/640x480]'
          v4l2-ctl -z platform:vimc -d "RGB/YUV Capture" -v width=1920,height=1440
     v4l2-ctl -z platform:vimc -d "Raw Capture 0" -v pixelformat=BA81 v4l2-ctl -z platform:vimc -d "Raw Capture 1" -v pixelformat=BA81 v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video1 v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video2 v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video3
Quoted 6 lines
Quoted 2 lines
The third patch in the series fixes a general protection fault found when rmmod is done while stream is active.
I applied your patch on top of media_tree/master and I did some testing. Not sure if I did something wrong, but just adding and removing the module generated a kernel panic:

Quoted 5 lines

Thanks for testing.
Odd. I tested modprobe and rmmod both.I was working on Linux 5.3-rc2. I will apply these to media latest and work from there. I have to rebase these on top of the reverts from Lucas and Helen

Ok, please let me know if I succeeded to reproduce.

Quoted 83 lines

Quoted 77 lines
~# modprobe vimc
~# rmmod vimc
[   16.452974] stack segment: 0000 [#1] SMP PTI
[   16.453688] CPU: 0 PID: 2038 Comm: rmmod Not tainted 5.3.0-rc2+ #36 [   16.454678] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-20181126_142135-anatol 04/01/2014
[   16.456191] RIP: 0010:kfree+0x4d/0x240
<registers values...>
[   16.469188] Call Trace:
[   16.469666] vimc_remove+0x35/0x90 [vimc]
[   16.470436] platform_drv_remove+0x1f/0x40
[   16.471233] device_release_driver_internal+0xd3/0x1b0 [   16.472184] driver_detach+0x37/0x6b
[   16.472882] bus_remove_driver+0x50/0xc1
[   16.473569] vimc_exit+0xc/0xca0 [vimc]
[   16.474231] __x64_sys_delete_module+0x18d/0x240 [   16.475036] do_syscall_64+0x43/0x110
[   16.475656] entry_SYSCALL_64_after_hwframe+0x44/0xa9 [   16.476504] RIP: 0033:0x7fceb8dafa4b
<registers values...>
[   16.484853] Modules linked in: vimc(-) videobuf2_vmalloc videobuf2_memops v4l2_tpg videobuf2_v4l2 videobuf2_common [   16.486187] ---[ end trace 91e5e0894e254d49 ]--- [   16.486758] RIP: 0010:kfree+0x4d/0x240
<registers values...>
fish: “rmmod vimc” terminated by signal SIGSEGV (Address boundary error)
I just added the module after booting, no other action was made. Here is how my git log --oneline looks like:
897d708e922b media: vimc: Fix gpf in rmmod path when stream is active 2e4a5ad8ad6d media: vimc: Collapse component structure into a single monolithic driver
7c8da1687e92 media: vimc: move private defines to a common header 97299a303532 media: Remove dev_err() usage after platform_get_irq() 25a3d6bac6b9 media: adv7511/cobalt: rename driver name to adv7511-v4l2
...
Quoted 23 lines
vimc_print_dot (--print-dot) topology after this change:
digraph board {
    rankdir=TB
    n00000001 [label="{{} | Sensor A\n/dev/v4l-subdev0 | {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green] n00000001:port0 -> n00000005:port0 [style=bold] n00000001:port0 -> n0000000b [style=bold]
    n00000003 [label="{{} | Sensor B\n/dev/v4l-subdev1 | {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green] n00000003:port0 -> n00000008:port0 [style=bold] n00000003:port0 -> n0000000f [style=bold]
    n00000005 [label="{{<port0> 0} | Debayer A\n/dev/v4l-subdev2 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green] n00000005:port1 -> n00000015:port0
    n00000008 [label="{{<port0> 0} | Debayer B\n/dev/v4l-subdev3 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green] n00000008:port1 -> n00000015:port0 [style=dashed] n0000000b [label="Raw Capture 0\n/dev/video1", shape=box, style=filled, fillcolor=yellow]
    n0000000f [label="Raw Capture 1\n/dev/video2", shape=box, style=filled, fillcolor=yellow]
    n00000013 [label="{{} | RGB/YUV Input\n/dev/v4l-subdev4 |
Quoted 8 lines
{<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green] n00000013:port0 -> n00000015:port0 [style=dashed] n00000015 [label="{{<port0> 0} | Scaler\n/dev/v4l-subdev5 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green] n00000015:port1 -> n00000018 [style=bold] n00000018 [label="RGB/YUV Capture\n/dev/video3", shape=box, style=filled, fillcolor=yellow]
}
Quoted 5 lines
Since the topology changed, it would be nice to change in the documentation as well. The current dot file can be found at Documentation/media/v4l-drivers/vimc.dot and it's rendered at this
page: https://www.kernel.org/doc/html/latest/media/v4l-drivers/vimc.html

Quoted 3 lines

Topology shouldn't have changed. No changes to links or pads etc. I will take a look to be sure. I agree that if topology changes document should be updated.

If you "diff" the current dot with the dot you generated, you will see some differences. The main difference is that "RGB/YUV Input" was a device "/dev/video2/", and now it a subdevice "/dev/v4l-subdev4".

Quoted 3 lines

thanks,
-- Shuah

Shuah Khan

author

2019-08-10 00:48 UTC #15 lore raw

Quoted 164 lines · André Almeida

Quoted 72 lines · Shuah Khan
Hi Andre,
Quoted 62 lines · André Almeida
Hello Shuah,
Thanks for the patch, I did some comments below.
Quoted 57 lines · Shuah Khan
vimc uses Component API to split the driver into functional components. The real hardware resembles a monolith structure than component and component structure added a level of complexity making it hard to maintain without adding any real benefit.
     The sensor is one vimc component that would makes sense to be a
separate
module to closely align with the real hardware. It would be easier to collapse vimc into single monolithic driver first and then split the sensor off as a separate module.
This patch series emoves the component API and makes minimal changes to the code base preserving the functional division of the code structure. Preserving the functional structure allows us to split the sensor off as a separate module in the future.
Major design elements in this change are:
     - Use existing struct vimc_ent_config and struct
vimc_pipeline_config
       to drive the initialization of the functional components.
     - Make vimc_ent_config global by moving it to vimc.h
     - Add two new hooks add and rm to initialize and register,
unregister
       and free subdevs.
     - All component API is now gone and bind and unbind hooks are
modified
       to do "add" and "rm" with minimal changes to just add and rm
subdevs.
     - vimc-core's bind and unbind are now register and unregister.
     - vimc-core invokes "add" hooks from its vimc_register_devices(). The "add" hooks remain the same and register subdevs. They don't create platform devices of their own and use vimc's pdev.dev as their reference device. The "add" hooks save their
vimc_ent_device(s)
       in the corresponding vimc_ent_config.
     - vimc-core invokes "rm" hooks from its unregister to unregister
subdevs
       and cleanup.
     - vimc-core invokes "add" and "rm" hooks with pointer to struct
vimc_device
       and the corresponding struct vimc_ent_config pointer. The following configure and stream test works on all devices. media-ctl -d platform:vimc -V '"Sensor
A":0[fmt:SBGGR8_1X8/640x480]'
     media-ctl -d platform:vimc -V '"Debayer
A":0[fmt:SBGGR8_1X8/640x480]'
     media-ctl -d platform:vimc -V '"Sensor
B":0[fmt:SBGGR8_1X8/640x480]'
     media-ctl -d platform:vimc -V '"Debayer
B":0[fmt:SBGGR8_1X8/640x480]'
          v4l2-ctl -z platform:vimc -d "RGB/YUV Capture" -v
width=1920,height=1440
     v4l2-ctl -z platform:vimc -d "Raw Capture 0" -v pixelformat=BA81 v4l2-ctl -z platform:vimc -d "Raw Capture 1" -v pixelformat=BA81 v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video1 v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video2 v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video3
Quoted 6 lines
Quoted 2 lines
The third patch in the series fixes a general protection fault found when rmmod is done while stream is active.
I applied your patch on top of media_tree/master and I did some testing. Not sure if I did something wrong, but just adding and removing the module generated a kernel panic:
Quoted 5 lines
Thanks for testing.
Odd. I tested modprobe and rmmod both.I was working on Linux 5.3-rc2. I will apply these to media latest and work from there. I have to rebase these on top of the reverts from Lucas and Helen
Ok, please let me know if I succeeded to reproduce.
Quoted 83 lines
Quoted 82 lines
~# modprobe vimc
~# rmmod vimc
[   16.452974] stack segment: 0000 [#1] SMP PTI
[   16.453688] CPU: 0 PID: 2038 Comm: rmmod Not tainted 5.3.0-rc2+ #36 [   16.454678] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-20181126_142135-anatol 04/01/2014
[   16.456191] RIP: 0010:kfree+0x4d/0x240
<registers values...>
[   16.469188] Call Trace:
[   16.469666] vimc_remove+0x35/0x90 [vimc]
[   16.470436] platform_drv_remove+0x1f/0x40
[   16.471233] device_release_driver_internal+0xd3/0x1b0 [   16.472184] driver_detach+0x37/0x6b
[   16.472882] bus_remove_driver+0x50/0xc1
[   16.473569] vimc_exit+0xc/0xca0 [vimc]
[   16.474231] __x64_sys_delete_module+0x18d/0x240 [   16.475036] do_syscall_64+0x43/0x110
[   16.475656] entry_SYSCALL_64_after_hwframe+0x44/0xa9 [   16.476504] RIP: 0033:0x7fceb8dafa4b
<registers values...>
[   16.484853] Modules linked in: vimc(-) videobuf2_vmalloc videobuf2_memops v4l2_tpg videobuf2_v4l2 videobuf2_common [   16.486187] ---[ end trace 91e5e0894e254d49 ]--- [   16.486758] RIP: 0010:kfree+0x4d/0x240
<registers values...>
fish: “rmmod vimc” terminated by signal SIGSEGV (Address boundary error)
I just added the module after booting, no other action was made. Here is how my git log --oneline looks like:
897d708e922b media: vimc: Fix gpf in rmmod path when stream is active 2e4a5ad8ad6d media: vimc: Collapse component structure into a single monolithic driver
7c8da1687e92 media: vimc: move private defines to a common header 97299a303532 media: Remove dev_err() usage after platform_get_irq() 25a3d6bac6b9 media: adv7511/cobalt: rename driver name to adv7511-v4l2
...
Quoted 23 lines
vimc_print_dot (--print-dot) topology after this change:
digraph board {
    rankdir=TB
    n00000001 [label="{{} | Sensor A\n/dev/v4l-subdev0 | {<port0>
0}}", shape=Mrecord, style=filled, fillcolor=green]
    n00000001:port0 -> n00000005:port0 [style=bold] n00000001:port0 -> n0000000b [style=bold]
    n00000003 [label="{{} | Sensor B\n/dev/v4l-subdev1 | {<port0>
0}}", shape=Mrecord, style=filled, fillcolor=green]
    n00000003:port0 -> n00000008:port0 [style=bold] n00000003:port0 -> n0000000f [style=bold]
    n00000005 [label="{{<port0> 0} | Debayer A\n/dev/v4l-subdev2 |
{<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
    n00000005:port1 -> n00000015:port0
    n00000008 [label="{{<port0> 0} | Debayer B\n/dev/v4l-subdev3 |
{<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
    n00000008:port1 -> n00000015:port0 [style=dashed] n0000000b [label="Raw Capture 0\n/dev/video1", shape=box,
style=filled, fillcolor=yellow]
    n0000000f [label="Raw Capture 1\n/dev/video2", shape=box,
style=filled, fillcolor=yellow]
    n00000013 [label="{{} | RGB/YUV Input\n/dev/v4l-subdev4 |
Quoted 8 lines
{<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green]
    n00000013:port0 -> n00000015:port0 [style=dashed] n00000015 [label="{{<port0> 0} | Scaler\n/dev/v4l-subdev5 |
{<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
    n00000015:port1 -> n00000018 [style=bold] n00000018 [label="RGB/YUV Capture\n/dev/video3", shape=box,
style=filled, fillcolor=yellow]
}
Since the topology changed, it would be nice to change in the documentation as well. The current dot file can be found at Documentation/media/v4l-drivers/vimc.dot and it's rendered at this
page: https://www.kernel.org/doc/html/latest/media/v4l-drivers/vimc.html

Quoted 8 lines

Quoted 3 lines
Topology shouldn't have changed. No changes to links or pads etc. I will take a look to be sure. I agree that if topology changes document should be updated.
If you "diff" the current dot with the dot you generated, you will see some differences. The main difference is that "RGB/YUV Input" was a device "/dev/video2/", and now it a subdevice "/dev/v4l-subdev4".

Yeah. I should have saved dot before my changes and compare. The goal is to not change anything. I will make sure topology doesn't change.

Thanks again for testing these patches quickly.

thanks,
-- Shuah

Helen Koike

2019-08-10 03:51 UTC #16 lore raw

In reply to André Almeida

Hi Andre,

Thanks for testing this.

Quoted 73 lines · André Almeida

Quoted 72 lines · Shuah Khan
Hi Andre,
Quoted 68 lines · André Almeida
Hello Shuah,
Thanks for the patch, I did some comments below.
Quoted 57 lines · Shuah Khan
vimc uses Component API to split the driver into functional components. The real hardware resembles a monolith structure than component and component structure added a level of complexity making it hard to maintain without adding any real benefit.
     The sensor is one vimc component that would makes sense to be a separate
module to closely align with the real hardware. It would be easier to collapse vimc into single monolithic driver first and then split the sensor off as a separate module.
This patch series emoves the component API and makes minimal changes to the code base preserving the functional division of the code structure. Preserving the functional structure allows us to split the sensor off as a separate module in the future.
Major design elements in this change are:
     - Use existing struct vimc_ent_config and struct vimc_pipeline_config
       to drive the initialization of the functional components.
     - Make vimc_ent_config global by moving it to vimc.h
     - Add two new hooks add and rm to initialize and register, unregister
       and free subdevs.
     - All component API is now gone and bind and unbind hooks are modified
       to do "add" and "rm" with minimal changes to just add and rm subdevs.
     - vimc-core's bind and unbind are now register and unregister.
     - vimc-core invokes "add" hooks from its vimc_register_devices(). The "add" hooks remain the same and register subdevs. They don't create platform devices of their own and use vimc's pdev.dev as their reference device. The "add" hooks save their vimc_ent_device(s)
       in the corresponding vimc_ent_config.
     - vimc-core invokes "rm" hooks from its unregister to unregister subdevs
       and cleanup.
     - vimc-core invokes "add" and "rm" hooks with pointer to struct vimc_device
       and the corresponding struct vimc_ent_config pointer. The following configure and stream test works on all devices. media-ctl -d platform:vimc -V '"Sensor A":0[fmt:SBGGR8_1X8/640x480]'
     media-ctl -d platform:vimc -V '"Debayer A":0[fmt:SBGGR8_1X8/640x480]'
     media-ctl -d platform:vimc -V '"Sensor B":0[fmt:SBGGR8_1X8/640x480]'
     media-ctl -d platform:vimc -V '"Debayer B":0[fmt:SBGGR8_1X8/640x480]'
          v4l2-ctl -z platform:vimc -d "RGB/YUV Capture" -v width=1920,height=1440
     v4l2-ctl -z platform:vimc -d "Raw Capture 0" -v pixelformat=BA81 v4l2-ctl -z platform:vimc -d "Raw Capture 1" -v pixelformat=BA81 v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video1 v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video2 v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video3
Quoted 2 lines
The third patch in the series fixes a general protection fault found when rmmod is done while stream is active.
I applied your patch on top of media_tree/master and I did some testing. Not sure if I did something wrong, but just adding and removing the module generated a kernel panic:

Quoted 51 lines

Quoted 5 lines
Thanks for testing.
Odd. I tested modprobe and rmmod both.I was working on Linux 5.3-rc2. I will apply these to media latest and work from there. I have to rebase these on top of the reverts from Lucas and Helen
Ok, please let me know if I succeeded to reproduce.
Quoted 43 lines
Quoted 43 lines
~# modprobe vimc
~# rmmod vimc
[   16.452974] stack segment: 0000 [#1] SMP PTI
[   16.453688] CPU: 0 PID: 2038 Comm: rmmod Not tainted 5.3.0-rc2+ #36 [   16.454678] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-20181126_142135-anatol 04/01/2014
[   16.456191] RIP: 0010:kfree+0x4d/0x240
<registers values...>
[   16.469188] Call Trace:
[   16.469666] vimc_remove+0x35/0x90 [vimc]
[   16.470436] platform_drv_remove+0x1f/0x40
[   16.471233] device_release_driver_internal+0xd3/0x1b0 [   16.472184] driver_detach+0x37/0x6b
[   16.472882] bus_remove_driver+0x50/0xc1
[   16.473569] vimc_exit+0xc/0xca0 [vimc]
[   16.474231] __x64_sys_delete_module+0x18d/0x240 [   16.475036] do_syscall_64+0x43/0x110
[   16.475656] entry_SYSCALL_64_after_hwframe+0x44/0xa9 [   16.476504] RIP: 0033:0x7fceb8dafa4b
<registers values...>
[   16.484853] Modules linked in: vimc(-) videobuf2_vmalloc videobuf2_memops v4l2_tpg videobuf2_v4l2 videobuf2_common [   16.486187] ---[ end trace 91e5e0894e254d49 ]--- [   16.486758] RIP: 0010:kfree+0x4d/0x240
<registers values...>
fish: “rmmod vimc” terminated by signal SIGSEGV (Address boundary error)
I just added the module after booting, no other action was made. Here is how my git log --oneline looks like:
897d708e922b media: vimc: Fix gpf in rmmod path when stream is active 2e4a5ad8ad6d media: vimc: Collapse component structure into a single monolithic driver
7c8da1687e92 media: vimc: move private defines to a common header 97299a303532 media: Remove dev_err() usage after platform_get_irq() 25a3d6bac6b9 media: adv7511/cobalt: rename driver name to adv7511-v4l2

I couldn't reproduce the error, my tree looks the same:

[I] koike@floko ~/m/o/linux> git log --oneline
e3345155c8ed (HEAD) media: vimc: Fix gpf in rmmod path when stream is active 43e9e2fe761f media: vimc: Collapse component structure into a single monolithic driver 8a6d0b9adde0 media: vimc: move private defines to a common header 97299a303532 (media/master) media: Remove dev_err() usage after platform_get_irq() 25a3d6bac6b9 media: adv7511/cobalt: rename driver name to adv7511-v4l2

André, is this deterministic? Or it just happens sometimes?

Quoted 40 lines

Quoted 40 lines
Quoted 39 lines
...
Quoted 23 lines
vimc_print_dot (--print-dot) topology after this change:
digraph board {
    rankdir=TB
    n00000001 [label="{{} | Sensor A\n/dev/v4l-subdev0 | {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green] n00000001:port0 -> n00000005:port0 [style=bold] n00000001:port0 -> n0000000b [style=bold]
    n00000003 [label="{{} | Sensor B\n/dev/v4l-subdev1 | {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green] n00000003:port0 -> n00000008:port0 [style=bold] n00000003:port0 -> n0000000f [style=bold]
    n00000005 [label="{{<port0> 0} | Debayer A\n/dev/v4l-subdev2 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green] n00000005:port1 -> n00000015:port0
    n00000008 [label="{{<port0> 0} | Debayer B\n/dev/v4l-subdev3 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green] n00000008:port1 -> n00000015:port0 [style=dashed] n0000000b [label="Raw Capture 0\n/dev/video1", shape=box, style=filled, fillcolor=yellow]
    n0000000f [label="Raw Capture 1\n/dev/video2", shape=box, style=filled, fillcolor=yellow]
    n00000013 [label="{{} | RGB/YUV Input\n/dev/v4l-subdev4 |
Quoted 8 lines
{<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green] n00000013:port0 -> n00000015:port0 [style=dashed] n00000015 [label="{{<port0> 0} | Scaler\n/dev/v4l-subdev5 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green] n00000015:port1 -> n00000018 [style=bold] n00000018 [label="RGB/YUV Capture\n/dev/video3", shape=box, style=filled, fillcolor=yellow]
}
Since the topology changed, it would be nice to change in the documentation as well. The current dot file can be found at Documentation/media/v4l-drivers/vimc.dot and it's rendered at this
page: https://www.kernel.org/doc/html/latest/media/v4l-drivers/vimc.html

Quoted 7 lines

Quoted 3 lines
Topology shouldn't have changed. No changes to links or pads etc. I will take a look to be sure. I agree that if topology changes document should be updated.
If you "diff" the current dot with the dot you generated, you will see some differences. The main difference is that "RGB/YUV Input" was a device "/dev/video2/", and now it a subdevice "/dev/v4l-subdev4".

hmm, I just generated the topology for media/master, and it is /dev/v4l-subdev4. As we don't have an implementation of the output device yet, we used the sensor as a place holder and that is why it appears as "/dev/v4l-subdev4", what is in the docs is the ideal version after we get the output merged. We should update the docs in any case.

Regards,
Helen

Quoted 5 lines

Quoted 4 lines
thanks,
-- Shuah

Shuah Khan

author

2019-08-12 14:08 UTC #17 lore raw

Quoted 130 lines · Helen Koike

Hi Andre,
Thanks for testing this.
Quoted 73 lines · André Almeida
Quoted 72 lines · Shuah Khan
Hi Andre,
Quoted 68 lines · André Almeida
Hello Shuah,
Thanks for the patch, I did some comments below.
Quoted 57 lines · Shuah Khan
vimc uses Component API to split the driver into functional components. The real hardware resembles a monolith structure than component and component structure added a level of complexity making it hard to maintain without adding any real benefit.
     The sensor is one vimc component that would makes sense to be a
separate
module to closely align with the real hardware. It would be easier to collapse vimc into single monolithic driver first and then split the sensor off as a separate module.
This patch series emoves the component API and makes minimal changes to the code base preserving the functional division of the code structure. Preserving the functional structure allows us to split the sensor off as a separate module in the future.
Major design elements in this change are:
     - Use existing struct vimc_ent_config and struct
vimc_pipeline_config
       to drive the initialization of the functional components.
     - Make vimc_ent_config global by moving it to vimc.h
     - Add two new hooks add and rm to initialize and register,
unregister
       and free subdevs.
     - All component API is now gone and bind and unbind hooks are
modified
       to do "add" and "rm" with minimal changes to just add and rm
subdevs.
     - vimc-core's bind and unbind are now register and unregister.
     - vimc-core invokes "add" hooks from its vimc_register_devices(). The "add" hooks remain the same and register subdevs. They don't create platform devices of their own and use vimc's pdev.dev as their reference device. The "add" hooks save their
vimc_ent_device(s)
       in the corresponding vimc_ent_config.
     - vimc-core invokes "rm" hooks from its unregister to unregister
subdevs
       and cleanup.
     - vimc-core invokes "add" and "rm" hooks with pointer to struct
vimc_device
       and the corresponding struct vimc_ent_config pointer. The following configure and stream test works on all devices. media-ctl -d platform:vimc -V '"Sensor
A":0[fmt:SBGGR8_1X8/640x480]'
     media-ctl -d platform:vimc -V '"Debayer
A":0[fmt:SBGGR8_1X8/640x480]'
     media-ctl -d platform:vimc -V '"Sensor
B":0[fmt:SBGGR8_1X8/640x480]'
     media-ctl -d platform:vimc -V '"Debayer
B":0[fmt:SBGGR8_1X8/640x480]'
          v4l2-ctl -z platform:vimc -d "RGB/YUV Capture" -v
width=1920,height=1440
     v4l2-ctl -z platform:vimc -d "Raw Capture 0" -v pixelformat=BA81 v4l2-ctl -z platform:vimc -d "Raw Capture 1" -v pixelformat=BA81 v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video1 v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video2 v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video3
Quoted 2 lines
The third patch in the series fixes a general protection fault found when rmmod is done while stream is active.
I applied your patch on top of media_tree/master and I did some testing. Not sure if I did something wrong, but just adding and removing the module generated a kernel panic:
Quoted 51 lines
Quoted 5 lines
Thanks for testing.
Odd. I tested modprobe and rmmod both.I was working on Linux 5.3-rc2. I will apply these to media latest and work from there. I have to rebase these on top of the reverts from Lucas and Helen
Ok, please let me know if I succeeded to reproduce.
Quoted 43 lines
Quoted 43 lines
~# modprobe vimc
~# rmmod vimc
[   16.452974] stack segment: 0000 [#1] SMP PTI
[   16.453688] CPU: 0 PID: 2038 Comm: rmmod Not tainted 5.3.0-rc2+ #36 [   16.454678] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-20181126_142135-anatol 04/01/2014
[   16.456191] RIP: 0010:kfree+0x4d/0x240
<registers values...>
[   16.469188] Call Trace:
[   16.469666] vimc_remove+0x35/0x90 [vimc]
[   16.470436] platform_drv_remove+0x1f/0x40
[   16.471233] device_release_driver_internal+0xd3/0x1b0 [   16.472184] driver_detach+0x37/0x6b
[   16.472882] bus_remove_driver+0x50/0xc1
[   16.473569] vimc_exit+0xc/0xca0 [vimc]
[   16.474231] __x64_sys_delete_module+0x18d/0x240 [   16.475036] do_syscall_64+0x43/0x110
[   16.475656] entry_SYSCALL_64_after_hwframe+0x44/0xa9 [   16.476504] RIP: 0033:0x7fceb8dafa4b
<registers values...>
[   16.484853] Modules linked in: vimc(-) videobuf2_vmalloc videobuf2_memops v4l2_tpg videobuf2_v4l2 videobuf2_common [   16.486187] ---[ end trace 91e5e0894e254d49 ]--- [   16.486758] RIP: 0010:kfree+0x4d/0x240
<registers values...>
fish: “rmmod vimc” terminated by signal SIGSEGV (Address boundary error)
I just added the module after booting, no other action was made. Here is how my git log --oneline looks like:
897d708e922b media: vimc: Fix gpf in rmmod path when stream is active 2e4a5ad8ad6d media: vimc: Collapse component structure into a single monolithic driver
7c8da1687e92 media: vimc: move private defines to a common header 97299a303532 media: Remove dev_err() usage after platform_get_irq() 25a3d6bac6b9 media: adv7511/cobalt: rename driver name to adv7511-v4l2

Quoted 8 lines

I couldn't reproduce the error, my tree looks the same:
[I] koike@floko ~/m/o/linux> git log --oneline
e3345155c8ed (HEAD) media: vimc: Fix gpf in rmmod path when stream is active 43e9e2fe761f media: vimc: Collapse component structure into a single monolithic driver 8a6d0b9adde0 media: vimc: move private defines to a common header 97299a303532 (media/master) media: Remove dev_err() usage after platform_get_irq() 25a3d6bac6b9 media: adv7511/cobalt: rename driver name to adv7511-v4l2

Thanks Helen for trying to reproduce and sharing the result.

Quoted 43 lines

André, is this deterministic? Or it just happens sometimes?
Quoted 40 lines
Quoted 40 lines
Quoted 39 lines
...
Quoted 23 lines
vimc_print_dot (--print-dot) topology after this change:
digraph board {
    rankdir=TB
    n00000001 [label="{{} | Sensor A\n/dev/v4l-subdev0 | {<port0>
0}}", shape=Mrecord, style=filled, fillcolor=green]
    n00000001:port0 -> n00000005:port0 [style=bold] n00000001:port0 -> n0000000b [style=bold]
    n00000003 [label="{{} | Sensor B\n/dev/v4l-subdev1 | {<port0>
0}}", shape=Mrecord, style=filled, fillcolor=green]
    n00000003:port0 -> n00000008:port0 [style=bold] n00000003:port0 -> n0000000f [style=bold]
    n00000005 [label="{{<port0> 0} | Debayer A\n/dev/v4l-subdev2 |
{<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
    n00000005:port1 -> n00000015:port0
    n00000008 [label="{{<port0> 0} | Debayer B\n/dev/v4l-subdev3 |
{<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
    n00000008:port1 -> n00000015:port0 [style=dashed] n0000000b [label="Raw Capture 0\n/dev/video1", shape=box,
style=filled, fillcolor=yellow]
    n0000000f [label="Raw Capture 1\n/dev/video2", shape=box,
style=filled, fillcolor=yellow]
    n00000013 [label="{{} | RGB/YUV Input\n/dev/v4l-subdev4 |
Quoted 8 lines
{<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green]
    n00000013:port0 -> n00000015:port0 [style=dashed] n00000015 [label="{{<port0> 0} | Scaler\n/dev/v4l-subdev5 |
{<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
    n00000015:port1 -> n00000018 [style=bold] n00000018 [label="RGB/YUV Capture\n/dev/video3", shape=box,
style=filled, fillcolor=yellow]
}
Since the topology changed, it would be nice to change in the documentation as well. The current dot file can be found at Documentation/media/v4l-drivers/vimc.dot and it's rendered at this
page: https://www.kernel.org/doc/html/latest/media/v4l-drivers/vimc.html

Thanks Andre! Yes this is the one I am using as a reference.

Quoted 8 lines

Quoted 7 lines
Quoted 3 lines
Topology shouldn't have changed. No changes to links or pads etc. I will take a look to be sure. I agree that if topology changes document should be updated.
If you "diff" the current dot with the dot you generated, you will see some differences. The main difference is that "RGB/YUV Input" was a device "/dev/video2/", and now it a subdevice "/dev/v4l-subdev4".

Quoted 6 lines

hmm, I just generated the topology for media/master, and it is /dev/v4l-subdev4. As we don't have an implementation of the output device yet, we used the sensor as a place holder and that is why it appears as "/dev/v4l-subdev4", what is in the docs is the ideal version after we get the output merged. We should update the docs in any case.

Helen! Does the master match what I generated. In any case, I will do the diff of before and after my patches for sure.

I can update the document with what it should be.

thanks,
-- Shuah

André Almeida

2019-08-12 18:53 UTC #18 lore raw

Hi Shuah,

Quoted 155 lines · Shuah Khan

Quoted 144 lines · Helen Koike
Hi Andre,
Thanks for testing this.
Quoted 83 lines · André Almeida
Quoted 82 lines · Shuah Khan
Hi Andre,
Quoted 78 lines · André Almeida
Hello Shuah,
Thanks for the patch, I did some comments below.
Quoted 66 lines · Shuah Khan
vimc uses Component API to split the driver into functional components.
The real hardware resembles a monolith structure than component and component structure added a level of complexity making it hard to maintain without adding any real benefit.
      The sensor is one vimc component that would makes sense to be a separate
module to closely align with the real hardware. It would be easier to collapse vimc into single monolithic driver first and then split the sensor off as a separate module.
This patch series emoves the component API and makes minimal changes to
the code base preserving the functional division of the code structure.
Preserving the functional structure allows us to split the sensor off as a separate module in the future.
Major design elements in this change are:
      - Use existing struct vimc_ent_config and struct vimc_pipeline_config
        to drive the initialization of the functional components.
      - Make vimc_ent_config global by moving it to vimc.h
      - Add two new hooks add and rm to initialize and register, unregister
        and free subdevs.
      - All component API is now gone and bind and unbind hooks are modified
        to do "add" and "rm" with minimal changes to just add and rm subdevs.
      - vimc-core's bind and unbind are now register and unregister.
      - vimc-core invokes "add" hooks from its vimc_register_devices().
        The "add" hooks remain the same and register subdevs. They don't
        create platform devices of their own and use vimc's pdev.dev as
        their reference device. The "add" hooks save their vimc_ent_device(s)
        in the corresponding vimc_ent_config.
      - vimc-core invokes "rm" hooks from its unregister to unregister
subdevs
        and cleanup.
      - vimc-core invokes "add" and "rm" hooks with pointer to struct vimc_device
        and the corresponding struct vimc_ent_config pointer. The following configure and stream test works on all devices. media-ctl -d platform:vimc -V '"Sensor A":0[fmt:SBGGR8_1X8/640x480]'
      media-ctl -d platform:vimc -V '"Debayer A":0[fmt:SBGGR8_1X8/640x480]'
      media-ctl -d platform:vimc -V '"Sensor B":0[fmt:SBGGR8_1X8/640x480]'
      media-ctl -d platform:vimc -V '"Debayer B":0[fmt:SBGGR8_1X8/640x480]'
           v4l2-ctl -z platform:vimc -d "RGB/YUV Capture" -v width=1920,height=1440
      v4l2-ctl -z platform:vimc -d "Raw Capture 0" -v pixelformat=BA81
      v4l2-ctl -z platform:vimc -d "Raw Capture 1" -v pixelformat=BA81
           v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video1 v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video2 v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video3
Quoted 2 lines
The third patch in the series fixes a general protection fault found when rmmod is done while stream is active.
I applied your patch on top of media_tree/master and I did some testing.
Not sure if I did something wrong, but just adding and removing the module generated a kernel panic:
Quoted 53 lines
Quoted 5 lines
Thanks for testing.
Odd. I tested modprobe and rmmod both.I was working on Linux 5.3-rc2. I will apply these to media latest and work from there. I have to rebase these on top of the reverts from Lucas and Helen
Ok, please let me know if I succeeded to reproduce.
Quoted 45 lines
Quoted 45 lines
~# modprobe vimc
~# rmmod vimc
[   16.452974] stack segment: 0000 [#1] SMP PTI
[   16.453688] CPU: 0 PID: 2038 Comm: rmmod Not tainted 5.3.0-rc2+ #36 [   16.454678] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-20181126_142135-anatol 04/01/2014
[   16.456191] RIP: 0010:kfree+0x4d/0x240
<registers values...>
[   16.469188] Call Trace:
[   16.469666] vimc_remove+0x35/0x90 [vimc]
[   16.470436] platform_drv_remove+0x1f/0x40
[   16.471233] device_release_driver_internal+0xd3/0x1b0 [   16.472184] driver_detach+0x37/0x6b
[   16.472882] bus_remove_driver+0x50/0xc1
[   16.473569] vimc_exit+0xc/0xca0 [vimc]
[   16.474231] __x64_sys_delete_module+0x18d/0x240 [   16.475036] do_syscall_64+0x43/0x110
[   16.475656] entry_SYSCALL_64_after_hwframe+0x44/0xa9 [   16.476504] RIP: 0033:0x7fceb8dafa4b
<registers values...>
[   16.484853] Modules linked in: vimc(-) videobuf2_vmalloc videobuf2_memops v4l2_tpg videobuf2_v4l2 videobuf2_common [   16.486187] ---[ end trace 91e5e0894e254d49 ]--- [   16.486758] RIP: 0010:kfree+0x4d/0x240
<registers values...>
fish: “rmmod vimc” terminated by signal SIGSEGV (Address boundary error)
I just added the module after booting, no other action was made. Here is
how my git log --oneline looks like:
897d708e922b media: vimc: Fix gpf in rmmod path when stream is active 2e4a5ad8ad6d media: vimc: Collapse component structure into a single monolithic driver
7c8da1687e92 media: vimc: move private defines to a common header 97299a303532 media: Remove dev_err() usage after platform_get_irq() 25a3d6bac6b9 media: adv7511/cobalt: rename driver name to adv7511-v4l2
I couldn't reproduce the error, my tree looks the same:
Quoted 9 lines
[I] koike@floko ~/m/o/linux> git log --oneline
e3345155c8ed (HEAD) media: vimc: Fix gpf in rmmod path when stream is active
43e9e2fe761f media: vimc: Collapse component structure into a single monolithic driver
8a6d0b9adde0 media: vimc: move private defines to a common header 97299a303532 (media/master) media: Remove dev_err() usage after platform_get_irq()
25a3d6bac6b9 media: adv7511/cobalt: rename driver name to adv7511-v4l2

Quoted 1 line

Thanks Helen for trying to reproduce and sharing the result.

Me and Helen found out what is the problem. If you follow this call trace:

vimc_ent_sd_unregister()
v4l2_device_unregister_subdev()
v4l2_subdev_release()

You'll notice that this last function calls the release callback implementation of the subdevice. For instance, the release of vimc-sensor is this one:

static void vimc_sen_release(struct v4l2_subdev *sd)

{
	struct vimc_sen_device *vsen =
				container_of(sd, struct vimc_sen_device, sd);

	v4l2_ctrl_handler_free(&vsen->hdl);
	tpg_free(&vsen->tpg);
	kfree(vsen);
}

And then you can see that vsen has been freed. Back to vimc_ent_sd_unregister(), after v4l2_device_unregister_subdev(), the function will call vimc_pads_cleanup(). This is basically a kfree(ved->pads), but ved has just been freed at v4l2_subdev_release(), producing a memory fault.

To fix that, we found two options:

- place the kfree(ved->pads) inside the release callback of each subdevice and removing vimc_pads_cleanup() from vimc_ent_sd_unregister()
- use a auxiliary variable to hold the address of the pads, for instance:

void vimc_ent_sd_unregister(...)

{
    struct media_pad *pads = ved->pads;
    ...
    vimc_pads_cleanup(pads);
}

Thanks,
André

Quoted 70 lines

Quoted 44 lines
André, is this deterministic? Or it just happens sometimes?
Quoted 41 lines
Quoted 41 lines
Quoted 40 lines
...
Quoted 23 lines
vimc_print_dot (--print-dot) topology after this change:
digraph board {
     rankdir=TB
     n00000001 [label="{{} | Sensor A\n/dev/v4l-subdev0 | {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green] n00000001:port0 -> n00000005:port0 [style=bold] n00000001:port0 -> n0000000b [style=bold]
     n00000003 [label="{{} | Sensor B\n/dev/v4l-subdev1 | {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green] n00000003:port0 -> n00000008:port0 [style=bold] n00000003:port0 -> n0000000f [style=bold]
     n00000005 [label="{{<port0> 0} | Debayer A\n/dev/v4l-subdev2 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green] n00000005:port1 -> n00000015:port0
     n00000008 [label="{{<port0> 0} | Debayer B\n/dev/v4l-subdev3 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green] n00000008:port1 -> n00000015:port0 [style=dashed] n0000000b [label="Raw Capture 0\n/dev/video1", shape=box, style=filled, fillcolor=yellow]
     n0000000f [label="Raw Capture 1\n/dev/video2", shape=box, style=filled, fillcolor=yellow]
     n00000013 [label="{{} | RGB/YUV Input\n/dev/v4l-subdev4 |
Quoted 8 lines
{<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green] n00000013:port0 -> n00000015:port0 [style=dashed] n00000015 [label="{{<port0> 0} | Scaler\n/dev/v4l-subdev5 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green] n00000015:port1 -> n00000018 [style=bold] n00000018 [label="RGB/YUV Capture\n/dev/video3", shape=box, style=filled, fillcolor=yellow]
}
Since the topology changed, it would be nice to change in the documentation as well. The current dot file can be found at Documentation/media/v4l-drivers/vimc.dot and it's rendered at this page:
https://www.kernel.org/doc/html/latest/media/v4l-drivers/vimc.html
Thanks Andre! Yes this is the one I am using as a reference.
Quoted 8 lines
Quoted 7 lines
Quoted 3 lines
Topology shouldn't have changed. No changes to links or pads etc. I will take a look to be sure. I agree that if topology changes document should be updated.
If you "diff" the current dot with the dot you generated, you will see some differences. The main difference is that "RGB/YUV Input" was a device "/dev/video2/", and now it a subdevice "/dev/v4l-subdev4".
Quoted 6 lines
hmm, I just generated the topology for media/master, and it is /dev/v4l-subdev4. As we don't have an implementation of the output device yet, we used the sensor as a place holder and that is why it appears as "/dev/v4l-subdev4", what is in the docs is the ideal version after we get the output merged. We should update the docs in any case.
Helen! Does the master match what I generated. In any case, I will do the diff of before and after my patches for sure.
I can update the document with what it should be.
thanks,
-- Shuah

Shuah Khan

author

2019-08-12 19:10 UTC #19 lore raw

Quoted 193 lines · André Almeida

Hi Shuah,
Quoted 155 lines · Shuah Khan
Quoted 144 lines · Helen Koike
Hi Andre,
Thanks for testing this.
Quoted 83 lines · André Almeida
Quoted 82 lines · Shuah Khan
Hi Andre,
Quoted 78 lines · André Almeida
Hello Shuah,
Thanks for the patch, I did some comments below.
Quoted 66 lines · Shuah Khan
vimc uses Component API to split the driver into functional components.
The real hardware resembles a monolith structure than component and component structure added a level of complexity making it hard to maintain without adding any real benefit.
      The sensor is one vimc component that would makes sense to be a
separate
module to closely align with the real hardware. It would be easier to collapse vimc into single monolithic driver first and then split the sensor off as a separate module.
This patch series emoves the component API and makes minimal changes to
the code base preserving the functional division of the code structure.
Preserving the functional structure allows us to split the sensor off as a separate module in the future.
Major design elements in this change are:
      - Use existing struct vimc_ent_config and struct
vimc_pipeline_config
        to drive the initialization of the functional components.
      - Make vimc_ent_config global by moving it to vimc.h
      - Add two new hooks add and rm to initialize and register,
unregister
        and free subdevs.
      - All component API is now gone and bind and unbind hooks are
modified
        to do "add" and "rm" with minimal changes to just add and rm
subdevs.
      - vimc-core's bind and unbind are now register and unregister.
      - vimc-core invokes "add" hooks from its
vimc_register_devices().
        The "add" hooks remain the same and register subdevs. They
don't
        create platform devices of their own and use vimc's
pdev.dev as
        their reference device. The "add" hooks save their
vimc_ent_device(s)
        in the corresponding vimc_ent_config.
      - vimc-core invokes "rm" hooks from its unregister to
unregister
subdevs
        and cleanup.
      - vimc-core invokes "add" and "rm" hooks with pointer to struct
vimc_device
        and the corresponding struct vimc_ent_config pointer. The following configure and stream test works on all devices. media-ctl -d platform:vimc -V '"Sensor
A":0[fmt:SBGGR8_1X8/640x480]'
      media-ctl -d platform:vimc -V '"Debayer
A":0[fmt:SBGGR8_1X8/640x480]'
      media-ctl -d platform:vimc -V '"Sensor
B":0[fmt:SBGGR8_1X8/640x480]'
      media-ctl -d platform:vimc -V '"Debayer
B":0[fmt:SBGGR8_1X8/640x480]'
           v4l2-ctl -z platform:vimc -d "RGB/YUV Capture" -v
width=1920,height=1440
      v4l2-ctl -z platform:vimc -d "Raw Capture 0" -v
pixelformat=BA81
      v4l2-ctl -z platform:vimc -d "Raw Capture 1" -v
pixelformat=BA81
           v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video1 v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video2 v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video3
Quoted 2 lines
The third patch in the series fixes a general protection fault found when rmmod is done while stream is active.
I applied your patch on top of media_tree/master and I did some testing.
Not sure if I did something wrong, but just adding and removing the module generated a kernel panic:
Quoted 53 lines
Quoted 5 lines
Thanks for testing.
Odd. I tested modprobe and rmmod both.I was working on Linux 5.3-rc2. I will apply these to media latest and work from there. I have to rebase these on top of the reverts from Lucas and Helen
Ok, please let me know if I succeeded to reproduce.
Quoted 45 lines
Quoted 45 lines
~# modprobe vimc
~# rmmod vimc
[   16.452974] stack segment: 0000 [#1] SMP PTI
[   16.453688] CPU: 0 PID: 2038 Comm: rmmod Not tainted 5.3.0-rc2+ #36 [   16.454678] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-20181126_142135-anatol 04/01/2014
[   16.456191] RIP: 0010:kfree+0x4d/0x240
<registers values...>
[   16.469188] Call Trace:
[   16.469666] vimc_remove+0x35/0x90 [vimc]
[   16.470436] platform_drv_remove+0x1f/0x40
[   16.471233] device_release_driver_internal+0xd3/0x1b0 [   16.472184] driver_detach+0x37/0x6b
[   16.472882] bus_remove_driver+0x50/0xc1
[   16.473569] vimc_exit+0xc/0xca0 [vimc]
[   16.474231] __x64_sys_delete_module+0x18d/0x240 [   16.475036] do_syscall_64+0x43/0x110
[   16.475656] entry_SYSCALL_64_after_hwframe+0x44/0xa9 [   16.476504] RIP: 0033:0x7fceb8dafa4b
<registers values...>
[   16.484853] Modules linked in: vimc(-) videobuf2_vmalloc videobuf2_memops v4l2_tpg videobuf2_v4l2 videobuf2_common [   16.486187] ---[ end trace 91e5e0894e254d49 ]--- [   16.486758] RIP: 0010:kfree+0x4d/0x240
<registers values...>
fish: “rmmod vimc” terminated by signal SIGSEGV (Address boundary error)
I just added the module after booting, no other action was made. Here is
how my git log --oneline looks like:
897d708e922b media: vimc: Fix gpf in rmmod path when stream is active 2e4a5ad8ad6d media: vimc: Collapse component structure into a single monolithic driver
7c8da1687e92 media: vimc: move private defines to a common header 97299a303532 media: Remove dev_err() usage after platform_get_irq() 25a3d6bac6b9 media: adv7511/cobalt: rename driver name to adv7511-v4l2
I couldn't reproduce the error, my tree looks the same:
Quoted 9 lines
[I] koike@floko ~/m/o/linux> git log --oneline
e3345155c8ed (HEAD) media: vimc: Fix gpf in rmmod path when stream is active
43e9e2fe761f media: vimc: Collapse component structure into a single monolithic driver
8a6d0b9adde0 media: vimc: move private defines to a common header 97299a303532 (media/master) media: Remove dev_err() usage after platform_get_irq()
25a3d6bac6b9 media: adv7511/cobalt: rename driver name to adv7511-v4l2
Quoted 1 line
Thanks Helen for trying to reproduce and sharing the result.
Me and Helen found out what is the problem. If you follow this call trace:
vimc_ent_sd_unregister()
v4l2_device_unregister_subdev()
v4l2_subdev_release()
You'll notice that this last function calls the release callback implementation of the subdevice. For instance, the release of vimc-sensor is this one:
static void vimc_sen_release(struct v4l2_subdev *sd)
{
	struct vimc_sen_device *vsen =
				container_of(sd, struct vimc_sen_device, sd);

	v4l2_ctrl_handler_free(&vsen->hdl);
	tpg_free(&vsen->tpg);
	kfree(vsen);
}
And then you can see that vsen has been freed. Back to vimc_ent_sd_unregister(), after v4l2_device_unregister_subdev(), the function will call vimc_pads_cleanup(). This is basically a kfree(ved->pads), but ved has just been freed at v4l2_subdev_release(), producing a memory fault.
To fix that, we found two options:
- place the kfree(ved->pads) inside the release callback of each subdevice and removing vimc_pads_cleanup() from vimc_ent_sd_unregister()
- use a auxiliary variable to hold the address of the pads, for instance:

Quoted 8 lines

void vimc_ent_sd_unregister(...)

{
     struct media_pad *pads = ved->pads;
     ...
     vimc_pads_cleanup(pads);
}

I fixed a problem in the thirds patch. vimc-capture uses the first approach - placing the kfree(ved->pads) inside the release callback.

I am debugging another such problem in unbind path while streaming. I am working on v2 and I will look for the rmmod problem and fix it.

thanks again for testing and finding the root cause. -- Shuah

Shuah Khan

author

2019-08-12 22:14 UTC #20 lore raw

Quoted 203 lines · Shuah Khan

Quoted 193 lines · André Almeida
Hi Shuah,
Quoted 155 lines · Shuah Khan
Quoted 144 lines · Helen Koike
Hi Andre,
Thanks for testing this.
Quoted 83 lines · André Almeida
Quoted 82 lines · Shuah Khan
Hi Andre,
Quoted 78 lines · André Almeida
Hello Shuah,
Thanks for the patch, I did some comments below.
Quoted 66 lines · Shuah Khan
vimc uses Component API to split the driver into functional components.
The real hardware resembles a monolith structure than component and component structure added a level of complexity making it hard to maintain without adding any real benefit.
       The sensor is one vimc component that would makes sense to be a separate
module to closely align with the real hardware. It would be easier to collapse vimc into single monolithic driver first and then split the sensor off as a separate module.
This patch series emoves the component API and makes minimal changes to
the code base preserving the functional division of the code structure.
Preserving the functional structure allows us to split the sensor off as a separate module in the future.
Major design elements in this change are:
       - Use existing struct vimc_ent_config and struct vimc_pipeline_config
         to drive the initialization of the functional components.
       - Make vimc_ent_config global by moving it to vimc.h
       - Add two new hooks add and rm to initialize and register, unregister
         and free subdevs.
       - All component API is now gone and bind and unbind hooks are modified
         to do "add" and "rm" with minimal changes to just add and rm subdevs.
       - vimc-core's bind and unbind are now register and unregister.
       - vimc-core invokes "add" hooks from its vimc_register_devices().
         The "add" hooks remain the same and register subdevs. They don't
         create platform devices of their own and use vimc's pdev.dev as
         their reference device. The "add" hooks save their vimc_ent_device(s)
         in the corresponding vimc_ent_config.
       - vimc-core invokes "rm" hooks from its unregister to unregister
subdevs
         and cleanup.
       - vimc-core invokes "add" and "rm" hooks with pointer to struct vimc_device
         and the corresponding struct vimc_ent_config pointer. The following configure and stream test works on all devices. media-ctl -d platform:vimc -V '"Sensor A":0[fmt:SBGGR8_1X8/640x480]'
       media-ctl -d platform:vimc -V '"Debayer A":0[fmt:SBGGR8_1X8/640x480]'
       media-ctl -d platform:vimc -V '"Sensor B":0[fmt:SBGGR8_1X8/640x480]'
       media-ctl -d platform:vimc -V '"Debayer B":0[fmt:SBGGR8_1X8/640x480]'
            v4l2-ctl -z platform:vimc -d "RGB/YUV Capture" -v width=1920,height=1440
       v4l2-ctl -z platform:vimc -d "Raw Capture 0" -v pixelformat=BA81
       v4l2-ctl -z platform:vimc -d "Raw Capture 1" -v pixelformat=BA81
            v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video1 v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video2 v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video3
Quoted 2 lines
The third patch in the series fixes a general protection fault found when rmmod is done while stream is active.
I applied your patch on top of media_tree/master and I did some testing.
Not sure if I did something wrong, but just adding and removing the module generated a kernel panic:
Quoted 53 lines
Quoted 5 lines
Thanks for testing.
Odd. I tested modprobe and rmmod both.I was working on Linux 5.3-rc2. I will apply these to media latest and work from there. I have to rebase these on top of the reverts from Lucas and Helen
Ok, please let me know if I succeeded to reproduce.
Quoted 45 lines
Quoted 45 lines
~# modprobe vimc
~# rmmod vimc
[   16.452974] stack segment: 0000 [#1] SMP PTI
[   16.453688] CPU: 0 PID: 2038 Comm: rmmod Not tainted 5.3.0-rc2+ #36 [   16.454678] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-20181126_142135-anatol 04/01/2014
[   16.456191] RIP: 0010:kfree+0x4d/0x240
<registers values...>
[   16.469188] Call Trace:
[   16.469666] vimc_remove+0x35/0x90 [vimc]
[   16.470436] platform_drv_remove+0x1f/0x40
[   16.471233] device_release_driver_internal+0xd3/0x1b0 [   16.472184] driver_detach+0x37/0x6b
[   16.472882] bus_remove_driver+0x50/0xc1
[   16.473569] vimc_exit+0xc/0xca0 [vimc]
[   16.474231] __x64_sys_delete_module+0x18d/0x240 [   16.475036] do_syscall_64+0x43/0x110
[   16.475656] entry_SYSCALL_64_after_hwframe+0x44/0xa9 [   16.476504] RIP: 0033:0x7fceb8dafa4b
<registers values...>
[   16.484853] Modules linked in: vimc(-) videobuf2_vmalloc videobuf2_memops v4l2_tpg videobuf2_v4l2 videobuf2_common [   16.486187] ---[ end trace 91e5e0894e254d49 ]--- [   16.486758] RIP: 0010:kfree+0x4d/0x240
<registers values...>
fish: “rmmod vimc” terminated by signal SIGSEGV (Address boundary error)
I just added the module after booting, no other action was made. Here is
how my git log --oneline looks like:
897d708e922b media: vimc: Fix gpf in rmmod path when stream is active 2e4a5ad8ad6d media: vimc: Collapse component structure into a single monolithic driver
7c8da1687e92 media: vimc: move private defines to a common header 97299a303532 media: Remove dev_err() usage after platform_get_irq() 25a3d6bac6b9 media: adv7511/cobalt: rename driver name to adv7511-v4l2
I couldn't reproduce the error, my tree looks the same:
Quoted 9 lines
[I] koike@floko ~/m/o/linux> git log --oneline
e3345155c8ed (HEAD) media: vimc: Fix gpf in rmmod path when stream is active
43e9e2fe761f media: vimc: Collapse component structure into a single monolithic driver
8a6d0b9adde0 media: vimc: move private defines to a common header 97299a303532 (media/master) media: Remove dev_err() usage after platform_get_irq()
25a3d6bac6b9 media: adv7511/cobalt: rename driver name to adv7511-v4l2
Quoted 1 line
Thanks Helen for trying to reproduce and sharing the result.
Me and Helen found out what is the problem. If you follow this call trace:
vimc_ent_sd_unregister()
v4l2_device_unregister_subdev()
v4l2_subdev_release()
You'll notice that this last function calls the release callback implementation of the subdevice. For instance, the release of vimc-sensor is this one:
static void vimc_sen_release(struct v4l2_subdev *sd) {
    struct vimc_sen_device *vsen =
                container_of(sd, struct vimc_sen_device, sd);
    v4l2_ctrl_handler_free(&vsen->hdl);
    tpg_free(&vsen->tpg);
    kfree(vsen);
}
And then you can see that vsen has been freed. Back to vimc_ent_sd_unregister(), after v4l2_device_unregister_subdev(), the function will call vimc_pads_cleanup(). This is basically a kfree(ved->pads), but ved has just been freed at v4l2_subdev_release(), producing a memory fault.
To fix that, we found two options:
- place the kfree(ved->pads) inside the release callback of each subdevice and removing vimc_pads_cleanup() from vimc_ent_sd_unregister()
- use a auxiliary variable to hold the address of the pads, for instance:
Quoted 8 lines
void vimc_ent_sd_unregister(...)
{
     struct media_pad *pads = ved->pads;
     ...
     vimc_pads_cleanup(pads);
}

Quoted 8 lines

I fixed a problem in the thirds patch. vimc-capture uses the first approach - placing the kfree(ved->pads) inside the release callback.
I am debugging another such problem in unbind path while streaming. I am working on v2 and I will look for the rmmod problem and fix it.
thanks again for testing and finding the root cause. -- Shuah

Hi Andre,

Here is what's happening.

Before this change, you can't really do rmmod vimc, because vimc is in use by other component drivers. With the collapse, now you can actually do rmmod on vimc and this problem in vimc_ent_sd_unregister() that frees pads first and the does v4l2_device_unregister_subdev().

I fixed this in the 3/3 patch. I can reproduce the problem with patches 1 and 2, and patch 3 fixes it.

Did you test with the third patch in this series?

thanks,
-- Shuah

Helen Koike

2019-08-12 23:41 UTC #21 lore raw

Hi Shuah,

Quoted 217 lines · Shuah Khan

Quoted 203 lines · Shuah Khan
Quoted 193 lines · André Almeida
Hi Shuah,
Quoted 155 lines · Shuah Khan
Quoted 144 lines · Helen Koike
Hi Andre,
Thanks for testing this.
Quoted 83 lines · André Almeida
Quoted 82 lines · Shuah Khan
Hi Andre,
Quoted 78 lines · André Almeida
Hello Shuah,
Thanks for the patch, I did some comments below.
Quoted 66 lines · Shuah Khan
vimc uses Component API to split the driver into functional components.
The real hardware resembles a monolith structure than component and component structure added a level of complexity making it hard to maintain without adding any real benefit.
       The sensor is one vimc component that would makes sense to be a separate
module to closely align with the real hardware. It would be easier to collapse vimc into single monolithic driver first and then split the sensor off as a separate module.
This patch series emoves the component API and makes minimal changes to
the code base preserving the functional division of the code structure.
Preserving the functional structure allows us to split the sensor off as a separate module in the future.
Major design elements in this change are:
       - Use existing struct vimc_ent_config and struct vimc_pipeline_config
         to drive the initialization of the functional components.
       - Make vimc_ent_config global by moving it to vimc.h
       - Add two new hooks add and rm to initialize and register, unregister
         and free subdevs.
       - All component API is now gone and bind and unbind hooks are modified
         to do "add" and "rm" with minimal changes to just add and rm subdevs.
       - vimc-core's bind and unbind are now register and unregister.
       - vimc-core invokes "add" hooks from its vimc_register_devices().
         The "add" hooks remain the same and register subdevs. They don't
         create platform devices of their own and use vimc's pdev.dev as
         their reference device. The "add" hooks save their vimc_ent_device(s)
         in the corresponding vimc_ent_config.
       - vimc-core invokes "rm" hooks from its unregister to unregister
subdevs
         and cleanup.
       - vimc-core invokes "add" and "rm" hooks with pointer to struct vimc_device
         and the corresponding struct vimc_ent_config pointer. The following configure and stream test works on all devices. media-ctl -d platform:vimc -V '"Sensor A":0[fmt:SBGGR8_1X8/640x480]'
       media-ctl -d platform:vimc -V '"Debayer A":0[fmt:SBGGR8_1X8/640x480]'
       media-ctl -d platform:vimc -V '"Sensor B":0[fmt:SBGGR8_1X8/640x480]'
       media-ctl -d platform:vimc -V '"Debayer B":0[fmt:SBGGR8_1X8/640x480]'
            v4l2-ctl -z platform:vimc -d "RGB/YUV Capture" -v width=1920,height=1440
       v4l2-ctl -z platform:vimc -d "Raw Capture 0" -v pixelformat=BA81
       v4l2-ctl -z platform:vimc -d "Raw Capture 1" -v pixelformat=BA81
            v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video1 v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video2 v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video3
Quoted 2 lines
The third patch in the series fixes a general protection fault found when rmmod is done while stream is active.
I applied your patch on top of media_tree/master and I did some testing.
Not sure if I did something wrong, but just adding and removing the module generated a kernel panic:
Quoted 53 lines
Quoted 5 lines
Thanks for testing.
Odd. I tested modprobe and rmmod both.I was working on Linux 5.3-rc2. I will apply these to media latest and work from there. I have to rebase these on top of the reverts from Lucas and Helen
Ok, please let me know if I succeeded to reproduce.
Quoted 45 lines
Quoted 45 lines
~# modprobe vimc
~# rmmod vimc
[   16.452974] stack segment: 0000 [#1] SMP PTI
[   16.453688] CPU: 0 PID: 2038 Comm: rmmod Not tainted 5.3.0-rc2+ #36 [   16.454678] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-20181126_142135-anatol 04/01/2014
[   16.456191] RIP: 0010:kfree+0x4d/0x240
<registers values...>
[   16.469188] Call Trace:
[   16.469666] vimc_remove+0x35/0x90 [vimc]
[   16.470436] platform_drv_remove+0x1f/0x40
[   16.471233] device_release_driver_internal+0xd3/0x1b0 [   16.472184] driver_detach+0x37/0x6b
[   16.472882] bus_remove_driver+0x50/0xc1
[   16.473569] vimc_exit+0xc/0xca0 [vimc]
[   16.474231] __x64_sys_delete_module+0x18d/0x240 [   16.475036] do_syscall_64+0x43/0x110
[   16.475656] entry_SYSCALL_64_after_hwframe+0x44/0xa9 [   16.476504] RIP: 0033:0x7fceb8dafa4b
<registers values...>
[   16.484853] Modules linked in: vimc(-) videobuf2_vmalloc videobuf2_memops v4l2_tpg videobuf2_v4l2 videobuf2_common [   16.486187] ---[ end trace 91e5e0894e254d49 ]--- [   16.486758] RIP: 0010:kfree+0x4d/0x240
<registers values...>
fish: “rmmod vimc” terminated by signal SIGSEGV (Address boundary error)
I just added the module after booting, no other action was made. Here is
how my git log --oneline looks like:
897d708e922b media: vimc: Fix gpf in rmmod path when stream is active 2e4a5ad8ad6d media: vimc: Collapse component structure into a single monolithic driver
7c8da1687e92 media: vimc: move private defines to a common header 97299a303532 media: Remove dev_err() usage after platform_get_irq() 25a3d6bac6b9 media: adv7511/cobalt: rename driver name to adv7511-v4l2
I couldn't reproduce the error, my tree looks the same:
Quoted 9 lines
[I] koike@floko ~/m/o/linux> git log --oneline
e3345155c8ed (HEAD) media: vimc: Fix gpf in rmmod path when stream is active
43e9e2fe761f media: vimc: Collapse component structure into a single monolithic driver
8a6d0b9adde0 media: vimc: move private defines to a common header 97299a303532 (media/master) media: Remove dev_err() usage after platform_get_irq()
25a3d6bac6b9 media: adv7511/cobalt: rename driver name to adv7511-v4l2
Quoted 1 line
Thanks Helen for trying to reproduce and sharing the result.
Me and Helen found out what is the problem. If you follow this call trace:
vimc_ent_sd_unregister()
v4l2_device_unregister_subdev()
v4l2_subdev_release()
You'll notice that this last function calls the release callback implementation of the subdevice. For instance, the release of vimc-sensor is this one:
static void vimc_sen_release(struct v4l2_subdev *sd) {
    struct vimc_sen_device *vsen =
                container_of(sd, struct vimc_sen_device, sd);
    v4l2_ctrl_handler_free(&vsen->hdl);
    tpg_free(&vsen->tpg);
    kfree(vsen);
}
And then you can see that vsen has been freed. Back to vimc_ent_sd_unregister(), after v4l2_device_unregister_subdev(), the function will call vimc_pads_cleanup(). This is basically a kfree(ved->pads), but ved has just been freed at v4l2_subdev_release(), producing a memory fault.
To fix that, we found two options:
- place the kfree(ved->pads) inside the release callback of each subdevice and removing vimc_pads_cleanup() from vimc_ent_sd_unregister()
- use a auxiliary variable to hold the address of the pads, for instance:
Quoted 8 lines
void vimc_ent_sd_unregister(...)
{
     struct media_pad *pads = ved->pads;
     ...
     vimc_pads_cleanup(pads);
}
Quoted 8 lines
I fixed a problem in the thirds patch. vimc-capture uses the first approach - placing the kfree(ved->pads) inside the release callback.
I am debugging another such problem in unbind path while streaming. I am working on v2 and I will look for the rmmod problem and fix it.
thanks again for testing and finding the root cause. -- Shuah
Hi Andre,
Here is what's happening.

Quoted 8 lines

Before this change, you can't really do rmmod vimc, because vimc is in use by other component drivers. With the collapse, now you can actually do rmmod on vimc and this problem in vimc_ent_sd_unregister() that frees pads first and the does v4l2_device_unregister_subdev().
I fixed this in the 3/3 patch. I can reproduce the problem with patches 1 and 2, and patch 3 fixes it.
Did you test with the third patch in this series?

yes, we tested with 3/3, but the new problem now is when doing the following in this order:

    v4l2_device_unregister_subdev(sd);
    vimc_pads_cleanup(ved->pads);

v4l2_device_unregister_subdev() calls the release function of the subdevice that frees the ved object, so ved->pads is not valid anymore. That is why André suggested a temporary variable to hold ved->pads and to be able to free it later:

    struct media_pad *pads = ved->pads;

    v4l2_device_unregister_subdev(sd);
    vimc_pads_cleanup(pads); // So we don't use the ved object here anymore.

Regards,
Helen

Quoted 3 lines

thanks,
-- Shuah

Shuah Khan

author

2019-08-13 00:58 UTC #22 lore raw

Quoted 236 lines · Helen Koike

Hi Shuah,
Quoted 217 lines · Shuah Khan
Quoted 203 lines · Shuah Khan
Quoted 193 lines · André Almeida
Hi Shuah,
Quoted 155 lines · Shuah Khan
Quoted 144 lines · Helen Koike
Hi Andre,
Thanks for testing this.
Quoted 83 lines · André Almeida
Quoted 82 lines · Shuah Khan
Hi Andre,
Quoted 78 lines · André Almeida
Hello Shuah,
Thanks for the patch, I did some comments below.
Quoted 66 lines · Shuah Khan
vimc uses Component API to split the driver into functional components.
The real hardware resembles a monolith structure than component and component structure added a level of complexity making it hard to maintain without adding any real benefit.
       The sensor is one vimc component that would makes sense to be a
separate
module to closely align with the real hardware. It would be easier to collapse vimc into single monolithic driver first and then split the sensor off as a separate module.
This patch series emoves the component API and makes minimal changes to
the code base preserving the functional division of the code structure.
Preserving the functional structure allows us to split the sensor off as a separate module in the future.
Major design elements in this change are:
       - Use existing struct vimc_ent_config and struct
vimc_pipeline_config
         to drive the initialization of the functional components.
       - Make vimc_ent_config global by moving it to vimc.h
       - Add two new hooks add and rm to initialize and register,
unregister
         and free subdevs.
       - All component API is now gone and bind and unbind hooks are
modified
         to do "add" and "rm" with minimal changes to just add and rm
subdevs.
       - vimc-core's bind and unbind are now register and unregister.
       - vimc-core invokes "add" hooks from its
vimc_register_devices().
         The "add" hooks remain the same and register subdevs. They
don't
         create platform devices of their own and use vimc's
pdev.dev as
         their reference device. The "add" hooks save their
vimc_ent_device(s)
         in the corresponding vimc_ent_config.
       - vimc-core invokes "rm" hooks from its unregister to
unregister
subdevs
         and cleanup.
       - vimc-core invokes "add" and "rm" hooks with pointer to struct
vimc_device
         and the corresponding struct vimc_ent_config pointer. The following configure and stream test works on all devices. media-ctl -d platform:vimc -V '"Sensor
A":0[fmt:SBGGR8_1X8/640x480]'
       media-ctl -d platform:vimc -V '"Debayer
A":0[fmt:SBGGR8_1X8/640x480]'
       media-ctl -d platform:vimc -V '"Sensor
B":0[fmt:SBGGR8_1X8/640x480]'
       media-ctl -d platform:vimc -V '"Debayer
B":0[fmt:SBGGR8_1X8/640x480]'
            v4l2-ctl -z platform:vimc -d "RGB/YUV Capture" -v
width=1920,height=1440
       v4l2-ctl -z platform:vimc -d "Raw Capture 0" -v
pixelformat=BA81
       v4l2-ctl -z platform:vimc -d "Raw Capture 1" -v
pixelformat=BA81
            v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video1 v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video2 v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video3
Quoted 2 lines
The third patch in the series fixes a general protection fault found when rmmod is done while stream is active.
I applied your patch on top of media_tree/master and I did some testing.
Not sure if I did something wrong, but just adding and removing the module generated a kernel panic:
Quoted 53 lines
Quoted 5 lines
Thanks for testing.
Odd. I tested modprobe and rmmod both.I was working on Linux 5.3-rc2. I will apply these to media latest and work from there. I have to rebase these on top of the reverts from Lucas and Helen
Ok, please let me know if I succeeded to reproduce.
Quoted 45 lines
Quoted 45 lines
~# modprobe vimc
~# rmmod vimc
[   16.452974] stack segment: 0000 [#1] SMP PTI
[   16.453688] CPU: 0 PID: 2038 Comm: rmmod Not tainted 5.3.0-rc2+ #36 [   16.454678] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-20181126_142135-anatol 04/01/2014
[   16.456191] RIP: 0010:kfree+0x4d/0x240
<registers values...>
[   16.469188] Call Trace:
[   16.469666] vimc_remove+0x35/0x90 [vimc]
[   16.470436] platform_drv_remove+0x1f/0x40
[   16.471233] device_release_driver_internal+0xd3/0x1b0 [   16.472184] driver_detach+0x37/0x6b
[   16.472882] bus_remove_driver+0x50/0xc1
[   16.473569] vimc_exit+0xc/0xca0 [vimc]
[   16.474231] __x64_sys_delete_module+0x18d/0x240 [   16.475036] do_syscall_64+0x43/0x110
[   16.475656] entry_SYSCALL_64_after_hwframe+0x44/0xa9 [   16.476504] RIP: 0033:0x7fceb8dafa4b
<registers values...>
[   16.484853] Modules linked in: vimc(-) videobuf2_vmalloc videobuf2_memops v4l2_tpg videobuf2_v4l2 videobuf2_common [   16.486187] ---[ end trace 91e5e0894e254d49 ]--- [   16.486758] RIP: 0010:kfree+0x4d/0x240
<registers values...>
fish: “rmmod vimc” terminated by signal SIGSEGV (Address boundary error)
I just added the module after booting, no other action was made. Here is
how my git log --oneline looks like:
897d708e922b media: vimc: Fix gpf in rmmod path when stream is active 2e4a5ad8ad6d media: vimc: Collapse component structure into a single monolithic driver
7c8da1687e92 media: vimc: move private defines to a common header 97299a303532 media: Remove dev_err() usage after platform_get_irq() 25a3d6bac6b9 media: adv7511/cobalt: rename driver name to adv7511-v4l2
I couldn't reproduce the error, my tree looks the same:
Quoted 9 lines
[I] koike@floko ~/m/o/linux> git log --oneline
e3345155c8ed (HEAD) media: vimc: Fix gpf in rmmod path when stream is active
43e9e2fe761f media: vimc: Collapse component structure into a single monolithic driver
8a6d0b9adde0 media: vimc: move private defines to a common header 97299a303532 (media/master) media: Remove dev_err() usage after platform_get_irq()
25a3d6bac6b9 media: adv7511/cobalt: rename driver name to adv7511-v4l2
Quoted 1 line
Thanks Helen for trying to reproduce and sharing the result.
Me and Helen found out what is the problem. If you follow this call trace:
vimc_ent_sd_unregister()
v4l2_device_unregister_subdev()
v4l2_subdev_release()
You'll notice that this last function calls the release callback implementation of the subdevice. For instance, the release of vimc-sensor is this one:
static void vimc_sen_release(struct v4l2_subdev *sd) {
    struct vimc_sen_device *vsen =
                container_of(sd, struct vimc_sen_device, sd);
    v4l2_ctrl_handler_free(&vsen->hdl);
    tpg_free(&vsen->tpg);
    kfree(vsen);
}
And then you can see that vsen has been freed. Back to vimc_ent_sd_unregister(), after v4l2_device_unregister_subdev(), the function will call vimc_pads_cleanup(). This is basically a kfree(ved->pads), but ved has just been freed at v4l2_subdev_release(), producing a memory fault.
To fix that, we found two options:
- place the kfree(ved->pads) inside the release callback of each subdevice and removing vimc_pads_cleanup() from vimc_ent_sd_unregister()
- use a auxiliary variable to hold the address of the pads, for instance:
Quoted 8 lines
void vimc_ent_sd_unregister(...)
{
     struct media_pad *pads = ved->pads;
     ...
     vimc_pads_cleanup(pads);
}
Quoted 8 lines
I fixed a problem in the thirds patch. vimc-capture uses the first approach - placing the kfree(ved->pads) inside the release callback.
I am debugging another such problem in unbind path while streaming. I am working on v2 and I will look for the rmmod problem and fix it.
thanks again for testing and finding the root cause. -- Shuah
Hi Andre,
Here is what's happening.
Quoted 8 lines
Before this change, you can't really do rmmod vimc, because vimc is in use by other component drivers. With the collapse, now you can actually do rmmod on vimc and this problem in vimc_ent_sd_unregister() that frees pads first and the does v4l2_device_unregister_subdev().
I fixed this in the 3/3 patch. I can reproduce the problem with patches 1 and 2, and patch 3 fixes it.
Did you test with the third patch in this series?
yes, we tested with 3/3, but the new problem now is when doing the following in this order:
     v4l2_device_unregister_subdev(sd);
     vimc_pads_cleanup(ved->pads);

Quoted 10 lines

v4l2_device_unregister_subdev() calls the release function of the subdevice that frees the ved object, so ved->pads is not valid anymore. That is why André suggested a temporary variable to hold ved->pads and to be able to free it later:
     struct media_pad *pads = ved->pads;

     v4l2_device_unregister_subdev(sd);
     vimc_pads_cleanup(pads); // So we don't use the ved object here anymore.

Got it.

thanks,
-- Shuah

Laurent Pinchart

2019-08-13 09:57 UTC #23 lore raw

In reply to Helen Koike

Hi Helen,

Quoted 222 lines · Helen Koike

Quoted 205 lines · Shuah Khan
Quoted 191 lines · Shuah Khan
Quoted 181 lines · André Almeida
Quoted 145 lines · Shuah Khan
Quoted 134 lines · Helen Koike
Quoted 77 lines · André Almeida
Quoted 76 lines · Shuah Khan
Quoted 74 lines · André Almeida
Quoted 66 lines · Shuah Khan
vimc uses Component API to split the driver into functional components.
The real hardware resembles a monolith structure than component and component structure added a level of complexity making it hard to maintain without adding any real benefit.
       The sensor is one vimc component that would makes sense to be a separate
module to closely align with the real hardware. It would be easier to collapse vimc into single monolithic driver first and then split the sensor off as a separate module.
This patch series emoves the component API and makes minimal changes to
the code base preserving the functional division of the code structure.
Preserving the functional structure allows us to split the sensor off as a separate module in the future.
Major design elements in this change are:
       - Use existing struct vimc_ent_config and struct vimc_pipeline_config
         to drive the initialization of the functional components.
       - Make vimc_ent_config global by moving it to vimc.h
       - Add two new hooks add and rm to initialize and register, unregister
         and free subdevs.
       - All component API is now gone and bind and unbind hooks are modified
         to do "add" and "rm" with minimal changes to just add and rm subdevs.
       - vimc-core's bind and unbind are now register and unregister.
       - vimc-core invokes "add" hooks from its vimc_register_devices().
         The "add" hooks remain the same and register subdevs. They don't
         create platform devices of their own and use vimc's pdev.dev as
         their reference device. The "add" hooks save their vimc_ent_device(s)
         in the corresponding vimc_ent_config.
       - vimc-core invokes "rm" hooks from its unregister to unregister
subdevs
         and cleanup.
       - vimc-core invokes "add" and "rm" hooks with pointer to struct vimc_device
         and the corresponding struct vimc_ent_config pointer. The following configure and stream test works on all devices. media-ctl -d platform:vimc -V '"Sensor A":0[fmt:SBGGR8_1X8/640x480]'
       media-ctl -d platform:vimc -V '"Debayer A":0[fmt:SBGGR8_1X8/640x480]'
       media-ctl -d platform:vimc -V '"Sensor B":0[fmt:SBGGR8_1X8/640x480]'
       media-ctl -d platform:vimc -V '"Debayer B":0[fmt:SBGGR8_1X8/640x480]'
            v4l2-ctl -z platform:vimc -d "RGB/YUV Capture" -v width=1920,height=1440
       v4l2-ctl -z platform:vimc -d "Raw Capture 0" -v pixelformat=BA81
       v4l2-ctl -z platform:vimc -d "Raw Capture 1" -v pixelformat=BA81
            v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video1 v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video2 v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video3
Quoted 2 lines
The third patch in the series fixes a general protection fault found when rmmod is done while stream is active.
I applied your patch on top of media_tree/master and I did some testing.
Not sure if I did something wrong, but just adding and removing the module generated a kernel panic:
Quoted 53 lines
Quoted 5 lines
Thanks for testing.
Odd. I tested modprobe and rmmod both.I was working on Linux 5.3-rc2. I will apply these to media latest and work from there. I have to rebase these on top of the reverts from Lucas and Helen
Ok, please let me know if I succeeded to reproduce.
Quoted 45 lines
Quoted 45 lines
~# modprobe vimc
~# rmmod vimc
[   16.452974] stack segment: 0000 [#1] SMP PTI
[   16.453688] CPU: 0 PID: 2038 Comm: rmmod Not tainted 5.3.0-rc2+ #36 [   16.454678] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-20181126_142135-anatol 04/01/2014
[   16.456191] RIP: 0010:kfree+0x4d/0x240
<registers values...>
[   16.469188] Call Trace:
[   16.469666] vimc_remove+0x35/0x90 [vimc]
[   16.470436] platform_drv_remove+0x1f/0x40
[   16.471233] device_release_driver_internal+0xd3/0x1b0 [   16.472184] driver_detach+0x37/0x6b
[   16.472882] bus_remove_driver+0x50/0xc1
[   16.473569] vimc_exit+0xc/0xca0 [vimc]
[   16.474231] __x64_sys_delete_module+0x18d/0x240 [   16.475036] do_syscall_64+0x43/0x110
[   16.475656] entry_SYSCALL_64_after_hwframe+0x44/0xa9 [   16.476504] RIP: 0033:0x7fceb8dafa4b
<registers values...>
[   16.484853] Modules linked in: vimc(-) videobuf2_vmalloc videobuf2_memops v4l2_tpg videobuf2_v4l2 videobuf2_common [   16.486187] ---[ end trace 91e5e0894e254d49 ]--- [   16.486758] RIP: 0010:kfree+0x4d/0x240
<registers values...>
fish: “rmmod vimc” terminated by signal SIGSEGV (Address boundary error)
I just added the module after booting, no other action was made. Here is
how my git log --oneline looks like:
897d708e922b media: vimc: Fix gpf in rmmod path when stream is active 2e4a5ad8ad6d media: vimc: Collapse component structure into a single monolithic driver
7c8da1687e92 media: vimc: move private defines to a common header 97299a303532 media: Remove dev_err() usage after platform_get_irq() 25a3d6bac6b9 media: adv7511/cobalt: rename driver name to adv7511-v4l2
I couldn't reproduce the error, my tree looks the same:
Quoted 9 lines
[I] koike@floko ~/m/o/linux> git log --oneline
e3345155c8ed (HEAD) media: vimc: Fix gpf in rmmod path when stream is active
43e9e2fe761f media: vimc: Collapse component structure into a single monolithic driver
8a6d0b9adde0 media: vimc: move private defines to a common header 97299a303532 (media/master) media: Remove dev_err() usage after platform_get_irq()
25a3d6bac6b9 media: adv7511/cobalt: rename driver name to adv7511-v4l2
Quoted 1 line
Thanks Helen for trying to reproduce and sharing the result.
Me and Helen found out what is the problem. If you follow this call trace:
vimc_ent_sd_unregister()
v4l2_device_unregister_subdev()
v4l2_subdev_release()
You'll notice that this last function calls the release callback implementation of the subdevice. For instance, the release of vimc-sensor is this one:
static void vimc_sen_release(struct v4l2_subdev *sd) {
    struct vimc_sen_device *vsen =
                container_of(sd, struct vimc_sen_device, sd);
    v4l2_ctrl_handler_free(&vsen->hdl);
    tpg_free(&vsen->tpg);
    kfree(vsen);
}
And then you can see that vsen has been freed. Back to vimc_ent_sd_unregister(), after v4l2_device_unregister_subdev(), the function will call vimc_pads_cleanup(). This is basically a kfree(ved->pads), but ved has just been freed at v4l2_subdev_release(), producing a memory fault.
To fix that, we found two options:
- place the kfree(ved->pads) inside the release callback of each subdevice and removing vimc_pads_cleanup() from vimc_ent_sd_unregister()
- use a auxiliary variable to hold the address of the pads, for instance:
Quoted 8 lines
void vimc_ent_sd_unregister(...)
{
     struct media_pad *pads = ved->pads;
     ...
     vimc_pads_cleanup(pads);
}
Quoted 8 lines
I fixed a problem in the thirds patch. vimc-capture uses the first approach - placing the kfree(ved->pads) inside the release callback.
I am debugging another such problem in unbind path while streaming. I am working on v2 and I will look for the rmmod problem and fix it.
thanks again for testing and finding the root cause. -- Shuah
Hi Andre,
Here is what's happening.
Quoted 8 lines
Before this change, you can't really do rmmod vimc, because vimc is in use by other component drivers. With the collapse, now you can actually do rmmod on vimc and this problem in vimc_ent_sd_unregister() that frees pads first and the does v4l2_device_unregister_subdev().
I fixed this in the 3/3 patch. I can reproduce the problem with patches 1 and 2, and patch 3 fixes it.
Did you test with the third patch in this series?
yes, we tested with 3/3, but the new problem now is when doing the following in this order:
    v4l2_device_unregister_subdev(sd);
    vimc_pads_cleanup(ved->pads);

Quoted 8 lines

v4l2_device_unregister_subdev() calls the release function of the subdevice that frees the ved object, so ved->pads is not valid anymore. That is why André suggested a temporary variable to hold ved->pads and to be able to free it later:
    struct media_pad *pads = ved->pads;

    v4l2_device_unregister_subdev(sd);
    vimc_pads_cleanup(pads); // So we don't use the ved object here anymore.

Can't you simply call vimc_pads_cleanup() in the release function of the subdevice before freeing the ved object ?

Signature · 3 lines

Regards,

Laurent Pinchart

Helen Koike

2019-08-13 12:25 UTC #24 lore raw

Quoted 229 lines · Laurent Pinchart

Hi Helen,
Quoted 226 lines · Helen Koike
Quoted 205 lines · Shuah Khan
Quoted 191 lines · Shuah Khan
Quoted 181 lines · André Almeida
Quoted 145 lines · Shuah Khan
Quoted 134 lines · Helen Koike
Quoted 77 lines · André Almeida
Quoted 76 lines · Shuah Khan
Quoted 74 lines · André Almeida
Quoted 66 lines · Shuah Khan
vimc uses Component API to split the driver into functional components.
The real hardware resembles a monolith structure than component and component structure added a level of complexity making it hard to maintain without adding any real benefit.
       The sensor is one vimc component that would makes sense to be a separate
module to closely align with the real hardware. It would be easier to collapse vimc into single monolithic driver first and then split the sensor off as a separate module.
This patch series emoves the component API and makes minimal changes to
the code base preserving the functional division of the code structure.
Preserving the functional structure allows us to split the sensor off as a separate module in the future.
Major design elements in this change are:
       - Use existing struct vimc_ent_config and struct vimc_pipeline_config
         to drive the initialization of the functional components.
       - Make vimc_ent_config global by moving it to vimc.h
       - Add two new hooks add and rm to initialize and register, unregister
         and free subdevs.
       - All component API is now gone and bind and unbind hooks are modified
         to do "add" and "rm" with minimal changes to just add and rm subdevs.
       - vimc-core's bind and unbind are now register and unregister.
       - vimc-core invokes "add" hooks from its vimc_register_devices().
         The "add" hooks remain the same and register subdevs. They don't
         create platform devices of their own and use vimc's pdev.dev as
         their reference device. The "add" hooks save their vimc_ent_device(s)
         in the corresponding vimc_ent_config.
       - vimc-core invokes "rm" hooks from its unregister to unregister
subdevs
         and cleanup.
       - vimc-core invokes "add" and "rm" hooks with pointer to struct vimc_device
         and the corresponding struct vimc_ent_config pointer. The following configure and stream test works on all devices. media-ctl -d platform:vimc -V '"Sensor A":0[fmt:SBGGR8_1X8/640x480]'
       media-ctl -d platform:vimc -V '"Debayer A":0[fmt:SBGGR8_1X8/640x480]'
       media-ctl -d platform:vimc -V '"Sensor B":0[fmt:SBGGR8_1X8/640x480]'
       media-ctl -d platform:vimc -V '"Debayer B":0[fmt:SBGGR8_1X8/640x480]'
            v4l2-ctl -z platform:vimc -d "RGB/YUV Capture" -v width=1920,height=1440
       v4l2-ctl -z platform:vimc -d "Raw Capture 0" -v pixelformat=BA81
       v4l2-ctl -z platform:vimc -d "Raw Capture 1" -v pixelformat=BA81
            v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video1 v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video2 v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video3
Quoted 2 lines
The third patch in the series fixes a general protection fault found when rmmod is done while stream is active.
I applied your patch on top of media_tree/master and I did some testing.
Not sure if I did something wrong, but just adding and removing the module generated a kernel panic:
Quoted 53 lines
Quoted 5 lines
Thanks for testing.
Odd. I tested modprobe and rmmod both.I was working on Linux 5.3-rc2. I will apply these to media latest and work from there. I have to rebase these on top of the reverts from Lucas and Helen
Ok, please let me know if I succeeded to reproduce.
Quoted 45 lines
Quoted 45 lines
~# modprobe vimc
~# rmmod vimc
[   16.452974] stack segment: 0000 [#1] SMP PTI
[   16.453688] CPU: 0 PID: 2038 Comm: rmmod Not tainted 5.3.0-rc2+ #36 [   16.454678] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-20181126_142135-anatol 04/01/2014
[   16.456191] RIP: 0010:kfree+0x4d/0x240
<registers values...>
[   16.469188] Call Trace:
[   16.469666] vimc_remove+0x35/0x90 [vimc]
[   16.470436] platform_drv_remove+0x1f/0x40
[   16.471233] device_release_driver_internal+0xd3/0x1b0 [   16.472184] driver_detach+0x37/0x6b
[   16.472882] bus_remove_driver+0x50/0xc1
[   16.473569] vimc_exit+0xc/0xca0 [vimc]
[   16.474231] __x64_sys_delete_module+0x18d/0x240 [   16.475036] do_syscall_64+0x43/0x110
[   16.475656] entry_SYSCALL_64_after_hwframe+0x44/0xa9 [   16.476504] RIP: 0033:0x7fceb8dafa4b
<registers values...>
[   16.484853] Modules linked in: vimc(-) videobuf2_vmalloc videobuf2_memops v4l2_tpg videobuf2_v4l2 videobuf2_common [   16.486187] ---[ end trace 91e5e0894e254d49 ]--- [   16.486758] RIP: 0010:kfree+0x4d/0x240
<registers values...>
fish: “rmmod vimc” terminated by signal SIGSEGV (Address boundary error)
I just added the module after booting, no other action was made. Here is
how my git log --oneline looks like:
897d708e922b media: vimc: Fix gpf in rmmod path when stream is active 2e4a5ad8ad6d media: vimc: Collapse component structure into a single monolithic driver
7c8da1687e92 media: vimc: move private defines to a common header 97299a303532 media: Remove dev_err() usage after platform_get_irq() 25a3d6bac6b9 media: adv7511/cobalt: rename driver name to adv7511-v4l2
I couldn't reproduce the error, my tree looks the same:
Quoted 9 lines
[I] koike@floko ~/m/o/linux> git log --oneline
e3345155c8ed (HEAD) media: vimc: Fix gpf in rmmod path when stream is active
43e9e2fe761f media: vimc: Collapse component structure into a single monolithic driver
8a6d0b9adde0 media: vimc: move private defines to a common header 97299a303532 (media/master) media: Remove dev_err() usage after platform_get_irq()
25a3d6bac6b9 media: adv7511/cobalt: rename driver name to adv7511-v4l2
Quoted 1 line
Thanks Helen for trying to reproduce and sharing the result.
Me and Helen found out what is the problem. If you follow this call trace:
vimc_ent_sd_unregister()
v4l2_device_unregister_subdev()
v4l2_subdev_release()
You'll notice that this last function calls the release callback implementation of the subdevice. For instance, the release of vimc-sensor is this one:
static void vimc_sen_release(struct v4l2_subdev *sd) {
    struct vimc_sen_device *vsen =
                container_of(sd, struct vimc_sen_device, sd);
    v4l2_ctrl_handler_free(&vsen->hdl);
    tpg_free(&vsen->tpg);
    kfree(vsen);
}
And then you can see that vsen has been freed. Back to vimc_ent_sd_unregister(), after v4l2_device_unregister_subdev(), the function will call vimc_pads_cleanup(). This is basically a kfree(ved->pads), but ved has just been freed at v4l2_subdev_release(), producing a memory fault.
To fix that, we found two options:
- place the kfree(ved->pads) inside the release callback of each subdevice and removing vimc_pads_cleanup() from vimc_ent_sd_unregister()
- use a auxiliary variable to hold the address of the pads, for instance:
Quoted 8 lines
void vimc_ent_sd_unregister(...)
{
     struct media_pad *pads = ved->pads;
     ...
     vimc_pads_cleanup(pads);
}
Quoted 8 lines
I fixed a problem in the thirds patch. vimc-capture uses the first approach - placing the kfree(ved->pads) inside the release callback.
I am debugging another such problem in unbind path while streaming. I am working on v2 and I will look for the rmmod problem and fix it.
thanks again for testing and finding the root cause. -- Shuah
Hi Andre,
Here is what's happening.
Quoted 8 lines
Before this change, you can't really do rmmod vimc, because vimc is in use by other component drivers. With the collapse, now you can actually do rmmod on vimc and this problem in vimc_ent_sd_unregister() that frees pads first and the does v4l2_device_unregister_subdev().
I fixed this in the 3/3 patch. I can reproduce the problem with patches 1 and 2, and patch 3 fixes it.
Did you test with the third patch in this series?
yes, we tested with 3/3, but the new problem now is when doing the following in this order:
    v4l2_device_unregister_subdev(sd);
    vimc_pads_cleanup(ved->pads);
v4l2_device_unregister_subdev() calls the release function of the subdevice that frees the ved object, so ved->pads is not valid anymore. That is why André suggested a temporary variable to hold ved->pads and to be able to free it later:

Quoted 8 lines

Quoted 4 lines
    struct media_pad *pads = ved->pads;

    v4l2_device_unregister_subdev(sd);
    vimc_pads_cleanup(pads); // So we don't use the ved object here anymore.
Can't you simply call vimc_pads_cleanup() in the release function of the subdevice before freeing the ved object ?

Yes we can, that is the other option Andre suggested.
The advantage of doing it inside vimc_ent_sd_unregister() is that who allocated the memory in the first place was vimc_ent_sd_register(), and also, this is a common code to all subdevs, so letting it in vimc_ent_sd_unregister() will make sure no subdevs ever forget to free this memory.
But saving the pointer to a variable to free it later is not that pretty either.

Helen

Laurent Pinchart

2019-08-13 12:37 UTC #25 lore raw

Hi Helen,

Quoted 236 lines · Helen Koike

Quoted 227 lines · Laurent Pinchart
Quoted 226 lines · Helen Koike
Quoted 205 lines · Shuah Khan
Quoted 191 lines · Shuah Khan
Quoted 181 lines · André Almeida
Quoted 145 lines · Shuah Khan
Quoted 134 lines · Helen Koike
Quoted 77 lines · André Almeida
Quoted 76 lines · Shuah Khan
Quoted 74 lines · André Almeida
Quoted 66 lines · Shuah Khan
vimc uses Component API to split the driver into functional components.
The real hardware resembles a monolith structure than component and component structure added a level of complexity making it hard to maintain without adding any real benefit.
       The sensor is one vimc component that would makes sense to be a separate
module to closely align with the real hardware. It would be easier to collapse vimc into single monolithic driver first and then split the sensor off as a separate module.
This patch series emoves the component API and makes minimal changes to
the code base preserving the functional division of the code structure.
Preserving the functional structure allows us to split the sensor off as a separate module in the future.
Major design elements in this change are:
       - Use existing struct vimc_ent_config and struct vimc_pipeline_config
         to drive the initialization of the functional components.
       - Make vimc_ent_config global by moving it to vimc.h
       - Add two new hooks add and rm to initialize and register, unregister
         and free subdevs.
       - All component API is now gone and bind and unbind hooks are modified
         to do "add" and "rm" with minimal changes to just add and rm subdevs.
       - vimc-core's bind and unbind are now register and unregister.
       - vimc-core invokes "add" hooks from its vimc_register_devices().
         The "add" hooks remain the same and register subdevs. They don't
         create platform devices of their own and use vimc's pdev.dev as
         their reference device. The "add" hooks save their vimc_ent_device(s)
         in the corresponding vimc_ent_config.
       - vimc-core invokes "rm" hooks from its unregister to unregister
subdevs
         and cleanup.
       - vimc-core invokes "add" and "rm" hooks with pointer to struct vimc_device
         and the corresponding struct vimc_ent_config pointer. The following configure and stream test works on all devices. media-ctl -d platform:vimc -V '"Sensor A":0[fmt:SBGGR8_1X8/640x480]'
       media-ctl -d platform:vimc -V '"Debayer A":0[fmt:SBGGR8_1X8/640x480]'
       media-ctl -d platform:vimc -V '"Sensor B":0[fmt:SBGGR8_1X8/640x480]'
       media-ctl -d platform:vimc -V '"Debayer B":0[fmt:SBGGR8_1X8/640x480]'
            v4l2-ctl -z platform:vimc -d "RGB/YUV Capture" -v width=1920,height=1440
       v4l2-ctl -z platform:vimc -d "Raw Capture 0" -v pixelformat=BA81
       v4l2-ctl -z platform:vimc -d "Raw Capture 1" -v pixelformat=BA81
            v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video1 v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video2 v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video3
Quoted 2 lines
The third patch in the series fixes a general protection fault found when rmmod is done while stream is active.
I applied your patch on top of media_tree/master and I did some testing.
Not sure if I did something wrong, but just adding and removing the module generated a kernel panic:
Quoted 53 lines
Quoted 5 lines
Thanks for testing.
Odd. I tested modprobe and rmmod both.I was working on Linux 5.3-rc2. I will apply these to media latest and work from there. I have to rebase these on top of the reverts from Lucas and Helen
Ok, please let me know if I succeeded to reproduce.
Quoted 45 lines
Quoted 45 lines
~# modprobe vimc
~# rmmod vimc
[   16.452974] stack segment: 0000 [#1] SMP PTI
[   16.453688] CPU: 0 PID: 2038 Comm: rmmod Not tainted 5.3.0-rc2+ #36 [   16.454678] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-20181126_142135-anatol 04/01/2014
[   16.456191] RIP: 0010:kfree+0x4d/0x240
<registers values...>
[   16.469188] Call Trace:
[   16.469666] vimc_remove+0x35/0x90 [vimc]
[   16.470436] platform_drv_remove+0x1f/0x40
[   16.471233] device_release_driver_internal+0xd3/0x1b0 [   16.472184] driver_detach+0x37/0x6b
[   16.472882] bus_remove_driver+0x50/0xc1
[   16.473569] vimc_exit+0xc/0xca0 [vimc]
[   16.474231] __x64_sys_delete_module+0x18d/0x240 [   16.475036] do_syscall_64+0x43/0x110
[   16.475656] entry_SYSCALL_64_after_hwframe+0x44/0xa9 [   16.476504] RIP: 0033:0x7fceb8dafa4b
<registers values...>
[   16.484853] Modules linked in: vimc(-) videobuf2_vmalloc videobuf2_memops v4l2_tpg videobuf2_v4l2 videobuf2_common [   16.486187] ---[ end trace 91e5e0894e254d49 ]--- [   16.486758] RIP: 0010:kfree+0x4d/0x240
<registers values...>
fish: “rmmod vimc” terminated by signal SIGSEGV (Address boundary error)
I just added the module after booting, no other action was made. Here is
how my git log --oneline looks like:
897d708e922b media: vimc: Fix gpf in rmmod path when stream is active 2e4a5ad8ad6d media: vimc: Collapse component structure into a single monolithic driver
7c8da1687e92 media: vimc: move private defines to a common header 97299a303532 media: Remove dev_err() usage after platform_get_irq() 25a3d6bac6b9 media: adv7511/cobalt: rename driver name to adv7511-v4l2
I couldn't reproduce the error, my tree looks the same:
Quoted 9 lines
[I] koike@floko ~/m/o/linux> git log --oneline
e3345155c8ed (HEAD) media: vimc: Fix gpf in rmmod path when stream is active
43e9e2fe761f media: vimc: Collapse component structure into a single monolithic driver
8a6d0b9adde0 media: vimc: move private defines to a common header 97299a303532 (media/master) media: Remove dev_err() usage after platform_get_irq()
25a3d6bac6b9 media: adv7511/cobalt: rename driver name to adv7511-v4l2
Quoted 1 line
Thanks Helen for trying to reproduce and sharing the result.
Me and Helen found out what is the problem. If you follow this call trace:
vimc_ent_sd_unregister()
v4l2_device_unregister_subdev()
v4l2_subdev_release()
You'll notice that this last function calls the release callback implementation of the subdevice. For instance, the release of vimc-sensor is this one:
static void vimc_sen_release(struct v4l2_subdev *sd) {
    struct vimc_sen_device *vsen =
                container_of(sd, struct vimc_sen_device, sd);
    v4l2_ctrl_handler_free(&vsen->hdl);
    tpg_free(&vsen->tpg);
    kfree(vsen);
}
And then you can see that vsen has been freed. Back to vimc_ent_sd_unregister(), after v4l2_device_unregister_subdev(), the function will call vimc_pads_cleanup(). This is basically a kfree(ved->pads), but ved has just been freed at v4l2_subdev_release(), producing a memory fault.
To fix that, we found two options:
- place the kfree(ved->pads) inside the release callback of each subdevice and removing vimc_pads_cleanup() from vimc_ent_sd_unregister()
- use a auxiliary variable to hold the address of the pads, for instance:
Quoted 8 lines
void vimc_ent_sd_unregister(...)
{
     struct media_pad *pads = ved->pads;
     ...
     vimc_pads_cleanup(pads);
}
Quoted 8 lines
I fixed a problem in the thirds patch. vimc-capture uses the first approach - placing the kfree(ved->pads) inside the release callback.
I am debugging another such problem in unbind path while streaming. I am working on v2 and I will look for the rmmod problem and fix it.
thanks again for testing and finding the root cause. -- Shuah
Hi Andre,
Here is what's happening.
Quoted 8 lines
Before this change, you can't really do rmmod vimc, because vimc is in use by other component drivers. With the collapse, now you can actually do rmmod on vimc and this problem in vimc_ent_sd_unregister() that frees pads first and the does v4l2_device_unregister_subdev().
I fixed this in the 3/3 patch. I can reproduce the problem with patches 1 and 2, and patch 3 fixes it.
Did you test with the third patch in this series?
yes, we tested with 3/3, but the new problem now is when doing the following in this order:
    v4l2_device_unregister_subdev(sd);
    vimc_pads_cleanup(ved->pads);
v4l2_device_unregister_subdev() calls the release function of the subdevice that frees the ved object, so ved->pads is not valid anymore. That is why André suggested a temporary variable to hold ved->pads and to be able to free it later:
Quoted 7 lines
Quoted 4 lines
    struct media_pad *pads = ved->pads;

    v4l2_device_unregister_subdev(sd);
    vimc_pads_cleanup(pads); // So we don't use the ved object here anymore.
Can't you simply call vimc_pads_cleanup() in the release function of the subdevice before freeing the ved object ?

Quoted 7 lines

Yes we can, that is the other option Andre suggested.
The advantage of doing it inside vimc_ent_sd_unregister() is that who allocated the memory in the first place was vimc_ent_sd_register(), and also, this is a common code to all subdevs, so letting it in vimc_ent_sd_unregister() will make sure no subdevs ever forget to free this memory.
But saving the pointer to a variable to free it later is not that pretty either.

The release function of a subdevice is meant to be used as a destructor, it should free all the resources allocated for the subdevice. Your subdevices (such as vimc_sca_device) are essentially objects derived from a common vimc subdevice object (vimc_ent_device), I thus suggest providing four functions to deal with vimc_ent_device:

- vimc_ent_sd_init() to initialise the base vimc_ent_device object (most of the code is currently in vimc_ent_sd_register()), to be called by the init function of each subdevice type (the bind functions in the current code)

- vimc_ent_sd_register() to only register the vimc_ent_device once it has been fully initialised

- vimc_end_sd_unregister() to unregister the vimc_ent_device

- vimc_end_sd_cleanup() to free the resources allocated by vimc_ent_sd_init() (including calling vimc_pads_cleanup()), to be called by the subdevice release handler

Signature · 3 lines

Regards,

Laurent Pinchart

Shuah Khan

author

2019-08-13 23:22 UTC #26 lore raw

Quoted 258 lines · Laurent Pinchart

Hi Helen,
Quoted 236 lines · Helen Koike
Quoted 227 lines · Laurent Pinchart
Quoted 226 lines · Helen Koike
Quoted 205 lines · Shuah Khan
Quoted 191 lines · Shuah Khan
Quoted 181 lines · André Almeida
Quoted 145 lines · Shuah Khan
Quoted 134 lines · Helen Koike
Quoted 77 lines · André Almeida
Quoted 76 lines · Shuah Khan
Quoted 74 lines · André Almeida
Quoted 68 lines · Shuah Khan
vimc uses Component API to split the driver into functional
components.
The real hardware resembles a monolith structure than component and
component structure added a level of complexity making it hard to
maintain without adding any real benefit.
       The sensor is one vimc component that would makes sense to be a
separate
module to closely align with the real hardware. It would be easier to
collapse vimc into single monolithic driver first and then split the
sensor off as a separate module.
This patch series emoves the component API and makes minimal
changes to
the code base preserving the functional division of the code
structure.
Preserving the functional structure allows us to split the sensor off
as a separate module in the future.
Major design elements in this change are:
       - Use existing struct vimc_ent_config and struct
vimc_pipeline_config
         to drive the initialization of the functional components.
       - Make vimc_ent_config global by moving it to vimc.h
       - Add two new hooks add and rm to initialize and register,
unregister
         and free subdevs.
       - All component API is now gone and bind and unbind hooks are
modified
         to do "add" and "rm" with minimal changes to just add and rm
subdevs.
       - vimc-core's bind and unbind are now register and unregister.
       - vimc-core invokes "add" hooks from its
vimc_register_devices().
         The "add" hooks remain the same and register subdevs. They
don't
         create platform devices of their own and use vimc's
pdev.dev as
         their reference device. The "add" hooks save their
vimc_ent_device(s)
         in the corresponding vimc_ent_config.
       - vimc-core invokes "rm" hooks from its unregister to
unregister
subdevs
         and cleanup.
       - vimc-core invokes "add" and "rm" hooks with pointer to struct
vimc_device
         and the corresponding struct vimc_ent_config pointer.
       The following configure and stream test works on all devices.
            media-ctl -d platform:vimc -V '"Sensor
A":0[fmt:SBGGR8_1X8/640x480]'
       media-ctl -d platform:vimc -V '"Debayer
A":0[fmt:SBGGR8_1X8/640x480]'
       media-ctl -d platform:vimc -V '"Sensor
B":0[fmt:SBGGR8_1X8/640x480]'
       media-ctl -d platform:vimc -V '"Debayer
B":0[fmt:SBGGR8_1X8/640x480]'
            v4l2-ctl -z platform:vimc -d "RGB/YUV Capture" -v
width=1920,height=1440
       v4l2-ctl -z platform:vimc -d "Raw Capture 0" -v
pixelformat=BA81
       v4l2-ctl -z platform:vimc -d "Raw Capture 1" -v
pixelformat=BA81
            v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video1
       v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video2
       v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video3
The third patch in the series fixes a general protection fault found
when rmmod is done while stream is active.
I applied your patch on top of media_tree/master and I did some testing.
Not sure if I did something wrong, but just adding and removing the module generated a kernel panic:
Quoted 53 lines
Quoted 5 lines
Thanks for testing.
Odd. I tested modprobe and rmmod both.I was working on Linux 5.3-rc2. I will apply these to media latest and work from there. I have to rebase these on top of the reverts from Lucas and Helen
Ok, please let me know if I succeeded to reproduce.
Quoted 45 lines
Quoted 45 lines
~# modprobe vimc
~# rmmod vimc
[   16.452974] stack segment: 0000 [#1] SMP PTI
[   16.453688] CPU: 0 PID: 2038 Comm: rmmod Not tainted 5.3.0-rc2+ #36 [   16.454678] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-20181126_142135-anatol 04/01/2014
[   16.456191] RIP: 0010:kfree+0x4d/0x240
<registers values...>
[   16.469188] Call Trace:
[   16.469666] vimc_remove+0x35/0x90 [vimc]
[   16.470436] platform_drv_remove+0x1f/0x40
[   16.471233] device_release_driver_internal+0xd3/0x1b0 [   16.472184] driver_detach+0x37/0x6b
[   16.472882] bus_remove_driver+0x50/0xc1
[   16.473569] vimc_exit+0xc/0xca0 [vimc]
[   16.474231] __x64_sys_delete_module+0x18d/0x240 [   16.475036] do_syscall_64+0x43/0x110
[   16.475656] entry_SYSCALL_64_after_hwframe+0x44/0xa9 [   16.476504] RIP: 0033:0x7fceb8dafa4b
<registers values...>
[   16.484853] Modules linked in: vimc(-) videobuf2_vmalloc videobuf2_memops v4l2_tpg videobuf2_v4l2 videobuf2_common [   16.486187] ---[ end trace 91e5e0894e254d49 ]--- [   16.486758] RIP: 0010:kfree+0x4d/0x240
<registers values...>
fish: “rmmod vimc” terminated by signal SIGSEGV (Address boundary error)
I just added the module after booting, no other action was made. Here is
how my git log --oneline looks like:
897d708e922b media: vimc: Fix gpf in rmmod path when stream is active 2e4a5ad8ad6d media: vimc: Collapse component structure into a single monolithic driver
7c8da1687e92 media: vimc: move private defines to a common header 97299a303532 media: Remove dev_err() usage after platform_get_irq() 25a3d6bac6b9 media: adv7511/cobalt: rename driver name to adv7511-v4l2
I couldn't reproduce the error, my tree looks the same:
Quoted 9 lines
[I] koike@floko ~/m/o/linux> git log --oneline
e3345155c8ed (HEAD) media: vimc: Fix gpf in rmmod path when stream is active
43e9e2fe761f media: vimc: Collapse component structure into a single monolithic driver
8a6d0b9adde0 media: vimc: move private defines to a common header 97299a303532 (media/master) media: Remove dev_err() usage after platform_get_irq()
25a3d6bac6b9 media: adv7511/cobalt: rename driver name to adv7511-v4l2
Quoted 1 line
Thanks Helen for trying to reproduce and sharing the result.
Me and Helen found out what is the problem. If you follow this call trace:
vimc_ent_sd_unregister()
v4l2_device_unregister_subdev()
v4l2_subdev_release()
You'll notice that this last function calls the release callback implementation of the subdevice. For instance, the release of vimc-sensor is this one:
static void vimc_sen_release(struct v4l2_subdev *sd) {
    struct vimc_sen_device *vsen =
                container_of(sd, struct vimc_sen_device, sd);
    v4l2_ctrl_handler_free(&vsen->hdl);
    tpg_free(&vsen->tpg);
    kfree(vsen);
}
And then you can see that vsen has been freed. Back to vimc_ent_sd_unregister(), after v4l2_device_unregister_subdev(), the function will call vimc_pads_cleanup(). This is basically a kfree(ved->pads), but ved has just been freed at v4l2_subdev_release(), producing a memory fault.
To fix that, we found two options:
- place the kfree(ved->pads) inside the release callback of each subdevice and removing vimc_pads_cleanup() from vimc_ent_sd_unregister()
- use a auxiliary variable to hold the address of the pads, for instance:
Quoted 8 lines
void vimc_ent_sd_unregister(...)
{
     struct media_pad *pads = ved->pads;
     ...
     vimc_pads_cleanup(pads);
}
Quoted 8 lines
I fixed a problem in the thirds patch. vimc-capture uses the first approach - placing the kfree(ved->pads) inside the release callback.
I am debugging another such problem in unbind path while streaming. I am working on v2 and I will look for the rmmod problem and fix it.
thanks again for testing and finding the root cause. -- Shuah
Hi Andre,
Here is what's happening.
Quoted 8 lines
Before this change, you can't really do rmmod vimc, because vimc is in use by other component drivers. With the collapse, now you can actually do rmmod on vimc and this problem in vimc_ent_sd_unregister() that frees pads first and the does v4l2_device_unregister_subdev().
I fixed this in the 3/3 patch. I can reproduce the problem with patches 1 and 2, and patch 3 fixes it.
Did you test with the third patch in this series?
yes, we tested with 3/3, but the new problem now is when doing the following in this order:
     v4l2_device_unregister_subdev(sd);
     vimc_pads_cleanup(ved->pads);
v4l2_device_unregister_subdev() calls the release function of the subdevice that frees the ved object, so ved->pads is not valid anymore. That is why André suggested a temporary variable to hold ved->pads and to be able to free it later:
Quoted 7 lines
Quoted 4 lines
     struct media_pad *pads = ved->pads;

     v4l2_device_unregister_subdev(sd);
     vimc_pads_cleanup(pads); // So we don't use the ved object here anymore.
Can't you simply call vimc_pads_cleanup() in the release function of the subdevice before freeing the ved object ?
Quoted 7 lines
Yes we can, that is the other option Andre suggested.
The advantage of doing it inside vimc_ent_sd_unregister() is that who allocated the memory in the first place was vimc_ent_sd_register(), and also, this is a common code to all subdevs, so letting it in vimc_ent_sd_unregister() will make sure no subdevs ever forget to free this memory.
But saving the pointer to a variable to free it later is not that pretty either.
The release function of a subdevice is meant to be used as a destructor, it should free all the resources allocated for the subdevice. Your subdevices (such as vimc_sca_device) are essentially objects derived from a common vimc subdevice object (vimc_ent_device), I thus suggest providing four functions to deal with vimc_ent_device:
- vimc_ent_sd_init() to initialise the base vimc_ent_device object (most of the code is currently in vimc_ent_sd_register()), to be called by the init function of each subdevice type (the bind functions in the current code)

Quoted 9 lines

- vimc_ent_sd_register() to only register the vimc_ent_device once it has been fully initialised
- vimc_end_sd_unregister() to unregister the vimc_ent_device
- vimc_end_sd_cleanup() to free the resources allocated by vimc_ent_sd_init() (including calling vimc_pads_cleanup()), to be called by the subdevice release handler

We kind of mostly there. We have release handlers free'ing resources with an exception of pads. I changed the release handlers to release pads and with a comment where pads_init is called to say that pads should be released from the release handler.

vimc-capture does this already. I changed the rest. rmmod works with no issues now. rmmod when streaming comes back with in use.

There is one outstanding issue remaining in unbind path. I will fix that in a separate patch. I decided to collapse the first two patches into one and added common defines (vimc_device and vimc_ent_config) to vimc-common.h based on our discussion.

I will send the patch series out soon.

thanks,
-- Shuah