Collapse vimc into single monolithic driver

vimc uses Component API to split the driver into functional components. The real hardware resembles a monolith structure than component and component structure added a level of complexity making it hard to maintain without adding any real benefit.

The sensor is one vimc component that would makes sense to be a separate module to closely align with the real hardware. It would be easier to collapse vimc into single monolithic driver first and then split the sensor off as a separate module.

This patch series emoves the component API and makes minimal changes to the code base preserving the functional division of the code structure. Preserving the functional structure allows us to split the sensor off as a separate module in the future.

Major design elements in this change are:
- Use existing struct vimc_ent_config and struct vimc_pipeline_config to drive the initialization of the functional components.
- Make vimc_ent_config global by moving it to vimc.h
- Add two new hooks add and rm to initialize and register, unregister and free subdevs.
- All component API is now gone and bind and unbind hooks are modified to do "add" and "rm" with minimal changes to just add and rm subdevs.
- vimc-core's bind and unbind are now register and unregister.
- vimc-core invokes "add" hooks from its vimc_register_devices(). The "add" hooks remain the same and register subdevs. They don't create platform devices of their own and use vimc's pdev.dev as their reference device. The "add" hooks save their vimc_ent_device(s) in the corresponding vimc_ent_config.
- vimc-core invokes "rm" hooks from its unregister to unregister subdevs and cleanup.
- vimc-core invokes "add" and "rm" hooks with pointer to struct vimc_device and the corresponding struct vimc_ent_config pointer.

The following configure and stream test works on all devices.

    media-ctl -d platform:vimc -V '"Sensor A":0[fmt:SBGGR8_1X8/640x480]'
    media-ctl -d platform:vimc -V '"Debayer A":0[fmt:SBGGR8_1X8/640x480]'
    media-ctl -d platform:vimc -V '"Sensor B":0[fmt:SBGGR8_1X8/640x480]'
    media-ctl -d platform:vimc -V '"Debayer B":0[fmt:SBGGR8_1X8/640x480]'
    
    v4l2-ctl -z platform:vimc -d "RGB/YUV Capture" -v width=1920,height=1440
    v4l2-ctl -z platform:vimc -d "Raw Capture 0" -v pixelformat=BA81
    v4l2-ctl -z platform:vimc -d "Raw Capture 1" -v pixelformat=BA81
    
    v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video1
    v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video2
    v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video3

The third patch in the series fixes a general protection fault found when rmmod is done while stream is active.

vimc_print_dot (--print-dot) topology after this change:

digraph board {
	rankdir=TB
	n00000001 [label="{{} | Sensor A\n/dev/v4l-subdev0 | {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green]
	n00000001:port0 -> n00000005:port0 [style=bold]
	n00000001:port0 -> n0000000b [style=bold]
	n00000003 [label="{{} | Sensor B\n/dev/v4l-subdev1 | {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green]
	n00000003:port0 -> n00000008:port0 [style=bold]
	n00000003:port0 -> n0000000f [style=bold]
	n00000005 [label="{{<port0> 0} | Debayer A\n/dev/v4l-subdev2 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
	n00000005:port1 -> n00000015:port0
	n00000008 [label="{{<port0> 0} | Debayer B\n/dev/v4l-subdev3 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
	n00000008:port1 -> n00000015:port0 [style=dashed]

	n0000000b [label="Raw Capture 0\n/dev/video1", shape=box, style=filled, fillcolor=yellow]
	n0000000f [label="Raw Capture 1\n/dev/video2", shape=box, style=filled, fillcolor=yellow]
	n00000013 [label="{{} | RGB/YUV Input\n/dev/v4l-subdev4 | {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green]
	n00000013:port0 -> n00000015:port0 [style=dashed]
	n00000015 [label="{{<port0> 0} | Scaler\n/dev/v4l-subdev5 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
	n00000015:port1 -> n00000018 [style=bold]
	n00000018 [label="RGB/YUV Capture\n/dev/video3", shape=box, style=filled, fillcolor=yellow]
}

Shuah Khan (3):
media: vimc: move private defines to a common header
media: vimc: Collapse component structure into a single monolithic
driver
media: vimc: Fix gpf in rmmod path when stream is active

 drivers/media/platform/vimc/Makefile       |   7 +-
 drivers/media/platform/vimc/vimc-capture.c |  96 ++----------
 drivers/media/platform/vimc/vimc-common.c  |   2 +-
 drivers/media/platform/vimc/vimc-core.c    | 174 +++++++--------------
 drivers/media/platform/vimc/vimc-debayer.c |  84 ++--------
 drivers/media/platform/vimc/vimc-scaler.c  |  83 ++--------
 drivers/media/platform/vimc/vimc-sensor.c  |  82 ++--------
 drivers/media/platform/vimc/vimc.h         | 121 ++++++++++++++
 8 files changed, 230 insertions(+), 419 deletions(-)
 create mode 100644 drivers/media/platform/vimc/vimc.h

Hello Shuah,

Thanks for the patch, I did some comments below.

Quoted 49 lines · Shuah Khan

vimc uses Component API to split the driver into functional components. The real hardware resembles a monolith structure than component and component structure added a level of complexity making it hard to maintain without adding any real benefit.

The sensor is one vimc component that would makes sense to be a separate module to closely align with the real hardware. It would be easier to collapse vimc into single monolithic driver first and then split the sensor off as a separate module.

This patch series emoves the component API and makes minimal changes to the code base preserving the functional division of the code structure. Preserving the functional structure allows us to split the sensor off as a separate module in the future.

Major design elements in this change are:
- Use existing struct vimc_ent_config and struct vimc_pipeline_config to drive the initialization of the functional components.
- Make vimc_ent_config global by moving it to vimc.h
- Add two new hooks add and rm to initialize and register, unregister and free subdevs.
- All component API is now gone and bind and unbind hooks are modified to do "add" and "rm" with minimal changes to just add and rm subdevs.
- vimc-core's bind and unbind are now register and unregister.
- vimc-core invokes "add" hooks from its vimc_register_devices(). The "add" hooks remain the same and register subdevs. They don't create platform devices of their own and use vimc's pdev.dev as their reference device. The "add" hooks save their vimc_ent_device(s) in the corresponding vimc_ent_config.
- vimc-core invokes "rm" hooks from its unregister to unregister subdevs and cleanup.
- vimc-core invokes "add" and "rm" hooks with pointer to struct vimc_device and the corresponding struct vimc_ent_config pointer.

The following configure and stream test works on all devices.

     media-ctl -d platform:vimc -V '"Sensor A":0[fmt:SBGGR8_1X8/640x480]'
     media-ctl -d platform:vimc -V '"Debayer A":0[fmt:SBGGR8_1X8/640x480]'
     media-ctl -d platform:vimc -V '"Sensor B":0[fmt:SBGGR8_1X8/640x480]'
     media-ctl -d platform:vimc -V '"Debayer B":0[fmt:SBGGR8_1X8/640x480]'
     
     v4l2-ctl -z platform:vimc -d "RGB/YUV Capture" -v width=1920,height=1440
     v4l2-ctl -z platform:vimc -d "Raw Capture 0" -v pixelformat=BA81
     v4l2-ctl -z platform:vimc -d "Raw Capture 1" -v pixelformat=BA81
     
     v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video1
     v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video2
     v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video3

Quoted 2 lines

The third patch in the series fixes a general protection fault found when rmmod is done while stream is active.

I applied your patch on top of media_tree/master and I did some testing. Not sure if I did something wrong, but just adding and removing the module generated a kernel panic:

~# modprobe vimc
~# rmmod vimc

[   16.452974] stack segment: 0000 [#1] SMP PTI
[   16.453688] CPU: 0 PID: 2038 Comm: rmmod Not tainted 5.3.0-rc2+ #36
[   16.454678] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996),
BIOS 1.12.0-20181126_142135-anatol 04/01/2014
[   16.456191] RIP: 0010:kfree+0x4d/0x240

<registers values...>

[   16.469188] Call Trace:
[   16.469666]  vimc_remove+0x35/0x90 [vimc]
[   16.470436]  platform_drv_remove+0x1f/0x40
[   16.471233]  device_release_driver_internal+0xd3/0x1b0
[   16.472184]  driver_detach+0x37/0x6b
[   16.472882]  bus_remove_driver+0x50/0xc1
[   16.473569]  vimc_exit+0xc/0xca0 [vimc]
[   16.474231]  __x64_sys_delete_module+0x18d/0x240
[   16.475036]  do_syscall_64+0x43/0x110
[   16.475656]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
[   16.476504] RIP: 0033:0x7fceb8dafa4b

<registers values...>

[   16.484853] Modules linked in: vimc(-) videobuf2_vmalloc
videobuf2_memops v4l2_tpg videobuf2_v4l2 videobuf2_common
[   16.486187] ---[ end trace 91e5e0894e254d49 ]---
[   16.486758] RIP: 0010:kfree+0x4d/0x240

<registers values...>

fish: “rmmod vimc” terminated by signal SIGSEGV (Address boundary error)

I just added the module after booting, no other action was made. Here is how my git log --oneline looks like:

897d708e922b media: vimc: Fix gpf in rmmod path when stream is active 2e4a5ad8ad6d media: vimc: Collapse component structure into a single monolithic driver
7c8da1687e92 media: vimc: move private defines to a common header 97299a303532 media: Remove dev_err() usage after platform_get_irq() 25a3d6bac6b9 media: adv7511/cobalt: rename driver name to adv7511-v4l2
...

Quoted 14 lines

vimc_print_dot (--print-dot) topology after this change:

digraph board {
	rankdir=TB
	n00000001 [label="{{} | Sensor A\n/dev/v4l-subdev0 | {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green]
	n00000001:port0 -> n00000005:port0 [style=bold]
	n00000001:port0 -> n0000000b [style=bold]
	n00000003 [label="{{} | Sensor B\n/dev/v4l-subdev1 | {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green]
	n00000003:port0 -> n00000008:port0 [style=bold]
	n00000003:port0 -> n0000000f [style=bold]
	n00000005 [label="{{<port0> 0} | Debayer A\n/dev/v4l-subdev2 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
	n00000005:port1 -> n00000015:port0
	n00000008 [label="{{<port0> 0} | Debayer B\n/dev/v4l-subdev3 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
	n00000008:port1 -> n00000015:port0 [style=dashed]

Quoted 8 lines

	n0000000b [label="Raw Capture 0\n/dev/video1", shape=box, style=filled, fillcolor=yellow]
	n0000000f [label="Raw Capture 1\n/dev/video2", shape=box, style=filled, fillcolor=yellow]
	n00000013 [label="{{} | RGB/YUV Input\n/dev/v4l-subdev4 | {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green]
	n00000013:port0 -> n00000015:port0 [style=dashed]
	n00000015 [label="{{<port0> 0} | Scaler\n/dev/v4l-subdev5 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
	n00000015:port1 -> n00000018 [style=bold]
	n00000018 [label="RGB/YUV Capture\n/dev/video3", shape=box, style=filled, fillcolor=yellow]
}

Since the topology changed, it would be nice to change in the documentation as well. The current dot file can be found at Documentation/media/v4l-drivers/vimc.dot and it's rendered at this
page: https://www.kernel.org/doc/html/latest/media/v4l-drivers/vimc.html

Hi Andre,

Quoted 62 lines · André Almeida

Hello Shuah,

Thanks for the patch, I did some comments below.

Quoted 57 lines · Shuah Khan

vimc uses Component API to split the driver into functional components. The real hardware resembles a monolith structure than component and component structure added a level of complexity making it hard to maintain without adding any real benefit.
     The sensor is one vimc component that would makes sense to be a separate
module to closely align with the real hardware. It would be easier to collapse vimc into single monolithic driver first and then split the sensor off as a separate module.

This patch series emoves the component API and makes minimal changes to the code base preserving the functional division of the code structure. Preserving the functional structure allows us to split the sensor off as a separate module in the future.

Major design elements in this change are:
     - Use existing struct vimc_ent_config and struct vimc_pipeline_config
       to drive the initialization of the functional components.
     - Make vimc_ent_config global by moving it to vimc.h
     - Add two new hooks add and rm to initialize and register, unregister
       and free subdevs.
     - All component API is now gone and bind and unbind hooks are modified
       to do "add" and "rm" with minimal changes to just add and rm subdevs.
     - vimc-core's bind and unbind are now register and unregister.
     - vimc-core invokes "add" hooks from its vimc_register_devices(). The "add" hooks remain the same and register subdevs. They don't create platform devices of their own and use vimc's pdev.dev as their reference device. The "add" hooks save their vimc_ent_device(s)
       in the corresponding vimc_ent_config.
     - vimc-core invokes "rm" hooks from its unregister to unregister subdevs
       and cleanup.
     - vimc-core invokes "add" and "rm" hooks with pointer to struct vimc_device
       and the corresponding struct vimc_ent_config pointer. The following configure and stream test works on all devices. media-ctl -d platform:vimc -V '"Sensor A":0[fmt:SBGGR8_1X8/640x480]'
     media-ctl -d platform:vimc -V '"Debayer A":0[fmt:SBGGR8_1X8/640x480]'
     media-ctl -d platform:vimc -V '"Sensor B":0[fmt:SBGGR8_1X8/640x480]'
     media-ctl -d platform:vimc -V '"Debayer B":0[fmt:SBGGR8_1X8/640x480]'
          v4l2-ctl -z platform:vimc -d "RGB/YUV Capture" -v width=1920,height=1440
     v4l2-ctl -z platform:vimc -d "Raw Capture 0" -v pixelformat=BA81 v4l2-ctl -z platform:vimc -d "Raw Capture 1" -v pixelformat=BA81 v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video1 v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video2 v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video3

Quoted 6 lines

Quoted 2 lines

The third patch in the series fixes a general protection fault found when rmmod is done while stream is active.

I applied your patch on top of media_tree/master and I did some testing. Not sure if I did something wrong, but just adding and removing the module generated a kernel panic:

Thanks for testing.

Odd. I tested modprobe and rmmod both.I was working on Linux 5.3-rc2. I will apply these to media latest and work from there. I have to rebase these on top of the reverts from Lucas and Helen

Quoted 77 lines

~# modprobe vimc
~# rmmod vimc
[   16.452974] stack segment: 0000 [#1] SMP PTI
[   16.453688] CPU: 0 PID: 2038 Comm: rmmod Not tainted 5.3.0-rc2+ #36 [   16.454678] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-20181126_142135-anatol 04/01/2014
[   16.456191] RIP: 0010:kfree+0x4d/0x240

<registers values...>

[   16.469188] Call Trace:
[   16.469666]  vimc_remove+0x35/0x90 [vimc]
[   16.470436]  platform_drv_remove+0x1f/0x40
[   16.471233]  device_release_driver_internal+0xd3/0x1b0 [   16.472184]  driver_detach+0x37/0x6b
[   16.472882]  bus_remove_driver+0x50/0xc1
[   16.473569]  vimc_exit+0xc/0xca0 [vimc]
[   16.474231]  __x64_sys_delete_module+0x18d/0x240 [   16.475036]  do_syscall_64+0x43/0x110
[   16.475656]  entry_SYSCALL_64_after_hwframe+0x44/0xa9 [   16.476504] RIP: 0033:0x7fceb8dafa4b

<registers values...>

[   16.484853] Modules linked in: vimc(-) videobuf2_vmalloc videobuf2_memops v4l2_tpg videobuf2_v4l2 videobuf2_common [   16.486187] ---[ end trace 91e5e0894e254d49 ]--- [   16.486758] RIP: 0010:kfree+0x4d/0x240

<registers values...>

fish: “rmmod vimc” terminated by signal SIGSEGV (Address boundary error)

I just added the module after booting, no other action was made. Here is how my git log --oneline looks like:

897d708e922b media: vimc: Fix gpf in rmmod path when stream is active 2e4a5ad8ad6d media: vimc: Collapse component structure into a single monolithic driver
7c8da1687e92 media: vimc: move private defines to a common header 97299a303532 media: Remove dev_err() usage after platform_get_irq() 25a3d6bac6b9 media: adv7511/cobalt: rename driver name to adv7511-v4l2
...

Quoted 23 lines

vimc_print_dot (--print-dot) topology after this change:
digraph board {
    rankdir=TB
    n00000001 [label="{{} | Sensor A\n/dev/v4l-subdev0 | {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green] n00000001:port0 -> n00000005:port0 [style=bold] n00000001:port0 -> n0000000b [style=bold]
    n00000003 [label="{{} | Sensor B\n/dev/v4l-subdev1 | {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green] n00000003:port0 -> n00000008:port0 [style=bold] n00000003:port0 -> n0000000f [style=bold]
    n00000005 [label="{{<port0> 0} | Debayer A\n/dev/v4l-subdev2 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green] n00000005:port1 -> n00000015:port0
    n00000008 [label="{{<port0> 0} | Debayer B\n/dev/v4l-subdev3 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green] n00000008:port1 -> n00000015:port0 [style=dashed] n0000000b [label="Raw Capture 0\n/dev/video1", shape=box, style=filled, fillcolor=yellow]
    n0000000f [label="Raw Capture 1\n/dev/video2", shape=box, style=filled, fillcolor=yellow]
    n00000013 [label="{{} | RGB/YUV Input\n/dev/v4l-subdev4 |

Quoted 8 lines

{<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green] n00000013:port0 -> n00000015:port0 [style=dashed] n00000015 [label="{{<port0> 0} | Scaler\n/dev/v4l-subdev5 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green] n00000015:port1 -> n00000018 [style=bold] n00000018 [label="RGB/YUV Capture\n/dev/video3", shape=box, style=filled, fillcolor=yellow]
}

Quoted 5 lines

Since the topology changed, it would be nice to change in the documentation as well. The current dot file can be found at Documentation/media/v4l-drivers/vimc.dot and it's rendered at this
page: https://www.kernel.org/doc/html/latest/media/v4l-drivers/vimc.html

Topology shouldn't have changed. No changes to links or pads etc. I will take a look to be sure. I agree that if topology changes document should be updated.

thanks,
-- Shuah

Quoted 72 lines · Shuah Khan

Hi Andre,

Quoted 62 lines · André Almeida

Hello Shuah,

Thanks for the patch, I did some comments below.

Quoted 57 lines · Shuah Khan

vimc uses Component API to split the driver into functional components. The real hardware resembles a monolith structure than component and component structure added a level of complexity making it hard to maintain without adding any real benefit.
     The sensor is one vimc component that would makes sense to be a
separate
module to closely align with the real hardware. It would be easier to collapse vimc into single monolithic driver first and then split the sensor off as a separate module.

This patch series emoves the component API and makes minimal changes to the code base preserving the functional division of the code structure. Preserving the functional structure allows us to split the sensor off as a separate module in the future.

Major design elements in this change are:
     - Use existing struct vimc_ent_config and struct
vimc_pipeline_config
       to drive the initialization of the functional components.
     - Make vimc_ent_config global by moving it to vimc.h
     - Add two new hooks add and rm to initialize and register,
unregister
       and free subdevs.
     - All component API is now gone and bind and unbind hooks are
modified
       to do "add" and "rm" with minimal changes to just add and rm
subdevs.
     - vimc-core's bind and unbind are now register and unregister.
     - vimc-core invokes "add" hooks from its vimc_register_devices(). The "add" hooks remain the same and register subdevs. They don't create platform devices of their own and use vimc's pdev.dev as their reference device. The "add" hooks save their
vimc_ent_device(s)
       in the corresponding vimc_ent_config.
     - vimc-core invokes "rm" hooks from its unregister to unregister
subdevs
       and cleanup.
     - vimc-core invokes "add" and "rm" hooks with pointer to struct
vimc_device
       and the corresponding struct vimc_ent_config pointer. The following configure and stream test works on all devices. media-ctl -d platform:vimc -V '"Sensor
A":0[fmt:SBGGR8_1X8/640x480]'
     media-ctl -d platform:vimc -V '"Debayer
A":0[fmt:SBGGR8_1X8/640x480]'
     media-ctl -d platform:vimc -V '"Sensor
B":0[fmt:SBGGR8_1X8/640x480]'
     media-ctl -d platform:vimc -V '"Debayer
B":0[fmt:SBGGR8_1X8/640x480]'
          v4l2-ctl -z platform:vimc -d "RGB/YUV Capture" -v
width=1920,height=1440
     v4l2-ctl -z platform:vimc -d "Raw Capture 0" -v pixelformat=BA81 v4l2-ctl -z platform:vimc -d "Raw Capture 1" -v pixelformat=BA81 v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video1 v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video2 v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video3

Quoted 6 lines

Quoted 2 lines

The third patch in the series fixes a general protection fault found when rmmod is done while stream is active.

I applied your patch on top of media_tree/master and I did some testing. Not sure if I did something wrong, but just adding and removing the module generated a kernel panic:

Quoted 5 lines

Thanks for testing.

Odd. I tested modprobe and rmmod both.I was working on Linux 5.3-rc2. I will apply these to media latest and work from there. I have to rebase these on top of the reverts from Lucas and Helen

Ok, please let me know if I succeeded to reproduce.

Quoted 83 lines

Quoted 82 lines

~# modprobe vimc
~# rmmod vimc
[   16.452974] stack segment: 0000 [#1] SMP PTI
[   16.453688] CPU: 0 PID: 2038 Comm: rmmod Not tainted 5.3.0-rc2+ #36 [   16.454678] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-20181126_142135-anatol 04/01/2014
[   16.456191] RIP: 0010:kfree+0x4d/0x240

<registers values...>

[   16.469188] Call Trace:
[   16.469666]  vimc_remove+0x35/0x90 [vimc]
[   16.470436]  platform_drv_remove+0x1f/0x40
[   16.471233]  device_release_driver_internal+0xd3/0x1b0 [   16.472184]  driver_detach+0x37/0x6b
[   16.472882]  bus_remove_driver+0x50/0xc1
[   16.473569]  vimc_exit+0xc/0xca0 [vimc]
[   16.474231]  __x64_sys_delete_module+0x18d/0x240 [   16.475036]  do_syscall_64+0x43/0x110
[   16.475656]  entry_SYSCALL_64_after_hwframe+0x44/0xa9 [   16.476504] RIP: 0033:0x7fceb8dafa4b

<registers values...>

[   16.484853] Modules linked in: vimc(-) videobuf2_vmalloc videobuf2_memops v4l2_tpg videobuf2_v4l2 videobuf2_common [   16.486187] ---[ end trace 91e5e0894e254d49 ]--- [   16.486758] RIP: 0010:kfree+0x4d/0x240

<registers values...>

fish: “rmmod vimc” terminated by signal SIGSEGV (Address boundary error)

I just added the module after booting, no other action was made. Here is how my git log --oneline looks like:

897d708e922b media: vimc: Fix gpf in rmmod path when stream is active 2e4a5ad8ad6d media: vimc: Collapse component structure into a single monolithic driver
7c8da1687e92 media: vimc: move private defines to a common header 97299a303532 media: Remove dev_err() usage after platform_get_irq() 25a3d6bac6b9 media: adv7511/cobalt: rename driver name to adv7511-v4l2
...

Quoted 23 lines

vimc_print_dot (--print-dot) topology after this change:
digraph board {
    rankdir=TB
    n00000001 [label="{{} | Sensor A\n/dev/v4l-subdev0 | {<port0>
0}}", shape=Mrecord, style=filled, fillcolor=green]
    n00000001:port0 -> n00000005:port0 [style=bold] n00000001:port0 -> n0000000b [style=bold]
    n00000003 [label="{{} | Sensor B\n/dev/v4l-subdev1 | {<port0>
0}}", shape=Mrecord, style=filled, fillcolor=green]
    n00000003:port0 -> n00000008:port0 [style=bold] n00000003:port0 -> n0000000f [style=bold]
    n00000005 [label="{{<port0> 0} | Debayer A\n/dev/v4l-subdev2 |
{<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
    n00000005:port1 -> n00000015:port0
    n00000008 [label="{{<port0> 0} | Debayer B\n/dev/v4l-subdev3 |
{<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
    n00000008:port1 -> n00000015:port0 [style=dashed] n0000000b [label="Raw Capture 0\n/dev/video1", shape=box,
style=filled, fillcolor=yellow]
    n0000000f [label="Raw Capture 1\n/dev/video2", shape=box,
style=filled, fillcolor=yellow]
    n00000013 [label="{{} | RGB/YUV Input\n/dev/v4l-subdev4 |

Quoted 8 lines

{<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green]
    n00000013:port0 -> n00000015:port0 [style=dashed] n00000015 [label="{{<port0> 0} | Scaler\n/dev/v4l-subdev5 |
{<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
    n00000015:port1 -> n00000018 [style=bold] n00000018 [label="RGB/YUV Capture\n/dev/video3", shape=box,
style=filled, fillcolor=yellow]
}

Since the topology changed, it would be nice to change in the documentation as well. The current dot file can be found at Documentation/media/v4l-drivers/vimc.dot and it's rendered at this
page: https://www.kernel.org/doc/html/latest/media/v4l-drivers/vimc.html

Quoted 3 lines

Topology shouldn't have changed. No changes to links or pads etc. I will take a look to be sure. I agree that if topology changes document should be updated.

If you "diff" the current dot with the dot you generated, you will see some differences. The main difference is that "RGB/YUV Input" was a device "/dev/video2/", and now it a subdevice "/dev/v4l-subdev4".

drivers/media/platform/vimc/vimc.h new file +102 -0

@@ -0,0 +1,102 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ *
+ * Copyright (C) 2019 Shuah Khan <[email protected]>
+ *
+ */
+
+/*
+ * This file defines vimc driver device and sub-device structures.
+ */
+
+#ifndef _VIMC_H_
+#define _VIMC_H_
+
+#include <linux/platform_device.h>
+#include <media/media-device.h>
+#include <media/v4l2-device.h>
+#include <media/videobuf2-core.h>
+#include <media/tpg/v4l2-tpg.h>
+#include <media/v4l2-ctrls.h>
+
+#include "vimc-common.h"
+
+struct vimc_cap_device {
+	struct vimc_ent_device ved;
+	struct video_device vdev;
+	struct device *dev;
+	struct v4l2_pix_format format;
+	struct vb2_queue queue;
+	struct list_head buf_list;
+	/*
+	 * NOTE: in a real driver, a spin lock must be used to access the
+	 * queue because the frames are generated from a hardware interruption
+	 * and the isr is not allowed to sleep.
+	 * Even if it is not necessary a spinlock in the vimc driver, we
+	 * use it here as a code reference
+	 */
+	spinlock_t qlock;
+	struct mutex lock;
+	u32 sequence;
+	struct vimc_stream stream;
+};
+
+struct vimc_sca_device {
+	struct vimc_ent_device ved;
+	struct v4l2_subdev sd;
+	struct device *dev;
+	/* NOTE: the source fmt is the same as the sink
+	 * with the width and hight multiplied by mult
+	 */
+	struct v4l2_mbus_framefmt sink_fmt;
+	/* Values calculated when the stream starts */
+	u8 *src_frame;
+	unsigned int src_line_size;
+	unsigned int bpp;
+};
+
+struct vimc_deb_device {
+	struct vimc_ent_device ved;
+	struct v4l2_subdev sd;
+	struct device *dev;
+	/* The active format */
+	struct v4l2_mbus_framefmt sink_fmt;
+	u32 src_code;
+	void (*set_rgb_src)(struct vimc_deb_device *vdeb, unsigned int lin,
+			    unsigned int col, unsigned int rgb[3]);
+	/* Values calculated when the stream starts */
+	u8 *src_frame;
+	const struct vimc_deb_pix_map *sink_pix_map;
+	unsigned int sink_bpp;
+};
+
+struct vimc_sen_device {
+	struct vimc_ent_device ved;
+	struct v4l2_subdev sd;
+	struct device *dev;
+	struct tpg_data tpg;
+	struct task_struct *kthread_sen;
+	u8 *frame;
+	/* The active format */
+	struct v4l2_mbus_framefmt mbus_format;
+	struct v4l2_ctrl_handler hdl;
+};
+
+struct vimc_device {
+	/* The platform device */
+	struct platform_device pdev;
+
+	/* The pipeline configuration */
+	const struct vimc_pipeline_config *pipe_cfg;
+
+	/* The Associated media_device parent */
+	struct media_device mdev;
+
+	/* Internal v4l2 parent device*/
+	struct v4l2_device v4l2_dev;
+
+	/* Subdevices */
+	struct platform_device **subdevs;
+};
+
+#endif

Quoted 72 lines · Shuah Khan

Hi Andre,

Quoted 68 lines · André Almeida

Hello Shuah,

Thanks for the patch, I did some comments below.

Quoted 57 lines · Shuah Khan

vimc uses Component API to split the driver into functional components. The real hardware resembles a monolith structure than component and component structure added a level of complexity making it hard to maintain without adding any real benefit.
     The sensor is one vimc component that would makes sense to be a separate
module to closely align with the real hardware. It would be easier to collapse vimc into single monolithic driver first and then split the sensor off as a separate module.

This patch series emoves the component API and makes minimal changes to the code base preserving the functional division of the code structure. Preserving the functional structure allows us to split the sensor off as a separate module in the future.

Major design elements in this change are:
     - Use existing struct vimc_ent_config and struct vimc_pipeline_config
       to drive the initialization of the functional components.
     - Make vimc_ent_config global by moving it to vimc.h
     - Add two new hooks add and rm to initialize and register, unregister
       and free subdevs.
     - All component API is now gone and bind and unbind hooks are modified
       to do "add" and "rm" with minimal changes to just add and rm subdevs.
     - vimc-core's bind and unbind are now register and unregister.
     - vimc-core invokes "add" hooks from its vimc_register_devices(). The "add" hooks remain the same and register subdevs. They don't create platform devices of their own and use vimc's pdev.dev as their reference device. The "add" hooks save their vimc_ent_device(s)
       in the corresponding vimc_ent_config.
     - vimc-core invokes "rm" hooks from its unregister to unregister subdevs
       and cleanup.
     - vimc-core invokes "add" and "rm" hooks with pointer to struct vimc_device
       and the corresponding struct vimc_ent_config pointer. The following configure and stream test works on all devices. media-ctl -d platform:vimc -V '"Sensor A":0[fmt:SBGGR8_1X8/640x480]'
     media-ctl -d platform:vimc -V '"Debayer A":0[fmt:SBGGR8_1X8/640x480]'
     media-ctl -d platform:vimc -V '"Sensor B":0[fmt:SBGGR8_1X8/640x480]'
     media-ctl -d platform:vimc -V '"Debayer B":0[fmt:SBGGR8_1X8/640x480]'
          v4l2-ctl -z platform:vimc -d "RGB/YUV Capture" -v width=1920,height=1440
     v4l2-ctl -z platform:vimc -d "Raw Capture 0" -v pixelformat=BA81 v4l2-ctl -z platform:vimc -d "Raw Capture 1" -v pixelformat=BA81 v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video1 v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video2 v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video3

Quoted 2 lines

The third patch in the series fixes a general protection fault found when rmmod is done while stream is active.

I applied your patch on top of media_tree/master and I did some testing. Not sure if I did something wrong, but just adding and removing the module generated a kernel panic:

Quoted 5 lines

Thanks for testing.

Odd. I tested modprobe and rmmod both.I was working on Linux 5.3-rc2. I will apply these to media latest and work from there. I have to rebase these on top of the reverts from Lucas and Helen

Ok, please let me know if I succeeded to reproduce.

Quoted 43 lines

Quoted 43 lines

~# modprobe vimc
~# rmmod vimc
[   16.452974] stack segment: 0000 [#1] SMP PTI
[   16.453688] CPU: 0 PID: 2038 Comm: rmmod Not tainted 5.3.0-rc2+ #36 [   16.454678] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-20181126_142135-anatol 04/01/2014
[   16.456191] RIP: 0010:kfree+0x4d/0x240

<registers values...>

[   16.469188] Call Trace:
[   16.469666]  vimc_remove+0x35/0x90 [vimc]
[   16.470436]  platform_drv_remove+0x1f/0x40
[   16.471233]  device_release_driver_internal+0xd3/0x1b0 [   16.472184]  driver_detach+0x37/0x6b
[   16.472882]  bus_remove_driver+0x50/0xc1
[   16.473569]  vimc_exit+0xc/0xca0 [vimc]
[   16.474231]  __x64_sys_delete_module+0x18d/0x240 [   16.475036]  do_syscall_64+0x43/0x110
[   16.475656]  entry_SYSCALL_64_after_hwframe+0x44/0xa9 [   16.476504] RIP: 0033:0x7fceb8dafa4b

<registers values...>

[   16.484853] Modules linked in: vimc(-) videobuf2_vmalloc videobuf2_memops v4l2_tpg videobuf2_v4l2 videobuf2_common [   16.486187] ---[ end trace 91e5e0894e254d49 ]--- [   16.486758] RIP: 0010:kfree+0x4d/0x240

<registers values...>

fish: “rmmod vimc” terminated by signal SIGSEGV (Address boundary error)

I just added the module after booting, no other action was made. Here is how my git log --oneline looks like:

897d708e922b media: vimc: Fix gpf in rmmod path when stream is active 2e4a5ad8ad6d media: vimc: Collapse component structure into a single monolithic driver
7c8da1687e92 media: vimc: move private defines to a common header 97299a303532 media: Remove dev_err() usage after platform_get_irq() 25a3d6bac6b9 media: adv7511/cobalt: rename driver name to adv7511-v4l2

Quoted 40 lines

Quoted 39 lines

...

Quoted 23 lines

vimc_print_dot (--print-dot) topology after this change:
digraph board {
    rankdir=TB
    n00000001 [label="{{} | Sensor A\n/dev/v4l-subdev0 | {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green] n00000001:port0 -> n00000005:port0 [style=bold] n00000001:port0 -> n0000000b [style=bold]
    n00000003 [label="{{} | Sensor B\n/dev/v4l-subdev1 | {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green] n00000003:port0 -> n00000008:port0 [style=bold] n00000003:port0 -> n0000000f [style=bold]
    n00000005 [label="{{<port0> 0} | Debayer A\n/dev/v4l-subdev2 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green] n00000005:port1 -> n00000015:port0
    n00000008 [label="{{<port0> 0} | Debayer B\n/dev/v4l-subdev3 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green] n00000008:port1 -> n00000015:port0 [style=dashed] n0000000b [label="Raw Capture 0\n/dev/video1", shape=box, style=filled, fillcolor=yellow]
    n0000000f [label="Raw Capture 1\n/dev/video2", shape=box, style=filled, fillcolor=yellow]
    n00000013 [label="{{} | RGB/YUV Input\n/dev/v4l-subdev4 |

Quoted 8 lines

{<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green] n00000013:port0 -> n00000015:port0 [style=dashed] n00000015 [label="{{<port0> 0} | Scaler\n/dev/v4l-subdev5 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green] n00000015:port1 -> n00000018 [style=bold] n00000018 [label="RGB/YUV Capture\n/dev/video3", shape=box, style=filled, fillcolor=yellow]
}

Since the topology changed, it would be nice to change in the documentation as well. The current dot file can be found at Documentation/media/v4l-drivers/vimc.dot and it's rendered at this
page: https://www.kernel.org/doc/html/latest/media/v4l-drivers/vimc.html

Quoted 3 lines

Topology shouldn't have changed. No changes to links or pads etc. I will take a look to be sure. I agree that if topology changes document should be updated.

If you "diff" the current dot with the dot you generated, you will see some differences. The main difference is that "RGB/YUV Input" was a device "/dev/video2/", and now it a subdevice "/dev/v4l-subdev4".

Quoted 4 lines

thanks,
-- Shuah

vimc uses Component API to split the driver into functional components. The real hardware resembles a monolith structure than component and component structure added a level of complexity making it hard to maintain without adding any real benefit.

The sensor is one vimc component that would makes sense to be a separate module to closely align with the real hardware. It would be easier to collapse vimc into single monolithic driver first and then split the sensor off as a separate module.

Collapse it into a single monolithic driver removing the Component API. This patch removes the component API and makes minimal changes to the code base preserving the functional division of the code structure. Preserving the functional structure allows us to split the sensor off as a separate module in the future.

Major design elements in this change are:
- Use existing struct vimc_ent_config and struct vimc_pipeline_config to drive the initialization of the functional components.
- Make vimc_ent_config global by moving it to vimc.h
- Add two new hooks add and rm to initialize and register, unregister and free subdevs.
- All component API is now gone and bind and unbind hooks are modified to do "add" and "rm" with minimal changes to just add and rm subdevs.
- vimc-core's bind and unbind are now register and unregister.
- vimc-core invokes "add" hooks from its vimc_register_devices(). The "add" hooks remain the same and register subdevs. They don't create platform devices of their own and use vimc's pdev.dev as their reference device. The "add" hooks save their vimc_ent_device(s) in the corresponding vimc_ent_config.
- vimc-core invokes "rm" hooks from its unregister to unregister subdevs and cleanup.
- vimc-core invokes "add" and "rm" hooks with pointer to struct vimc_device and the corresponding struct vimc_ent_config pointer.

The following configure and stream test works on all devices.

media-ctl -d platform:vimc -V '"Sensor A":0[fmt:SBGGR8_1X8/640x480]' media-ctl -d platform:vimc -V '"Debayer A":0[fmt:SBGGR8_1X8/640x480]' media-ctl -d platform:vimc -V '"Sensor B":0[fmt:SBGGR8_1X8/640x480]' media-ctl -d platform:vimc -V '"Debayer B":0[fmt:SBGGR8_1X8/640x480]'

v4l2-ctl -z platform:vimc -d "RGB/YUV Capture" -v width=1920,height=1440 v4l2-ctl -z platform:vimc -d "Raw Capture 0" -v pixelformat=BA81 v4l2-ctl -z platform:vimc -d "Raw Capture 1" -v pixelformat=BA81

v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video1 v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video2 v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video3

Signed-off-by Shuah Khan <[email protected]>

---
 drivers/media/platform/vimc/Makefile       |   7 +-
 drivers/media/platform/vimc/vimc-capture.c |  75 ++--------
 drivers/media/platform/vimc/vimc-core.c    | 156 ++++++++-------------
 drivers/media/platform/vimc/vimc-debayer.c |  68 ++-------
 drivers/media/platform/vimc/vimc-scaler.c  |  68 ++-------
 drivers/media/platform/vimc/vimc-sensor.c  |  69 ++-------
 drivers/media/platform/vimc/vimc.h         |  25 +++-
 7 files changed, 125 insertions(+), 343 deletions(-)

drivers/media/platform/vimc/Makefile +4 -3

index 96d06f030c31..a53b2b532e9f 100644
@@ -1,5 +1,6 @@
 # SPDX-License-Identifier: GPL-2.0
-vimc-y := vimc-core.o vimc-common.o vimc-streamer.o
+vimc-y := vimc-core.o vimc-common.o vimc-streamer.o vimc-capture.o \
+		vimc-debayer.o vimc-scaler.o vimc-sensor.o
+
+obj-$(CONFIG_VIDEO_VIMC) += vimc.o
 
-obj-$(CONFIG_VIDEO_VIMC) += vimc.o vimc-capture.o vimc-debayer.o \
-                vimc-scaler.o vimc-sensor.o

drivers/media/platform/vimc/vimc-capture.c +13 -62

index c52fc5d97c2d..b7b2d3c3d4f8 100644
@@ -5,10 +5,6 @@
  * Copyright (C) 2015-2017 Helen Koike <[email protected]>
  */
 
-#include <linux/component.h>
-#include <linux/module.h>
-#include <linux/mod_devicetable.h>
-#include <linux/platform_device.h>
 #include <media/v4l2-ioctl.h>
 #include <media/videobuf2-core.h>
 #include <media/videobuf2-vmalloc.h>

@@ -17,8 +13,6 @@
 #include "vimc-common.h"
 #include "vimc-streamer.h"
 
-#define VIMC_CAP_DRV_NAME "vimc-capture"
-
 static const u32 vimc_cap_supported_pixfmt[] = {
 	V4L2_PIX_FMT_BGR24,
 	V4L2_PIX_FMT_RGB24,

@@ -348,11 +342,11 @@ static void vimc_cap_release(struct video_device *vdev)
 	kfree(vcap);
 }
 
-static void vimc_cap_comp_unbind(struct device *comp, struct device *master,
-				 void *master_data)
+void vimc_cap_rm(struct vimc_device *vimc, struct vimc_ent_config *vent)
 {
-	struct vimc_ent_device *ved = dev_get_drvdata(comp);
-	struct vimc_cap_device *vcap = container_of(ved, struct vimc_cap_device,
+	struct vimc_ent_device *ved = vent->ved;
+	struct vimc_cap_device *vcap = container_of(ved,
+						    struct vimc_cap_device,
 						    ved);
 
 	vb2_queue_release(&vcap->queue);

@@ -399,11 +393,9 @@ static void *vimc_cap_process_frame(struct vimc_ent_device *ved,
 	return NULL;
 }
 
-static int vimc_cap_comp_bind(struct device *comp, struct device *master,
-			      void *master_data)
+int vimc_cap_add(struct vimc_device *vimc, struct vimc_ent_config *vent)
 {
-	struct v4l2_device *v4l2_dev = master_data;
-	struct vimc_platform_data *pdata = comp->platform_data;
+	struct v4l2_device *v4l2_dev = &vimc->v4l2_dev;
 	struct vimc_cap_device *vcap;
 	struct video_device *vdev;
 	struct vb2_queue *q;

@@ -423,7 +415,7 @@ static int vimc_cap_comp_bind(struct device *comp, struct device *master,
 	}
 
 	/* Initialize the media entity */
-	vcap->vdev.entity.name = pdata->entity_name;
+	vcap->vdev.entity.name = vent->name;
 	vcap->vdev.entity.function = MEDIA_ENT_F_IO_V4L;
 	ret = media_entity_pads_init(&vcap->vdev.entity,
 				     1, vcap->ved.pads);

@@ -447,8 +439,8 @@ static int vimc_cap_comp_bind(struct device *comp, struct device *master,
 
 	ret = vb2_queue_init(q);
 	if (ret) {
-		dev_err(comp, "%s: vb2 queue init failed (err=%d)\n",
-			pdata->entity_name, ret);
+		dev_err(&vimc->pdev.dev, "%s: vb2 queue init failed (err=%d)\n",
+			vent->name, ret);
 		goto err_clean_m_ent;
 	}
 

@@ -465,8 +457,7 @@ static int vimc_cap_comp_bind(struct device *comp, struct device *master,
 	vcap->ved.ent = &vcap->vdev.entity;
 	vcap->ved.process_frame = vimc_cap_process_frame;
 	vcap->ved.vdev_get_format = vimc_cap_get_format;
-	dev_set_drvdata(comp, &vcap->ved);
-	vcap->dev = comp;
+	vcap->dev = &vimc->pdev.dev;
 
 	/* Initialize the video_device struct */
 	vdev = &vcap->vdev;

@@ -479,17 +470,18 @@ static int vimc_cap_comp_bind(struct device *comp, struct device *master,
 	vdev->queue = q;
 	vdev->v4l2_dev = v4l2_dev;
 	vdev->vfl_dir = VFL_DIR_RX;
-	strscpy(vdev->name, pdata->entity_name, sizeof(vdev->name));
+	strscpy(vdev->name, vent->name, sizeof(vdev->name));
 	video_set_drvdata(vdev, &vcap->ved);
 
 	/* Register the video_device with the v4l2 and the media framework */
 	ret = video_register_device(vdev, VFL_TYPE_GRABBER, -1);
 	if (ret) {
-		dev_err(comp, "%s: video register failed (err=%d)\n",
+		dev_err(&vimc->pdev.dev, "%s: video register failed (err=%d)\n",
 			vcap->vdev.name, ret);
 		goto err_release_queue;
 	}
 
+	vent->ved = &vcap->ved;
 	return 0;
 
 err_release_queue:

@@ -503,44 +495,3 @@ static int vimc_cap_comp_bind(struct device *comp, struct device *master,
 
 	return ret;
 }
-
-static const struct component_ops vimc_cap_comp_ops = {
-	.bind = vimc_cap_comp_bind,
-	.unbind = vimc_cap_comp_unbind,
-};
-
-static int vimc_cap_probe(struct platform_device *pdev)
-{
-	return component_add(&pdev->dev, &vimc_cap_comp_ops);
-}
-
-static int vimc_cap_remove(struct platform_device *pdev)
-{
-	component_del(&pdev->dev, &vimc_cap_comp_ops);
-
-	return 0;
-}
-
-static const struct platform_device_id vimc_cap_driver_ids[] = {
-	{
-		.name           = VIMC_CAP_DRV_NAME,
-	},
-	{ }
-};
-
-static struct platform_driver vimc_cap_pdrv = {
-	.probe		= vimc_cap_probe,
-	.remove		= vimc_cap_remove,
-	.id_table	= vimc_cap_driver_ids,
-	.driver		= {
-		.name	= VIMC_CAP_DRV_NAME,
-	},
-};
-
-module_platform_driver(vimc_cap_pdrv);
-
-MODULE_DEVICE_TABLE(platform, vimc_cap_driver_ids);
-
-MODULE_DESCRIPTION("Virtual Media Controller Driver (VIMC) Capture");
-MODULE_AUTHOR("Helen Mae Koike Fornazier <[email protected]>");
-MODULE_LICENSE("GPL");

drivers/media/platform/vimc/vimc-core.c +20 -9

index c9b351472272..3bd89cf38a90 100644
@@ -5,7 +5,6 @@
  * Copyright (C) 2015-2017 Helen Koike <[email protected]>
  */
 
-#include <linux/component.h>
 #include <linux/init.h>
 #include <linux/module.h>
 #include <linux/platform_device.h>

@@ -25,12 +24,6 @@
 	.flags = link_flags,					\
 }
 
-/* Structure which describes individual configuration for each entity */
-struct vimc_ent_config {
-	const char *name;
-	const char *drv;
-};
-
 /* Structure which describes links between entities */
 struct vimc_ent_link {
 	unsigned int src_ent;

@@ -42,7 +35,7 @@ struct vimc_ent_link {
 
 /* Structure which describes the whole topology */
 struct vimc_pipeline_config {
-	const struct vimc_ent_config *ents;
+	struct vimc_ent_config *ents;
 	size_t num_ents;
 	const struct vimc_ent_link *links;
 	size_t num_links;

@@ -52,43 +45,61 @@ struct vimc_pipeline_config {
  * Topology Configuration
  */
 
-static const struct vimc_ent_config ent_config[] = {
+static struct vimc_ent_config ent_config[] = {
 	{
 		.name = "Sensor A",
 		.drv = "vimc-sensor",
+		.add = vimc_sen_add,
+		.rm = vimc_sen_rm,
 	},
 	{
 		.name = "Sensor B",
 		.drv = "vimc-sensor",
+		.add = vimc_sen_add,
+		.rm = vimc_sen_rm,
 	},
 	{
 		.name = "Debayer A",
 		.drv = "vimc-debayer",
+		.add = vimc_deb_add,
+		.rm = vimc_deb_rm,
 	},
 	{
 		.name = "Debayer B",
 		.drv = "vimc-debayer",
+		.add = vimc_deb_add,
+		.rm = vimc_deb_rm,
 	},
 	{
 		.name = "Raw Capture 0",
 		.drv = "vimc-capture",
+		.add = vimc_cap_add,
+		.rm = vimc_cap_rm,
 	},
 	{
 		.name = "Raw Capture 1",
 		.drv = "vimc-capture",
+		.add = vimc_cap_add,
+		.rm = vimc_cap_rm,
 	},
 	{
 		.name = "RGB/YUV Input",
 		/* TODO: change this to vimc-input when it is implemented */
 		.drv = "vimc-sensor",
+		.add = vimc_sen_add,
+		.rm = vimc_sen_rm,
 	},
 	{
 		.name = "Scaler",
 		.drv = "vimc-scaler",
+		.add = vimc_sca_add,
+		.rm = vimc_sca_rm,
 	},
 	{
 		.name = "RGB/YUV Capture",
 		.drv = "vimc-capture",
+		.add = vimc_cap_add,
+		.rm = vimc_cap_rm,
 	},
 };
 

drivers/media/platform/vimc/vimc-core.c +3 -6

 	.ents		= ent_config,
 	.num_ents	= ARRAY_SIZE(ent_config),
 	.links		= ent_links,

@@ -128,14 +139,11 @@ static int vimc_create_links(struct vimc_device *vimc)
 	/* Initialize the links between entities */
 	for (i = 0; i < vimc->pipe_cfg->num_links; i++) {
 		const struct vimc_ent_link *link = &vimc->pipe_cfg->links[i];
-		/*
-		 * TODO: Check another way of retrieving ved struct without
-		 * relying on platform_get_drvdata
-		 */
+
 		struct vimc_ent_device *ved_src =
-			platform_get_drvdata(vimc->subdevs[link->src_ent]);
+			vimc->pipe_cfg->ents[link->src_ent].ved;
 		struct vimc_ent_device *ved_sink =
-			platform_get_drvdata(vimc->subdevs[link->sink_ent]);
+			vimc->pipe_cfg->ents[link->sink_ent].ved;
 
 		ret = media_create_pad_link(ved_src->ent, link->src_pad,
 					    ved_sink->ent, link->sink_pad,

drivers/media/platform/vimc/vimc-core.c +28 -81

+
+	for (i = 0; i < vimc->pipe_cfg->num_ents; i++) {
+		dev_dbg(&vimc->pdev.dev, "new entity for %s\n",
+			vimc->pipe_cfg->ents[i].name);
+		ret = vimc->pipe_cfg->ents[i].add(vimc,
+				&vimc->pipe_cfg->ents[i]);
+		if (ret) {
+			dev_err(&vimc->pdev.dev, "add new entity for %s\n",
+				vimc->pipe_cfg->ents[i].name);
+			return ret;
+		}
+	}
+	return 0;
+}
 
-	dev_dbg(master, "bind");
+static int vimc_register_devices(struct vimc_device *vimc)
+{
+	int ret;
 
 	/* Register the v4l2 struct */
 	ret = v4l2_device_register(vimc->mdev.dev, &vimc->v4l2_dev);

@@ -163,22 +186,20 @@ static int vimc_comp_bind(struct device *master)
 		return ret;
 	}
 
-	/* Bind subdevices */
-	ret = component_bind_all(master, &vimc->v4l2_dev);
-	if (ret)
-		goto err_v4l2_unregister;
+	/* Invoke enitity config hooks to initialize and register subdevs */
+	vimc_add_subdevs(vimc);
 
 	/* Initialize links */
 	ret = vimc_create_links(vimc);
 	if (ret)
-		goto err_comp_unbind_all;
+		goto err_v4l2_unregister;
 
 	/* Register the media device */
 	ret = media_device_register(&vimc->mdev);
 	if (ret) {
 		dev_err(vimc->mdev.dev,
 			"media device register failed (err=%d)\n", ret);
-		goto err_comp_unbind_all;
+		goto err_v4l2_unregister;
 	}
 
 	/* Expose all subdev's nodes*/

@@ -195,98 +216,36 @@ static int vimc_comp_bind(struct device *master)
 err_mdev_unregister:
 	media_device_unregister(&vimc->mdev);
 	media_device_cleanup(&vimc->mdev);
-err_comp_unbind_all:
-	component_unbind_all(master, NULL);
 err_v4l2_unregister:
 	v4l2_device_unregister(&vimc->v4l2_dev);
 
 	return ret;
 }
 
-static void vimc_comp_unbind(struct device *master)
+static void vimc_unregister(struct vimc_device *vimc)
 {
-	struct vimc_device *vimc = container_of(to_platform_device(master),
-						struct vimc_device, pdev);
-
-	dev_dbg(master, "unbind");
-
 	media_device_unregister(&vimc->mdev);
 	media_device_cleanup(&vimc->mdev);
-	component_unbind_all(master, NULL);
 	v4l2_device_unregister(&vimc->v4l2_dev);
 }
 
-static int vimc_comp_compare(struct device *comp, void *data)
-{
-	return comp == data;
-}
-
-static struct component_match *vimc_add_subdevs(struct vimc_device *vimc)
-{
-	struct component_match *match = NULL;
-	struct vimc_platform_data pdata;
-	int i;
-
-	for (i = 0; i < vimc->pipe_cfg->num_ents; i++) {
-		dev_dbg(&vimc->pdev.dev, "new pdev for %s\n",
-			vimc->pipe_cfg->ents[i].drv);
-
-		strscpy(pdata.entity_name, vimc->pipe_cfg->ents[i].name,
-			sizeof(pdata.entity_name));
-
-		vimc->subdevs[i] = platform_device_register_data(&vimc->pdev.dev,
-						vimc->pipe_cfg->ents[i].drv,
-						PLATFORM_DEVID_AUTO,
-						&pdata,
-						sizeof(pdata));
-		if (IS_ERR(vimc->subdevs[i])) {
-			match = ERR_CAST(vimc->subdevs[i]);
-			while (--i >= 0)
-				platform_device_unregister(vimc->subdevs[i]);
-
-			return match;
-		}
-
-		component_match_add(&vimc->pdev.dev, &match, vimc_comp_compare,
-				    &vimc->subdevs[i]->dev);
-	}
-
-	return match;
-}
-
 static void vimc_rm_subdevs(struct vimc_device *vimc)
 {
 	unsigned int i;
 
 	for (i = 0; i < vimc->pipe_cfg->num_ents; i++)
-		platform_device_unregister(vimc->subdevs[i]);
+		vimc->pipe_cfg->ents[i].rm(vimc, &vimc->pipe_cfg->ents[i]);
 }
 
-static const struct component_master_ops vimc_comp_ops = {
-	.bind = vimc_comp_bind,
-	.unbind = vimc_comp_unbind,
-};
-
 static int vimc_probe(struct platform_device *pdev)
 {
 	struct vimc_device *vimc = container_of(pdev, struct vimc_device, pdev);
-	struct component_match *match = NULL;
-	int ret;
+	int ret = 0;
 
 	dev_dbg(&pdev->dev, "probe");
 
 	memset(&vimc->mdev, 0, sizeof(vimc->mdev));
 
-	/* Create platform_device for each entity in the topology*/
-	vimc->subdevs = devm_kcalloc(&vimc->pdev.dev, vimc->pipe_cfg->num_ents,
-				     sizeof(*vimc->subdevs), GFP_KERNEL);
-	if (!vimc->subdevs)
-		return -ENOMEM;
-
-	match = vimc_add_subdevs(vimc);
-	if (IS_ERR(match))
-		return PTR_ERR(match);
-
 	/* Link the media device within the v4l2_device */
 	vimc->v4l2_dev.mdev = &vimc->mdev;
 

@@ -298,16 +257,11 @@ static int vimc_probe(struct platform_device *pdev)
 	vimc->mdev.dev = &pdev->dev;
 	media_device_init(&vimc->mdev);
 
-	/* Add self to the component system */
-	ret = component_master_add_with_match(&pdev->dev, &vimc_comp_ops,
-					      match);
-	if (ret) {
+	ret = vimc_register_devices(vimc);
+	if (ret)
 		media_device_cleanup(&vimc->mdev);
-		vimc_rm_subdevs(vimc);
-		return ret;
-	}
 
-	return 0;
+	return ret;
 }
 
 static int vimc_remove(struct platform_device *pdev)

@@ -316,8 +270,8 @@ static int vimc_remove(struct platform_device *pdev)
 
 	dev_dbg(&pdev->dev, "remove");
 
-	component_master_del(&pdev->dev, &vimc_comp_ops);
 	vimc_rm_subdevs(vimc);
+	vimc_unregister(vimc);
 
 	return 0;
 }

drivers/media/platform/vimc/vimc-debayer.c +10 -58

index 750752bb173c..b58e0946ac2a 100644
@@ -5,10 +5,7 @@
  * Copyright (C) 2015-2017 Helen Koike <[email protected]>
  */
 
-#include <linux/component.h>
 #include <linux/module.h>
-#include <linux/mod_devicetable.h>
-#include <linux/platform_device.h>
 #include <linux/vmalloc.h>
 #include <linux/v4l2-mediabus.h>
 #include <media/v4l2-subdev.h>

@@ -16,9 +13,7 @@
 #include "vimc.h"
 #include "vimc-common.h"
 
-#define VIMC_DEB_DRV_NAME "vimc-debayer"
-/* This module only supports transforming a bayer format
- * to V4L2_PIX_FMT_RGB24
+/* Supports transforming a bayer format to V4L2_PIX_FMT_RGB24
  */
 #define VIMC_DEB_SRC_PIXFMT V4L2_PIX_FMT_RGB24
 #define VIMC_DEB_SRC_MBUS_FMT_DEFAULT MEDIA_BUS_FMT_RGB888_1X24

@@ -499,21 +494,19 @@ static const struct v4l2_subdev_internal_ops vimc_deb_int_ops = {
 	.release = vimc_deb_release,
 };
 
-static void vimc_deb_comp_unbind(struct device *comp, struct device *master,
-				 void *master_data)
+void vimc_deb_rm(struct vimc_device *vimc, struct vimc_ent_config *vent)
 {
-	struct vimc_ent_device *ved = dev_get_drvdata(comp);
-	struct vimc_deb_device *vdeb = container_of(ved, struct vimc_deb_device,
+	struct vimc_ent_device *ved = vent->ved;
+	struct vimc_deb_device *vdeb = container_of(ved,
+						    struct vimc_deb_device,
 						    ved);
 
 	vimc_ent_sd_unregister(ved, &vdeb->sd);
 }
 
-static int vimc_deb_comp_bind(struct device *comp, struct device *master,
-			      void *master_data)
+int vimc_deb_add(struct vimc_device *vimc, struct vimc_ent_config *vent)
 {
-	struct v4l2_device *v4l2_dev = master_data;
-	struct vimc_platform_data *pdata = comp->platform_data;
+	struct v4l2_device *v4l2_dev = &vimc->v4l2_dev;
 	struct vimc_deb_device *vdeb;
 	int ret;
 

@@ -524,7 +517,7 @@ static int vimc_deb_comp_bind(struct device *comp, struct device *master,
 
 	/* Initialize ved and sd */
 	ret = vimc_ent_sd_register(&vdeb->ved, &vdeb->sd, v4l2_dev,
-				   pdata->entity_name,
+				   vent->name,
 				   MEDIA_ENT_F_PROC_VIDEO_PIXEL_ENC_CONV, 2,
 				   (const unsigned long[2]) {MEDIA_PAD_FL_SINK,
 				   MEDIA_PAD_FL_SOURCE},

@@ -535,8 +528,7 @@ static int vimc_deb_comp_bind(struct device *comp, struct device *master,
 	}
 
 	vdeb->ved.process_frame = vimc_deb_process_frame;
-	dev_set_drvdata(comp, &vdeb->ved);
-	vdeb->dev = comp;
+	vdeb->dev = &vimc->pdev.dev;
 
 	/* Initialize the frame format */
 	vdeb->sink_fmt = sink_fmt_default;

@@ -549,46 +541,6 @@ static int vimc_deb_comp_bind(struct device *comp, struct device *master,
 	 */
 	vdeb->set_rgb_src = vimc_deb_set_rgb_pix_rgb24;
 
+	vent->ved = &vdeb->ved;
 	return 0;
 }
-
-static const struct component_ops vimc_deb_comp_ops = {
-	.bind = vimc_deb_comp_bind,
-	.unbind = vimc_deb_comp_unbind,
-};
-
-static int vimc_deb_probe(struct platform_device *pdev)
-{
-	return component_add(&pdev->dev, &vimc_deb_comp_ops);
-}
-
-static int vimc_deb_remove(struct platform_device *pdev)
-{
-	component_del(&pdev->dev, &vimc_deb_comp_ops);
-
-	return 0;
-}
-
-static const struct platform_device_id vimc_deb_driver_ids[] = {
-	{
-		.name           = VIMC_DEB_DRV_NAME,
-	},
-	{ }
-};
-
-static struct platform_driver vimc_deb_pdrv = {
-	.probe		= vimc_deb_probe,
-	.remove		= vimc_deb_remove,
-	.id_table	= vimc_deb_driver_ids,
-	.driver		= {
-		.name	= VIMC_DEB_DRV_NAME,
-	},
-};
-
-module_platform_driver(vimc_deb_pdrv);
-
-MODULE_DEVICE_TABLE(platform, vimc_deb_driver_ids);
-
-MODULE_DESCRIPTION("Virtual Media Controller Driver (VIMC) Debayer");
-MODULE_AUTHOR("Helen Mae Koike Fornazier <[email protected]>");
-MODULE_LICENSE("GPL");

drivers/media/platform/vimc/vimc-scaler.c +11 -57

index fe99b9102ada..7cd478fee3ae 100644
@@ -5,10 +5,7 @@
  * Copyright (C) 2015-2017 Helen Koike <[email protected]>
  */
 
-#include <linux/component.h>
 #include <linux/module.h>
-#include <linux/mod_devicetable.h>
-#include <linux/platform_device.h>
 #include <linux/vmalloc.h>
 #include <linux/v4l2-mediabus.h>
 #include <media/v4l2-subdev.h>

@@ -16,8 +13,6 @@
 #include "vimc.h"
 #include "vimc-common.h"
 
-#define VIMC_SCA_DRV_NAME "vimc-scaler"
-
 static unsigned int sca_mult = 3;
 module_param(sca_mult, uint, 0000);
 MODULE_PARM_DESC(sca_mult, " the image size multiplier");

@@ -331,22 +326,21 @@ static const struct v4l2_subdev_internal_ops vimc_sca_int_ops = {
 	.release = vimc_sca_release,
 };
 
-static void vimc_sca_comp_unbind(struct device *comp, struct device *master,
-				 void *master_data)
+void vimc_sca_rm(struct vimc_device *vimc, struct vimc_ent_config *vent)
 {
-	struct vimc_ent_device *ved = dev_get_drvdata(comp);
-	struct vimc_sca_device *vsca = container_of(ved, struct vimc_sca_device,
+	struct vimc_ent_device *ved = vent->ved;
+	struct vimc_sca_device *vsca = container_of(ved,
+						    struct vimc_sca_device,
 						    ved);
 
+
 	vimc_ent_sd_unregister(ved, &vsca->sd);
 }
 
 
-static int vimc_sca_comp_bind(struct device *comp, struct device *master,
-			      void *master_data)
+int vimc_sca_add(struct vimc_device *vimc, struct vimc_ent_config *vent)
 {
-	struct v4l2_device *v4l2_dev = master_data;
-	struct vimc_platform_data *pdata = comp->platform_data;
+	struct v4l2_device *v4l2_dev = &vimc->v4l2_dev;
 	struct vimc_sca_device *vsca;
 	int ret;
 

@@ -357,7 +351,7 @@ static int vimc_sca_comp_bind(struct device *comp, struct device *master,
 
 	/* Initialize ved and sd */
 	ret = vimc_ent_sd_register(&vsca->ved, &vsca->sd, v4l2_dev,
-				   pdata->entity_name,
+				   vent->name,
 				   MEDIA_ENT_F_PROC_VIDEO_SCALER, 2,
 				   (const unsigned long[2]) {MEDIA_PAD_FL_SINK,
 				   MEDIA_PAD_FL_SOURCE},

@@ -368,52 +362,12 @@ static int vimc_sca_comp_bind(struct device *comp, struct device *master,
 	}
 
 	vsca->ved.process_frame = vimc_sca_process_frame;
-	dev_set_drvdata(comp, &vsca->ved);
-	vsca->dev = comp;
+	vsca->dev = &vimc->pdev.dev;
+
+	vent->ved = &vsca->ved;
 
 	/* Initialize the frame format */
 	vsca->sink_fmt = sink_fmt_default;
 
 	return 0;
 }
-
-static const struct component_ops vimc_sca_comp_ops = {
-	.bind = vimc_sca_comp_bind,
-	.unbind = vimc_sca_comp_unbind,
-};
-
-static int vimc_sca_probe(struct platform_device *pdev)
-{
-	return component_add(&pdev->dev, &vimc_sca_comp_ops);
-}
-
-static int vimc_sca_remove(struct platform_device *pdev)
-{
-	component_del(&pdev->dev, &vimc_sca_comp_ops);
-
-	return 0;
-}
-
-static const struct platform_device_id vimc_sca_driver_ids[] = {
-	{
-		.name           = VIMC_SCA_DRV_NAME,
-	},
-	{ }
-};
-
-static struct platform_driver vimc_sca_pdrv = {
-	.probe		= vimc_sca_probe,
-	.remove		= vimc_sca_remove,
-	.id_table	= vimc_sca_driver_ids,
-	.driver		= {
-		.name	= VIMC_SCA_DRV_NAME,
-	},
-};
-
-module_platform_driver(vimc_sca_pdrv);
-
-MODULE_DEVICE_TABLE(platform, vimc_sca_driver_ids);
-
-MODULE_DESCRIPTION("Virtual Media Controller Driver (VIMC) Scaler");
-MODULE_AUTHOR("Helen Mae Koike Fornazier <[email protected]>");
-MODULE_LICENSE("GPL");

drivers/media/platform/vimc/vimc-sensor.c +10 -59

index 6c57b1e262f9..bd237e030a1b 100644
@@ -5,10 +5,6 @@
  * Copyright (C) 2015-2017 Helen Koike <[email protected]>
  */
 
-#include <linux/component.h>
-#include <linux/module.h>
-#include <linux/mod_devicetable.h>
-#include <linux/platform_device.h>
 #include <linux/v4l2-mediabus.h>
 #include <linux/vmalloc.h>
 #include <media/v4l2-ctrls.h>

@@ -19,8 +15,6 @@
 #include "vimc.h"
 #include "vimc-common.h"
 
-#define VIMC_SEN_DRV_NAME "vimc-sensor"
-
 static const struct v4l2_mbus_framefmt fmt_default = {
 	.width = 640,
 	.height = 480,

@@ -268,12 +262,12 @@ static const struct v4l2_subdev_internal_ops vimc_sen_int_ops = {
 	.release = vimc_sen_release,
 };
 
-static void vimc_sen_comp_unbind(struct device *comp, struct device *master,
-				 void *master_data)
+void vimc_sen_rm(struct vimc_device *vimc, struct vimc_ent_config *vent)
 {
-	struct vimc_ent_device *ved = dev_get_drvdata(comp);
-	struct vimc_sen_device *vsen =
-				container_of(ved, struct vimc_sen_device, ved);
+	struct vimc_ent_device *ved = vent->ved;
+	struct vimc_sen_device *vsen = container_of(ved,
+						    struct vimc_sen_device,
+						    ved);
 
 	vimc_ent_sd_unregister(ved, &vsen->sd);
 }

@@ -295,11 +289,9 @@ static const struct v4l2_ctrl_config vimc_sen_ctrl_test_pattern = {
 	.qmenu = tpg_pattern_strings,
 };
 
-static int vimc_sen_comp_bind(struct device *comp, struct device *master,
-			      void *master_data)
+int vimc_sen_add(struct vimc_device *vimc, struct vimc_ent_config *vent)
 {
-	struct v4l2_device *v4l2_dev = master_data;
-	struct vimc_platform_data *pdata = comp->platform_data;
+	struct v4l2_device *v4l2_dev = &vimc->v4l2_dev;
 	struct vimc_sen_device *vsen;
 	int ret;
 

@@ -332,7 +324,7 @@ static int vimc_sen_comp_bind(struct device *comp, struct device *master,
 
 	/* Initialize ved and sd */
 	ret = vimc_ent_sd_register(&vsen->ved, &vsen->sd, v4l2_dev,
-				   pdata->entity_name,
+				   vent->name,
 				   MEDIA_ENT_F_CAM_SENSOR, 1,
 				   (const unsigned long[1]) {MEDIA_PAD_FL_SOURCE},
 				   &vimc_sen_int_ops, &vimc_sen_ops);

@@ -340,8 +332,7 @@ static int vimc_sen_comp_bind(struct device *comp, struct device *master,
 		goto err_free_hdl;
 
 	vsen->ved.process_frame = vimc_sen_process_frame;
-	dev_set_drvdata(comp, &vsen->ved);
-	vsen->dev = comp;
+	vsen->dev = &vimc->pdev.dev;
 
 	/* Initialize the frame format */
 	vsen->mbus_format = fmt_default;

@@ -353,6 +344,7 @@ static int vimc_sen_comp_bind(struct device *comp, struct device *master,
 	if (ret)
 		goto err_unregister_ent_sd;
 
+	vent->ved = &vsen->ved;
 	return 0;
 
 err_unregister_ent_sd:

@@ -364,44 +356,3 @@ static int vimc_sen_comp_bind(struct device *comp, struct device *master,
 
 	return ret;
 }
-
-static const struct component_ops vimc_sen_comp_ops = {
-	.bind = vimc_sen_comp_bind,
-	.unbind = vimc_sen_comp_unbind,
-};
-
-static int vimc_sen_probe(struct platform_device *pdev)
-{
-	return component_add(&pdev->dev, &vimc_sen_comp_ops);
-}
-
-static int vimc_sen_remove(struct platform_device *pdev)
-{
-	component_del(&pdev->dev, &vimc_sen_comp_ops);
-
-	return 0;
-}
-
-static const struct platform_device_id vimc_sen_driver_ids[] = {
-	{
-		.name           = VIMC_SEN_DRV_NAME,
-	},
-	{ }
-};
-
-static struct platform_driver vimc_sen_pdrv = {
-	.probe		= vimc_sen_probe,
-	.remove		= vimc_sen_remove,
-	.id_table	= vimc_sen_driver_ids,
-	.driver		= {
-		.name	= VIMC_SEN_DRV_NAME,
-	},
-};
-
-module_platform_driver(vimc_sen_pdrv);
-
-MODULE_DEVICE_TABLE(platform, vimc_sen_driver_ids);
-
-MODULE_DESCRIPTION("Virtual Media Controller Driver (VIMC) Sensor");
-MODULE_AUTHOR("Helen Mae Koike Fornazier <[email protected]>");
-MODULE_LICENSE("GPL");

drivers/media/platform/vimc/vimc.h +0 -1

index a5adebdda941..cd527c6fe6fc 100644
@@ -92,11 +92,30 @@ struct vimc_device {
 	/* The Associated media_device parent */
 	struct media_device mdev;
 
-	/* Internal v4l2 parent device*/

drivers/media/platform/vimc/vimc.h +16 -0

+	struct vimc_ent_device *ved;
+	int (*add)(struct vimc_device *vimc, struct vimc_ent_config *vent);
+	void (*rm)(struct vimc_device *vimc, struct vimc_ent_config *vent);
 };
 
+/* prototypes for vimc_ent_config add and rm hooks */
+int vimc_cap_add(struct vimc_device *vimc, struct vimc_ent_config *vent);
+void vimc_cap_rm(struct vimc_device *vimc, struct vimc_ent_config *vent);
+
+int vimc_deb_add(struct vimc_device *vimc, struct vimc_ent_config *vent);
+void vimc_deb_rm(struct vimc_device *vimc, struct vimc_ent_config *vent);
+
+int vimc_sca_add(struct vimc_device *vimc, struct vimc_ent_config *vent);
+void vimc_sca_rm(struct vimc_device *vimc, struct vimc_ent_config *vent);
+
+int vimc_sen_add(struct vimc_device *vimc, struct vimc_ent_config *vent);
+void vimc_sen_rm(struct vimc_device *vimc, struct vimc_ent_config *vent);
+
 #endif

In preparation for collapsing the component driver structure into a monolith, move private device structure defines to a new common header file.

Signed-off-by Shuah Khan <[email protected]>

---
 drivers/media/platform/vimc/vimc-capture.c |  21 +----
 drivers/media/platform/vimc/vimc-core.c    |  18 +---
 drivers/media/platform/vimc/vimc-debayer.c |  16 +---
 drivers/media/platform/vimc/vimc-scaler.c  |  15 +--
 drivers/media/platform/vimc/vimc-sensor.c  |  13 +--
 drivers/media/platform/vimc/vimc.h         | 102 +++++++++++++++++++++
 6 files changed, 107 insertions(+), 78 deletions(-)
 create mode 100644 drivers/media/platform/vimc/vimc.h

drivers/media/platform/vimc/vimc-capture.c +1 -20

index 664855708fdf..c52fc5d97c2d 100644
@@ -13,6 +13,7 @@
 #include <media/videobuf2-core.h>
 #include <media/videobuf2-vmalloc.h>
 
+#include "vimc.h"
 #include "vimc-common.h"
 #include "vimc-streamer.h"
 

@@ -44,26 +45,6 @@ static const u32 vimc_cap_supported_pixfmt[] = {
 	V4L2_PIX_FMT_SRGGB12,
 };
 
-struct vimc_cap_device {
-	struct vimc_ent_device ved;
-	struct video_device vdev;
-	struct device *dev;
-	struct v4l2_pix_format format;
-	struct vb2_queue queue;
-	struct list_head buf_list;
-	/*
-	 * NOTE: in a real driver, a spin lock must be used to access the
-	 * queue because the frames are generated from a hardware interruption
-	 * and the isr is not allowed to sleep.
-	 * Even if it is not necessary a spinlock in the vimc driver, we
-	 * use it here as a code reference
-	 */
-	spinlock_t qlock;
-	struct mutex lock;
-	u32 sequence;
-	struct vimc_stream stream;
-};
-
 static const struct v4l2_pix_format fmt_default = {
 	.width = 640,
 	.height = 480,

drivers/media/platform/vimc/vimc-core.c +1 -17

index 571c55aa0e16..c9b351472272 100644
@@ -12,6 +12,7 @@
 #include <media/media-device.h>
 #include <media/v4l2-device.h>
 
+#include "vimc.h"
 #include "vimc-common.h"
 
 #define VIMC_MDEV_MODEL_NAME "VIMC MDEV"

@@ -24,23 +25,6 @@
 	.flags = link_flags,					\
 }
 
-struct vimc_device {
-	/* The platform device */
-	struct platform_device pdev;
-
-	/* The pipeline configuration */
-	const struct vimc_pipeline_config *pipe_cfg;
-
-	/* The Associated media_device parent */
-	struct media_device mdev;
-
-	/* Internal v4l2 parent device*/
-	struct v4l2_device v4l2_dev;
-
-	/* Subdevices */
-	struct platform_device **subdevs;
-};
-
 /* Structure which describes individual configuration for each entity */
 struct vimc_ent_config {
 	const char *name;

drivers/media/platform/vimc/vimc-debayer.c +1 -15

index 00598fbf3cba..750752bb173c 100644
@@ -13,6 +13,7 @@
 #include <linux/v4l2-mediabus.h>
 #include <media/v4l2-subdev.h>
 
+#include "vimc.h"
 #include "vimc-common.h"
 
 #define VIMC_DEB_DRV_NAME "vimc-debayer"

@@ -44,21 +45,6 @@ struct vimc_deb_pix_map {
 	enum vimc_deb_rgb_colors order[2][2];
 };
 
-struct vimc_deb_device {
-	struct vimc_ent_device ved;
-	struct v4l2_subdev sd;
-	struct device *dev;
-	/* The active format */
-	struct v4l2_mbus_framefmt sink_fmt;
-	u32 src_code;
-	void (*set_rgb_src)(struct vimc_deb_device *vdeb, unsigned int lin,
-			    unsigned int col, unsigned int rgb[3]);
-	/* Values calculated when the stream starts */
-	u8 *src_frame;
-	const struct vimc_deb_pix_map *sink_pix_map;
-	unsigned int sink_bpp;
-};
-
 static const struct v4l2_mbus_framefmt sink_fmt_default = {
 	.width = 640,
 	.height = 480,

drivers/media/platform/vimc/vimc-scaler.c +1 -14

index c7123a45c55b..fe99b9102ada 100644
@@ -13,6 +13,7 @@
 #include <linux/v4l2-mediabus.h>
 #include <media/v4l2-subdev.h>
 
+#include "vimc.h"
 #include "vimc-common.h"
 
 #define VIMC_SCA_DRV_NAME "vimc-scaler"

@@ -31,20 +32,6 @@ static const u32 vimc_sca_supported_pixfmt[] = {
 	V4L2_PIX_FMT_ARGB32,
 };
 
-struct vimc_sca_device {
-	struct vimc_ent_device ved;
-	struct v4l2_subdev sd;
-	struct device *dev;
-	/* NOTE: the source fmt is the same as the sink
-	 * with the width and hight multiplied by mult
-	 */
-	struct v4l2_mbus_framefmt sink_fmt;
-	/* Values calculated when the stream starts */
-	u8 *src_frame;
-	unsigned int src_line_size;
-	unsigned int bpp;
-};
-
 static const struct v4l2_mbus_framefmt sink_fmt_default = {
 	.width = 640,
 	.height = 480,

drivers/media/platform/vimc/vimc-sensor.c +1 -12

index 51359472eef2..6c57b1e262f9 100644
@@ -16,22 +16,11 @@
 #include <media/v4l2-subdev.h>
 #include <media/tpg/v4l2-tpg.h>
 
+#include "vimc.h"
 #include "vimc-common.h"
 
 #define VIMC_SEN_DRV_NAME "vimc-sensor"
 
-struct vimc_sen_device {
-	struct vimc_ent_device ved;
-	struct v4l2_subdev sd;
-	struct device *dev;
-	struct tpg_data tpg;
-	struct task_struct *kthread_sen;
-	u8 *frame;
-	/* The active format */
-	struct v4l2_mbus_framefmt mbus_format;
-	struct v4l2_ctrl_handler hdl;
-};
-
 static const struct v4l2_mbus_framefmt fmt_default = {
 	.width = 640,
 	.height = 480,

drivers/media/platform/vimc/vimc.h new file +102 -0

new file mode 100644
index 000000000000..a5adebdda941
@@ -0,0 +1,102 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ *
+ * Copyright (C) 2019 Shuah Khan <[email protected]>
+ *
+ */
+
+/*
+ * This file defines vimc driver device and sub-device structures.
+ */
+
+#ifndef _VIMC_H_
+#define _VIMC_H_
+
+#include <linux/platform_device.h>
+#include <media/media-device.h>
+#include <media/v4l2-device.h>
+#include <media/videobuf2-core.h>
+#include <media/tpg/v4l2-tpg.h>
+#include <media/v4l2-ctrls.h>
+
+#include "vimc-common.h"
+
+struct vimc_cap_device {
+	struct vimc_ent_device ved;
+	struct video_device vdev;
+	struct device *dev;
+	struct v4l2_pix_format format;
+	struct vb2_queue queue;
+	struct list_head buf_list;
+	/*
+	 * NOTE: in a real driver, a spin lock must be used to access the
+	 * queue because the frames are generated from a hardware interruption
+	 * and the isr is not allowed to sleep.
+	 * Even if it is not necessary a spinlock in the vimc driver, we
+	 * use it here as a code reference
+	 */
+	spinlock_t qlock;
+	struct mutex lock;
+	u32 sequence;
+	struct vimc_stream stream;
+};
+
+struct vimc_sca_device {
+	struct vimc_ent_device ved;
+	struct v4l2_subdev sd;
+	struct device *dev;
+	/* NOTE: the source fmt is the same as the sink
+	 * with the width and hight multiplied by mult
+	 */
+	struct v4l2_mbus_framefmt sink_fmt;
+	/* Values calculated when the stream starts */
+	u8 *src_frame;
+	unsigned int src_line_size;
+	unsigned int bpp;
+};
+
+struct vimc_deb_device {
+	struct vimc_ent_device ved;
+	struct v4l2_subdev sd;
+	struct device *dev;
+	/* The active format */
+	struct v4l2_mbus_framefmt sink_fmt;
+	u32 src_code;
+	void (*set_rgb_src)(struct vimc_deb_device *vdeb, unsigned int lin,
+			    unsigned int col, unsigned int rgb[3]);
+	/* Values calculated when the stream starts */
+	u8 *src_frame;
+	const struct vimc_deb_pix_map *sink_pix_map;
+	unsigned int sink_bpp;
+};
+
+struct vimc_sen_device {
+	struct vimc_ent_device ved;
+	struct v4l2_subdev sd;
+	struct device *dev;
+	struct tpg_data tpg;
+	struct task_struct *kthread_sen;
+	u8 *frame;
+	/* The active format */
+	struct v4l2_mbus_framefmt mbus_format;
+	struct v4l2_ctrl_handler hdl;
+};
+
+struct vimc_device {
+	/* The platform device */
+	struct platform_device pdev;
+
+	/* The pipeline configuration */
+	const struct vimc_pipeline_config *pipe_cfg;
+
+	/* The Associated media_device parent */
+	struct media_device mdev;
+
+	/* Internal v4l2 parent device*/
+	struct v4l2_device v4l2_dev;
+
+	/* Subdevices */
+	struct platform_device **subdevs;
+};
+
+#endif

Hi Shuah,

Thanks for the patch.

Quoted 186 lines · Shuah Khan

In preparation for collapsing the component driver structure into a monolith, move private device structure defines to a new common header file.

Signed-off-by Shuah Khan <[email protected]>

---
  drivers/media/platform/vimc/vimc-capture.c |  21 +----
  drivers/media/platform/vimc/vimc-core.c    |  18 +---
  drivers/media/platform/vimc/vimc-debayer.c |  16 +---
  drivers/media/platform/vimc/vimc-scaler.c  |  15 +--
  drivers/media/platform/vimc/vimc-sensor.c  |  13 +--
  drivers/media/platform/vimc/vimc.h         | 102 +++++++++++++++++++++
  6 files changed, 107 insertions(+), 78 deletions(-)

create mode 100644 drivers/media/platform/vimc/vimc.h

drivers/media/platform/vimc/vimc-capture.c +1 -20

index 664855708fdf..c52fc5d97c2d 100644
@@ -13,6 +13,7 @@
  #include <media/videobuf2-core.h>
  #include <media/videobuf2-vmalloc.h>
 
+#include "vimc.h"
  #include "vimc-common.h"
  #include "vimc-streamer.h"
 

@@ -44,26 +45,6 @@ static const u32 vimc_cap_supported_pixfmt[] = {
  	V4L2_PIX_FMT_SRGGB12,
  };
 
-struct vimc_cap_device {
-	struct vimc_ent_device ved;
-	struct video_device vdev;
-	struct device *dev;
-	struct v4l2_pix_format format;
-	struct vb2_queue queue;
-	struct list_head buf_list;
-	/*
-	 * NOTE: in a real driver, a spin lock must be used to access the
-	 * queue because the frames are generated from a hardware interruption
-	 * and the isr is not allowed to sleep.
-	 * Even if it is not necessary a spinlock in the vimc driver, we
-	 * use it here as a code reference
-	 */
-	spinlock_t qlock;
-	struct mutex lock;
-	u32 sequence;
-	struct vimc_stream stream;
-};
-
  static const struct v4l2_pix_format fmt_default = {
  	.width = 640,
  	.height = 480,

drivers/media/platform/vimc/vimc-core.c +1 -17

index 571c55aa0e16..c9b351472272 100644
@@ -12,6 +12,7 @@
  #include <media/media-device.h>
  #include <media/v4l2-device.h>
 
+#include "vimc.h"
  #include "vimc-common.h"
 
  #define VIMC_MDEV_MODEL_NAME "VIMC MDEV"

@@ -24,23 +25,6 @@
  	.flags = link_flags,					\
  }
 
-struct vimc_device {
-	/* The platform device */
-	struct platform_device pdev;
-
-	/* The pipeline configuration */
-	const struct vimc_pipeline_config *pipe_cfg;
-
-	/* The Associated media_device parent */
-	struct media_device mdev;
-
-	/* Internal v4l2 parent device*/
-	struct v4l2_device v4l2_dev;
-
-	/* Subdevices */
-	struct platform_device **subdevs;
-};
-
  /* Structure which describes individual configuration for each entity */
  struct vimc_ent_config {
  	const char *name;

drivers/media/platform/vimc/vimc-debayer.c +1 -15

index 00598fbf3cba..750752bb173c 100644
@@ -13,6 +13,7 @@
  #include <linux/v4l2-mediabus.h>
  #include <media/v4l2-subdev.h>
 
+#include "vimc.h"
  #include "vimc-common.h"
 
  #define VIMC_DEB_DRV_NAME "vimc-debayer"

@@ -44,21 +45,6 @@ struct vimc_deb_pix_map {
  	enum vimc_deb_rgb_colors order[2][2];
  };
 
-struct vimc_deb_device {
-	struct vimc_ent_device ved;
-	struct v4l2_subdev sd;
-	struct device *dev;
-	/* The active format */
-	struct v4l2_mbus_framefmt sink_fmt;
-	u32 src_code;
-	void (*set_rgb_src)(struct vimc_deb_device *vdeb, unsigned int lin,
-			    unsigned int col, unsigned int rgb[3]);
-	/* Values calculated when the stream starts */
-	u8 *src_frame;
-	const struct vimc_deb_pix_map *sink_pix_map;
-	unsigned int sink_bpp;
-};
-
  static const struct v4l2_mbus_framefmt sink_fmt_default = {
  	.width = 640,
  	.height = 480,

drivers/media/platform/vimc/vimc-scaler.c +1 -14

index c7123a45c55b..fe99b9102ada 100644
@@ -13,6 +13,7 @@
  #include <linux/v4l2-mediabus.h>
  #include <media/v4l2-subdev.h>
 
+#include "vimc.h"
  #include "vimc-common.h"
 
  #define VIMC_SCA_DRV_NAME "vimc-scaler"

@@ -31,20 +32,6 @@ static const u32 vimc_sca_supported_pixfmt[] = {
  	V4L2_PIX_FMT_ARGB32,
  };
 
-struct vimc_sca_device {
-	struct vimc_ent_device ved;
-	struct v4l2_subdev sd;
-	struct device *dev;
-	/* NOTE: the source fmt is the same as the sink
-	 * with the width and hight multiplied by mult
-	 */
-	struct v4l2_mbus_framefmt sink_fmt;
-	/* Values calculated when the stream starts */
-	u8 *src_frame;
-	unsigned int src_line_size;
-	unsigned int bpp;
-};
-
  static const struct v4l2_mbus_framefmt sink_fmt_default = {
  	.width = 640,
  	.height = 480,

drivers/media/platform/vimc/vimc-sensor.c +1 -12

index 51359472eef2..6c57b1e262f9 100644
@@ -16,22 +16,11 @@
  #include <media/v4l2-subdev.h>
  #include <media/tpg/v4l2-tpg.h>
 
+#include "vimc.h"
  #include "vimc-common.h"
 
  #define VIMC_SEN_DRV_NAME "vimc-sensor"
 
-struct vimc_sen_device {
-	struct vimc_ent_device ved;
-	struct v4l2_subdev sd;
-	struct device *dev;
-	struct tpg_data tpg;
-	struct task_struct *kthread_sen;
-	u8 *frame;
-	/* The active format */
-	struct v4l2_mbus_framefmt mbus_format;
-	struct v4l2_ctrl_handler hdl;
-};
-
  static const struct v4l2_mbus_framefmt fmt_default = {
  	.width = 640,
  	.height = 480,

drivers/media/platform/vimc/vimc.h +0 -0

drivers/media/platform/vimc/vimc.h

Quoted 4 lines

new file mode 100644
index 000000000000..a5adebdda941

drivers/media/platform/vimc/vimc.h new file +0 -0

I was wondering if instead of creating a vimc.h, if we could move to vimc-common.h,just because it is not clear to me the difference between the two now, what do you think?

Hi Andre,

Thanks for testing this.

Quoted 73 lines · André Almeida

Quoted 72 lines · Shuah Khan

Hi Andre,

Quoted 68 lines · André Almeida

Hello Shuah,

Thanks for the patch, I did some comments below.

Quoted 57 lines · Shuah Khan

vimc uses Component API to split the driver into functional components. The real hardware resembles a monolith structure than component and component structure added a level of complexity making it hard to maintain without adding any real benefit.
     The sensor is one vimc component that would makes sense to be a
separate
module to closely align with the real hardware. It would be easier to collapse vimc into single monolithic driver first and then split the sensor off as a separate module.

This patch series emoves the component API and makes minimal changes to the code base preserving the functional division of the code structure. Preserving the functional structure allows us to split the sensor off as a separate module in the future.

Major design elements in this change are:
     - Use existing struct vimc_ent_config and struct
vimc_pipeline_config
       to drive the initialization of the functional components.
     - Make vimc_ent_config global by moving it to vimc.h
     - Add two new hooks add and rm to initialize and register,
unregister
       and free subdevs.
     - All component API is now gone and bind and unbind hooks are
modified
       to do "add" and "rm" with minimal changes to just add and rm
subdevs.
     - vimc-core's bind and unbind are now register and unregister.
     - vimc-core invokes "add" hooks from its vimc_register_devices(). The "add" hooks remain the same and register subdevs. They don't create platform devices of their own and use vimc's pdev.dev as their reference device. The "add" hooks save their
vimc_ent_device(s)
       in the corresponding vimc_ent_config.
     - vimc-core invokes "rm" hooks from its unregister to unregister
subdevs
       and cleanup.
     - vimc-core invokes "add" and "rm" hooks with pointer to struct
vimc_device
       and the corresponding struct vimc_ent_config pointer. The following configure and stream test works on all devices. media-ctl -d platform:vimc -V '"Sensor
A":0[fmt:SBGGR8_1X8/640x480]'
     media-ctl -d platform:vimc -V '"Debayer
A":0[fmt:SBGGR8_1X8/640x480]'
     media-ctl -d platform:vimc -V '"Sensor
B":0[fmt:SBGGR8_1X8/640x480]'
     media-ctl -d platform:vimc -V '"Debayer
B":0[fmt:SBGGR8_1X8/640x480]'
          v4l2-ctl -z platform:vimc -d "RGB/YUV Capture" -v
width=1920,height=1440
     v4l2-ctl -z platform:vimc -d "Raw Capture 0" -v pixelformat=BA81 v4l2-ctl -z platform:vimc -d "Raw Capture 1" -v pixelformat=BA81 v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video1 v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video2 v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video3

Quoted 2 lines

The third patch in the series fixes a general protection fault found when rmmod is done while stream is active.

I applied your patch on top of media_tree/master and I did some testing. Not sure if I did something wrong, but just adding and removing the module generated a kernel panic:

Quoted 51 lines

Quoted 5 lines

Thanks for testing.

Odd. I tested modprobe and rmmod both.I was working on Linux 5.3-rc2. I will apply these to media latest and work from there. I have to rebase these on top of the reverts from Lucas and Helen

Ok, please let me know if I succeeded to reproduce.

Quoted 43 lines

Quoted 43 lines

~# modprobe vimc
~# rmmod vimc
[   16.452974] stack segment: 0000 [#1] SMP PTI
[   16.453688] CPU: 0 PID: 2038 Comm: rmmod Not tainted 5.3.0-rc2+ #36 [   16.454678] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-20181126_142135-anatol 04/01/2014
[   16.456191] RIP: 0010:kfree+0x4d/0x240

<registers values...>

[   16.469188] Call Trace:
[   16.469666]  vimc_remove+0x35/0x90 [vimc]
[   16.470436]  platform_drv_remove+0x1f/0x40
[   16.471233]  device_release_driver_internal+0xd3/0x1b0 [   16.472184]  driver_detach+0x37/0x6b
[   16.472882]  bus_remove_driver+0x50/0xc1
[   16.473569]  vimc_exit+0xc/0xca0 [vimc]
[   16.474231]  __x64_sys_delete_module+0x18d/0x240 [   16.475036]  do_syscall_64+0x43/0x110
[   16.475656]  entry_SYSCALL_64_after_hwframe+0x44/0xa9 [   16.476504] RIP: 0033:0x7fceb8dafa4b

<registers values...>

[   16.484853] Modules linked in: vimc(-) videobuf2_vmalloc videobuf2_memops v4l2_tpg videobuf2_v4l2 videobuf2_common [   16.486187] ---[ end trace 91e5e0894e254d49 ]--- [   16.486758] RIP: 0010:kfree+0x4d/0x240

<registers values...>

fish: “rmmod vimc” terminated by signal SIGSEGV (Address boundary error)

I just added the module after booting, no other action was made. Here is how my git log --oneline looks like:

897d708e922b media: vimc: Fix gpf in rmmod path when stream is active 2e4a5ad8ad6d media: vimc: Collapse component structure into a single monolithic driver
7c8da1687e92 media: vimc: move private defines to a common header 97299a303532 media: Remove dev_err() usage after platform_get_irq() 25a3d6bac6b9 media: adv7511/cobalt: rename driver name to adv7511-v4l2

I couldn't reproduce the error, my tree looks the same:

[I] koike@floko ~/m/o/linux> git log --oneline
e3345155c8ed (HEAD) media: vimc: Fix gpf in rmmod path when stream is active 43e9e2fe761f media: vimc: Collapse component structure into a single monolithic driver 8a6d0b9adde0 media: vimc: move private defines to a common header 97299a303532 (media/master) media: Remove dev_err() usage after platform_get_irq() 25a3d6bac6b9 media: adv7511/cobalt: rename driver name to adv7511-v4l2

André, is this deterministic? Or it just happens sometimes?

Quoted 40 lines

Quoted 40 lines

Quoted 39 lines

...

Quoted 23 lines

vimc_print_dot (--print-dot) topology after this change:
digraph board {
    rankdir=TB
    n00000001 [label="{{} | Sensor A\n/dev/v4l-subdev0 | {<port0>
0}}", shape=Mrecord, style=filled, fillcolor=green]
    n00000001:port0 -> n00000005:port0 [style=bold] n00000001:port0 -> n0000000b [style=bold]
    n00000003 [label="{{} | Sensor B\n/dev/v4l-subdev1 | {<port0>
0}}", shape=Mrecord, style=filled, fillcolor=green]
    n00000003:port0 -> n00000008:port0 [style=bold] n00000003:port0 -> n0000000f [style=bold]
    n00000005 [label="{{<port0> 0} | Debayer A\n/dev/v4l-subdev2 |
{<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
    n00000005:port1 -> n00000015:port0
    n00000008 [label="{{<port0> 0} | Debayer B\n/dev/v4l-subdev3 |
{<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
    n00000008:port1 -> n00000015:port0 [style=dashed] n0000000b [label="Raw Capture 0\n/dev/video1", shape=box,
style=filled, fillcolor=yellow]
    n0000000f [label="Raw Capture 1\n/dev/video2", shape=box,
style=filled, fillcolor=yellow]
    n00000013 [label="{{} | RGB/YUV Input\n/dev/v4l-subdev4 |

Quoted 8 lines

{<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green]
    n00000013:port0 -> n00000015:port0 [style=dashed] n00000015 [label="{{<port0> 0} | Scaler\n/dev/v4l-subdev5 |
{<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
    n00000015:port1 -> n00000018 [style=bold] n00000018 [label="RGB/YUV Capture\n/dev/video3", shape=box,
style=filled, fillcolor=yellow]
}

Since the topology changed, it would be nice to change in the documentation as well. The current dot file can be found at Documentation/media/v4l-drivers/vimc.dot and it's rendered at this
page: https://www.kernel.org/doc/html/latest/media/v4l-drivers/vimc.html

Quoted 7 lines

Quoted 3 lines

Topology shouldn't have changed. No changes to links or pads etc. I will take a look to be sure. I agree that if topology changes document should be updated.

If you "diff" the current dot with the dot you generated, you will see some differences. The main difference is that "RGB/YUV Input" was a device "/dev/video2/", and now it a subdevice "/dev/v4l-subdev4".

hmm, I just generated the topology for media/master, and it is /dev/v4l-subdev4. As we don't have an implementation of the output device yet, we used the sensor as a place holder and that is why it appears as "/dev/v4l-subdev4", what is in the docs is the ideal version after we get the output merged. We should update the docs in any case.

Hi Shuah,

Thanks for the patch, just some small comments.

Quoted 322 lines · Shuah Khan

vimc uses Component API to split the driver into functional components. The real hardware resembles a monolith structure than component and component structure added a level of complexity making it hard to maintain without adding any real benefit.

The sensor is one vimc component that would makes sense to be a separate module to closely align with the real hardware. It would be easier to collapse vimc into single monolithic driver first and then split the sensor off as a separate module.

Collapse it into a single monolithic driver removing the Component API. This patch removes the component API and makes minimal changes to the code base preserving the functional division of the code structure. Preserving the functional structure allows us to split the sensor off as a separate module in the future.

Major design elements in this change are:
- Use existing struct vimc_ent_config and struct vimc_pipeline_config to drive the initialization of the functional components.
- Make vimc_ent_config global by moving it to vimc.h
- Add two new hooks add and rm to initialize and register, unregister and free subdevs.
- All component API is now gone and bind and unbind hooks are modified to do "add" and "rm" with minimal changes to just add and rm subdevs.
- vimc-core's bind and unbind are now register and unregister.
- vimc-core invokes "add" hooks from its vimc_register_devices(). The "add" hooks remain the same and register subdevs. They don't create platform devices of their own and use vimc's pdev.dev as their reference device. The "add" hooks save their vimc_ent_device(s) in the corresponding vimc_ent_config.
- vimc-core invokes "rm" hooks from its unregister to unregister subdevs and cleanup.
- vimc-core invokes "add" and "rm" hooks with pointer to struct vimc_device and the corresponding struct vimc_ent_config pointer.

The following configure and stream test works on all devices.

media-ctl -d platform:vimc -V '"Sensor A":0[fmt:SBGGR8_1X8/640x480]' media-ctl -d platform:vimc -V '"Debayer A":0[fmt:SBGGR8_1X8/640x480]' media-ctl -d platform:vimc -V '"Sensor B":0[fmt:SBGGR8_1X8/640x480]' media-ctl -d platform:vimc -V '"Debayer B":0[fmt:SBGGR8_1X8/640x480]'

v4l2-ctl -z platform:vimc -d "RGB/YUV Capture" -v width=1920,height=1440 v4l2-ctl -z platform:vimc -d "Raw Capture 0" -v pixelformat=BA81 v4l2-ctl -z platform:vimc -d "Raw Capture 1" -v pixelformat=BA81

v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video1 v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video2 v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video3

Signed-off-by Shuah Khan <[email protected]>

---
  drivers/media/platform/vimc/Makefile       |   7 +-
  drivers/media/platform/vimc/vimc-capture.c |  75 ++--------
  drivers/media/platform/vimc/vimc-core.c    | 156 ++++++++-------------
  drivers/media/platform/vimc/vimc-debayer.c |  68 ++-------
  drivers/media/platform/vimc/vimc-scaler.c  |  68 ++-------
  drivers/media/platform/vimc/vimc-sensor.c  |  69 ++-------
  drivers/media/platform/vimc/vimc.h         |  25 +++-
  7 files changed, 125 insertions(+), 343 deletions(-)

drivers/media/platform/vimc/Makefile +4 -3

index 96d06f030c31..a53b2b532e9f 100644
@@ -1,5 +1,6 @@
  # SPDX-License-Identifier: GPL-2.0
-vimc-y := vimc-core.o vimc-common.o vimc-streamer.o
+vimc-y := vimc-core.o vimc-common.o vimc-streamer.o vimc-capture.o \
+		vimc-debayer.o vimc-scaler.o vimc-sensor.o
+
+obj-$(CONFIG_VIDEO_VIMC) += vimc.o
 
-obj-$(CONFIG_VIDEO_VIMC) += vimc.o vimc-capture.o vimc-debayer.o \
-                vimc-scaler.o vimc-sensor.o

drivers/media/platform/vimc/vimc-capture.c +13 -62

index c52fc5d97c2d..b7b2d3c3d4f8 100644
@@ -5,10 +5,6 @@
   * Copyright (C) 2015-2017 Helen Koike <[email protected]>
   */
 
-#include <linux/component.h>
-#include <linux/module.h>
-#include <linux/mod_devicetable.h>
-#include <linux/platform_device.h>
  #include <media/v4l2-ioctl.h>
  #include <media/videobuf2-core.h>
  #include <media/videobuf2-vmalloc.h>

@@ -17,8 +13,6 @@
  #include "vimc-common.h"
  #include "vimc-streamer.h"
 
-#define VIMC_CAP_DRV_NAME "vimc-capture"
-
  static const u32 vimc_cap_supported_pixfmt[] = {
  	V4L2_PIX_FMT_BGR24,
  	V4L2_PIX_FMT_RGB24,

@@ -348,11 +342,11 @@ static void vimc_cap_release(struct video_device *vdev)
  	kfree(vcap);
  }
 
-static void vimc_cap_comp_unbind(struct device *comp, struct device *master,
-				 void *master_data)
+void vimc_cap_rm(struct vimc_device *vimc, struct vimc_ent_config *vent)
  {
-	struct vimc_ent_device *ved = dev_get_drvdata(comp);
-	struct vimc_cap_device *vcap = container_of(ved, struct vimc_cap_device,
+	struct vimc_ent_device *ved = vent->ved;
+	struct vimc_cap_device *vcap = container_of(ved,
+						    struct vimc_cap_device,
  						    ved);
 
  	vb2_queue_release(&vcap->queue);

@@ -399,11 +393,9 @@ static void *vimc_cap_process_frame(struct vimc_ent_device *ved,
  	return NULL;
  }
 
-static int vimc_cap_comp_bind(struct device *comp, struct device *master,
-			      void *master_data)
+int vimc_cap_add(struct vimc_device *vimc, struct vimc_ent_config *vent)
  {
-	struct v4l2_device *v4l2_dev = master_data;
-	struct vimc_platform_data *pdata = comp->platform_data;
+	struct v4l2_device *v4l2_dev = &vimc->v4l2_dev;
  	struct vimc_cap_device *vcap;
  	struct video_device *vdev;
  	struct vb2_queue *q;

@@ -423,7 +415,7 @@ static int vimc_cap_comp_bind(struct device *comp, struct device *master,
  	}
 
  	/* Initialize the media entity */
-	vcap->vdev.entity.name = pdata->entity_name;
+	vcap->vdev.entity.name = vent->name;
  	vcap->vdev.entity.function = MEDIA_ENT_F_IO_V4L;
  	ret = media_entity_pads_init(&vcap->vdev.entity,
  				     1, vcap->ved.pads);

@@ -447,8 +439,8 @@ static int vimc_cap_comp_bind(struct device *comp, struct device *master,
 
  	ret = vb2_queue_init(q);
  	if (ret) {
-		dev_err(comp, "%s: vb2 queue init failed (err=%d)\n",
-			pdata->entity_name, ret);
+		dev_err(&vimc->pdev.dev, "%s: vb2 queue init failed (err=%d)\n",
+			vent->name, ret);
  		goto err_clean_m_ent;
  	}
 

@@ -465,8 +457,7 @@ static int vimc_cap_comp_bind(struct device *comp, struct device *master,
  	vcap->ved.ent = &vcap->vdev.entity;
  	vcap->ved.process_frame = vimc_cap_process_frame;
  	vcap->ved.vdev_get_format = vimc_cap_get_format;
-	dev_set_drvdata(comp, &vcap->ved);
-	vcap->dev = comp;
+	vcap->dev = &vimc->pdev.dev;
 
  	/* Initialize the video_device struct */
  	vdev = &vcap->vdev;

@@ -479,17 +470,18 @@ static int vimc_cap_comp_bind(struct device *comp, struct device *master,
  	vdev->queue = q;
  	vdev->v4l2_dev = v4l2_dev;
  	vdev->vfl_dir = VFL_DIR_RX;
-	strscpy(vdev->name, pdata->entity_name, sizeof(vdev->name));
+	strscpy(vdev->name, vent->name, sizeof(vdev->name));
  	video_set_drvdata(vdev, &vcap->ved);
 
  	/* Register the video_device with the v4l2 and the media framework */
  	ret = video_register_device(vdev, VFL_TYPE_GRABBER, -1);
  	if (ret) {
-		dev_err(comp, "%s: video register failed (err=%d)\n",
+		dev_err(&vimc->pdev.dev, "%s: video register failed (err=%d)\n",
  			vcap->vdev.name, ret);
  		goto err_release_queue;
  	}
 
+	vent->ved = &vcap->ved;
  	return 0;
 
  err_release_queue:

@@ -503,44 +495,3 @@ static int vimc_cap_comp_bind(struct device *comp, struct device *master,
 
  	return ret;
  }
-
-static const struct component_ops vimc_cap_comp_ops = {
-	.bind = vimc_cap_comp_bind,
-	.unbind = vimc_cap_comp_unbind,
-};
-
-static int vimc_cap_probe(struct platform_device *pdev)
-{
-	return component_add(&pdev->dev, &vimc_cap_comp_ops);
-}
-
-static int vimc_cap_remove(struct platform_device *pdev)
-{
-	component_del(&pdev->dev, &vimc_cap_comp_ops);
-
-	return 0;
-}
-
-static const struct platform_device_id vimc_cap_driver_ids[] = {
-	{
-		.name           = VIMC_CAP_DRV_NAME,
-	},
-	{ }
-};
-
-static struct platform_driver vimc_cap_pdrv = {
-	.probe		= vimc_cap_probe,
-	.remove		= vimc_cap_remove,
-	.id_table	= vimc_cap_driver_ids,
-	.driver		= {
-		.name	= VIMC_CAP_DRV_NAME,
-	},
-};
-
-module_platform_driver(vimc_cap_pdrv);
-
-MODULE_DEVICE_TABLE(platform, vimc_cap_driver_ids);
-
-MODULE_DESCRIPTION("Virtual Media Controller Driver (VIMC) Capture");
-MODULE_AUTHOR("Helen Mae Koike Fornazier <[email protected]>");
-MODULE_LICENSE("GPL");

drivers/media/platform/vimc/vimc-core.c +20 -9

index c9b351472272..3bd89cf38a90 100644
@@ -5,7 +5,6 @@
   * Copyright (C) 2015-2017 Helen Koike <[email protected]>
   */
 
-#include <linux/component.h>
  #include <linux/init.h>
  #include <linux/module.h>
  #include <linux/platform_device.h>

@@ -25,12 +24,6 @@
  	.flags = link_flags,					\
  }
 
-/* Structure which describes individual configuration for each entity */
-struct vimc_ent_config {
-	const char *name;
-	const char *drv;
-};
-
  /* Structure which describes links between entities */
  struct vimc_ent_link {
  	unsigned int src_ent;

@@ -42,7 +35,7 @@ struct vimc_ent_link {
 
  /* Structure which describes the whole topology */
  struct vimc_pipeline_config {
-	const struct vimc_ent_config *ents;
+	struct vimc_ent_config *ents;
  	size_t num_ents;
  	const struct vimc_ent_link *links;
  	size_t num_links;

@@ -52,43 +45,61 @@ struct vimc_pipeline_config {
   * Topology Configuration
   */
 
-static const struct vimc_ent_config ent_config[] = {
+static struct vimc_ent_config ent_config[] = {
  	{
  		.name = "Sensor A",
  		.drv = "vimc-sensor",
+		.add = vimc_sen_add,
+		.rm = vimc_sen_rm,
  	},
  	{
  		.name = "Sensor B",
  		.drv = "vimc-sensor",
+		.add = vimc_sen_add,
+		.rm = vimc_sen_rm,
  	},
  	{
  		.name = "Debayer A",
  		.drv = "vimc-debayer",
+		.add = vimc_deb_add,
+		.rm = vimc_deb_rm,
  	},
  	{
  		.name = "Debayer B",
  		.drv = "vimc-debayer",
+		.add = vimc_deb_add,
+		.rm = vimc_deb_rm,
  	},
  	{
  		.name = "Raw Capture 0",
  		.drv = "vimc-capture",
+		.add = vimc_cap_add,
+		.rm = vimc_cap_rm,
  	},
  	{
  		.name = "Raw Capture 1",
  		.drv = "vimc-capture",
+		.add = vimc_cap_add,
+		.rm = vimc_cap_rm,
  	},
  	{
  		.name = "RGB/YUV Input",
  		/* TODO: change this to vimc-input when it is implemented */
  		.drv = "vimc-sensor",
+		.add = vimc_sen_add,
+		.rm = vimc_sen_rm,
  	},
  	{
  		.name = "Scaler",
  		.drv = "vimc-scaler",
+		.add = vimc_sca_add,
+		.rm = vimc_sca_rm,
  	},
  	{
  		.name = "RGB/YUV Capture",
  		.drv = "vimc-capture",
+		.add = vimc_cap_add,
+		.rm = vimc_cap_rm,
  	},
  };
 

drivers/media/platform/vimc/vimc-core.c static const struct vimc_ent_link ent_links[] = { · line 125

Quoted 6 lines

drivers/media/platform/vimc/vimc-core.c +1 -1

@@ -111,7 +122,7 @@ static const struct vimc_ent_link ent_links[] = {
  	VIMC_ENT_LINK(7, 1, 8, 0, MEDIA_LNK_FL_ENABLED | MEDIA_LNK_FL_IMMUTABLE),
  };
 
-static const struct vimc_pipeline_config pipe_cfg = {
+static struct vimc_pipeline_config pipe_cfg = {

Any special reason why the const was dropped here?

Quoted 22 lines

drivers/media/platform/vimc/vimc-core.c +3 -6

  	.ents		= ent_config,
  	.num_ents	= ARRAY_SIZE(ent_config),
  	.links		= ent_links,

@@ -128,14 +139,11 @@ static int vimc_create_links(struct vimc_device *vimc)
  	/* Initialize the links between entities */
  	for (i = 0; i < vimc->pipe_cfg->num_links; i++) {
  		const struct vimc_ent_link *link = &vimc->pipe_cfg->links[i];
-		/*
-		 * TODO: Check another way of retrieving ved struct without
-		 * relying on platform_get_drvdata
-		 */
+
  		struct vimc_ent_device *ved_src =
-			platform_get_drvdata(vimc->subdevs[link->src_ent]);
+			vimc->pipe_cfg->ents[link->src_ent].ved;
  		struct vimc_ent_device *ved_sink =
-			platform_get_drvdata(vimc->subdevs[link->sink_ent]);
+			vimc->pipe_cfg->ents[link->sink_ent].ved;
 
  		ret = media_create_pad_link(ved_src->ent, link->src_pad,
  					    ved_sink->ent, link->sink_pad,

drivers/media/platform/vimc/vimc-core.c static int vimc_create_links(struct vimc_device *vimc) · line 160

Quoted 10 lines

drivers/media/platform/vimc/vimc-core.c +2 -4

@@ -147,13 +155,28 @@ static int vimc_create_links(struct vimc_device *vimc)
  	return 0;
  }
 
-static int vimc_comp_bind(struct device *master)
+static int vimc_add_subdevs(struct vimc_device *vimc)
  {
-	struct vimc_device *vimc = container_of(to_platform_device(master),
-						struct vimc_device, pdev);
-	int ret;
+	int i;

unsigned int

drivers/media/platform/vimc/vimc-core.c static int vimc_create_links(struct vimc_device *vimc) · line 161

Quoted 1 line

drivers/media/platform/vimc/vimc-core.c +1 -0

+	int ret = 0;

no need to init to 0, but not a problem either.

Quoted 562 lines

drivers/media/platform/vimc/vimc-core.c +28 -81

+
+	for (i = 0; i < vimc->pipe_cfg->num_ents; i++) {
+		dev_dbg(&vimc->pdev.dev, "new entity for %s\n",
+			vimc->pipe_cfg->ents[i].name);
+		ret = vimc->pipe_cfg->ents[i].add(vimc,
+				&vimc->pipe_cfg->ents[i]);
+		if (ret) {
+			dev_err(&vimc->pdev.dev, "add new entity for %s\n",
+				vimc->pipe_cfg->ents[i].name);
+			return ret;
+		}
+	}
+	return 0;
+}
 
-	dev_dbg(master, "bind");
+static int vimc_register_devices(struct vimc_device *vimc)
+{
+	int ret;
 
  	/* Register the v4l2 struct */
  	ret = v4l2_device_register(vimc->mdev.dev, &vimc->v4l2_dev);

@@ -163,22 +186,20 @@ static int vimc_comp_bind(struct device *master)
  		return ret;
  	}
 
-	/* Bind subdevices */
-	ret = component_bind_all(master, &vimc->v4l2_dev);
-	if (ret)
-		goto err_v4l2_unregister;
+	/* Invoke enitity config hooks to initialize and register subdevs */
+	vimc_add_subdevs(vimc);
 
  	/* Initialize links */
  	ret = vimc_create_links(vimc);
  	if (ret)
-		goto err_comp_unbind_all;
+		goto err_v4l2_unregister;
 
  	/* Register the media device */
  	ret = media_device_register(&vimc->mdev);
  	if (ret) {
  		dev_err(vimc->mdev.dev,
  			"media device register failed (err=%d)\n", ret);
-		goto err_comp_unbind_all;
+		goto err_v4l2_unregister;
  	}
 
  	/* Expose all subdev's nodes*/

@@ -195,98 +216,36 @@ static int vimc_comp_bind(struct device *master)
  err_mdev_unregister:
  	media_device_unregister(&vimc->mdev);
  	media_device_cleanup(&vimc->mdev);
-err_comp_unbind_all:
-	component_unbind_all(master, NULL);
  err_v4l2_unregister:
  	v4l2_device_unregister(&vimc->v4l2_dev);
 
  	return ret;
  }
 
-static void vimc_comp_unbind(struct device *master)
+static void vimc_unregister(struct vimc_device *vimc)
  {
-	struct vimc_device *vimc = container_of(to_platform_device(master),
-						struct vimc_device, pdev);
-
-	dev_dbg(master, "unbind");
-
  	media_device_unregister(&vimc->mdev);
  	media_device_cleanup(&vimc->mdev);
-	component_unbind_all(master, NULL);
  	v4l2_device_unregister(&vimc->v4l2_dev);
  }
 
-static int vimc_comp_compare(struct device *comp, void *data)
-{
-	return comp == data;
-}
-
-static struct component_match *vimc_add_subdevs(struct vimc_device *vimc)
-{
-	struct component_match *match = NULL;
-	struct vimc_platform_data pdata;
-	int i;
-
-	for (i = 0; i < vimc->pipe_cfg->num_ents; i++) {
-		dev_dbg(&vimc->pdev.dev, "new pdev for %s\n",
-			vimc->pipe_cfg->ents[i].drv);
-
-		strscpy(pdata.entity_name, vimc->pipe_cfg->ents[i].name,
-			sizeof(pdata.entity_name));
-
-		vimc->subdevs[i] = platform_device_register_data(&vimc->pdev.dev,
-						vimc->pipe_cfg->ents[i].drv,
-						PLATFORM_DEVID_AUTO,
-						&pdata,
-						sizeof(pdata));
-		if (IS_ERR(vimc->subdevs[i])) {
-			match = ERR_CAST(vimc->subdevs[i]);
-			while (--i >= 0)
-				platform_device_unregister(vimc->subdevs[i]);
-
-			return match;
-		}
-
-		component_match_add(&vimc->pdev.dev, &match, vimc_comp_compare,
-				    &vimc->subdevs[i]->dev);
-	}
-
-	return match;
-}
-
  static void vimc_rm_subdevs(struct vimc_device *vimc)
  {
  	unsigned int i;
 
  	for (i = 0; i < vimc->pipe_cfg->num_ents; i++)
-		platform_device_unregister(vimc->subdevs[i]);
+		vimc->pipe_cfg->ents[i].rm(vimc, &vimc->pipe_cfg->ents[i]);
  }
 
-static const struct component_master_ops vimc_comp_ops = {
-	.bind = vimc_comp_bind,
-	.unbind = vimc_comp_unbind,
-};
-
  static int vimc_probe(struct platform_device *pdev)
  {
  	struct vimc_device *vimc = container_of(pdev, struct vimc_device, pdev);
-	struct component_match *match = NULL;
-	int ret;
+	int ret = 0;
 
  	dev_dbg(&pdev->dev, "probe");
 
  	memset(&vimc->mdev, 0, sizeof(vimc->mdev));
 
-	/* Create platform_device for each entity in the topology*/
-	vimc->subdevs = devm_kcalloc(&vimc->pdev.dev, vimc->pipe_cfg->num_ents,
-				     sizeof(*vimc->subdevs), GFP_KERNEL);
-	if (!vimc->subdevs)
-		return -ENOMEM;
-
-	match = vimc_add_subdevs(vimc);
-	if (IS_ERR(match))
-		return PTR_ERR(match);
-
  	/* Link the media device within the v4l2_device */
  	vimc->v4l2_dev.mdev = &vimc->mdev;
 

@@ -298,16 +257,11 @@ static int vimc_probe(struct platform_device *pdev)
  	vimc->mdev.dev = &pdev->dev;
  	media_device_init(&vimc->mdev);
 
-	/* Add self to the component system */
-	ret = component_master_add_with_match(&pdev->dev, &vimc_comp_ops,
-					      match);
-	if (ret) {
+	ret = vimc_register_devices(vimc);
+	if (ret)
  		media_device_cleanup(&vimc->mdev);
-		vimc_rm_subdevs(vimc);
-		return ret;
-	}
 
-	return 0;
+	return ret;
  }
 
  static int vimc_remove(struct platform_device *pdev)

@@ -316,8 +270,8 @@ static int vimc_remove(struct platform_device *pdev)
 
  	dev_dbg(&pdev->dev, "remove");
 
-	component_master_del(&pdev->dev, &vimc_comp_ops);
  	vimc_rm_subdevs(vimc);
+	vimc_unregister(vimc);
 
  	return 0;
  }

drivers/media/platform/vimc/vimc-debayer.c +10 -58

index 750752bb173c..b58e0946ac2a 100644
@@ -5,10 +5,7 @@
   * Copyright (C) 2015-2017 Helen Koike <[email protected]>
   */
 
-#include <linux/component.h>
  #include <linux/module.h>
-#include <linux/mod_devicetable.h>
-#include <linux/platform_device.h>
  #include <linux/vmalloc.h>
  #include <linux/v4l2-mediabus.h>
  #include <media/v4l2-subdev.h>

@@ -16,9 +13,7 @@
  #include "vimc.h"
  #include "vimc-common.h"
 
-#define VIMC_DEB_DRV_NAME "vimc-debayer"
-/* This module only supports transforming a bayer format
- * to V4L2_PIX_FMT_RGB24
+/* Supports transforming a bayer format to V4L2_PIX_FMT_RGB24
   */
  #define VIMC_DEB_SRC_PIXFMT V4L2_PIX_FMT_RGB24
  #define VIMC_DEB_SRC_MBUS_FMT_DEFAULT MEDIA_BUS_FMT_RGB888_1X24

@@ -499,21 +494,19 @@ static const struct v4l2_subdev_internal_ops vimc_deb_int_ops = {
  	.release = vimc_deb_release,
  };
 
-static void vimc_deb_comp_unbind(struct device *comp, struct device *master,
-				 void *master_data)
+void vimc_deb_rm(struct vimc_device *vimc, struct vimc_ent_config *vent)
  {
-	struct vimc_ent_device *ved = dev_get_drvdata(comp);
-	struct vimc_deb_device *vdeb = container_of(ved, struct vimc_deb_device,
+	struct vimc_ent_device *ved = vent->ved;
+	struct vimc_deb_device *vdeb = container_of(ved,
+						    struct vimc_deb_device,
  						    ved);
 
  	vimc_ent_sd_unregister(ved, &vdeb->sd);
  }
 
-static int vimc_deb_comp_bind(struct device *comp, struct device *master,
-			      void *master_data)
+int vimc_deb_add(struct vimc_device *vimc, struct vimc_ent_config *vent)
  {
-	struct v4l2_device *v4l2_dev = master_data;
-	struct vimc_platform_data *pdata = comp->platform_data;
+	struct v4l2_device *v4l2_dev = &vimc->v4l2_dev;
  	struct vimc_deb_device *vdeb;
  	int ret;
 

@@ -524,7 +517,7 @@ static int vimc_deb_comp_bind(struct device *comp, struct device *master,
 
  	/* Initialize ved and sd */
  	ret = vimc_ent_sd_register(&vdeb->ved, &vdeb->sd, v4l2_dev,
-				   pdata->entity_name,
+				   vent->name,
  				   MEDIA_ENT_F_PROC_VIDEO_PIXEL_ENC_CONV, 2,
  				   (const unsigned long[2]) {MEDIA_PAD_FL_SINK,
  				   MEDIA_PAD_FL_SOURCE},

@@ -535,8 +528,7 @@ static int vimc_deb_comp_bind(struct device *comp, struct device *master,
  	}
 
  	vdeb->ved.process_frame = vimc_deb_process_frame;
-	dev_set_drvdata(comp, &vdeb->ved);
-	vdeb->dev = comp;
+	vdeb->dev = &vimc->pdev.dev;
 
  	/* Initialize the frame format */
  	vdeb->sink_fmt = sink_fmt_default;

@@ -549,46 +541,6 @@ static int vimc_deb_comp_bind(struct device *comp, struct device *master,
  	 */
  	vdeb->set_rgb_src = vimc_deb_set_rgb_pix_rgb24;
 
+	vent->ved = &vdeb->ved;
  	return 0;
  }
-
-static const struct component_ops vimc_deb_comp_ops = {
-	.bind = vimc_deb_comp_bind,
-	.unbind = vimc_deb_comp_unbind,
-};
-
-static int vimc_deb_probe(struct platform_device *pdev)
-{
-	return component_add(&pdev->dev, &vimc_deb_comp_ops);
-}
-
-static int vimc_deb_remove(struct platform_device *pdev)
-{
-	component_del(&pdev->dev, &vimc_deb_comp_ops);
-
-	return 0;
-}
-
-static const struct platform_device_id vimc_deb_driver_ids[] = {
-	{
-		.name           = VIMC_DEB_DRV_NAME,
-	},
-	{ }
-};
-
-static struct platform_driver vimc_deb_pdrv = {
-	.probe		= vimc_deb_probe,
-	.remove		= vimc_deb_remove,
-	.id_table	= vimc_deb_driver_ids,
-	.driver		= {
-		.name	= VIMC_DEB_DRV_NAME,
-	},
-};
-
-module_platform_driver(vimc_deb_pdrv);
-
-MODULE_DEVICE_TABLE(platform, vimc_deb_driver_ids);
-
-MODULE_DESCRIPTION("Virtual Media Controller Driver (VIMC) Debayer");
-MODULE_AUTHOR("Helen Mae Koike Fornazier <[email protected]>");
-MODULE_LICENSE("GPL");

drivers/media/platform/vimc/vimc-scaler.c +11 -57

index fe99b9102ada..7cd478fee3ae 100644
@@ -5,10 +5,7 @@
   * Copyright (C) 2015-2017 Helen Koike <[email protected]>
   */
 
-#include <linux/component.h>
  #include <linux/module.h>
-#include <linux/mod_devicetable.h>
-#include <linux/platform_device.h>
  #include <linux/vmalloc.h>
  #include <linux/v4l2-mediabus.h>
  #include <media/v4l2-subdev.h>

@@ -16,8 +13,6 @@
  #include "vimc.h"
  #include "vimc-common.h"
 
-#define VIMC_SCA_DRV_NAME "vimc-scaler"
-
  static unsigned int sca_mult = 3;
  module_param(sca_mult, uint, 0000);
  MODULE_PARM_DESC(sca_mult, " the image size multiplier");

@@ -331,22 +326,21 @@ static const struct v4l2_subdev_internal_ops vimc_sca_int_ops = {
  	.release = vimc_sca_release,
  };
 
-static void vimc_sca_comp_unbind(struct device *comp, struct device *master,
-				 void *master_data)
+void vimc_sca_rm(struct vimc_device *vimc, struct vimc_ent_config *vent)
  {
-	struct vimc_ent_device *ved = dev_get_drvdata(comp);
-	struct vimc_sca_device *vsca = container_of(ved, struct vimc_sca_device,
+	struct vimc_ent_device *ved = vent->ved;
+	struct vimc_sca_device *vsca = container_of(ved,
+						    struct vimc_sca_device,
  						    ved);
 
+
  	vimc_ent_sd_unregister(ved, &vsca->sd);
  }
 
 
-static int vimc_sca_comp_bind(struct device *comp, struct device *master,
-			      void *master_data)
+int vimc_sca_add(struct vimc_device *vimc, struct vimc_ent_config *vent)
  {
-	struct v4l2_device *v4l2_dev = master_data;
-	struct vimc_platform_data *pdata = comp->platform_data;
+	struct v4l2_device *v4l2_dev = &vimc->v4l2_dev;
  	struct vimc_sca_device *vsca;
  	int ret;
 

@@ -357,7 +351,7 @@ static int vimc_sca_comp_bind(struct device *comp, struct device *master,
 
  	/* Initialize ved and sd */
  	ret = vimc_ent_sd_register(&vsca->ved, &vsca->sd, v4l2_dev,
-				   pdata->entity_name,
+				   vent->name,
  				   MEDIA_ENT_F_PROC_VIDEO_SCALER, 2,
  				   (const unsigned long[2]) {MEDIA_PAD_FL_SINK,
  				   MEDIA_PAD_FL_SOURCE},

@@ -368,52 +362,12 @@ static int vimc_sca_comp_bind(struct device *comp, struct device *master,
  	}
 
  	vsca->ved.process_frame = vimc_sca_process_frame;
-	dev_set_drvdata(comp, &vsca->ved);
-	vsca->dev = comp;
+	vsca->dev = &vimc->pdev.dev;
+
+	vent->ved = &vsca->ved;
 
  	/* Initialize the frame format */
  	vsca->sink_fmt = sink_fmt_default;
 
  	return 0;
  }
-
-static const struct component_ops vimc_sca_comp_ops = {
-	.bind = vimc_sca_comp_bind,
-	.unbind = vimc_sca_comp_unbind,
-};
-
-static int vimc_sca_probe(struct platform_device *pdev)
-{
-	return component_add(&pdev->dev, &vimc_sca_comp_ops);
-}
-
-static int vimc_sca_remove(struct platform_device *pdev)
-{
-	component_del(&pdev->dev, &vimc_sca_comp_ops);
-
-	return 0;
-}
-
-static const struct platform_device_id vimc_sca_driver_ids[] = {
-	{
-		.name           = VIMC_SCA_DRV_NAME,
-	},
-	{ }
-};
-
-static struct platform_driver vimc_sca_pdrv = {
-	.probe		= vimc_sca_probe,
-	.remove		= vimc_sca_remove,
-	.id_table	= vimc_sca_driver_ids,
-	.driver		= {
-		.name	= VIMC_SCA_DRV_NAME,
-	},
-};
-
-module_platform_driver(vimc_sca_pdrv);
-
-MODULE_DEVICE_TABLE(platform, vimc_sca_driver_ids);
-
-MODULE_DESCRIPTION("Virtual Media Controller Driver (VIMC) Scaler");
-MODULE_AUTHOR("Helen Mae Koike Fornazier <[email protected]>");
-MODULE_LICENSE("GPL");

drivers/media/platform/vimc/vimc-sensor.c +10 -59

index 6c57b1e262f9..bd237e030a1b 100644
@@ -5,10 +5,6 @@
   * Copyright (C) 2015-2017 Helen Koike <[email protected]>
   */
 
-#include <linux/component.h>
-#include <linux/module.h>
-#include <linux/mod_devicetable.h>
-#include <linux/platform_device.h>
  #include <linux/v4l2-mediabus.h>
  #include <linux/vmalloc.h>
  #include <media/v4l2-ctrls.h>

@@ -19,8 +15,6 @@
  #include "vimc.h"
  #include "vimc-common.h"
 
-#define VIMC_SEN_DRV_NAME "vimc-sensor"
-
  static const struct v4l2_mbus_framefmt fmt_default = {
  	.width = 640,
  	.height = 480,

@@ -268,12 +262,12 @@ static const struct v4l2_subdev_internal_ops vimc_sen_int_ops = {
  	.release = vimc_sen_release,
  };
 
-static void vimc_sen_comp_unbind(struct device *comp, struct device *master,
-				 void *master_data)
+void vimc_sen_rm(struct vimc_device *vimc, struct vimc_ent_config *vent)
  {
-	struct vimc_ent_device *ved = dev_get_drvdata(comp);
-	struct vimc_sen_device *vsen =
-				container_of(ved, struct vimc_sen_device, ved);
+	struct vimc_ent_device *ved = vent->ved;
+	struct vimc_sen_device *vsen = container_of(ved,
+						    struct vimc_sen_device,
+						    ved);
 
  	vimc_ent_sd_unregister(ved, &vsen->sd);
  }

@@ -295,11 +289,9 @@ static const struct v4l2_ctrl_config vimc_sen_ctrl_test_pattern = {
  	.qmenu = tpg_pattern_strings,
  };
 
-static int vimc_sen_comp_bind(struct device *comp, struct device *master,
-			      void *master_data)
+int vimc_sen_add(struct vimc_device *vimc, struct vimc_ent_config *vent)
  {
-	struct v4l2_device *v4l2_dev = master_data;
-	struct vimc_platform_data *pdata = comp->platform_data;
+	struct v4l2_device *v4l2_dev = &vimc->v4l2_dev;
  	struct vimc_sen_device *vsen;
  	int ret;
 

@@ -332,7 +324,7 @@ static int vimc_sen_comp_bind(struct device *comp, struct device *master,
 
  	/* Initialize ved and sd */
  	ret = vimc_ent_sd_register(&vsen->ved, &vsen->sd, v4l2_dev,
-				   pdata->entity_name,
+				   vent->name,
  				   MEDIA_ENT_F_CAM_SENSOR, 1,
  				   (const unsigned long[1]) {MEDIA_PAD_FL_SOURCE},
  				   &vimc_sen_int_ops, &vimc_sen_ops);

@@ -340,8 +332,7 @@ static int vimc_sen_comp_bind(struct device *comp, struct device *master,
  		goto err_free_hdl;
 
  	vsen->ved.process_frame = vimc_sen_process_frame;
-	dev_set_drvdata(comp, &vsen->ved);
-	vsen->dev = comp;
+	vsen->dev = &vimc->pdev.dev;
 
  	/* Initialize the frame format */
  	vsen->mbus_format = fmt_default;

@@ -353,6 +344,7 @@ static int vimc_sen_comp_bind(struct device *comp, struct device *master,
  	if (ret)
  		goto err_unregister_ent_sd;
 
+	vent->ved = &vsen->ved;
  	return 0;
 
  err_unregister_ent_sd:

@@ -364,44 +356,3 @@ static int vimc_sen_comp_bind(struct device *comp, struct device *master,
 
  	return ret;
  }
-
-static const struct component_ops vimc_sen_comp_ops = {
-	.bind = vimc_sen_comp_bind,
-	.unbind = vimc_sen_comp_unbind,
-};
-
-static int vimc_sen_probe(struct platform_device *pdev)
-{
-	return component_add(&pdev->dev, &vimc_sen_comp_ops);
-}
-
-static int vimc_sen_remove(struct platform_device *pdev)
-{
-	component_del(&pdev->dev, &vimc_sen_comp_ops);
-
-	return 0;
-}
-
-static const struct platform_device_id vimc_sen_driver_ids[] = {
-	{
-		.name           = VIMC_SEN_DRV_NAME,
-	},
-	{ }
-};
-
-static struct platform_driver vimc_sen_pdrv = {
-	.probe		= vimc_sen_probe,
-	.remove		= vimc_sen_remove,
-	.id_table	= vimc_sen_driver_ids,
-	.driver		= {
-		.name	= VIMC_SEN_DRV_NAME,
-	},
-};
-
-module_platform_driver(vimc_sen_pdrv);
-
-MODULE_DEVICE_TABLE(platform, vimc_sen_driver_ids);
-
-MODULE_DESCRIPTION("Virtual Media Controller Driver (VIMC) Sensor");
-MODULE_AUTHOR("Helen Mae Koike Fornazier <[email protected]>");
-MODULE_LICENSE("GPL");

drivers/media/platform/vimc/vimc.h +2 -1

index a5adebdda941..cd527c6fe6fc 100644
@@ -92,11 +92,30 @@ struct vimc_device {
  	/* The Associated media_device parent */
  	struct media_device mdev;
 
-	/* Internal v4l2 parent device*/
+	/* Internal v4l2 parent device */
  	struct v4l2_device v4l2_dev;
+};
 

drivers/media/platform/vimc/vimc.h struct vimc_device { · line 102

Quoted 6 lines

drivers/media/platform/vimc/vimc.h +4 -2

-	/* Subdevices */
-	struct platform_device **subdevs;
+/* Structure which describes individual configuration for each entity */
+struct vimc_ent_config {
+	const char *name;
+	const char *drv;

I was tempted in suggesting to remove this field as it is not being used anymore, but it will be usefull for the configfs patch.

Hi Shuah,

Thank you for the patch.

Quoted 3 lines · Shuah Khan

In preparation for collapsing the component driver structure into a monolith, move private device structure defines to a new common header file.

Apart from the vimc_device structure, this doesn't seem to be needed. I'd rather keep each structure private to the .c file that handles it, and only share vimc_device globally.

Quoted 8 lines · Laurent Pinchart

Quoted 3 lines · Shuah Khan

In preparation for collapsing the component driver structure into a monolith, move private device structure defines to a new common header file.

Apart from the vimc_device structure, this doesn't seem to be needed. I'd rather keep each structure private to the .c file that handles it, and only share vimc_device globally.

Right. I initially thought that I needed these global. Once I completed the patches without needing these as global, I overlooked updating the patches.

I will take care of that. Any thoughts on vimc.h vs. adding vimc_device struct to existing vimc-common.h

As I explained to Helen in response to her comment about:

"My thinking is that vimc-common.h is common for all the subdevs and putting vimc-core defines and structures it shares it with the subdev files can be in a separate file.

It is more of design choice to keep structures and defined organized. Originally I was thinking all the subdev device structires need to be global, and my patch set I sent out as such doesn't need that. I just overlooked that when I sent the patches out.

This reduces the number of things that need to be common, I don't really have any strong reasons for either choice of adding common defines to vimc-common.h vs vimc.h - maybe with a slight tilt towards vimc.h"

Thanks all for a quick review and testing. I will work on v2 with your comments. I want to make sure topology either looks the same as what is in media master. I think it is, but I want to double check.

Hi Shua,

Quoted 28 lines · Shuah Khan

Quoted 8 lines · Laurent Pinchart

Quoted 3 lines · Shuah Khan

In preparation for collapsing the component driver structure into a monolith, move private device structure defines to a new common header file.

Apart from the vimc_device structure, this doesn't seem to be needed. I'd rather keep each structure private to the .c file that handles it, and only share vimc_device globally.

Right. I initially thought that I needed these global. Once I completed the patches without needing these as global, I overlooked updating the patches.

I will take care of that. Any thoughts on vimc.h vs. adding vimc_device struct to existing vimc-common.h

As I explained to Helen in response to her comment about:

"My thinking is that vimc-common.h is common for all the subdevs and putting vimc-core defines and structures it shares it with the subdev files can be in a separate file.

It is more of design choice to keep structures and defined organized. Originally I was thinking all the subdev device structires need to be global, and my patch set I sent out as such doesn't need that. I just overlooked that when I sent the patches out.

Quoted 3 lines

This reduces the number of things that need to be common, I don't really have any strong reasons for either choice of adding common defines to vimc-common.h vs vimc.h - maybe with a slight tilt towards vimc.h"

The vimc_device structure fits nicely in vimc-common.h in my opinion, as it's used by every component. I don't care much either way.

Quoted 144 lines · Helen Koike

Hi Andre,

Thanks for testing this.

Quoted 83 lines · André Almeida

Quoted 82 lines · Shuah Khan

Hi Andre,

Quoted 78 lines · André Almeida

Hello Shuah,

Thanks for the patch, I did some comments below.

Quoted 66 lines · Shuah Khan

vimc uses Component API to split the driver into functional components.
The real hardware resembles a monolith structure than component and component structure added a level of complexity making it hard to maintain without adding any real benefit.
      The sensor is one vimc component that would makes sense to be a separate
module to closely align with the real hardware. It would be easier to collapse vimc into single monolithic driver first and then split the sensor off as a separate module.

This patch series emoves the component API and makes minimal changes to
the code base preserving the functional division of the code structure.
Preserving the functional structure allows us to split the sensor off as a separate module in the future.

Major design elements in this change are:
      - Use existing struct vimc_ent_config and struct vimc_pipeline_config
        to drive the initialization of the functional components.
      - Make vimc_ent_config global by moving it to vimc.h
      - Add two new hooks add and rm to initialize and register, unregister
        and free subdevs.
      - All component API is now gone and bind and unbind hooks are modified
        to do "add" and "rm" with minimal changes to just add and rm subdevs.
      - vimc-core's bind and unbind are now register and unregister.
      - vimc-core invokes "add" hooks from its vimc_register_devices().
        The "add" hooks remain the same and register subdevs. They don't
        create platform devices of their own and use vimc's pdev.dev as
        their reference device. The "add" hooks save their vimc_ent_device(s)
        in the corresponding vimc_ent_config.
      - vimc-core invokes "rm" hooks from its unregister to unregister
subdevs
        and cleanup.
      - vimc-core invokes "add" and "rm" hooks with pointer to struct vimc_device
        and the corresponding struct vimc_ent_config pointer. The following configure and stream test works on all devices. media-ctl -d platform:vimc -V '"Sensor A":0[fmt:SBGGR8_1X8/640x480]'
      media-ctl -d platform:vimc -V '"Debayer A":0[fmt:SBGGR8_1X8/640x480]'
      media-ctl -d platform:vimc -V '"Sensor B":0[fmt:SBGGR8_1X8/640x480]'
      media-ctl -d platform:vimc -V '"Debayer B":0[fmt:SBGGR8_1X8/640x480]'
           v4l2-ctl -z platform:vimc -d "RGB/YUV Capture" -v width=1920,height=1440
      v4l2-ctl -z platform:vimc -d "Raw Capture 0" -v pixelformat=BA81
      v4l2-ctl -z platform:vimc -d "Raw Capture 1" -v pixelformat=BA81
           v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video1 v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video2 v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video3

Quoted 2 lines

The third patch in the series fixes a general protection fault found when rmmod is done while stream is active.

I applied your patch on top of media_tree/master and I did some testing.
Not sure if I did something wrong, but just adding and removing the module generated a kernel panic:

Quoted 53 lines

Quoted 5 lines

Thanks for testing.

Odd. I tested modprobe and rmmod both.I was working on Linux 5.3-rc2. I will apply these to media latest and work from there. I have to rebase these on top of the reverts from Lucas and Helen

Ok, please let me know if I succeeded to reproduce.

Quoted 45 lines

Quoted 45 lines

~# modprobe vimc
~# rmmod vimc
[   16.452974] stack segment: 0000 [#1] SMP PTI
[   16.453688] CPU: 0 PID: 2038 Comm: rmmod Not tainted 5.3.0-rc2+ #36 [   16.454678] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-20181126_142135-anatol 04/01/2014
[   16.456191] RIP: 0010:kfree+0x4d/0x240

<registers values...>

[   16.469188] Call Trace:
[   16.469666]  vimc_remove+0x35/0x90 [vimc]
[   16.470436]  platform_drv_remove+0x1f/0x40
[   16.471233]  device_release_driver_internal+0xd3/0x1b0 [   16.472184]  driver_detach+0x37/0x6b
[   16.472882]  bus_remove_driver+0x50/0xc1
[   16.473569]  vimc_exit+0xc/0xca0 [vimc]
[   16.474231]  __x64_sys_delete_module+0x18d/0x240 [   16.475036]  do_syscall_64+0x43/0x110
[   16.475656]  entry_SYSCALL_64_after_hwframe+0x44/0xa9 [   16.476504] RIP: 0033:0x7fceb8dafa4b

<registers values...>

[   16.484853] Modules linked in: vimc(-) videobuf2_vmalloc videobuf2_memops v4l2_tpg videobuf2_v4l2 videobuf2_common [   16.486187] ---[ end trace 91e5e0894e254d49 ]--- [   16.486758] RIP: 0010:kfree+0x4d/0x240

<registers values...>

fish: “rmmod vimc” terminated by signal SIGSEGV (Address boundary error)

I just added the module after booting, no other action was made. Here is
how my git log --oneline looks like:

897d708e922b media: vimc: Fix gpf in rmmod path when stream is active 2e4a5ad8ad6d media: vimc: Collapse component structure into a single monolithic driver
7c8da1687e92 media: vimc: move private defines to a common header 97299a303532 media: Remove dev_err() usage after platform_get_irq() 25a3d6bac6b9 media: adv7511/cobalt: rename driver name to adv7511-v4l2

I couldn't reproduce the error, my tree looks the same:

Quoted 9 lines

[I] koike@floko ~/m/o/linux> git log --oneline
e3345155c8ed (HEAD) media: vimc: Fix gpf in rmmod path when stream is active
43e9e2fe761f media: vimc: Collapse component structure into a single monolithic driver
8a6d0b9adde0 media: vimc: move private defines to a common header 97299a303532 (media/master) media: Remove dev_err() usage after platform_get_irq()
25a3d6bac6b9 media: adv7511/cobalt: rename driver name to adv7511-v4l2

Thanks Helen for trying to reproduce and sharing the result.

Quoted 44 lines

André, is this deterministic? Or it just happens sometimes?

Quoted 41 lines

Quoted 41 lines

Quoted 40 lines

...

Quoted 23 lines

vimc_print_dot (--print-dot) topology after this change:
digraph board {
     rankdir=TB
     n00000001 [label="{{} | Sensor A\n/dev/v4l-subdev0 | {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green] n00000001:port0 -> n00000005:port0 [style=bold] n00000001:port0 -> n0000000b [style=bold]
     n00000003 [label="{{} | Sensor B\n/dev/v4l-subdev1 | {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green] n00000003:port0 -> n00000008:port0 [style=bold] n00000003:port0 -> n0000000f [style=bold]
     n00000005 [label="{{<port0> 0} | Debayer A\n/dev/v4l-subdev2 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green] n00000005:port1 -> n00000015:port0
     n00000008 [label="{{<port0> 0} | Debayer B\n/dev/v4l-subdev3 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green] n00000008:port1 -> n00000015:port0 [style=dashed] n0000000b [label="Raw Capture 0\n/dev/video1", shape=box, style=filled, fillcolor=yellow]
     n0000000f [label="Raw Capture 1\n/dev/video2", shape=box, style=filled, fillcolor=yellow]
     n00000013 [label="{{} | RGB/YUV Input\n/dev/v4l-subdev4 |

Quoted 8 lines

{<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green] n00000013:port0 -> n00000015:port0 [style=dashed] n00000015 [label="{{<port0> 0} | Scaler\n/dev/v4l-subdev5 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green] n00000015:port1 -> n00000018 [style=bold] n00000018 [label="RGB/YUV Capture\n/dev/video3", shape=box, style=filled, fillcolor=yellow]
}

Since the topology changed, it would be nice to change in the documentation as well. The current dot file can be found at Documentation/media/v4l-drivers/vimc.dot and it's rendered at this page:
https://www.kernel.org/doc/html/latest/media/v4l-drivers/vimc.html

Thanks Andre! Yes this is the one I am using as a reference.

Quoted 8 lines

Quoted 7 lines

Quoted 3 lines

Topology shouldn't have changed. No changes to links or pads etc. I will take a look to be sure. I agree that if topology changes document should be updated.

If you "diff" the current dot with the dot you generated, you will see some differences. The main difference is that "RGB/YUV Input" was a device "/dev/video2/", and now it a subdevice "/dev/v4l-subdev4".

Quoted 6 lines

hmm, I just generated the topology for media/master, and it is /dev/v4l-subdev4. As we don't have an implementation of the output device yet, we used the sensor as a place holder and that is why it appears as "/dev/v4l-subdev4", what is in the docs is the ideal version after we get the output merged. We should update the docs in any case.

Helen! Does the master match what I generated. In any case, I will do the diff of before and after my patches for sure.

I can update the document with what it should be.

thanks,
-- Shuah

Hi Shuah,

Quoted 155 lines · Shuah Khan

Quoted 144 lines · Helen Koike

Hi Andre,

Thanks for testing this.

Quoted 83 lines · André Almeida

Quoted 82 lines · Shuah Khan

Hi Andre,

Quoted 78 lines · André Almeida

Hello Shuah,

Thanks for the patch, I did some comments below.

Quoted 66 lines · Shuah Khan

vimc uses Component API to split the driver into functional components.
The real hardware resembles a monolith structure than component and component structure added a level of complexity making it hard to maintain without adding any real benefit.
      The sensor is one vimc component that would makes sense to be a
separate
module to closely align with the real hardware. It would be easier to collapse vimc into single monolithic driver first and then split the sensor off as a separate module.

This patch series emoves the component API and makes minimal changes to
the code base preserving the functional division of the code structure.
Preserving the functional structure allows us to split the sensor off as a separate module in the future.

Major design elements in this change are:
      - Use existing struct vimc_ent_config and struct
vimc_pipeline_config
        to drive the initialization of the functional components.
      - Make vimc_ent_config global by moving it to vimc.h
      - Add two new hooks add and rm to initialize and register,
unregister
        and free subdevs.
      - All component API is now gone and bind and unbind hooks are
modified
        to do "add" and "rm" with minimal changes to just add and rm
subdevs.
      - vimc-core's bind and unbind are now register and unregister.
      - vimc-core invokes "add" hooks from its
vimc_register_devices().
        The "add" hooks remain the same and register subdevs. They
don't
        create platform devices of their own and use vimc's
pdev.dev as
        their reference device. The "add" hooks save their
vimc_ent_device(s)
        in the corresponding vimc_ent_config.
      - vimc-core invokes "rm" hooks from its unregister to
unregister
subdevs
        and cleanup.
      - vimc-core invokes "add" and "rm" hooks with pointer to struct
vimc_device
        and the corresponding struct vimc_ent_config pointer. The following configure and stream test works on all devices. media-ctl -d platform:vimc -V '"Sensor
A":0[fmt:SBGGR8_1X8/640x480]'
      media-ctl -d platform:vimc -V '"Debayer
A":0[fmt:SBGGR8_1X8/640x480]'
      media-ctl -d platform:vimc -V '"Sensor
B":0[fmt:SBGGR8_1X8/640x480]'
      media-ctl -d platform:vimc -V '"Debayer
B":0[fmt:SBGGR8_1X8/640x480]'
           v4l2-ctl -z platform:vimc -d "RGB/YUV Capture" -v
width=1920,height=1440
      v4l2-ctl -z platform:vimc -d "Raw Capture 0" -v
pixelformat=BA81
      v4l2-ctl -z platform:vimc -d "Raw Capture 1" -v
pixelformat=BA81
           v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video1 v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video2 v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video3

Quoted 2 lines

The third patch in the series fixes a general protection fault found when rmmod is done while stream is active.

I applied your patch on top of media_tree/master and I did some testing.
Not sure if I did something wrong, but just adding and removing the module generated a kernel panic:

Quoted 53 lines

Quoted 5 lines

Thanks for testing.

Odd. I tested modprobe and rmmod both.I was working on Linux 5.3-rc2. I will apply these to media latest and work from there. I have to rebase these on top of the reverts from Lucas and Helen

Ok, please let me know if I succeeded to reproduce.

Quoted 45 lines

Quoted 45 lines

~# modprobe vimc
~# rmmod vimc
[   16.452974] stack segment: 0000 [#1] SMP PTI
[   16.453688] CPU: 0 PID: 2038 Comm: rmmod Not tainted 5.3.0-rc2+ #36 [   16.454678] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-20181126_142135-anatol 04/01/2014
[   16.456191] RIP: 0010:kfree+0x4d/0x240

<registers values...>

[   16.469188] Call Trace:
[   16.469666]  vimc_remove+0x35/0x90 [vimc]
[   16.470436]  platform_drv_remove+0x1f/0x40
[   16.471233]  device_release_driver_internal+0xd3/0x1b0 [   16.472184]  driver_detach+0x37/0x6b
[   16.472882]  bus_remove_driver+0x50/0xc1
[   16.473569]  vimc_exit+0xc/0xca0 [vimc]
[   16.474231]  __x64_sys_delete_module+0x18d/0x240 [   16.475036]  do_syscall_64+0x43/0x110
[   16.475656]  entry_SYSCALL_64_after_hwframe+0x44/0xa9 [   16.476504] RIP: 0033:0x7fceb8dafa4b

<registers values...>

[   16.484853] Modules linked in: vimc(-) videobuf2_vmalloc videobuf2_memops v4l2_tpg videobuf2_v4l2 videobuf2_common [   16.486187] ---[ end trace 91e5e0894e254d49 ]--- [   16.486758] RIP: 0010:kfree+0x4d/0x240

<registers values...>

fish: “rmmod vimc” terminated by signal SIGSEGV (Address boundary error)

I just added the module after booting, no other action was made. Here is
how my git log --oneline looks like:

897d708e922b media: vimc: Fix gpf in rmmod path when stream is active 2e4a5ad8ad6d media: vimc: Collapse component structure into a single monolithic driver
7c8da1687e92 media: vimc: move private defines to a common header 97299a303532 media: Remove dev_err() usage after platform_get_irq() 25a3d6bac6b9 media: adv7511/cobalt: rename driver name to adv7511-v4l2

I couldn't reproduce the error, my tree looks the same:

Quoted 9 lines

[I] koike@floko ~/m/o/linux> git log --oneline
e3345155c8ed (HEAD) media: vimc: Fix gpf in rmmod path when stream is active
43e9e2fe761f media: vimc: Collapse component structure into a single monolithic driver
8a6d0b9adde0 media: vimc: move private defines to a common header 97299a303532 (media/master) media: Remove dev_err() usage after platform_get_irq()
25a3d6bac6b9 media: adv7511/cobalt: rename driver name to adv7511-v4l2

Quoted 1 line

Thanks Helen for trying to reproduce and sharing the result.

Me and Helen found out what is the problem. If you follow this call trace:

vimc_ent_sd_unregister()
v4l2_device_unregister_subdev()
v4l2_subdev_release()

You'll notice that this last function calls the release callback implementation of the subdevice. For instance, the release of vimc-sensor is this one:

static void vimc_sen_release(struct v4l2_subdev *sd)

{
	struct vimc_sen_device *vsen =
				container_of(sd, struct vimc_sen_device, sd);

	v4l2_ctrl_handler_free(&vsen->hdl);
	tpg_free(&vsen->tpg);
	kfree(vsen);
}

And then you can see that vsen has been freed. Back to vimc_ent_sd_unregister(), after v4l2_device_unregister_subdev(), the function will call vimc_pads_cleanup(). This is basically a kfree(ved->pads), but ved has just been freed at v4l2_subdev_release(), producing a memory fault.

To fix that, we found two options:

- place the kfree(ved->pads) inside the release callback of each subdevice and removing vimc_pads_cleanup() from vimc_ent_sd_unregister()
- use a auxiliary variable to hold the address of the pads, for instance: