media: Fault-Tolerant V4L2
Subsystems (6)
v2 cover media: Fault-Tolerant V4L2
On media-centric camera pipelines, every entity is known at boot time and described statically (for example via device-tree).
The V4L2 framework assumes that every entity of the camera pipeline is always present. This is a reasonable assumption for integrated cameras in laptops or phones. However, in automotive systems, cameras (e.g. rear-view or surround-view) can be damaged or disconnected. When the framework assumes all sensors are present, a single missing camera can prevent the entire pipeline from operating.
This is a known problem and has been discussed at various conferences:
* https://www.linuxtv.org/downloads/presentations/media_summit_2026/Michael%20-%20The%20Butterfly%20Effect.pdf
* https://www.linuxtv.org/news.php?entry=2022-11-14-0.hverkuil
* https://lore.kernel.org/linux-media/[email protected]/T/#u
For example, in the following pipeline:
"""
imx219 9-0010 -> ds90ub953 7-0044 -> ds90ub960 pad0 -+
|-- pad4 -> cdns_csi2rx -> ticsi2rx
imx219 10-0010 -> ds90ub953 7-0045 -> ds90ub960 pad1 -+
"""
If imx219 9-0010 fails to probe, media-ctl -p will show:
"""
ds90ub953 7-0044 -> ds90ub960 pad0 -+
|-- pad4 -> cdns_csi2rx -> ticsi2rx imx219 10-0010 -> ds90ub953 7-0045 -> ds90ub960 pad1 -+
"""
Now, if ds90ub953 7-0044 fails to probe(), media-ctl -p will only show the following topology:
"""
cdns_csi2rx -> ticsi2rx
"""
To solve this, we chose to go for a simple solution inspired by DRM connectors.
We created an ioctl for media entitities to report their current connection status.
This allows applications to discover an entity state when discovering the graph (using media-ctl -p, for example)
The entity status is then cached. When applications start streaming, the status is polled on a regular basis and reported as a media event every time it changes.
This mandates that the driver can probe even when the device is not connected.
This has the benefits of being fairly simple to implement, understand and support in drivers.
This new behaviour is opt-in. It is enabled via a module parameter for the Media Controler: mc.fault_tolerant.
Patch 1-9 Add event support at media device level Patch 10-14 Add entity status and ioctls to interact with it Patch 15-16 implement this pattern switch for the imx219 driver.
With this, we can hot-plug the sensor and get notified via the media device of the status:
[root@fedora]# ./media-event-monitor Subscribed to ENTITY_REGISTRATION and ENTITY_STATUS_CHANGE on /dev/media0 Waiting for events (Ctrl-C to quit)... event: type=2 (ENTITY_STATUS_CHANGE) id=0 seq=0 ts=403045107120 pending=0 entity_id=40 status=DISCONNECTED event: type=2 (ENTITY_STATUS_CHANGE) id=0 seq=1 ts=415292362587 pending=0 entity_id=40 status=CONNECTED
The code for this demo is available at:
https://gitlab.com/-/snippets/6077614
The media device event code is mostly Sakari's work which can be found at:
https://git.linuxtv.org/sailus/media_tree.git/log/?h=partial-reg
Current known limitations:
* Only leaf entities (typically sensor nodes) are covered. We should also address entities that are in the middle of the media graph
This has been tested on a SK-AM69 board on top of the following series:
* https://lore.kernel.org/all/[email protected]/
* https://lore.kernel.org/all/20261104-j721e-csi2rx-crash-sd-enable-streams-v1-1-b77ee38c8dcf@kernel.org/
Thanks,
Mattijs
---
Changes in v2:
- Notifications are now done at media device level (Laurent, Nicolas)
- Connection status is tracked at entity level (Laurent)
- imx219: Fail the probe if CHIP_ID does not match (Dave)
- Dropped patch 'media: imx219: Move LP-11 state switch to power_on()'
as we did not get any feedback from cadence on this
- Replaced VIDIOC_SUBDEV_G_CONNECTION_STATUS with a media controller
ioctl instead: MEDIA_IOC_G_ENTITY_STATUS
- Do status polling in generic framework (Dave)
- Increase polling interval to 10s (Dave)
- Polling/Fault tolerance is only enabled when mc.faul_tolerance=1
module param is set (Dave, Hans V, Sakari)
- Reworked .detect() polling to be only done when streaming is on. This
should address the privacy led blinking concerns (Dave, Hans D)
- Link to v1: https://lore.kernel.org/r/[email protected]
---
Laurent Pinchart (1):
media: mc: Add per-file-handle data support
Mattijs Korpershoek (7):
media: media.h: Add entity connection status
media: mc: Add .detect() to media_entity_operations
media: mc: Add MEDIA_IOC_G_ENTITY_STATUS ioctl
media: mc: Add MEDIA_EVENT_ENTITY_STATUS_CHANGE event
media: mc: Add detect polling helpers for drivers
media: imx219: Allow driver probe with missing sensor
media: imx219: Implement .detect() sensor operation
Sakari Ailus (8):
media: mc: Add struct media_device_fh pointer to media device IOCTL op
v4l: event: Separate event related information from struct v4l2_fh
v4l: event: Rework data structures for separating events and V4L2
v4l: event: Split the event framework into generic and V4L2 bits
media: mc: Maintain a list of open file handles in a media device
media: mc: Add MEDIA_IOC_DQEVENT
media: mc: Add MEDIA_EVENT_ENTITY_REGISTRATION
Documentation: media: Add Media controller event documentation
.../userspace-api/media/mediactl/media-funcs.rst | 3 + .../media/mediactl/media-ioc-dqevent.rst | 130 +++++++++ .../media/mediactl/media-ioc-g-entity-status.rst | 92 ++++++ .../media/mediactl/media-ioc-subscribe-event.rst | 112 +++++++ drivers/media/Kconfig | 3 + drivers/media/i2c/imx219.c | 117 +++++--- drivers/media/mc/Kconfig | 3 + drivers/media/mc/Makefile | 3 +- drivers/media/mc/mc-device.c | 236 ++++++++++++++- drivers/media/mc/mc-devnode.c | 19 +- drivers/media/mc/mc-entity.c | 59 ++++ drivers/media/mc/mc-evbase.c | 260 +++++++++++++++++ drivers/media/mc/mc-event.c | 244 ++++++++++++++++ drivers/media/pci/ivtv/ivtv-fileops.c | 2 +- drivers/media/usb/uvc/uvc_ctrl.c | 36 ++- drivers/media/v4l2-core/v4l2-ctrls-api.c | 16 +- drivers/media/v4l2-core/v4l2-ctrls-core.c | 20 +- drivers/media/v4l2-core/v4l2-event.c | 324 +++++++++------------ drivers/media/v4l2-core/v4l2-fh.c | 12 +- include/media/media-device.h | 24 ++ include/media/media-devnode.h | 18 +- include/media/media-entity.h | 77 +++++ include/media/media-evbase.h | 200 +++++++++++++ include/media/media-event.h | 181 ++++++++++++ include/media/media-fh.h | 37 +++ include/media/v4l2-event.h | 106 ++++--- include/media/v4l2-fh.h | 32 +- include/uapi/linux/media.h | 51 +++- 28 files changed, 2079 insertions(+), 338 deletions(-)
---
base-commit: 6628e7d4c6b25c8d18f1345e3a20d21829aae7e4
base-tree: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git master
change-id: 20260923-v4l2-sensor-detect-32a0b2936cc1
Best regards,
Signature · 1 line
Mattijs Korpershoek <[email protected]>
RFC PATCH v2 1/16 media: mc: Add struct media_device_fh pointer to media device IOCTL op
From: Sakari Ailus <[email protected]>
In order to use the struct media_device_fh pointer in IOCTL callbacks, add it to the callback, between the Media device and the IOCTL argument.
--- drivers/media/mc/mc-device.c | 30 +++++++++++++++++++----------- 1 file changed, 19 insertions(+), 11 deletions(-)
drivers/media/mc/mc-device.c +19 -11
@@ -55,7 +55,8 @@ static int media_device_close(struct file *filp)return 0 ; } -static long media_device_get_info (struct media_device *dev,void *arg) +static long media_device_get_info (struct media_device *dev, +struct file *file,void *arg) {struct media_device_info *info = arg;
@@ -95,7 +96,8 @@ static struct media_entity *find_entity(struct media_device *mdev, u32 id)return NULL ; } -static long media_device_enum_entities (struct media_device *mdev,void *arg) +static long media_device_enum_entities (struct media_device *mdev, +struct file *file,void *arg) {struct media_entity_desc *entd = arg;struct media_entity *ent;
@@ -148,7 +150,8 @@ static void media_device_kpad_to_upad(const struct media_pad *kpad, upad->flags = kpad->flags; } -static long media_device_enum_links (struct media_device *mdev,void *arg) +static long media_device_enum_links (struct media_device *mdev, +struct file *file,void *arg) {struct media_links_enum *links = arg;struct media_entity *entity;
@@ -197,7 +200,8 @@ static long media_device_enum_links(struct media_device *mdev, void *arg)return 0 ; } -static long media_device_setup_link (struct media_device *mdev,void *arg) +static long media_device_setup_link (struct media_device *mdev, +struct file *file,void *arg) {struct media_link_desc *linkd = arg;struct media_link *link =NULL ;
@@ -227,7 +231,8 @@ static long media_device_setup_link(struct media_device *mdev, void *arg)return __media_entity_setup_link (link, linkd->flags); } -static long media_device_get_topology (struct media_device *mdev,void *arg) +static long media_device_get_topology (struct media_device *mdev, +struct file *file,void *arg) {struct media_v2_topology *topo = arg;struct media_entity *entity;
@@ -370,7 +375,8 @@ static long media_device_get_topology(struct media_device *mdev, void *arg)return ret; } -static long media_device_request_alloc (struct media_device *mdev,void *arg) +static long media_device_request_alloc (struct media_device *mdev, +struct file *file,void *arg) {int *alloc_fd = arg;
@@ -417,7 +423,8 @@ static long copy_arg_to_user(void __user *uarg, void *karg, unsigned int cmd)struct media_ioctl_info {unsigned int cmd;unsigned short flags; -long (*fn)(struct media_device *dev,void *arg); +long (*fn)(struct media_device *dev,struct file *file, +void *arg);long (*arg_from_user)(void *karg,void __user *uarg,unsigned int cmd);long (*arg_to_user)(void __user *uarg,void *karg,unsigned int cmd); };
@@ -462,7 +469,7 @@ static long media_device_ioctl(struct file *filp, unsigned int cmd,if (info->flags &MEDIA_IOC_FL_GRAPH_MUTEX )mutex_lock (&dev->graph_mutex); - ret = info->fn (dev, karg); + ret = info->fn (dev, filp, karg);if (info->flags &MEDIA_IOC_FL_GRAPH_MUTEX )mutex_unlock (&dev->graph_mutex);
@@ -487,6 +494,7 @@ struct media_links_enum32 { };static long media_device_enum_links32 (struct media_device *mdev, +struct file *file,struct media_links_enum32 __user *ulinks) {struct media_links_enum links;
@@ -503,7 +511,7 @@ static long media_device_enum_links32(struct media_device *mdev, links.pads =compat_ptr (pads_ptr); links.links =compat_ptr (links_ptr); - ret =media_device_enum_links (mdev, &links); + ret =media_device_enum_links (mdev, file, &links);if (ret)return ret;
@@ -525,8 +533,8 @@ static long media_device_compat_ioctl(struct file *filp, unsigned int cmd,switch (cmd) {case MEDIA_IOC_ENUM_LINKS32 :mutex_lock (&dev->graph_mutex); - ret =media_device_enum_links32 (dev, - (struct media_links_enum32 __user *)arg); + ret =media_device_enum_links32 (dev, filp, + (struct media_links_enum32 __user *)arg);mutex_unlock (&dev->graph_mutex);break ;
Signature · 1 line
2.56.0
RFC PATCH v2 2/16 media: mc: Add per-file-handle data support
In reply to Mattijs KorpershoekFrom: Laurent Pinchart <[email protected]>
The media devnode core associates devnodes with files by storing the devnode pointer in the file structure private_data field. In order to allow tracking of per-file-handle data introduce a new media devnode file handle structure that stores the devnode pointer, and store a pointer to that structure in the file private_data field.
Users of the media devnode code (the only existing user being media_device) are responsible for managing their own subclass of the media_devnode_fh structure.
--- drivers/media/mc/mc-device.c | 21 +++++++++++++++++++++ drivers/media/mc/mc-devnode.c | 19 ++++++++++--------- include/media/media-devnode.h | 18 +++++++++++++++++- 3 files changed, 48 insertions(+), 10 deletions(-)
drivers/media/mc/mc-device.c +21 -0
@@ -36,6 +36,15 @@#define MEDIA_ENT_T_DEVNODE_UNKNOWN (MEDIA_ENT_F_OLD_BASE | \ MEDIA_ENT_SUBTYPE_MASK) +struct media_device_fh { +struct media_devnode_fh fh; +}; + +static inline struct media_device_fh *media_device_fh (struct file *filp) +{ + return container_of(filp->private_data, struct media_device_fh, fh); +} +/* ----------------------------------------------------------------------------- * Userspace API */
@@ -47,11 +56,23 @@ static inline void __user *media_get_uptr(__u64 arg)static int media_device_open (struct file *filp) { +struct media_device_fh *fh; + + fh =kzalloc_obj (*fh); +if (!fh) +return -ENOMEM ; + + filp->private_data = &fh->fh; +return 0 ; }static int media_device_close (struct file *filp) { +struct media_device_fh *fh =media_device_fh (filp); + +kfree (fh); +return 0 ; }
drivers/media/mc/mc-devnode.c +10 -9
@@ -140,6 +140,7 @@ static long media_compat_ioctl(struct file *filp, unsigned int cmd,static int media_open (struct inode *inode,struct file *filp) {struct media_devnode *devnode; +struct media_devnode_fh *fh;int ret;/* Check if the media device is available. This needs to be done with
@@ -162,15 +163,16 @@ static int media_open(struct inode *inode, struct file *filp) filp->private_data = devnode; -if (devnode->fops->open) { - ret = devnode->fops->open (filp); -if (ret) { -put_device (&devnode->dev); - filp->private_data =NULL ; -return ret; - } + ret = devnode->fops->open (filp); +if (ret) { +put_device (&devnode->dev); + filp->private_data =NULL ; +return ret; } + fh = filp->private_data; + fh->devnode = devnode; +return 0 ; }
@@ -179,8 +181,7 @@ static int media_release(struct inode *inode, struct file *filp) {struct media_devnode *devnode =media_devnode_data (filp); -if (devnode->fops->release) - devnode->fops->release (filp); + devnode->fops->release (filp); filp->private_data =NULL ;
include/media/media-devnode.h +17 -1
@@ -59,6 +59,20 @@ struct media_file_operations {int (*release) (struct file *); }; +/** +* struct media_devnode_fh - Media device node file handle +* @devnode: pointer to the media device node +* +* This structure serves as a base for per-file-handle data storage. Media +* device node users embed media_devnode_fh in their custom file handle data +* structures and store the media_devnode_fh in the file private_data in order +* to let the media device node core locate the media_devnode corresponding to a +* file handle. +*/ +struct media_devnode_fh { +struct media_devnode *devnode; +}; +/** * struct media_devnode - Media device node * @media_dev: pointer to struct &media_device
@@ -150,7 +164,9 @@ void media_devnode_unregister(struct media_devnode *devnode);*/ static inline struct media_devnode *media_devnode_data (struct file *filp) { -return filp->private_data; +struct media_devnode_fh *fh = filp->private_data; + +return fh->devnode; }/**
Signature · 1 line
2.56.0
RFC PATCH v2 3/16 v4l: event: Separate event related information from struct v4l2_fh
In reply to Mattijs KorpershoekFrom: Sakari Ailus <[email protected]>
Generalise event related file handle information to struct media_event_fh.
--- drivers/media/pci/ivtv/ivtv-fileops.c | 2 +- drivers/media/v4l2-core/v4l2-event.c | 72 ++++++++++++++++++----------------- drivers/media/v4l2-core/v4l2-fh.c | 12 +++--- include/media/media-evbase.h | 40 +++++++++++++++++++ include/media/v4l2-fh.h | 20 +++------- 5 files changed, 90 insertions(+), 56 deletions(-)
drivers/media/pci/ivtv/ivtv-fileops.c +1 -1
@@ -742,7 +742,7 @@ __poll_t ivtv_v4l2_dec_poll(struct file *filp, poll_table *wait)/* If there are subscribed events, then only use the new event API instead of the old video.h based API. */ -if (!list_empty (&id->fh.subscribed)) { +if (!list_empty (&id->fh.ev_fh.subscribed)) {poll_wait (filp, &id->fh.wait, wait);/* Turn off the old-style vsync events */ clear_bit (IVTV_F_I_EV_VSYNC_ENABLED , &itv->i_flags);
drivers/media/v4l2-core/v4l2-event.c +37 -35
201 lines not on this page · raw message
drivers/media/v4l2-core/v4l2-fh.c +7 -5
@@ -34,10 +34,12 @@ void v4l2_fh_init(struct v4l2_fh *fh, struct video_device *vdev)set_bit (_IOC_NR (VIDIOC_S_PRIORITY ), vdev->valid_ioctls); fh->prio =V4L2_PRIORITY_UNSET ;init_waitqueue_head (&fh->wait); -INIT_LIST_HEAD (&fh->available); -INIT_LIST_HEAD (&fh->subscribed); - fh->sequence =-1 ; -mutex_init (&fh->subscribe_lock); + fh->ev_fh.lock = &vdev->fh_lock; + fh->ev_fh.wait = &fh->wait; +INIT_LIST_HEAD (&fh->ev_fh.available); +INIT_LIST_HEAD (&fh->ev_fh.subscribed); + fh->ev_fh.sequence =-1 ; +mutex_init (&fh->ev_fh.subscribe_lock); }EXPORT_SYMBOL_GPL (v4l2_fh_init);
@@ -86,7 +88,7 @@ void v4l2_fh_exit(struct v4l2_fh *fh)return ;v4l_disable_media_source (fh->vdev);v4l2_event_unsubscribe_all (fh); -mutex_destroy (&fh->subscribe_lock); +mutex_destroy (&fh->ev_fh.subscribe_lock); fh->vdev =NULL ; }EXPORT_SYMBOL_GPL (v4l2_fh_exit);
include/media/media-evbase.h new file +40 -0
@@ -0,0 +1,40 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* +* media-evbase.h +* +* Copyright (C) 2018--2019 Intel Corporation +*/ + +#ifndef MEDIA_EVBASE_H +#define MEDIA_EVBASE_H + +#include <linux/spinlock.h> +#include <linux/wait.h> + +struct list_head ; + +/* +* struct media_evbase_fh - File handles for media subsystem events +* +* @lock: serialise access to fields and lists (apart from wait) +* @subscribe_lock: serialise changes to the subscribed list; guarantee that +* the add and del event callbacks are orderly called +* @wait: waiting list for the events +* @subscribed: list of subscribed events +* @available: list of events waiting to be dequeued +* @navailable: number of available events at @available list +* @sequence: event sequence number +* +*/ +struct media_evbase_fh { +spinlock_t *lock; +struct mutex subscribe_lock; +wait_queue_head_t *wait; + +struct list_head subscribed; +struct list_head available; +unsigned int navailable; +u32 sequence; +}; + +#endif /* MEDIA_EVBASE_H */
include/media/v4l2-fh.h +5 -15
@@ -18,6 +18,8 @@#include <linux/list.h> #include <linux/videodev2.h> +#include <media/media-evbase.h> +struct video_device ;struct v4l2_ctrl_handler ;
@@ -28,32 +30,20 @@ struct v4l2_ctrl_handler;* @vdev: pointer to &struct video_device * @ctrl_handler: pointer to &struct v4l2_ctrl_handler * @prio: priority of the file handler, as defined by &enum v4l2_priority -* -* @wait: event' s wait queue -* @subscribe_lock: serialise changes to the subscribed list; guarantee that -* the add and del event callbacks are orderly called -* @subscribed: list of subscribed events -* @available: list of events waiting to be dequeued -* @navailable: number of available events at @available list -* @sequence: event sequence number +* @wait: file handle's wait queue * * @m2m_ctx: pointer to &struct v4l2_m2m_ctx +* @ev_fh: file handle specific data for events */ struct v4l2_fh {struct list_head list;struct video_device *vdev;struct v4l2_ctrl_handler *ctrl_handler;enum v4l2_priority prio; - -/* Events */ wait_queue_head_t wait; -struct mutex subscribe_lock; -struct list_head subscribed; -struct list_head available; -unsigned int navailable; -u32 sequence;struct v4l2_m2m_ctx *m2m_ctx; +struct media_evbase_fh ev_fh; };/**
Signature · 1 line
2.56.0
RFC PATCH v2 4/16 v4l: event: Rework data structures for separating events and V4L2
In reply to Mattijs KorpershoekFrom: Sakari Ailus <[email protected]>
The V4L2 event framework will be functionality-wise useful for implementing events for the media controller. Separate the V4L2 dependent and V4L2 independent bits from data structures now.
--- drivers/media/usb/uvc/uvc_ctrl.c | 36 ++++---- drivers/media/v4l2-core/v4l2-ctrls-api.c | 16 ++-- drivers/media/v4l2-core/v4l2-ctrls-core.c | 20 +++-- drivers/media/v4l2-core/v4l2-event.c | 140 ++++++++++++++++++++++-------- include/media/media-evbase.h | 80 ++++++++++++++++- include/media/v4l2-event.h | 100 +++++++++++++-------- include/media/v4l2-fh.h | 12 +++ 7 files changed, 303 insertions(+), 101 deletions(-)
drivers/media/usb/uvc/uvc_ctrl.c +21 -15
@@ -2054,7 +2054,7 @@ static void uvc_ctrl_send_event(struct uvc_video_chain *chain, struct uvc_control_mapping *mapping, s32 value, u32 changes) { struct v4l2_fh *originator = handle ? &handle->vfh : NULL; -struct v4l2_subscribed_event *sev; +struct media_subscribed_evbase *mse;struct v4l2_event ev;if (list_empty (&mapping->ev_subs))
@@ -2062,11 +2062,15 @@ static void uvc_ctrl_send_event(struct uvc_video_chain *chain,uvc_ctrl_fill_event (chain, &ev, ctrl, mapping, value, changes); -list_for_each_entry (sev, &mapping->ev_subs, node) { -if (sev->fh != originator || - (sev->flags &V4L2_EVENT_SUB_FL_ALLOW_FEEDBACK ) || +list_for_each_entry (mse, &mapping->ev_subs, node) { +struct v4l2_subscribed_event *sev = +to_v4l2_subscribed_event (mse); + +if (media_evbase_fh_to_v4l2 (mse->fh) != originator || + (sev->evspec.flags &V4L2_EVENT_SUB_FL_ALLOW_FEEDBACK ) || (changes &V4L2_EVENT_CTRL_CH_FLAGS )) -v4l2_event_queue_fh (sev->fh, &ev); +v4l2_event_queue_fh (media_evbase_fh_to_v4l2 (mse->fh), + &ev); } }
@@ -2295,7 +2299,8 @@ static void uvc_ctrl_send_events(struct uvc_fh *handle,static int uvc_ctrl_add_event (struct v4l2_subscribed_event *sev,unsigned elems) { - struct uvc_fh *handle = container_of(sev->fh, struct uvc_fh, vfh); +struct uvc_fh *handle =container_of ( + media_evbase_fh_to_v4l2(sev->mse.fh), struct uvc_fh, vfh);struct uvc_control_mapping *mapping;struct uvc_control *ctrl;int ret;
@@ -2304,18 +2309,18 @@ static int uvc_ctrl_add_event(struct v4l2_subscribed_event *sev, unsigned elems)if (ret <0 )return -ERESTARTSYS ; -if (__uvc_query_v4l2_class (handle->chain, sev->id,0 ) >=0 ) { +if (__uvc_query_v4l2_class (handle->chain, sev->evspec.id,0 ) >=0 ) { ret =0 ; goto done; } - ctrl =uvc_find_control (handle->chain, sev->id, &mapping); + ctrl =uvc_find_control (handle->chain, sev->evspec.id, &mapping);if (ctrl ==NULL ) { ret = -EINVAL ; goto done; } -if (sev->flags &V4L2_EVENT_SUB_FL_SEND_INITIAL ) { +if (sev->evspec.flags &V4L2_EVENT_SUB_FL_SEND_INITIAL ) {struct v4l2_event ev;u32 changes =V4L2_EVENT_CTRL_CH_FLAGS ;s32 val =0 ;
@@ -2337,11 +2342,11 @@ static int uvc_ctrl_add_event(struct v4l2_subscribed_event *sev, unsigned elems)* Mark the queue as active, allowing this initial event to be * accepted. */ - sev->elems = elems; -v4l2_event_queue_fh (sev->fh, &ev); + sev->mse.elems = elems; +v4l2_event_queue_fh (media_evbase_fh_to_v4l2 (sev->mse.fh), &ev); } -list_add_tail (&sev->node, &mapping->ev_subs); +list_add_tail (&sev->mse.node, &mapping->ev_subs); done:mutex_unlock (&handle->chain->ctrl_mutex);
@@ -2350,12 +2355,13 @@ static int uvc_ctrl_add_event(struct v4l2_subscribed_event *sev, unsigned elems)static void uvc_ctrl_del_event (struct v4l2_subscribed_event *sev) { - struct uvc_fh *handle = container_of(sev->fh, struct uvc_fh, vfh); +struct uvc_fh *handle =container_of ( + media_evbase_fh_to_v4l2(sev->mse.fh), struct uvc_fh, vfh);mutex_lock (&handle->chain->ctrl_mutex); -if (__uvc_query_v4l2_class (handle->chain, sev->id,0 ) >=0 ) +if (__uvc_query_v4l2_class (handle->chain, sev->evspec.id,0 ) >=0 ) goto done; -list_del (&sev->node); +list_del (&sev->mse.node); done:mutex_unlock (&handle->chain->ctrl_mutex); }
drivers/media/v4l2-core/v4l2-ctrls-api.c +10 -6
@@ -1277,29 +1277,33 @@ EXPORT_SYMBOL(v4l2_ctrl_subdev_log_status);static int v4l2_ctrl_add_event (struct v4l2_subscribed_event *sev,unsigned int elems) { -struct v4l2_ctrl *ctrl =v4l2_ctrl_find (sev->fh->ctrl_handler, sev->id); +struct v4l2_fh *fh =media_evbase_fh_to_v4l2 (sev->mse.fh); +struct v4l2_ctrl *ctrl =v4l2_ctrl_find (fh->ctrl_handler, + sev->evspec.id);if (!ctrl)return -EINVAL ;v4l2_ctrl_lock (ctrl); -list_add_tail (&sev->node, &ctrl->ev_subs); +list_add_tail (&sev->mse.node, &ctrl->ev_subs);if (ctrl->type !=V4L2_CTRL_TYPE_CTRL_CLASS && - (sev->flags &V4L2_EVENT_SUB_FL_SEND_INITIAL )) -send_initial_event (sev->fh, ctrl); + (sev->evspec.flags &V4L2_EVENT_SUB_FL_SEND_INITIAL )) +send_initial_event (media_evbase_fh_to_v4l2 (sev->mse.fh), ctrl);v4l2_ctrl_unlock (ctrl);return 0 ; }static void v4l2_ctrl_del_event (struct v4l2_subscribed_event *sev) { -struct v4l2_ctrl *ctrl =v4l2_ctrl_find (sev->fh->ctrl_handler, sev->id); +struct v4l2_fh *fh =media_evbase_fh_to_v4l2 (sev->mse.fh); +struct v4l2_ctrl *ctrl =v4l2_ctrl_find (fh->ctrl_handler, + sev->evspec.id);if (!ctrl)return ;v4l2_ctrl_lock (ctrl); -list_del (&sev->node); +list_del (&sev->mse.node);v4l2_ctrl_unlock (ctrl); }
drivers/media/v4l2-core/v4l2-ctrls-core.c +12 -8
@@ -56,16 +56,20 @@ void send_initial_event(struct v4l2_fh *fh, struct v4l2_ctrl *ctrl)void send_event (struct v4l2_fh *fh,struct v4l2_ctrl *ctrl,u32 changes) {struct v4l2_event ev; -struct v4l2_subscribed_event *sev; +struct media_subscribed_evbase *mse;if (list_empty (&ctrl->ev_subs))return ;fill_event (&ev, ctrl, changes); -list_for_each_entry (sev, &ctrl->ev_subs, node) -if (sev->fh != fh || - (sev->flags &V4L2_EVENT_SUB_FL_ALLOW_FEEDBACK )) -v4l2_event_queue_fh (sev->fh, &ev); +list_for_each_entry (mse, &ctrl->ev_subs, node) { +struct v4l2_subscribed_event *sev = +to_v4l2_subscribed_event (mse); + +if (mse->fh != &fh->ev_fh || + (sev->evspec.flags &V4L2_EVENT_SUB_FL_ALLOW_FEEDBACK )) +v4l2_event_queue_fh (media_evbase_fh_to_v4l2 (mse->fh), &ev); + } } bool v4l2_ctrl_type_op_equal(const struct v4l2_ctrl *ctrl,
@@ -1821,7 +1825,7 @@ int v4l2_ctrl_handler_free(struct v4l2_ctrl_handler *hdl) {struct v4l2_ctrl_ref *ref, *next_ref;struct v4l2_ctrl *ctrl, *next_ctrl; -struct v4l2_subscribed_event *sev, *next_sev; +struct media_subscribed_evbase *mse, *next_mse;if (!hdl)return 0 ;
@@ -1842,8 +1846,8 @@ int v4l2_ctrl_handler_free(struct v4l2_ctrl_handler *hdl)/* Free all controls owned by the handler */ list_for_each_entry_safe (ctrl, next_ctrl, &hdl->ctrls, node) {list_del (&ctrl->node); -list_for_each_entry_safe (sev, next_sev, &ctrl->ev_subs, node) -list_del (&sev->node); +list_for_each_entry_safe (mse, next_mse, &ctrl->ev_subs, node) +list_del (&mse->node);kvfree (ctrl->p_array);kvfree (ctrl); }
drivers/media/v4l2-core/v4l2-event.c +104 -36
231 lines not on this page · raw message
include/media/media-evbase.h +79 -1
93 lines not on this page · raw message
include/media/v4l2-event.h +65 -35
122 lines not on this page · raw message
include/media/v4l2-fh.h +12 -0
@@ -168,4 +168,16 @@ static inline int v4l2_fh_is_singular_file(struct file *filp) return v4l2_fh_is_singular(filp->private_data); } +/** +* media_evbase_fh_to_v4l2 - Return the struct v4l2_fh pointer from the +* media event fh +* +* @fh: pointer to &struct media_evbase_fh +*/ +static inline struct v4l2_fh * +media_evbase_fh_to_v4l2 (struct media_evbase_fh *fh) +{ + return container_of(fh, struct v4l2_fh, ev_fh); +} +#endif /* V4L2_EVENT_H */
Signature · 1 line
2.56.0
RFC PATCH v2 5/16 v4l: event: Split the event framework into generic and V4L2 bits
In reply to Mattijs KorpershoekFrom: Sakari Ailus <[email protected]>
Split the functionality of the V4L2 event framework into generic and V4L2 specific parts. This enables using the same infrastructure for implementing events for the Media controller API.
--- drivers/media/Kconfig | 2 + drivers/media/mc/Kconfig | 3 + drivers/media/mc/Makefile | 1 + drivers/media/mc/mc-evbase.c | 260 ++++++++++++++++++++++++++++++++++ drivers/media/v4l2-core/v4l2-event.c | 264 ++++++++++------------------------- include/media/media-evbase.h | 82 +++++++++++ include/media/v4l2-event.h | 6 +- 7 files changed, 425 insertions(+), 193 deletions(-)
drivers/media/Kconfig +2 -0
@@ -162,6 +162,7 @@ config VIDEO_DEV default MEDIA_CAMERA_SUPPORT || MEDIA_ANALOG_TV_SUPPORT || MEDIA_RADIO_SUPPORT || MEDIA_SDR_SUPPORT || MEDIA_PLATFORM_SUPPORT || MEDIA_TEST_SUPPORT depends on (I2C || I2C=n) select RATIONAL + select MEDIA_EVENT select VIDEOBUF2_V4L2 if VIDEOBUF2_CORE help Enables the V4L2 API, used by cameras, analog TV, video grabbers,
@@ -170,6 +171,7 @@ config VIDEO_DEV config MEDIA_CONTROLLER bool "Media Controller API" default MEDIA_CAMERA_SUPPORT || MEDIA_ANALOG_TV_SUPPORT || MEDIA_DIGITAL_TV_SUPPORT || MEDIA_PLATFORM_SUPPORT + select MEDIA_EVENT help Enable the media controller API used to query media devices internal topology and configure it dynamically.
drivers/media/mc/Kconfig +3 -0
@@ -11,3 +11,6 @@ config MEDIA_CONTROLLER_DVB Enable the media controller API support for DVB. This is currently experimental. + +config MEDIA_EVENT + tristate
drivers/media/mc/Makefile +1 -0
@@ -8,3 +8,4 @@ ifneq ($(CONFIG_USB),) endif obj-$(CONFIG_MEDIA_SUPPORT) += mc.o +obj-$(CONFIG_MEDIA_EVENT) += mc-evbase.o
drivers/media/mc/mc-evbase.c new file +260 -0
260 lines not on this page · raw message
drivers/media/v4l2-core/v4l2-event.c +72 -192
371 lines not on this page · raw message
include/media/media-evbase.h +82 -0
98 lines not on this page · raw message
include/media/v4l2-event.h +5 -1
@@ -148,7 +148,11 @@ void v4l2_event_queue(struct video_device *vdev, const struct v4l2_event *ev);* The driver's only responsibility is to fill in the type and the data * fields. The other fields will be filled in by V4L2. */ -void v4l2_event_queue_fh (struct v4l2_fh *fh,const struct v4l2_event *ev); +static inline void v4l2_event_queue_fh (struct v4l2_fh *fh, +const struct v4l2_event *ev) +{ +media_evbase_queue_fh (&fh->ev_fh, ev,ktime_get_ns ()); +}/** * v4l2_event_wake_all - Wake all filehandles.
Signature · 1 line
2.56.0
RFC PATCH v2 6/16 media: mc: Maintain a list of open file handles in a media device
In reply to Mattijs KorpershoekFrom: Sakari Ailus <[email protected]>
The list of file handles is needed to deliver media events in the future.
--- drivers/media/mc/mc-device.c | 35 +++++++++++++++++++++++++---------- include/media/media-device.h | 7 +++++++ include/media/media-fh.h | 32 ++++++++++++++++++++++++++++++++ 3 files changed, 64 insertions(+), 10 deletions(-)
drivers/media/mc/mc-device.c +25 -10
@@ -22,6 +22,7 @@#include <media/media-device.h> #include <media/media-devnode.h> #include <media/media-entity.h> +#include <media/media-fh.h> #include <media/media-request.h> #ifdef CONFIG_MEDIA_CONTROLLER
@@ -35,16 +36,6 @@#define MEDIA_ENT_SUBTYPE_MASK 0x0000ffff#define MEDIA_ENT_T_DEVNODE_UNKNOWN (MEDIA_ENT_F_OLD_BASE | \ MEDIA_ENT_SUBTYPE_MASK) - -struct media_device_fh { -struct media_devnode_fh fh; -}; - -static inline struct media_device_fh *media_device_fh (struct file *filp) -{ - return container_of(filp->private_data, struct media_device_fh, fh); -} -/* ----------------------------------------------------------------------------- * Userspace API */
@@ -56,7 +47,10 @@ static inline void __user *media_get_uptr(__u64 arg)static int media_device_open (struct file *filp) { +struct media_devnode *devnode = filp->private_data; +struct media_device *mdev = devnode->media_dev;struct media_device_fh *fh; +unsigned long flags; fh =kzalloc_obj (*fh);if (!fh)
@@ -64,12 +58,25 @@ static int media_device_open(struct file *filp) filp->private_data = &fh->fh; +spin_lock_irqsave (&mdev->fh_list_lock, flags); +list_add (&fh->mdev_list, &mdev->fh_list); +spin_unlock_irqrestore (&mdev->fh_list_lock, flags); +return 0 ; }static int media_device_close (struct file *filp) { +struct media_devnode *devnode =media_devnode_data (filp); +struct media_device *mdev = devnode->media_dev;struct media_device_fh *fh =media_device_fh (filp); +unsigned long flags; + +if (mdev) { +spin_lock_irqsave (&mdev->fh_list_lock, flags); +list_del (&fh->mdev_list); +spin_unlock_irqrestore (&mdev->fh_list_lock, flags); + }kfree (fh);
@@ -732,12 +739,14 @@ void media_device_init(struct media_device *mdev)INIT_LIST_HEAD (&mdev->pads);INIT_LIST_HEAD (&mdev->links);INIT_LIST_HEAD (&mdev->entity_notify); +INIT_LIST_HEAD (&mdev->fh_list);mutex_init (&mdev->req_queue_mutex);mutex_init (&mdev->graph_mutex);ida_init (&mdev->entity_internal_idx);atomic_set (&mdev->request_id,0 ); +spin_lock_init (&mdev->fh_list_lock);if (!*mdev->bus_info) media_set_bus_info(mdev->bus_info, sizeof(mdev->bus_info),
@@ -843,6 +852,7 @@ void media_device_unregister(struct media_device *mdev)struct media_entity *next;struct media_interface *intf, *tmp_intf;struct media_entity_notify *notify, *nextp; +unsigned long flags;if (mdev ==NULL )return ;
@@ -883,6 +893,11 @@ void media_device_unregister(struct media_device *mdev)dev_dbg (mdev->dev,"Media device unregistered\n" );debugfs_remove_recursive (mdev->media_dir); + +spin_lock_irqsave (&mdev->fh_list_lock, flags); +list_del_init (&mdev->fh_list); +spin_unlock_irqrestore (&mdev->fh_list_lock, flags); +device_remove_file (&mdev->devnode->dev, &dev_attr_model);media_devnode_unregister (mdev->devnode);/* devnode free is handled in media_devnode_*() */
include/media/media-device.h +7 -0
@@ -112,6 +112,10 @@ struct media_device_ops {* @media_dir: DebugFS media directory * @request_id: Used to generate unique request IDs * +* @fh_list: List of file handles in the media device +* (struct media_device_fh.mdev_list). +* @fh_list_lock: Serialise access to fh_list list. +* * This structure represents an abstract high-level media device. It allows easy * access to entities and provides basic media device-level support. The * structure can be allocated directly or embedded in a larger structure.
@@ -189,6 +193,9 @@ struct media_device {/* debugfs */ struct dentry *media_dir;atomic_t request_id; + +struct list_head fh_list; +spinlock_t fh_list_lock; };/* We don't need to include usb.h here */
include/media/media-fh.h new file +32 -0
@@ -0,0 +1,32 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* +* Media device file handle +* +* Copyright (C) 2019 Intel Corporation +*/ + +#ifndef MEDIA_FH_H +#define MEDIA_FH_H + +#include <linux/list.h> +#include <linux/file.h> + +#include <media/media-devnode.h> + +/** +* struct media_device_fh - File handle specific information on MC +* +* @fh: The media device file handle +* @mdev_list: This file handle in media device's list of file handles +*/ +struct media_device_fh { +struct media_devnode_fh fh; +struct list_head mdev_list; +}; + +static inline struct media_device_fh *media_device_fh (struct file *filp) +{ + return container_of(filp->private_data, struct media_device_fh, fh); +} + +#endif /* MEDIA_FH_H */
Signature · 1 line
2.56.0
RFC PATCH v2 7/16 media: mc: Add MEDIA_IOC_DQEVENT
In reply to Mattijs KorpershoekFrom: Sakari Ailus <[email protected]>
Events on a media device tell about completion of requests. Blocking and non-blocking operation is supported.
--- drivers/media/Kconfig | 1 + drivers/media/mc/Makefile | 2 +- drivers/media/mc/mc-device.c | 48 +++++++++ drivers/media/mc/mc-event.c | 244 +++++++++++++++++++++++++++++++++++++++++++ include/media/media-device.h | 3 + include/media/media-event.h | 181 ++++++++++++++++++++++++++++++++ include/media/media-fh.h | 5 + include/uapi/linux/media.h | 24 +++++ 8 files changed, 507 insertions(+), 1 deletion(-)
drivers/media/Kconfig +1 -0
@@ -12,6 +12,7 @@ source "drivers/media/cec/Kconfig" menuconfig MEDIA_SUPPORT tristate "Multimedia support" depends on HAS_IOMEM + select MEDIA_EVENT help If you want to use media devices, including Webcams, Video grabber devices and/or TV devices, V4L2 codecs, etc, enable this option
drivers/media/mc/Makefile +1 -1
@@ -1,7 +1,7 @@ # SPDX-License-Identifier: GPL-2.0 mc-objs := mc-device.o mc-devnode.o mc-entity.o \ - mc-request.o + mc-request.o mc-event.o ifneq ($(CONFIG_USB),) mc-objs += mc-dev-allocator.o
drivers/media/mc/mc-device.c +48 -0
93 lines not on this page · raw message
drivers/media/mc/mc-event.c new file +244 -0
244 lines not on this page · raw message
include/media/media-device.h +3 -0
@@ -19,10 +19,13 @@#include <media/media-devnode.h> #include <media/media-entity.h> +#include <media/media-evbase.h> struct ida ;struct media_device ; +#include <uapi/linux/media.h> +/** * struct media_entity_notify - Media Entity Notify *
include/media/media-event.h new file +181 -0
181 lines not on this page · raw message
include/media/media-fh.h +5 -0
@@ -12,16 +12,21 @@#include <linux/file.h> #include <media/media-devnode.h> +#include <media/media-evbase.h> /** * struct media_device_fh - File handle specific information on MC * * @fh: The media device file handle * @mdev_list: This file handle in media device's list of file handles +* @ev_fh: File handle specific media event information +* @wait: File handle's wait queue */ struct media_device_fh {struct media_devnode_fh fh;struct list_head mdev_list; +struct media_evbase_fh ev_fh; +struct wait_queue_head wait; };static inline struct media_device_fh *media_device_fh (struct file *filp)
include/uapi/linux/media.h +24 -0
@@ -373,6 +373,27 @@ struct media_v2_topology {__u64 ptr_links; } __attribute__ ((packed)); +#define MEDIA_EVENT_ALL 0 + +struct media_event { +__u32 type; +union { +__u8 data[64 ]; + } u; +__u32 reserved[8 ]; +__u32 pending; +__u32 sequence; +__u64 timestamp; +__u32 id; +} __attribute__((packed)); + +struct media_event_subscription { +__u32 type; +__u32 id; +__u32 flags; +__u32 reserved[5 ]; +} __attribute__((packed)); +/* ioctls */ #define MEDIA_IOC_DEVICE_INFO _IOWR('|', 0x00, struct media_device_info)
@@ -381,6 +402,9 @@ struct media_v2_topology {#define MEDIA_IOC_SETUP_LINK _IOWR('|', 0x03, struct media_link_desc)#define MEDIA_IOC_G_TOPOLOGY _IOWR('|', 0x04, struct media_v2_topology)#define MEDIA_IOC_REQUEST_ALLOC _IOR ('|', 0x05, int) +#define MEDIA_IOC_SUBSCRIBE_EVENT _IOWR ('|', 0x06, struct media_event_subscription) +#define MEDIA_IOC_UNSUBSCRIBE_EVENT _IOWR ('|', 0x07, struct media_event_subscription) +#define MEDIA_IOC_DQEVENT _IOWR('|', 0x08, struct media_event)/* * These ioctls are called on the request file descriptor as returned
Signature · 1 line
2.56.0
RFC PATCH v2 8/16 media: mc: Add MEDIA_EVENT_ENTITY_REGISTRATION
In reply to Mattijs KorpershoekFrom: Sakari Ailus <[email protected]>
Add a media event to tell a new media entity has been registered.
--- drivers/media/mc/mc-device.c | 5 +++++ include/uapi/linux/media.h | 9 ++++++++- 2 files changed, 13 insertions(+), 1 deletion(-)
drivers/media/mc/mc-device.c +5 -0
@@ -746,6 +746,11 @@ int __must_check media_device_register_entity(struct media_device *mdev, }mutex_unlock (&mdev->graph_mutex); +media_event_queue (mdev, &(struct media_event ){ + .type =MEDIA_EVENT_ENTITY_REGISTRATION , + .u.entity_reg.entity_id = entity->graph_obj.id, + }); +return 0 ; }EXPORT_SYMBOL_GPL (media_device_register_entity);
include/uapi/linux/media.h +8 -1
@@ -373,11 +373,18 @@ struct media_v2_topology {__u64 ptr_links; } __attribute__ ((packed)); -#define MEDIA_EVENT_ALL 0 +#define MEDIA_EVENT_ALL 0 +#define MEDIA_EVENT_ENTITY_REGISTRATION 1 + +struct media_event_entity_reg { +__u32 entity_id; +__u32 pending; +} __attribute__ ((packed));struct media_event {__u32 type;union { +struct media_event_entity_reg entity_reg;__u8 data[64 ]; } u;__u32 reserved[8 ];
Signature · 1 line
2.56.0
RFC PATCH v2 9/16 Documentation: media: Add Media controller event documentation
In reply to Mattijs KorpershoekFrom: Sakari Ailus <[email protected]>
Add documentation for the Media controller event IOCTLs as well as an event, MEDIA_EVENT_ENTITY_REGISTRATION.
--- .../userspace-api/media/mediactl/media-funcs.rst | 2 + .../media/mediactl/media-ioc-dqevent.rst | 125 +++++++++++++++++++++ .../media/mediactl/media-ioc-subscribe-event.rst | 112 ++++++++++++++++++ 3 files changed, 239 insertions(+)
Documentation/userspace-api/media/mediactl/media-funcs.rst +2 -0
@@ -19,6 +19,8 @@ Function Reference media-ioc-enum-links media-ioc-setup-link media-ioc-request-alloc + media-ioc-subscribe-event + media-ioc-dqevent request-func-close request-func-ioctl request-func-poll
Documentation/userspace-api/media/mediactl/media-ioc-dqevent.rst new file +125 -0
125 lines not on this page · raw message
Documentation/userspace-api/media/mediactl/media-ioc-subscribe-event.rst new file +112 -0
112 lines not on this page · raw message
Signature · 1 line
2.56.0
RFC PATCH v2 10/16 media: media.h: Add entity connection status
In reply to Mattijs KorpershoekSome elements in the media graph (such as camera sensors) can be physically disconnected/reconnected. Drivers may still probe in that case to have a complete media graph.
Add a new entity field, named status, which reflects the connection status of that particular entity.
This field is retrieved by userspace via the MEDIA_IOC_G_TOPOLOGY and can be updated from drivers via media_entity_set_status().
When a status is not initialized, it's set to zero (STATUS_CONNECTED). This matches older media graphs were all entities are always connected.
--- drivers/media/mc/mc-device.c | 1 + drivers/media/mc/mc-entity.c | 29 +++++++++++++++++++++++++++++ include/media/media-entity.h | 20 ++++++++++++++++++++ include/uapi/linux/media.h | 6 +++++- 4 files changed, 55 insertions(+), 1 deletion(-)
drivers/media/mc/mc-device.c +1 -0
@@ -307,6 +307,7 @@ static long media_device_get_topology(struct media_device *mdev, kentity.id = entity->graph_obj.id; kentity.function = entity->function; kentity.flags = entity->flags; + kentity.status = entity->status;strscpy (kentity.name, entity->name,sizeof (kentity.name));
drivers/media/mc/mc-entity.c +29 -0
@@ -1530,6 +1530,35 @@ struct media_pipeline *media_entity_pipeline(struct media_entity *entity) }EXPORT_SYMBOL_GPL (media_entity_pipeline); +void __media_entity_set_status (struct media_entity *entity,u32 status) +{ +struct media_device *mdev = entity->graph_obj.mdev; + +/* Do nothing if the entity is not registered. */ +if (mdev ==NULL ) +return ; + +lockdep_assert_held (&mdev->graph_mutex); + +if (entity->status != status) + entity->status = status; +} +EXPORT_SYMBOL_GPL (__media_entity_set_status); + +void media_entity_set_status (struct media_entity *entity,u32 status) +{ +struct media_device *mdev = entity->graph_obj.mdev; + +/* Do nothing if the entity is not registered. */ +if (mdev ==NULL ) +return ; + +mutex_lock (&mdev->graph_mutex); +__media_entity_set_status (entity, status); +mutex_unlock (&mdev->graph_mutex); +} +EXPORT_SYMBOL_GPL (media_entity_set_status); +struct media_pipeline *media_pad_pipeline (struct media_pad *pad) {return pad->pipe;
include/media/media-entity.h +20 -0
@@ -325,6 +325,10 @@ enum media_entity_type {* @flags: Entity flags, as defined in * :ref:`include/uapi/linux/media.h <media_header>` * (seek for ``MEDIA_ENT_FL_*``) +* @status: Entity connection status, as defined in +* :ref:`include/uapi/linux/media.h <media_header>` +* (seek for ``MEDIA_ENT_STATUS_*``). +* Access with media_entity_set_status() * @num_pads: Number of sink and source pads. * @num_links: Total number of links, forward and back, enabled and disabled. * @num_backlinks: Number of backlinks
@@ -352,6 +356,7 @@ struct media_entity {enum media_entity_type obj_type;u32 function;unsigned long flags; +u32 status;u16 num_pads;u16 num_links;
@@ -1022,6 +1027,21 @@ static inline bool media_entity_is_streaming(const struct media_entity *entity)return false; } +void __media_entity_set_status (struct media_entity *entity,u32 status); + +/** +* media_entity_set_status - Set the connection status of an entity +* @entity: The entity +* @status: The new status (%MEDIA_ENT_STATUS_CONNECTED or +* %MEDIA_ENT_STATUS_DISCONNECTED) +* +* Update the entity's connection status. +* +* The entity must be registered with a media device. To set the initial +* status before registration, assign entity->status directly. +*/ +void media_entity_set_status (struct media_entity *entity,u32 status); +/** * media_entity_pipeline - Get the media pipeline an entity is part of * @entity: The entity
include/uapi/linux/media.h +5 -1
@@ -302,12 +302,16 @@ struct media_links_enum {#define MEDIA_V2_ENTITY_HAS_FLAGS (media_version) \ ((media_version) >= ((4U << 16) | (19U << 8) | 0U)) +#define MEDIA_ENT_STATUS_CONNECTED 0 +#define MEDIA_ENT_STATUS_DISCONNECTED 1 +struct media_v2_entity {__u32 id;char name[64 ];__u32 function;/* Main function of the entity */ __u32 flags; -__u32 reserved[5 ]; +__u32 status; +__u32 reserved[4 ]; } __attribute__ ((packed));/* Should match the specific fields at media_intf_devnode */
Signature · 1 line
2.56.0
RFC PATCH v2 11/16 media: mc: Add .detect() to media_entity_operations
In reply to Mattijs KorpershoekWhen mc.fault_tolerance = true, media drivers can probe without the underlying hardware being physically connected.
In that case, The media controller core needs to be able to retrieve the driver state in order to update the topology.
Add a new, optional, media_entity_operation named .detect().
To support fault tolerance, drivers must be modified and do the following when media_fault_tolerance_enabled() = true:
* Probe must succeed even when the underlying hardware is not present
* Probe must set initial entity connections status if disconnected
* The .detect() operation must be implemented
* .enable_streams() must query hardware for presence and must fail when it's not detected. Can be done using the media_entity_detect_or_fail() helper.
Note: this is based on the .detect() helpers from DRM connectors.
--- drivers/media/mc/mc-device.c | 11 +++++++++++ drivers/media/mc/mc-entity.c | 22 ++++++++++++++++++++++ include/media/media-device.h | 10 ++++++++++ include/media/media-entity.h | 24 ++++++++++++++++++++++++ 4 files changed, 67 insertions(+)
drivers/media/mc/mc-device.c +11 -0
@@ -14,6 +14,7 @@#include <linux/idr.h> #include <linux/ioctl.h> #include <linux/media.h> +#include <linux/module.h> #include <linux/slab.h> #include <linux/types.h> #include <linux/pci.h>
@@ -27,6 +28,16 @@#include <media/media-fh.h> #include <media/media-request.h> +static bool fault_tolerance; +module_param (fault_tolerance, bool,0444 ); +MODULE_PARM_DESC (fault_tolerance,"Enable fault-tolerant support" ); + +bool media_fault_tolerance_enabled (void ) +{ +return fault_tolerance; +} +EXPORT_SYMBOL_GPL (media_fault_tolerance_enabled); +#ifdef CONFIG_MEDIA_CONTROLLER /*
drivers/media/mc/mc-entity.c +22 -0
@@ -1559,6 +1559,28 @@ void media_entity_set_status(struct media_entity *entity, u32 status) }EXPORT_SYMBOL_GPL (media_entity_set_status); +int media_entity_detect_or_fail (struct media_entity *entity) +{ +int ret; + +if (!media_fault_tolerance_enabled ()) +return 0 ; + + ret =media_entity_call (entity, detect); +if (ret == -ENOIOCTLCMD ) +return 0 ; +if (ret <0 ) +return ret; + +media_entity_set_status (entity, ret); + +if (ret ==MEDIA_ENT_STATUS_DISCONNECTED ) +return -ENOLINK ; + +return 0 ; +} +EXPORT_SYMBOL_GPL (media_entity_detect_or_fail); +struct media_pipeline *media_pad_pipeline (struct media_pad *pad) {return pad->pipe;
include/media/media-device.h +10 -0
@@ -204,6 +204,16 @@ struct media_device {/* We don't need to include usb.h here */ struct usb_device ; +/** +* media_fault_tolerance_enabled - Check if media fault tolerance is enabled +* +* Returns true when the mc.fault_tolerance module parameter is set. +* Drivers should call this to decide whether to enable fault-tolerant +* behavior (e.g. probing without the underlying hardware being +* physically connected). +*/ +bool media_fault_tolerance_enabled (void ); +#ifdef CONFIG_MEDIA_CONTROLLER /* Supported link_notify @notification values. */
include/media/media-entity.h +24 -0
@@ -269,6 +269,12 @@ struct media_pad {* media_entity_has_pad_interdep(). * Optional: If the operation isn't implemented all pads * will be considered as interdependent. +* @detect: Query whether the entity's hardware is physically +* connected and reachable. Must not modify entity state. +* Returns a %MEDIA_ENT_STATUS_* value on success, or +* a negative error code on failure. +* Optional: If the operation isn't implemented the +* entity will be considered as connected * * .. note:: *
@@ -284,6 +290,7 @@ struct media_entity_operations {int (*link_validate)(struct media_link *link);bool (*has_pad_interdep)(struct media_entity *entity,unsigned int pad0,unsigned int pad1); +int (*detect)(struct media_entity *entity); };/**
@@ -1042,6 +1049,23 @@ void __media_entity_set_status(struct media_entity *entity, u32 status);*/ void media_entity_set_status (struct media_entity *entity,u32 status); +/** +* media_entity_detect_or_fail - Detect entity hardware and fail if disconnected +* @entity: The entity +* +* Call the entity's &media_entity_operations.detect() operation to probe +* the hardware. If the entity is disconnected, the entity status is updated +* and a %MEDIA_EVENT_ENTITY_STATUS_CHANGE event is queued. +* +* Intended to be called by drivers at the beginning of their +* &v4l2_subdev_pad_ops.enable_streams() callback +* +* Return: 0 if the entity is connected or does not implement .detect(), +* -ENOLINK if the entity is disconnected, or a negative error code on +* failure. +*/ +int media_entity_detect_or_fail (struct media_entity *entity); +/** * media_entity_pipeline - Get the media pipeline an entity is part of * @entity: The entity
Signature · 1 line
2.56.0
RFC PATCH v2 12/16 media: mc: Add MEDIA_IOC_G_ENTITY_STATUS ioctl
In reply to Mattijs KorpershoekUserspace can get an entity's connection status using 'media-ctl -p'. This calls MEDIA_IOC_G_TOPOLOGY which will also provide the entity status.
However, that status is from the cache. When fault tolerance is enabled, the status can be only be updated in 2 ways:
1. During driver probe
2. When a stream is started
Add MEDIA_IOC_G_ENTITY_STATUS which allows userspace to get the hardware state at any given time.
This can be useful for diagnostics, for example.
Calling MEDIA_IOC_G_ENTITY_STATUS will also update the topology's cached value.
--- .../userspace-api/media/mediactl/media-funcs.rst | 1 + .../media/mediactl/media-ioc-g-entity-status.rst | 92 ++++++++++++++++++++++ drivers/media/mc/mc-device.c | 23 ++++++ include/uapi/linux/media.h | 7 ++ 4 files changed, 123 insertions(+)
Documentation/userspace-api/media/mediactl/media-funcs.rst +1 -0
@@ -21,6 +21,7 @@ Function Reference media-ioc-request-alloc media-ioc-subscribe-event media-ioc-dqevent + media-ioc-g-entity-status request-func-close request-func-ioctl request-func-poll
Documentation/userspace-api/media/mediactl/media-ioc-g-entity-status.rst new file +92 -0
92 lines not on this page · raw message
drivers/media/mc/mc-device.c +23 -0
@@ -501,6 +501,28 @@ struct media_ioctl_info {long (*arg_to_user)(void __user *uarg,void *karg,unsigned int cmd); }; +static long media_device_get_entity_status (struct media_device *mdev, +struct file *file,void *arg) +{ +struct media_entity_status *estatus = arg; +struct media_entity *entity; +int ret; + + entity =find_entity (mdev, estatus->id); +if (!entity) +return -EINVAL ; + + ret =media_entity_call (entity, detect); +if (ret >=0 ) +__media_entity_set_status (entity, ret); + +memset (estatus->reserved,0 ,sizeof (estatus->reserved)); + estatus->id =media_entity_id (entity); + estatus->status = entity->status; + +return 0 ; +} +static const struct media_ioctl_info ioctl_info[] = {MEDIA_IOC (DEVICE_INFO , media_device_get_info,MEDIA_IOC_FL_GRAPH_MUTEX ),MEDIA_IOC (ENUM_ENTITIES , media_device_enum_entities,MEDIA_IOC_FL_GRAPH_MUTEX ),
@@ -511,6 +533,7 @@ static const struct media_ioctl_info ioctl_info[] = {MEDIA_IOC (DQEVENT , media_device_event_dequeue,0 ),MEDIA_IOC (SUBSCRIBE_EVENT , media_device_event_subscribe,0 ),MEDIA_IOC (UNSUBSCRIBE_EVENT , media_device_event_unsubscribe,0 ), + MEDIA_IOC(G_ENTITY_STATUS, media_device_get_entity_status, MEDIA_IOC_FL_GRAPH_MUTEX), }; static long media_device_ioctl(struct file *filp, unsigned int cmd,
include/uapi/linux/media.h +7 -0
@@ -377,6 +377,12 @@ struct media_v2_topology {__u64 ptr_links; } __attribute__ ((packed)); +struct media_entity_status { +__u32 id; +__u32 status; +__u32 reserved[6 ]; +} __attribute__ ((packed)); +#define MEDIA_EVENT_ALL 0#define MEDIA_EVENT_ENTITY_REGISTRATION 1
@@ -416,6 +422,7 @@ struct media_event_subscription {#define MEDIA_IOC_SUBSCRIBE_EVENT _IOWR ('|', 0x06, struct media_event_subscription)#define MEDIA_IOC_UNSUBSCRIBE_EVENT _IOWR ('|', 0x07, struct media_event_subscription)#define MEDIA_IOC_DQEVENT _IOWR('|', 0x08, struct media_event) +#define MEDIA_IOC_G_ENTITY_STATUS _IOWR('|', 0x09, struct media_entity_status)/* * These ioctls are called on the request file descriptor as returned
Signature · 1 line
2.56.0
RFC PATCH v2 13/16 media: mc: Add MEDIA_EVENT_ENTITY_STATUS_CHANGE event
In reply to Mattijs KorpershoekAdd a media event that notifies userspace when an entity's connection state changes. This allows userspace to react when a sensor gets disconnected. For example, by aborting an ongoing stream.
--- .../userspace-api/media/mediactl/media-ioc-dqevent.rst | 5 +++++ drivers/media/mc/mc-entity.c | 10 +++++++++- include/media/media-entity.h | 3 ++- include/uapi/linux/media.h | 7 +++++++ 4 files changed, 23 insertions(+), 2 deletions(-)
Documentation/userspace-api/media/mediactl/media-ioc-dqevent.rst +5 -0
@@ -112,6 +112,11 @@ subscribe the events. known to be still missing. This number may be less than the total number of all missing entities. However if this number is zero, then all entities have been registered to the media device. + * - ``MEDIA_EVENT_ENTITY_STATUS_CHANGE`` + - 2 + - Triggered when a media entity connection status changes. The entity + id is in ``entity_id`` and the status is encoded in ``status``. + See :ref:`MEDIA_IOC_G_ENTITY_STATUS` for details about entity status. Return Value ============
drivers/media/mc/mc-entity.c +9 -1
@@ -14,6 +14,7 @@#include <linux/slab.h> #include <media/media-entity.h> #include <media/media-device.h> +#include <media/media-event.h> static inline const char *intf_type (struct media_interface *intf) {
@@ -1540,8 +1541,15 @@ void __media_entity_set_status(struct media_entity *entity, u32 status)lockdep_assert_held (&mdev->graph_mutex); -if (entity->status != status) +if (entity->status != status) { entity->status = status; + +media_event_queue (mdev, &(struct media_event ){ + .type =MEDIA_EVENT_ENTITY_STATUS_CHANGE , + .u.entity_status.entity_id = entity->graph_obj.id, + .u.entity_status.status = status, + }); + } }EXPORT_SYMBOL_GPL (__media_entity_set_status);
include/media/media-entity.h +2 -1
@@ -1042,7 +1042,8 @@ void __media_entity_set_status(struct media_entity *entity, u32 status);* @status: The new status (%MEDIA_ENT_STATUS_CONNECTED or * %MEDIA_ENT_STATUS_DISCONNECTED) * -* Update the entity's connection status. +* Update the entity's connection status. If the status has changed, a +* %MEDIA_EVENT_ENTITY_STATUS_CHANGE event is queued to the media device. * * The entity must be registered with a media device. To set the initial * status before registration, assign entity->status directly.
include/uapi/linux/media.h +7 -0
@@ -385,16 +385,23 @@ struct media_entity_status {#define MEDIA_EVENT_ALL 0#define MEDIA_EVENT_ENTITY_REGISTRATION 1 +#define MEDIA_EVENT_ENTITY_STATUS_CHANGE 2struct media_event_entity_reg {__u32 entity_id;__u32 pending; } __attribute__ ((packed)); +struct media_event_entity_status { +__u32 entity_id; +__u32 status; +} __attribute__ ((packed)); +struct media_event {__u32 type;union {struct media_event_entity_reg entity_reg; +struct media_event_entity_status entity_status;__u8 data[64 ]; } u;__u32 reserved[8 ];
Signature · 1 line
2.56.0
RFC PATCH v2 14/16 media: mc: Add detect polling helpers for drivers
In reply to Mattijs KorpershoekIf a stream is ongoing and an entity gets disconnected from the media graph, userspace will hang on DQBUF without any additional information.
When fault tolerance is enabled, the media controller can notify userspace of such happening by periodically polling all entities that support the .detect() operation.
If an entity's status change, userspace gets notified via the MEDIA_EVENT_ENTITY_STATUS_CHANGE event.
Add media_detect_poll_enable/disable so that drivers can opt-in into polling for .detect().
--- drivers/media/mc/mc-device.c | 80 ++++++++++++++++++++++++++++++++++++++++++++ include/media/media-device.h | 4 +++ include/media/media-entity.h | 32 ++++++++++++++++++ 3 files changed, 116 insertions(+)
drivers/media/mc/mc-device.c +80 -0
@@ -20,6 +20,7 @@#include <linux/pci.h> #include <linux/usb.h> #include <linux/version.h> +#include <linux/workqueue.h> #include <media/media-device.h> #include <media/media-devnode.h>
@@ -700,6 +701,9 @@ static void __media_device_unregister_entity(struct media_entity *entity)ida_free (&mdev->entity_internal_idx, entity->internal_idx); +if (!list_empty (&entity->detect_list)) +list_del_init (&entity->detect_list); +/* Remove all interface links pointing to this entity */ list_for_each_entry (intf, &mdev->interfaces, graph_obj.list) {list_for_each_entry_safe (link, tmp, &intf->links, list) {
@@ -738,6 +742,7 @@ int __must_check media_device_register_entity(struct media_device *mdev,WARN_ON (entity->graph_obj.mdev !=NULL ); entity->graph_obj.mdev = mdev;INIT_LIST_HEAD (&entity->links); +INIT_LIST_HEAD (&entity->detect_list); entity->num_links =0 ; entity->num_backlinks =0 ;
@@ -820,6 +825,76 @@ static int media_device_requests(struct seq_file *file, void *priv) }#endif +#define MEDIA_DETECT_POLL_INTERVAL_MS 10000 + +static void media_detect_work_fn (struct work_struct *work) +{ +struct media_device *mdev =container_of (to_delayed_work (work), + struct media_device, + detect_work); +struct media_entity *entity; +int ret; + +if (!mutex_trylock (&mdev->graph_mutex)) { +schedule_delayed_work (&mdev->detect_work, +msecs_to_jiffies (MEDIA_DETECT_POLL_INTERVAL_MS )); +return ; + } + +list_for_each_entry (entity, &mdev->detect_entities, detect_list) { + ret =media_entity_call (entity, detect); +if (ret >=0 ) +__media_entity_set_status (entity, ret); + } + +if (!list_empty (&mdev->detect_entities)) +schedule_delayed_work (&mdev->detect_work, +msecs_to_jiffies (MEDIA_DETECT_POLL_INTERVAL_MS )); + +mutex_unlock (&mdev->graph_mutex); +} + +void media_detect_poll_enable (struct media_entity *entity) +{ +struct media_device *mdev = entity->graph_obj.mdev; + +if (!media_fault_tolerance_enabled ()) +return ; + +mutex_lock (&mdev->graph_mutex); + +if (!list_empty (&entity->detect_list)) + goto out; + +list_add_tail (&entity->detect_list, &mdev->detect_entities); +schedule_delayed_work (&mdev->detect_work, +msecs_to_jiffies (MEDIA_DETECT_POLL_INTERVAL_MS )); +out: +mutex_unlock (&mdev->graph_mutex); +} +EXPORT_SYMBOL_GPL (media_detect_poll_enable); + +void media_detect_poll_disable (struct media_entity *entity) +{ +struct media_device *mdev = entity->graph_obj.mdev; + +if (!media_fault_tolerance_enabled ()) +return ; + +mutex_lock (&mdev->graph_mutex); + +if (list_empty (&entity->detect_list)) + goto out; + +list_del_init (&entity->detect_list); + +if (list_empty (&mdev->detect_entities)) +cancel_delayed_work (&mdev->detect_work); +out: +mutex_unlock (&mdev->graph_mutex); +} +EXPORT_SYMBOL_GPL (media_detect_poll_disable); +void media_device_init (struct media_device *mdev) {INIT_LIST_HEAD (&mdev->entities);
@@ -828,6 +903,8 @@ void media_device_init(struct media_device *mdev)INIT_LIST_HEAD (&mdev->links);INIT_LIST_HEAD (&mdev->entity_notify);INIT_LIST_HEAD (&mdev->fh_list); +INIT_LIST_HEAD (&mdev->detect_entities); +INIT_DELAYED_WORK (&mdev->detect_work, media_detect_work_fn);mutex_init (&mdev->req_queue_mutex);mutex_init (&mdev->graph_mutex);
@@ -849,6 +926,7 @@ EXPORT_SYMBOL_GPL(media_device_init);void media_device_cleanup (struct media_device *mdev) { +cancel_delayed_work_sync (&mdev->detect_work);ida_destroy (&mdev->entity_internal_idx); mdev->entity_internal_idx_max =0 ;media_graph_walk_cleanup (&mdev->pm_count_walk);
@@ -945,6 +1023,8 @@ void media_device_unregister(struct media_device *mdev)if (mdev ==NULL )return ; +cancel_delayed_work_sync (&mdev->detect_work); +mutex_lock (&mdev->graph_mutex);/* Check if mdev was ever registered at all */
include/media/media-device.h +4 -0
@@ -16,6 +16,7 @@#include <linux/mutex.h> #include <linux/pci.h> #include <linux/platform_device.h> +#include <linux/workqueue.h> #include <media/media-devnode.h> #include <media/media-entity.h>
@@ -199,6 +200,9 @@ struct media_device {struct list_head fh_list;spinlock_t fh_list_lock; + +struct list_head detect_entities; +struct delayed_work detect_work; };/* We don't need to include usb.h here */
include/media/media-entity.h +32 -0
@@ -343,6 +343,9 @@ enum media_entity_type {* re-used if entities are unregistered or registered again. * @pads: Pads array with the size defined by @num_pads. * @links: List of data links. +* @detect_list: Entry in &media_device.detect_entities for poll-based +* detection. Managed by media_detect_poll_enable() / +* media_detect_poll_disable(). * @ops: Entity operations. * @use_count: Use count for the entity. * @info: Union with devnode information. Kept just for backward
@@ -372,6 +375,7 @@ struct media_entity {struct media_pad *pads;struct list_head links; +struct list_head detect_list;const struct media_entity_operations *ops;
@@ -1067,6 +1071,34 @@ void media_entity_set_status(struct media_entity *entity, u32 status);*/ int media_entity_detect_or_fail (struct media_entity *entity); +/** +* media_detect_poll_enable - Enable periodic .detect() polling for an entity +* @entity: The entity to poll +* +* Add @entity to the media device's detect poll list and schedule the +* detect worker if not already running. The worker periodically calls +* the entity's &media_entity_operations.detect() operation and updates +* the entity status on change. +* +* Intended to be called from the driver's enable_streams() callback, +* so that detection polling only runs while the entity is powered and +* streaming. +* +* The entity must implement the .detect() operation. +*/ +void media_detect_poll_enable (struct media_entity *entity); + +/** +* media_detect_poll_disable - Disable periodic .detect() polling for an entity +* @entity: The entity to stop polling +* +* Remove @entity from the media device's detect poll list. If no more +* entities are being polled, the detect worker is cancelled. +* +* Intended to be called from the driver's disable_streams() callback. +*/ +void media_detect_poll_disable (struct media_entity *entity); +/** * media_entity_pipeline - Get the media pipeline an entity is part of * @entity: The entity
Signature · 1 line
2.56.0
RFC PATCH v2 15/16 media: imx219: Allow driver probe with missing sensor
In reply to Mattijs KorpershoekWhen media fault tolerance is enabled, probe() should complete even when a sensor is disconnected.
Currently, the driver reads the CHIP_ID over i2c in the probe(). When we can't read CHIP_ID, the probe errors out, which result in the v4l2-subdev not being created.
Extract imx219_read_chip_id() from imx219_identify_module() so that probe can distinguish an unreachable sensor (I2C read failure) from a wrong sensor (CHIP_ID mismatch).
When the sensor is unreachable, skip the LP-11 state initialization and continue probe so the v4l2-subdev and media entity are created.
Note that a CHIP_ID mismatch remains a fatal error.
--- drivers/media/i2c/imx219.c | 74 ++++++++++++++++++++++------------------------ 1 file changed, 36 insertions(+), 38 deletions(-)
drivers/media/i2c/imx219.c +36 -38
@@ -1099,25 +1099,9 @@ static int imx219_get_regulators(struct imx219 *imx219) imx219->supplies); } -/* Verify chip ID */ -static int imx219_identify_module (struct imx219 *imx219) +static int imx219_read_chip_id (struct imx219 *imx219,u64 *chip_id) { -struct i2c_client *client =v4l2_get_subdevdata (&imx219->sd); -int ret; -u64 val; - - ret =cci_read (imx219->regmap,IMX219_REG_CHIP_ID , &val,NULL ); -if (ret) -return dev_err_probe (&client->dev, ret, -"failed to read chip id %x\n" , -IMX219_CHIP_ID ); - -if (val !=IMX219_CHIP_ID ) -return dev_err_probe (&client->dev, -EIO , -"chip id mismatch: %x!=%llx\n" , -IMX219_CHIP_ID , val); - -return 0 ; +return cci_read (imx219->regmap,IMX219_REG_CHIP_ID , chip_id,NULL ); }static int imx219_check_hwcfg (struct device *dev,struct imx219 *imx219)
@@ -1194,6 +1178,7 @@ static int imx219_probe(struct i2c_client *client)struct device *dev = &client->dev;struct imx219 *imx219;int ret; +u64 val; imx219 =devm_kzalloc (&client->dev,sizeof (*imx219),GFP_KERNEL );if (!imx219)
@@ -1235,36 +1220,49 @@ static int imx219_probe(struct i2c_client *client) "failed to get reset gpio\n");/* -* The sensor must be powered for imx219_identify_module() +* The sensor must be powered for imx219_read_chip_id() * to be able to read the CHIP_ID register */ ret =imx219_power_on (dev);if (ret)return ret; - ret =imx219_identify_module (imx219); -if (ret) - goto error_power_off; - -/* -* Sensor doesn't enter LP-11 state upon power up until and unless -* streaming is started, so upon power up switch the modes to: -* streaming -> standby -*/ - ret =cci_write (imx219->regmap,IMX219_REG_MODE_SELECT , -IMX219_MODE_STREAMING ,NULL ); -if (ret <0 ) +/* Verify chip ID */ + ret =imx219_read_chip_id (imx219, &val); +if (ret) { +if (!media_fault_tolerance_enabled ()) { +dev_err_probe (&client->dev, ret, +"failed to read chip id %x\n" , +IMX219_CHIP_ID ); + goto error_power_off; + } +dev_dbg (&client->dev,"failed to read chip id %x\n" ,IMX219_CHIP_ID ); + }else if (val !=IMX219_CHIP_ID ) { + ret =dev_err_probe (&client->dev, -EIO , +"chip id mismatch: %x!=%llx\n" , +IMX219_CHIP_ID , val); goto error_power_off; + }else { +/* +* Sensor doesn't enter LP-11 state upon power up until and unless +* streaming is started, so upon power up switch the modes to: +* streaming -> standby +*/ + ret =cci_write (imx219->regmap,IMX219_REG_MODE_SELECT , +IMX219_MODE_STREAMING ,NULL ); +if (ret <0 ) + goto error_power_off; -usleep_range (100 ,110 ); +usleep_range (100 ,110 ); -/* put sensor back to standby mode */ - ret =cci_write (imx219->regmap,IMX219_REG_MODE_SELECT , -IMX219_MODE_STANDBY ,NULL ); -if (ret <0 ) - goto error_power_off; +/* put sensor back to standby mode */ + ret =cci_write (imx219->regmap,IMX219_REG_MODE_SELECT , +IMX219_MODE_STANDBY ,NULL ); +if (ret <0 ) + goto error_power_off; -usleep_range (100 ,110 ); +usleep_range (100 ,110 ); + } ret =imx219_init_controls (imx219);if (ret)
Signature · 1 line
2.56.0
RFC PATCH v2 16/16 media: imx219: Implement .detect() sensor operation
In reply to Mattijs KorpershoekNow that the driver can probe() with a missing sensor, userspace should be able to query the sensor connection state via the media device.
Implement the .detect() sensor operation so that userspace can do that using the MEDIA_IOC_G_ENTITY_STATUS ioctl.
Also call media_detect_poll_enable() when streaming successfully started and poll_disable() when disabling streaming to make sure that the media device reports the status to userspace while streaming.
--- drivers/media/i2c/imx219.c | 47 +++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 42 insertions(+), 5 deletions(-)
drivers/media/i2c/imx219.c +42 -5
@@ -28,6 +28,7 @@#include <media/v4l2-device.h> #include <media/v4l2-fwnode.h> #include <media/v4l2-mediabus.h> +#include <media/v4l2-subdev.h> /* Chip ID */ #define IMX219_REG_CHIP_ID CCI_REG16(0x0000)
@@ -742,6 +743,10 @@ static int imx219_enable_streams(struct v4l2_subdev *sd,if (ret <0 )return ret; + ret =media_entity_detect_or_fail (&sd->entity); +if (ret) + goto err_rpm_put; +/* Send all registers that are common to all modes */ ret =cci_multi_reg_write (imx219->regmap, imx219_common_regs,ARRAY_SIZE (imx219_common_regs),NULL );
@@ -780,6 +785,8 @@ static int imx219_enable_streams(struct v4l2_subdev *sd,__v4l2_ctrl_grab (imx219->vflip, true);__v4l2_ctrl_grab (imx219->hflip, true); +media_detect_poll_enable (&sd->entity); +return 0 ; err_rpm_put:
@@ -795,6 +802,8 @@ static int imx219_disable_streams(struct v4l2_subdev *sd,struct i2c_client *client =v4l2_get_subdevdata (&imx219->sd);int ret; +media_detect_poll_disable (&sd->entity); +/* set stream off register */ ret =cci_write (imx219->regmap,IMX219_REG_MODE_SELECT ,IMX219_MODE_STANDBY ,NULL );
@@ -1006,6 +1015,33 @@ static int imx219_init_state(struct v4l2_subdev *sd,return imx219_set_pad_format (sd,NULL , state, &fmt); } +static int imx219_read_chip_id (struct imx219 *imx219,u64 *chip_id) +{ +return cci_read (imx219->regmap,IMX219_REG_CHIP_ID , chip_id,NULL ); +} + +static int imx219_detect (struct media_entity *entity) +{ +struct v4l2_subdev *sd =media_entity_to_v4l2_subdev (entity); +struct i2c_client *client =v4l2_get_subdevdata (sd); +struct imx219 *imx219 =to_imx219 (sd); +u64 chip_id; +int ret; + + ret =pm_runtime_resume_and_get (&client->dev); +if (ret <0 ) +return ret; + +if (imx219_read_chip_id (imx219, &chip_id)) + ret =MEDIA_ENT_STATUS_DISCONNECTED ; +else + ret =MEDIA_ENT_STATUS_CONNECTED ; + +pm_runtime_put_autosuspend (&client->dev); + +return ret; +} +static const struct v4l2_subdev_video_ops imx219_video_ops = { .s_stream = v4l2_subdev_s_stream_helper, };
@@ -1029,6 +1065,10 @@ static const struct v4l2_subdev_internal_ops imx219_internal_ops = { .init_state = imx219_init_state, }; +static const struct media_entity_operations imx219_entity_ops = { + .detect = imx219_detect, +}; +/* ----------------------------------------------------------------------------- * Power management */
@@ -1099,11 +1139,6 @@ static int imx219_get_regulators(struct imx219 *imx219) imx219->supplies); } -static int imx219_read_chip_id (struct imx219 *imx219,u64 *chip_id) -{ -return cci_read (imx219->regmap,IMX219_REG_CHIP_ID , chip_id,NULL ); -} -static int imx219_check_hwcfg (struct device *dev,struct imx219 *imx219) {struct fwnode_handle *endpoint;
@@ -1237,6 +1272,7 @@ static int imx219_probe(struct i2c_client *client) goto error_power_off; }dev_dbg (&client->dev,"failed to read chip id %x\n" ,IMX219_CHIP_ID ); + imx219->sd.entity.status =MEDIA_ENT_STATUS_DISCONNECTED ; }else if (val !=IMX219_CHIP_ID ) { ret = dev_err_probe(&client->dev, -EIO, "chip id mismatch: %x!=%llx\n",
@@ -1271,6 +1307,7 @@ static int imx219_probe(struct i2c_client *client)/* Initialize subdev */ imx219->sd.flags |=V4L2_SUBDEV_FL_HAS_DEVNODE ; imx219->sd.entity.function =MEDIA_ENT_F_CAM_SENSOR ; + imx219->sd.entity.ops = &imx219_entity_ops;/* Initialize source pad */ imx219->pad.flags =MEDIA_PAD_FL_SOURCE ;
Signature · 1 line
2.56.0