drivercore: Add driver probe deferral mechanism

Allow drivers to report at probe time that they cannot get all the resources required by the device, and should be retried at a later time.

I would appreciate
any feedback.

Allow drivers to report at probe time that they cannot get all the resources required by the device, and should be retried at a later time.

This should completely solve the problem of getting devices initialized in the right order. Right now this is mostly handled by mucking about with initcall ordering which is a complete hack, and doesn't even remotely handle the case where device drivers are in modules. This approach completely sidesteps the issues by allowing driver registration to occur in any order, and any driver can request to be retried after a few more other drivers get probed.

v3: - Hold off workqueue scheduling until late_initcall so that the bulk
of driver probes are complete before we start retrying deferred devices.
- Tested with simple use cases. Still needs more testing though. Using it to get rid of the gpio early_initcall madness, or to replace the ASoC internal probe deferral code would be ideal.
v2: - added locking so it should no longer be utterly broken in that regard
- remove device from deferred list at device_del time.
- Still completely untested with any real use case, but has been boot tested.

TODO: - Create a separate singlethread_workqueue so that drivers can't
mess things up by calling flush_work().
- Maybe this should be wrapped with a kconfig symbol so it can be compiled out on systems that don't care.

Signed-off-by Grant Likely<[email protected]>

Cc Greg Kroah-Hartman<[email protected]>

Cc Mark Brown<[email protected]>

Cc Arnd Bergmann<[email protected]>

Cc Dilan Lee<[email protected]>

Cc Manjunath GKondaiah<[email protected]>

---

Hi Manjunath,

Here's the current state of the patch. The major think that needs to be done is to convert it to use a separate workqueue as described in the TODO above. It also needs some users adapted to it. One of the gpio drivers would work; preferably one of the newer drivers that doesn't have a lot of drivers depending on the early_initcall() behaviour yet.

Mark Brown may also be able to suggest specific examples.

For everyone else; this is the current state of the patch. I think it is in pretty good shape other than the TODO item above. I'm turning it over to Manjunath to polish up for merging. I would appreciate any feedback.

g.

  drivers/base/base.h    |    1
  drivers/base/core.c    |    2 +
  drivers/base/dd.c      |  134 ++++++++++++++++++++++++++++++++++++++++++++++++
  include/linux/device.h |    5 ++
  4 files changed, 141 insertions(+), 1 deletions(-)

Definitely what is needed for some of the x86 SoC stuff and would let us rip out some of the special case magic for the SCU discovery.

First thing that strikes me is driver_bound kicks the processing queue again. That seems odd - surely this isn't needed because any driver that does initialise this time and may allow something else to get going will queue the kick itself. Thus this seems to just add overhead.

It all looks a bit O(N?) if we don't expect the drivers that might trigger something else binding to just say 'hey I'm one of the troublemakers'

Quoted 11 lines · Alan Cox

Definitely what is needed for some of the x86 SoC stuff and would let us rip out some of the special case magic for the SCU discovery.

First thing that strikes me is driver_bound kicks the processing queue again. That seems odd - surely this isn't needed because any driver that does initialise this time and may allow something else to get going will queue the kick itself. Thus this seems to just add overhead.

It all looks a bit O(N?) if we don't expect the drivers that might trigger something else binding to just say 'hey I'm one of the troublemakers'

The way I read it, absolute worst case is when every device but one depends on another device. In that case I believe it will be O(Nlog(N)). (Every device gets probed on the first pass, but only the last one gets probed. Then it goes through N-1 devices to the result of only 1 more device getting probed, then N-2, etc.).

Quoted 13 lines · Grant Likely

Quoted 11 lines · Alan Cox

Definitely what is needed for some of the x86 SoC stuff and would let us rip out some of the special case magic for the SCU discovery.

First thing that strikes me is driver_bound kicks the processing queue again. That seems odd - surely this isn't needed because any driver that does initialise this time and may allow something else to get going will queue the kick itself. Thus this seems to just add overhead.

It all looks a bit O(N?) if we don't expect the drivers that might trigger something else binding to just say 'hey I'm one of the troublemakers'

Quoted 5 lines

The way I read it, absolute worst case is when every device but one depends on another device. In that case I believe it will be O(Nlog(N)). (Every device gets probed on the first pass, but only the last one gets probed. Then it goes through N-1 devices to the result of only 1 more device getting probed, then N-2, etc.).

That is indeed O(N**2) not Nlog(N). The total number of probes is (N+1)(N)/2 To get it to O(Nlog(N)), you'd have to probe N devices the first pass, N/2 devices on the second pass, N/4 on the third, and so on.

Definitely what is needed for some of the x86 SoC stuff and would let us rip out some of the special case magic for the SCU discovery.

First thing that strikes me is driver_bound kicks the processing queue again. That seems odd - surely this isn't needed because any driver that does initialise this time and may allow something else to get going will queue the kick itself. Thus this seems to just add overhead.

It all looks a bit O(N²) if we don't expect the drivers that might trigger something else binding to just say 'hey I'm one of the troublemakers'

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

How would a given driver know that something else is waiting for it? Or would we add the explicit kick to each and every driver in the tree?

Quoted 2 lines

How would a given driver know that something else is waiting for it? Or would we add the explicit kick to each and every driver in the tree?

I think there are very very few drivers that have this property and don't already implicitly cause a probe by creating a new bus or device.

Those drivers that set something up for another device really should know what is going on because they are making a guarantee that they are ready for the other device to call into them or whatever is going on at some point, either explicitly in the kick or implicitly in returning from their probe method.

Quoted 2 lines

How would a given driver know that something else is waiting for it? Or would we add the explicit kick to each and every driver in the tree?

I think there are very very few drivers that have this property and don't already implicitly cause a probe by creating a new bus or device.

Those drivers that set something up for another device really should know what is going on because they are making a guarantee that they are ready for the other device to call into them or whatever is going on at some point, either explicitly in the kick or implicitly in returning from their probe method.

I know which I think is clearer and easier for a 3rd party to see and not miss completely when updating code.

Allow drivers to report at probe time that they cannot get all the resources required by the device, and should be retried at a later time.

This should completely solve the problem of getting devices initialized in the right order. Right now this is mostly handled by mucking about with initcall ordering which is a complete hack, and doesn't even remotely handle the case where device drivers are in modules. This approach completely sidesteps the issues by allowing driver registration to occur in any order, and any driver can request to be retried after a few more other drivers get probed.

Quoted 10 lines · Grant Likely

Allow drivers to report at probe time that they cannot get all the resources required by the device, and should be retried at a later time.

This should completely solve the problem of getting devices initialized in the right order. Right now this is mostly handled by mucking about with initcall ordering which is a complete hack, and doesn't even remotely handle the case where device drivers are in modules. This approach completely sidesteps the issues by allowing driver registration to occur in any order, and any driver can request to be retried after a few more other drivers get probed.

So, one issue I did think of the other day while putting some support in the regulator core for using this: what happens with devices which can optionally use a resource but don't rely on it? One example here is that a lot of the MMC drivers have an optional regulator to control some of the supplies for the cards. If the reglator isn't there it won't be used but it's not a blocker for anything. Devices doing this would need some way to figure out if they should -EBUSY or fail otherwise.

Quoted 2 lines · Mark Brown

used but it's not a blocker for anything. Devices doing this would need some way to figure out if they should -EBUSY or fail otherwise.

Just to avoid confusion - ITYM -EAGAIN there. -EBUSY is already used by drivers to mean "someone else claimed a resource I need" be it the IO region or an IRQ resource...

Quoted 8 lines · Russell King - ARM Linux

Quoted 2 lines · Mark Brown

used but it's not a blocker for anything. ?Devices doing this would need some way to figure out if they should -EBUSY or fail otherwise.

Just to avoid confusion - ITYM -EAGAIN there. ?-EBUSY is already used by drivers to mean "someone else claimed a resource I need" be it the IO region or an IRQ resource...

Yes, I do - sorry.

Actually, in the next iteration, I'm thinking it would be a good idea to create a new #define to only be used by probe deferral. I want it to be easy to find all the drivers that are using this mechanism without needing to filter all the unrelated hits. However, this is a kernel-only thing so it is probably not appropriate to add it to include/asm-generic/errno.h. I could use some guidance/advice as to the best way to handle this.

Quoted 7 lines · Grant Likely

Actually, in the next iteration, I'm thinking it would be a good idea to create a new #define to only be used by probe deferral. I want it to be easy to find all the drivers that are using this mechanism without needing to filter all the unrelated hits. However, this is a kernel-only thing so it is probably not appropriate to add it to include/asm-generic/errno.h. I could use some guidance/advice as to the best way to handle this.

include/linux/errno.h already has a bunch of kernel-internal error codes that are never supposed to be seen in user space. Just add one there, the next free one right now is 517, after ERESTART_RESTARTBLOCK.

Okay, will do. How does EPROBE_DEFER 518 sound?

Quoted 1 line · Grant Likely

Okay, will do. How does EPROBE_DEFER 518 sound?

Note that I'm not sure this answers the issue I was raising - the issue isn't that the caller doesn't know what the error code means, the issue is that in some cases the driver needs to take a decision about what failure to get a resource means. Does it mean that the driver can work fine and be slightly less featureful or should it cause a deferral?

Quoted 5 lines · Mark Brown

Note that I'm not sure this answers the issue I was raising - the issue isn't that the caller doesn't know what the error code means, the issue is that in some cases the driver needs to take a decision about what failure to get a resource means. Does it mean that the driver can work fine and be slightly less featureful or should it cause a deferral?

Can you think of cases where this information cannot be put into the device tree or platform_data? If a board provides an optional feature, I think that should be a property of the device that the driver gets, so it can return an error when that feature is not around, or continue when it knows that the feature will never become available.

Quoted 1 line · Grant Likely

Okay, will do. How does EPROBE_DEFER 518 sound?

Note that I'm not sure this answers the issue I was raising - the issue isn't that the caller doesn't know what the error code means, the issue is that in some cases the driver needs to take a decision about what failure to get a resource means. Does it mean that the driver can work fine and be slightly less featureful or should it cause a deferral?

For your question, I still think it is the driver that gets to make the decision. If it can proceed without a resource, then it should go ahead and succeed on the probe, and then arrange to either be notified of new gpio controller (or whatever) registrations, or poll for the resource to be set up.

Allow drivers to report at probe time that they cannot get all the resources required by the device, and should be retried at a later time.

This should completely solve the problem of getting devices initialized in the right order. Right now this is mostly handled by mucking about with initcall ordering which is a complete hack, and doesn't even remotely handle the case where device drivers are in modules. This approach completely sidesteps the issues by allowing driver registration to occur in any order, and any driver can request to be retried after a few more other drivers get probed.

Quoted 10 lines · Grant Likely

Allow drivers to report at probe time that they cannot get all the resources required by the device, and should be retried at a later time.

This should completely solve the problem of getting devices initialized in the right order. ?Right now this is mostly handled by mucking about with initcall ordering which is a complete hack, and doesn't even remotely handle the case where device drivers are in modules. ?This approach completely sidesteps the issues by allowing driver registration to occur in any order, and any driver can request to be retried after a few more other drivers get probed.

This is great work, thanks!

For the TODO list:

While the proposed patch should solve probe order dependencies, I don't think it will solve the suspend/resume ordering dependencies, which are typically the same.

Currenly suspend/resume order is based on the order devices are *added* (device_add() -> device_pm_add() -> device added to dpm_list), so unfortunately, deferring probe isn't going to affect suspend/resume ordering.

Extending this to also address suspend/resume ordering by also changing when the device is added to the dpm_list (or possibly creating another list) should probably be explored as well.

Allow drivers to report at probe time that they cannot get all the resources required by the device, and should be retried at a later time.

This should completely solve the problem of getting devices initialized in the right order. Right now this is mostly handled by mucking about with initcall ordering which is a complete hack, and doesn't even remotely handle the case where device drivers are in modules. This approach completely sidesteps the issues by allowing driver registration to occur in any order, and any driver can request to be retried after a few more other drivers get probed.

v3: - Hold off workqueue scheduling until late_initcall so that the bulk
of driver probes are complete before we start retrying deferred devices.
- Tested with simple use cases. Still needs more testing though. Using it to get rid of the gpio early_initcall madness, or to replace the ASoC internal probe deferral code would be ideal.
v2: - added locking so it should no longer be utterly broken in that regard
- remove device from deferred list at device_del time.
- Still completely untested with any real use case, but has been boot tested.

TODO: - Create a separate singlethread_workqueue so that drivers can't
mess things up by calling flush_work().
- Maybe this should be wrapped with a kconfig symbol so it can be compiled out on systems that don't care.

Signed-off-by Grant Likely <[email protected]>

Cc Greg Kroah-Hartman <[email protected]>

Cc Mark Brown <[email protected]>

Cc Arnd Bergmann <[email protected]>

Cc Dilan Lee <[email protected]>

Cc Manjunath GKondaiah <[email protected]>

---

Hi Manjunath,

Here's the current state of the patch. The major think that needs to be done is to convert it to use a separate workqueue as described in the TODO above. It also needs some users adapted to it. One of the gpio drivers would work; preferably one of the newer drivers that doesn't have a lot of drivers depending on the early_initcall() behaviour yet.

Mark Brown may also be able to suggest specific examples.

For everyone else; this is the current state of the patch. I think it is in pretty good shape other than the TODO item above. I'm turning it over to Manjunath to polish up for merging. I would appreciate any feedback.

g.

 drivers/base/base.h    |    1 
 drivers/base/core.c    |    2 +
 drivers/base/dd.c      |  134 ++++++++++++++++++++++++++++++++++++++++++++++++
 include/linux/device.h |    5 ++
 4 files changed, 141 insertions(+), 1 deletions(-)

drivers/base/base.h +1 -0

index a34dca0..9641309 100644
@@ -105,6 +105,7 @@ extern void bus_remove_driver(struct device_driver *drv);
 
 extern void driver_detach(struct device_driver *drv);
 extern int driver_probe_device(struct device_driver *drv, struct device *dev);
+extern void driver_deferred_probe_del(struct device *dev);
 static inline int driver_match_device(struct device_driver *drv,
 				      struct device *dev)
 {

drivers/base/core.c +2 -0

index bc8729d..0d37e18 100644
@@ -588,6 +588,7 @@ void device_initialize(struct device *dev)
 {
 	dev->kobj.kset = devices_kset;
 	kobject_init(&dev->kobj, &device_ktype);
+	INIT_LIST_HEAD(&dev->deferred_probe);
 	INIT_LIST_HEAD(&dev->dma_pools);
 	mutex_init(&dev->mutex);
 	lockdep_set_novalidate_class(&dev->mutex);

@@ -1119,6 +1120,7 @@ void device_del(struct device *dev)
 	device_remove_file(dev, &uevent_attr);
 	device_remove_attrs(dev);
 	bus_remove_device(dev);
+	driver_deferred_probe_del(dev);
 
 	/*
 	 * Some platform devices are driven without driver attached

drivers/base/dd.c +133 -1

index 6658da7..8be5b33 100644
@@ -28,6 +28,129 @@
 #include "base.h"
 #include "power/power.h"
 
+/*
+ * Deferred Probe infrastructure.
+ *
+ * Sometimes driver probe order matters, but the kernel doesn't always have
+ * dependency information which means some drivers will get probed before a
+ * resource it depends on is available.  For example, an SDHCI driver may
+ * first need a GPIO line from an i2c GPIO controller before it can be
+ * initialized.  If a required resource is not available yet, a driver can
+ * request probing to be deferred by returning -EAGAIN from its probe hook
+ *
+ * Deferred probe maintains two lists of devices, a pending list and an active
+ * list.  A driver returning -EAGAIN causes the device to be added to the
+ * pending list.
+ *
+ * The deferred_probe_mutex *must* be held any time the deferred_probe_*_list
+ * of the (struct device*)->deferred_probe pointers are manipulated
+ */
+static DEFINE_MUTEX(deferred_probe_mutex);
+static LIST_HEAD(deferred_probe_pending_list);
+static LIST_HEAD(deferred_probe_active_list);
+
+/**
+ * deferred_probe_work_func() - Retry probing devices in the active list.
+ */
+static void deferred_probe_work_func(struct work_struct *work)
+{
+	struct device *dev;
+	/*
+	 * This bit is tricky.  We want to process every device in the
+	 * deferred list, but devices can be removed from the list at any
+	 * time while inside this for-each loop.  There are two things that
+	 * need to be protected against:
+	 * - if the device is removed from the deferred_probe_list, then we
+	 *   loose our place in the loop.  Since any device can be removed
+	 *   asynchronously, list_for_each_entry_safe() wouldn't make things
+	 *   much better.  Simplest solution is to restart walking the list
+	 *   whenever the current device gets removed.  Not the most efficient,
+	 *   but is simple to implement and easy to audit for correctness.
+	 * - if the device is unregistered, and freed, then there is a risk
+	 *   of a null pointer dereference.  This code uses get/put_device()
+	 *   to ensure the device cannot disappear from under our feet.
+	 */
+	mutex_lock(&deferred_probe_mutex);
+	while (!list_empty(&deferred_probe_active_list)) {
+		dev = list_first_entry(&deferred_probe_active_list,
+					typeof(*dev), deferred_probe);
+		list_del_init(&dev->deferred_probe);
+
+		get_device(dev);
+
+		/* Drop the mutex while probing each device; the probe path
+		 * may manipulate the deferred list */
+		mutex_unlock(&deferred_probe_mutex);
+		dev_dbg(dev, "Retrying from deferred list\n");
+		bus_probe_device(dev);
+		mutex_lock(&deferred_probe_mutex);
+
+		put_device(dev);
+	}
+	mutex_unlock(&deferred_probe_mutex);
+}
+static DECLARE_WORK(deferred_probe_work, deferred_probe_work_func);
+
+static void driver_deferred_probe_add(struct device *dev)
+{
+	mutex_lock(&deferred_probe_mutex);
+	if (list_empty(&dev->deferred_probe)) {
+		dev_dbg(dev, "Added to deferred list\n");
+		list_add(&dev->deferred_probe, &deferred_probe_pending_list);
+	}
+	mutex_unlock(&deferred_probe_mutex);
+}
+
+void driver_deferred_probe_del(struct device *dev)
+{
+	mutex_lock(&deferred_probe_mutex);
+	if (!list_empty(&dev->deferred_probe)) {
+		dev_dbg(dev, "Removed from deferred list\n");
+		list_del_init(&dev->deferred_probe);
+	}
+	mutex_unlock(&deferred_probe_mutex);
+}
+
+static bool driver_deferred_probe_enable = false;
+/**
+ * driver_deferred_probe_trigger() - Kick off re-probing deferred devices
+ *
+ * This functions moves all devices from the pending list to the active
+ * list and schedules the deferred probe workqueue to process them.  It
+ * should be called anytime a driver is successfully bound to a device.
+ */
+static void driver_deferred_probe_trigger(void)
+{
+	if (!driver_deferred_probe_enable)
+		return;
+
+	/* A successful probe means that all the devices in the pending list
+	 * should be triggered to be reprobed.  Move all the deferred devices
+	 * into the active list so they can be retried by the workqueue */
+	mutex_lock(&deferred_probe_mutex);
+	list_splice_tail_init(&deferred_probe_pending_list,
+			      &deferred_probe_active_list);
+	mutex_unlock(&deferred_probe_mutex);
+
+	/* Kick the re-probe thread.  It may already be scheduled, but
+	 * it is safe to kick it again. */
+	schedule_work(&deferred_probe_work);
+}
+
+/**
+ * deferred_probe_initcall() - Enable probing of deferred devices
+ *
+ * We don't want to get in the way when the bulk of drivers are getting probed.
+ * Instead, this initcall makes sure that deferred probing is delayed until
+ * late_initcall time.
+ */
+static int deferred_probe_initcall(void)
+{
+	driver_deferred_probe_enable = true;
+	driver_deferred_probe_trigger();
+	return 0;
+}
+late_initcall(deferred_probe_initcall);
 
 static void driver_bound(struct device *dev)
 {

@@ -42,6 +165,11 @@ static void driver_bound(struct device *dev)
 
 	klist_add_tail(&dev->p->knode_driver, &dev->driver->p->klist_devices);
 
+	/* Make sure the device is no longer in one of the deferred lists
+	 * and kick off retrying all pending devices */
+	driver_deferred_probe_del(dev);
+	driver_deferred_probe_trigger();
+
 	if (dev->bus)
 		blocking_notifier_call_chain(&dev->bus->p->bus_notifier,
 					     BUS_NOTIFY_BOUND_DRIVER, dev);

@@ -142,7 +270,11 @@ probe_failed:
 	driver_sysfs_remove(dev);
 	dev->driver = NULL;
 
-	if (ret != -ENODEV && ret != -ENXIO) {
+	if (ret == -EAGAIN) {
+		/* Driver requested deferred probing */
+		dev_info(dev, "Driver %s requests probe deferral\n", drv->name);
+		driver_deferred_probe_add(dev);
+	} else if (ret != -ENODEV && ret != -ENXIO) {
 		/* driver matched but the probe failed */
 		printk(KERN_WARNING
 		       "%s: probe of %s failed with error %d\n",

include/linux/device.h +5 -0

index c20dfbf..dab93a4 100644
@@ -506,6 +506,10 @@ struct device_dma_parameters {
  * @mutex:	Mutex to synchronize calls to its driver.
  * @bus:	Type of bus device is on.
  * @driver:	Which driver has allocated this
+ * @deferred_probe: entry in deferred_probe_list which is used to retry the
+ * 		binding of drivers which were unable to get all the resources
+ * 		needed by the device; typically because it depends on another
+ * 		driver getting probed first.
  * @platform_data: Platform data specific to the device.
  * 		Example: For devices on custom boards, as typical of embedded
  * 		and SOC based hardware, Linux often uses platform_data to point

@@ -564,6 +568,7 @@ struct device {
 	struct bus_type	*bus;		/* type of bus device is on */
 	struct device_driver *driver;	/* which driver has allocated this
 					   device */
+	struct list_head	deferred_probe;
 	void		*platform_data;	/* Platform specific data, device
 					   core doesn't touch it */
 	struct dev_pm_info	power;

Quoted 8 lines · Grant Likely

Hi Manjunath,

Here's the current state of the patch. ?The major think that needs to be done is to convert it to use a separate workqueue as described in the TODO above. ?It also needs some users adapted to it. ?One of the gpio drivers would work; preferably one of the newer drivers that doesn't have a lot of drivers depending on the early_initcall() behaviour yet.

I have tested this patch on omap3 beagle board by making:
1. omap-gpio driver init as late_initcall instead of postcore_initcall
2. mmc driver probe will request gpio through gpio_request and gpio driver returns -EDEFER_PROBE which in turn makes mmc driver to request deferral probe.
3. When deferral probe gets activated, it scans driver entries and it will not find any match for mmc driver probe.

Quoted 10 lines · Manjunath Kondaiah

Quoted 8 lines · Grant Likely

Hi Manjunath,

Here's the current state of the patch. ?The major think that needs to be done is to convert it to use a separate workqueue as described in the TODO above. ?It also needs some users adapted to it. ?One of the gpio drivers would work; preferably one of the newer drivers that doesn't have a lot of drivers depending on the early_initcall() behaviour yet.

Quoted 6 lines

I have tested this patch on omap3 beagle board by making:
1. omap-gpio driver init as late_initcall instead of postcore_initcall
2. mmc driver probe will request gpio through gpio_request and gpio driver returns -EDEFER_PROBE which in turn makes mmc driver to request deferral probe.
3. When deferral probe gets activated, it scans driver entries and it will not find any match for mmc driver probe.

Looks like drivers/mmc/host/omap.c is using platform_driver_probe() instead of platform_driver_register(). Add the probe hook to the platform_driver structure and change it to call platform_driver_register() and it should work. Don't forget to change mmc_omap_probe from __init to __devinit.

Quoted 18 lines · Grant Likely

Quoted 10 lines · Manjunath Kondaiah

Quoted 8 lines · Grant Likely

Hi Manjunath,

Here's the current state of the patch. ?The major think that needs to be done is to convert it to use a separate workqueue as described in the TODO above. ?It also needs some users adapted to it. ?One of the gpio drivers would work; preferably one of the newer drivers that doesn't have a lot of drivers depending on the early_initcall() behaviour yet.

Quoted 6 lines

I have tested this patch on omap3 beagle board by making:
1. omap-gpio driver init as late_initcall instead of postcore_initcall
2. mmc driver probe will request gpio through gpio_request and gpio driver returns -EDEFER_PROBE which in turn makes mmc driver to request deferral probe.
3. When deferral probe gets activated, it scans driver entries and it will not find any match for mmc driver probe.

Quoted 5 lines

Looks like drivers/mmc/host/omap.c is using platform_driver_probe() instead of platform_driver_register(). Add the probe hook to the platform_driver structure and change it to call platform_driver_register() and it should work. Don't forget to change mmc_omap_probe from __init to __devinit.

Yes. After changing it into platform_driver_register, I can see mmc probe is getting completed from deferred probe list. But, MMC upper layer will check probe and it waits for ever since probe_count is not getting incremented.

Log:

[    1.807830] platform omap_hsmmc.0: Driver omap_hsmmc requests probe deferral

...

[    1.948760] omap_device: omap_gpio.0: new worst case activate latency 0:30517
[    1.959259] OMAP GPIO hardware version 2.5

...

[    2.000488] platform omap_hsmmc.0: Retrying from deferred list
[    2.008026] omap_device: omap_hsmmc.0: new worst case activate latency 0:244140
[    2.020080] input: gpio-keys as /devices/platform/gpio-keys/input/input1
[    2.035827] omap_hsmmc omap_hsmmc.0: Probe success...
[    2.042083] twl_rtc twl_rtc: setting system clock to 2000-01-01 01:06:35 UTC
(946688795)
[    2.056030] Waiting for root device /dev/mmcblk0p2...
[    2.061492] driver_probe_done: probe_count = 0
[    2.168518] driver_probe_done: probe_count = 0
[    2.277832] driver_probe_done: probe_count = 0
[    2.387207] driver_probe_done: probe_count = 0
[    2.496582] driver_probe_done: probe_count = 0

Waits for ever in init/do_mount.c:

	if ((ROOT_DEV == 0) && root_wait) {
		printk(KERN_INFO "Waiting for root device %s...\n",
			saved_root_name);
		while (driver_probe_done() != 0 ||
			(ROOT_DEV = name_to_dev_t(saved_root_name)) == 0)
			msleep(100);
		async_synchronize_full();
	}

-Manjunath

Quoted 54 lines · Manjunath Kondaiah

Quoted 18 lines · Grant Likely

Quoted 10 lines · Manjunath Kondaiah

Quoted 8 lines · Grant Likely

Hi Manjunath,

Here's the current state of the patch. ?The major think that needs to be done is to convert it to use a separate workqueue as described in the TODO above. ?It also needs some users adapted to it. ?One of the gpio drivers would work; preferably one of the newer drivers that doesn't have a lot of drivers depending on the early_initcall() behaviour yet.

Quoted 6 lines

I have tested this patch on omap3 beagle board by making:
1. omap-gpio driver init as late_initcall instead of postcore_initcall
2. mmc driver probe will request gpio through gpio_request and gpio driver returns -EDEFER_PROBE which in turn makes mmc driver to request deferral probe.
3. When deferral probe gets activated, it scans driver entries and it will not find any match for mmc driver probe.

Quoted 5 lines

Looks like drivers/mmc/host/omap.c is using platform_driver_probe() instead of platform_driver_register(). Add the probe hook to the platform_driver structure and change it to call platform_driver_register() and it should work. Don't forget to change mmc_omap_probe from __init to __devinit.

Yes. After changing it into platform_driver_register, I can see mmc probe is getting completed from deferred probe list. But, MMC upper layer will check probe and it waits for ever since probe_count is not getting incremented.

Log:

[    1.807830] platform omap_hsmmc.0: Driver omap_hsmmc requests probe deferral

...

[    1.948760] omap_device: omap_gpio.0: new worst case activate latency 0:30517
[    1.959259] OMAP GPIO hardware version 2.5

...

[    2.000488] platform omap_hsmmc.0: Retrying from deferred list
[    2.008026] omap_device: omap_hsmmc.0: new worst case activate latency 0:244140
[    2.020080] input: gpio-keys as /devices/platform/gpio-keys/input/input1
[    2.035827] omap_hsmmc omap_hsmmc.0: Probe success...
[    2.042083] twl_rtc twl_rtc: setting system clock to 2000-01-01 01:06:35 UTC
(946688795)
[    2.056030] Waiting for root device /dev/mmcblk0p2...
[    2.061492] driver_probe_done: probe_count = 0
[    2.168518] driver_probe_done: probe_count = 0
[    2.277832] driver_probe_done: probe_count = 0
[    2.387207] driver_probe_done: probe_count = 0
[    2.496582] driver_probe_done: probe_count = 0

Waits for ever in init/do_mount.c:

Quoted 8 lines

	if ((ROOT_DEV == 0) && root_wait) {
		printk(KERN_INFO "Waiting for root device %s...\n",
			saved_root_name);
		while (driver_probe_done() != 0 ||
			(ROOT_DEV = name_to_dev_t(saved_root_name)) == 0)
			msleep(100);
		async_synchronize_full();
	}

Okay, it looks like the driver deferral workqueue needs to get kicked off before the kernel starts waiting for the root filesystem. Check the initcall level that is being used by do_mount, and make sure the deferred probe starts before that...

Hmmm.... wait, that doesn't make sense because the probe is still successful. What triggers the exit conditions in that while loop? How would it be different from when the driver can probe successfully without deferral?

Allow drivers to report at probe time that they cannot get all the resources required by the device, and should be retried at a later time.

This should completely solve the problem of getting devices initialized in the right order. Right now this is mostly handled by mucking about with initcall ordering which is a complete hack, and doesn't even remotely handle the case where device drivers are in modules. This approach completely sidesteps the issues by allowing driver registration to occur in any order, and any driver can request to be retried after a few more other drivers get probed.

...

Quoted 10 lines · On Thu, 22 Sep 2011 12:51:23 -0600 Grant Likely<[email protected]>…

Allow drivers to report at probe time that they cannot get all the resources required by the device, and should be retried at a later time.

This should completely solve the problem of getting devices initialized in the right order. Right now this is mostly handled by mucking about with initcall ordering which is a complete hack, and doesn't even remotely handle the case where device drivers are in modules. This approach completely sidesteps the issues by allowing driver registration to occur in any order, and any driver can request to be retried after a few more other drivers get probed.

What happens is there is a circular dependency, or if a driver's preconditions are never met? AFAICT the code keeps running the probe function for ever.

If so: bad. The kernel should detect such situations, should exhaustively report them and if possible, fix them up and struggle onwards.

Quoted 14 lines · Andrew Morton

Quoted 10 lines · On Thu, 22 Sep 2011 12:51:23 -0600 Grant Likely<[email protected]>…

Allow drivers to report at probe time that they cannot get all the resources required by the device, and should be retried at a later time.

This should completely solve the problem of getting devices initialized in the right order. Right now this is mostly handled by mucking about with initcall ordering which is a complete hack, and doesn't even remotely handle the case where device drivers are in modules. This approach completely sidesteps the issues by allowing driver registration to occur in any order, and any driver can request to be retried after a few more other drivers get probed.

Quoted 4 lines

What happens is there is a circular dependency, or if a driver's preconditions are never met? AFAICT the code keeps running the probe function for ever.

The deferred probe functions are only run once per (other) driver binding event. So once you quit registering new drivers, no further probing is done. There is no endless loop happening here.

If the preconditions are never met, the driver will just sit in the list waiting.

Quoted 3 lines

If so: bad. The kernel should detect such situations, should exhaustively report them and if possible, fix them up and struggle onwards.

I don't think we should actively report anything, but being able to inspect the deferred probe list from user space might be useful for diagnosing problems