Skip to contentSkip to conversation

shut down devices asynchronously

PATCH Superseded Stuart Hayes 5 participants 10 messages May 16, 2024 – Jun 12, 2024

Subsystems (2)

v6 of 10 Superseded Latest is v10 →

lore
Share & tools
lore
#0 lore raw

v6 cover shut down devices asynchronously

This adds the ability for the kernel to shutdown devices asynchronously (this is V6).

It is in "safe" mode by default (where only devices whose drivers opt-in are shut down asynchronously), but it can be "on" (where only devices whose drivers opt-out are shut down synchronously), or "off" (legacy behavior).

This can dramatically reduce system shutdown/reboot time on systems that have multiple devices that take many seconds to shut down (like certain NVMe drives). On one system tested, the shutdown time went from 11 minutes without this patch to 55 seconds with the patch.

Changes from V5:

Separated into multiple patches to make review easier. Reworked some code to make it more readable
Made devices wait for consumers to shut down, not just children
(suggested by David Jeffery)

Changes from V4:

Change code to use cookies for synchronization rather than async domains Allow async shutdown to be disabled via sysfs, and allow driver opt-in or
opt-out of async shutdown (when not disabled), with ability to control driver opt-in/opt-out via sysfs

Changes from V3:

Bug fix (used "parent" not "dev->parent" in device_shutdown)

Changes from V2:

Removed recursive functions to schedule children to be shutdown before
parents, since existing device_shutdown loop will already do this

Changes from V1:

Rewritten using kernel async code (suggested by Lukas Wunner)

Stuart Hayes (4):
driver core: don't always lock parent in shutdown driver core: separate function to shutdown one device driver core: shut down devices asynchronously
nvme-pci: Make driver prefer asynchronous shutdown

 drivers/base/base.h           |   3 +
 drivers/base/bus.c            |  47 +++++++++
 drivers/base/core.c           | 185 +++++++++++++++++++++++++++++-----
 drivers/nvme/host/pci.c       |   1 +
 include/linux/device/driver.h |   8 ++
 5 files changed, 218 insertions(+), 26 deletions(-)
Signature · 1 line
2.39.3
#1 lore raw

PATCH v6 1/4 driver core: don't always lock parent in shutdown

Don't lock a parent device unless it is needed in device_shutdown. This is in preparation for making device shutdown asynchronous, when it will be needed to allow children of a common parent to shut down simultaneously.

---
 drivers/base/core.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/base/core.c +2 -2
index 5f4e03336e68..e63177314e86 100644
@@ -4856,7 +4856,7 @@ void device_shutdown(void)
 		spin_unlock(&devices_kset->list_lock);
 
 		/* hold lock to avoid race with probe/release */
-		if (parent)
+		if (parent && dev->bus && dev->bus->need_parent_lock)
 			device_lock(parent);
 		device_lock(dev);
 
@@ -4880,7 +4880,7 @@ void device_shutdown(void)
 		}
 
 		device_unlock(dev);
-		if (parent)
+		if (parent && dev->bus && dev->bus->need_parent_lock)
 			device_unlock(parent);
 
 		put_device(dev);
Signature · 1 line
2.39.3
#2 lore raw

PATCH v6 2/4 driver core: separate function to shutdown one device

In reply to Stuart Hayes

Make a separate function for the part of device_shutdown() that does the shutown for a single device. This is in preparation for making device shutdown asynchronous.

---
 drivers/base/core.c | 66 ++++++++++++++++++++++++---------------------
 1 file changed, 36 insertions(+), 30 deletions(-)
drivers/base/core.c +36 -30
index e63177314e86..e76cba51513a 100644
@@ -4819,6 +4819,41 @@ int device_change_owner(struct device *dev, kuid_t kuid, kgid_t kgid)
 }
 EXPORT_SYMBOL_GPL(device_change_owner);
 
+static void shutdown_one_device(struct device *dev)
+{
+	/* hold lock to avoid race with probe/release */
+	if (dev->parent && dev->bus && dev->bus->need_parent_lock)
+		device_lock(dev->parent);
+	device_lock(dev);
+
+	/* Don't allow any more runtime suspends */
+	pm_runtime_get_noresume(dev);
+	pm_runtime_barrier(dev);
+
+	if (dev->class && dev->class->shutdown_pre) {
+		if (initcall_debug)
+			dev_info(dev, "shutdown_pre\n");
+		dev->class->shutdown_pre(dev);
+	}
+	if (dev->bus && dev->bus->shutdown) {
+		if (initcall_debug)
+			dev_info(dev, "shutdown\n");
+		dev->bus->shutdown(dev);
+	} else if (dev->driver && dev->driver->shutdown) {
+		if (initcall_debug)
+			dev_info(dev, "shutdown\n");
+		dev->driver->shutdown(dev);
+	}
+
+	device_unlock(dev);
+	if (dev->parent && dev->bus && dev->bus->need_parent_lock)
+		device_unlock(dev->parent);
+
+	put_device(dev);
+	if (dev->parent)
+		put_device(dev->parent);
+}
+
 /**
  * device_shutdown - call ->shutdown() on each device to shutdown.
  */
@@ -4855,36 +4890,7 @@ void device_shutdown(void)
 		list_del_init(&dev->kobj.entry);
 		spin_unlock(&devices_kset->list_lock);
 
-		/* hold lock to avoid race with probe/release */
-		if (parent && dev->bus && dev->bus->need_parent_lock)
-			device_lock(parent);
-		device_lock(dev);
-
-		/* Don't allow any more runtime suspends */
-		pm_runtime_get_noresume(dev);
-		pm_runtime_barrier(dev);
-
-		if (dev->class && dev->class->shutdown_pre) {
-			if (initcall_debug)
-				dev_info(dev, "shutdown_pre\n");
-			dev->class->shutdown_pre(dev);
-		}
-		if (dev->bus && dev->bus->shutdown) {
-			if (initcall_debug)
-				dev_info(dev, "shutdown\n");
-			dev->bus->shutdown(dev);
-		} else if (dev->driver && dev->driver->shutdown) {
-			if (initcall_debug)
-				dev_info(dev, "shutdown\n");
-			dev->driver->shutdown(dev);
-		}
-
-		device_unlock(dev);
-		if (parent && dev->bus && dev->bus->need_parent_lock)
-			device_unlock(parent);
-
-		put_device(dev);
-		put_device(parent);
+		shutdown_one_device(dev);
 
 		spin_lock(&devices_kset->list_lock);
 	}
Signature · 1 line
2.39.3
#3 lore raw

PATCH v6 3/4 driver core: shut down devices asynchronously

In reply to Stuart Hayes

Add code to shut down devices asynchronously, while ensuring that each device is shut down before its parents & suppliers, and allowing devices that share a driver to be shutdown one at a time if necessary.

Add /sys/kernel/async_shutdown to allow user control of this feature:

safe: shut down all devices synchronously, unless driver prefers async
shutdown (driver opt-in) (default)
on: shut down all devices asynchronously, unless disabled by the driver
(driver opt-out)
off: shut down all devices synchronously

Add async_shutdown to struct device_driver, and expose it via sysfs. This will be used to view or change driver opt-in/opt-out of asynchronous shutdown, if it is globally enabled.

async: driver opt-in to async device shutdown (devices will be shut down
asynchronously if async_shutdown is "on" or "safe")
sync: driver opt-out of async device shutdown (devices will always be
shut down synchronously)
default: devices will be shutdown asynchronously if async_shutdown is "on"

This can dramatically reduce system shutdown/reboot time on systems that have multiple devices that take many seconds to shut down (like certain NVMe drives). On one system tested, the shutdown time went from 11 minutes without this patch to 55 seconds with the patch.

---
 drivers/base/base.h           |   3 +
 drivers/base/bus.c            |  47 +++++++++++++
 drivers/base/core.c           | 129 +++++++++++++++++++++++++++++++++-
 include/linux/device/driver.h |   8 +++
 4 files changed, 186 insertions(+), 1 deletion(-)
drivers/base/base.h +3 -0
index 0738ccad08b2..ab80a0721b2e 100644
@@ -10,6 +10,7 @@
  * shared outside of the drivers/base/ directory.
  *
  */
+#include <linux/async.h>
 #include <linux/notifier.h>
 
 /**
@@ -97,6 +98,7 @@ struct driver_private {
  *	the device; typically because it depends on another driver getting
  *	probed first.
  * @async_driver - pointer to device driver awaiting probe via async_probe
+ * @shutdown_after - used during async shutdown to ensure correct shutdown ordering.
  * @device - pointer back to the struct device that this structure is
  * associated with.
  * @dead - This device is currently either in the process of or has been
@@ -114,6 +116,7 @@ struct device_private {
 	struct list_head deferred_probe;
 	struct device_driver *async_driver;
 	char *deferred_probe_reason;
+	async_cookie_t shutdown_after;
 	struct device *device;
 	u8 dead:1;
 };
drivers/base/bus.c +47 -0

65 lines not on this page · raw message

drivers/base/core.c +128 -1

175 lines not on this page · raw message

include/linux/device/driver.h +8 -0
index 7738f458995f..f414c8a6f814 100644
@@ -48,6 +48,12 @@ enum probe_type {
 	PROBE_FORCE_SYNCHRONOUS,
 };
 
+enum shutdown_type {
+	SHUTDOWN_DEFAULT_STRATEGY,
+	SHUTDOWN_PREFER_ASYNCHRONOUS,
+	SHUTDOWN_FORCE_SYNCHRONOUS,
+};
+
 /**
  * struct device_driver - The basic device driver structure
  * @name:	Name of the device driver.
@@ -56,6 +62,7 @@ enum probe_type {
  * @mod_name:	Used for built-in modules.
  * @suppress_bind_attrs: Disables bind/unbind via sysfs.
  * @probe_type:	Type of the probe (synchronous or asynchronous) to use.
+ * @shutdown_type: Type of the shutdown (synchronous or asynchronous) to use.
  * @of_match_table: The open firmware table.
  * @acpi_match_table: The ACPI match table.
  * @probe:	Called to query the existence of a specific device,
@@ -102,6 +109,7 @@ struct device_driver {
 
 	bool suppress_bind_attrs;	/* disables bind/unbind via sysfs */
 	enum probe_type probe_type;
+	enum shutdown_type shutdown_type;
 
 	const struct of_device_id	*of_match_table;
 	const struct acpi_device_id	*acpi_match_table;
Signature · 1 line
2.39.3
#4 lore raw

Re: [PATCH v6 3/4] driver core: shut down devices asynchronously

Quoted 7 lines · Stuart Hayes

Add /sys/kernel/async_shutdown to allow user control of this feature:

safe: shut down all devices synchronously, unless driver prefers async
shutdown (driver opt-in) (default)
on: shut down all devices asynchronously, unless disabled by the driver
(driver opt-out)
off: shut down all devices synchronously

The on option seems very odd. IMHO safe is the only really sensible option, and maybe we have to support off as a bandaid due to userspace behavior dependent on synchronous shutdown, but I'd rather try even without that first.

#5 lore raw

Re: [PATCH v6 3/4] driver core: shut down devices asynchronously

Quoted 9 lines · Christoph Hellwig
Quoted 7 lines · Stuart Hayes

Add /sys/kernel/async_shutdown to allow user control of this feature:

safe: shut down all devices synchronously, unless driver prefers async
shutdown (driver opt-in) (default)
on: shut down all devices asynchronously, unless disabled by the driver
(driver opt-out)
off: shut down all devices synchronously

Quoted 5 lines

The on option seems very odd. IMHO safe is the only really sensible option, and maybe we have to support off as a bandaid due to userspace behavior dependent on synchronous shutdown, but I'd rather try even without that first.

I added the option because of comments from Greg K-H on the v4 submission of this patch--see

https://lore.kernel.org/lkml/2023102151-rejoicing-studio-6126@gregkh/T/#m5d0459480bc0fda4563040cab2036839bcbb79a8).

I thought it would be nice to have the option for testing, even if it gets removed later, but I'll certainly remove it now if necessary.

#6 lore raw

Re: [PATCH v6 3/4] driver core: shut down devices asynchronously

Quoted 18 lines · stuart hayes
Quoted 9 lines · Christoph Hellwig
Quoted 7 lines · Stuart Hayes

Add /sys/kernel/async_shutdown to allow user control of this feature:

safe: shut down all devices synchronously, unless driver prefers async
shutdown (driver opt-in) (default)
on: shut down all devices asynchronously, unless disabled by the driver
(driver opt-out)
off: shut down all devices synchronously

Quoted 5 lines

The on option seems very odd. IMHO safe is the only really sensible option, and maybe we have to support off as a bandaid due to userspace behavior dependent on synchronous shutdown, but I'd rather try even without that first.

Quoted 7 lines

I added the option because of comments from Greg K-H on the v4 submission of this patch--see

https://lore.kernel.org/lkml/2023102151-rejoicing-studio-6126@gregkh/T/#m5d0459480bc0fda4563040cab2036839bcbb79a8).

I thought it would be nice to have the option for testing, even if it gets removed later, but I'll certainly remove it now if necessary.

Opt-in is the requirement here, that's all I asked for. The "on" can probably be removed, and by doing that, you can make this option simpler as well.

thanks,

greg k-h

maintainer
#7 lore raw

Re: [PATCH v6 3/4] driver core: shut down devices asynchronously

In reply to Stuart Hayes
Quoted 22 lines · Stuart Hayes

Add code to shut down devices asynchronously, while ensuring that each device is shut down before its parents & suppliers, and allowing devices that share a driver to be shutdown one at a time if necessary.

Add /sys/kernel/async_shutdown to allow user control of this feature:

safe: shut down all devices synchronously, unless driver prefers async
shutdown (driver opt-in) (default)
on: shut down all devices asynchronously, unless disabled by the driver
(driver opt-out)
off: shut down all devices synchronously

Add async_shutdown to struct device_driver, and expose it via sysfs. This will be used to view or change driver opt-in/opt-out of asynchronous shutdown, if it is globally enabled.

async: driver opt-in to async device shutdown (devices will be shut down
asynchronously if async_shutdown is "on" or "safe")
sync: driver opt-out of async device shutdown (devices will always be
shut down synchronously)
default: devices will be shutdown asynchronously if async_shutdown is "on"

Quoted 4 lines

This can dramatically reduce system shutdown/reboot time on systems that have multiple devices that take many seconds to shut down (like certain NVMe drives). On one system tested, the shutdown time went from 11 minutes without this patch to 55 seconds with the patch.

I've successfully tested this out on a few systems, and noticing a very decent shutdown time on my nvme systems. I also like the current solution here, as the two-pass method was harder to follow.

So I think just remove the extra options that Christoph mentioned and always use the driver's preferred shutdown method, then this would all look good to me.

#8 lore raw

Re: [PATCH v6 3/4] driver core: shut down devices asynchronously

Quoted 28 lines · Keith Busch
Quoted 22 lines · Stuart Hayes

Add code to shut down devices asynchronously, while ensuring that each device is shut down before its parents & suppliers, and allowing devices that share a driver to be shutdown one at a time if necessary.

Add /sys/kernel/async_shutdown to allow user control of this feature:

safe: shut down all devices synchronously, unless driver prefers async
shutdown (driver opt-in) (default)
on: shut down all devices asynchronously, unless disabled by the driver
(driver opt-out)
off: shut down all devices synchronously

Add async_shutdown to struct device_driver, and expose it via sysfs. This will be used to view or change driver opt-in/opt-out of asynchronous shutdown, if it is globally enabled.

async: driver opt-in to async device shutdown (devices will be shut down
asynchronously if async_shutdown is "on" or "safe")
sync: driver opt-out of async device shutdown (devices will always be
shut down synchronously)
default: devices will be shutdown asynchronously if async_shutdown is "on"

Quoted 4 lines

This can dramatically reduce system shutdown/reboot time on systems that have multiple devices that take many seconds to shut down (like certain NVMe drives). On one system tested, the shutdown time went from 11 minutes without this patch to 55 seconds with the patch.

Quoted 7 lines

I've successfully tested this out on a few systems, and noticing a very decent shutdown time on my nvme systems. I also like the current solution here, as the two-pass method was harder to follow.

So I think just remove the extra options that Christoph mentioned and always use the driver's preferred shutdown method, then this would all look good to me.

Yes, I have tested this patch on my systems and am greatly in favour of this instead of the two-pass version I was trying to make work. It is easy to understand and fixes the problem for my NVME issue.

Jeremy.

#9 lore raw

PATCH v6 4/4 nvme-pci: Make driver prefer asynchronous shutdown

In reply to Stuart Hayes

Set the driver default to prefer asynchronous shutdown.

---
 drivers/nvme/host/pci.c | 1 +
 1 file changed, 1 insertion(+)
drivers/nvme/host/pci.c +1 -0
index 710043086dff..df7bd88f50f2 100644
@@ -3530,6 +3530,7 @@ static struct pci_driver nvme_driver = {
 	.shutdown	= nvme_shutdown,
 	.driver		= {
 		.probe_type	= PROBE_PREFER_ASYNCHRONOUS,
+		.shutdown_type	= SHUTDOWN_PREFER_ASYNCHRONOUS,
 #ifdef CONFIG_PM_SLEEP
 		.pm		= &nvme_dev_pm_ops,
 #endif
Signature · 1 line
2.39.3