shut down devices asynchronously
Subsystems (2)
v6 cover shut down devices asynchronously
This adds the ability for the kernel to shutdown devices asynchronously (this is V6).
It is in "safe" mode by default (where only devices whose drivers opt-in are shut down asynchronously), but it can be "on" (where only devices whose drivers opt-out are shut down synchronously), or "off" (legacy behavior).
This can dramatically reduce system shutdown/reboot time on systems that have multiple devices that take many seconds to shut down (like certain NVMe drives). On one system tested, the shutdown time went from 11 minutes without this patch to 55 seconds with the patch.
Changes from V5:
Separated into multiple patches to make review easier. Reworked some code to make it more readable
Made devices wait for consumers to shut down, not just children
(suggested by David Jeffery)
Changes from V4:
Change code to use cookies for synchronization rather than async domains Allow async shutdown to be disabled via sysfs, and allow driver opt-in or
opt-out of async shutdown (when not disabled), with ability to control driver opt-in/opt-out via sysfs
Changes from V3:
Bug fix (used "parent" not "dev->parent" in device_shutdown)
Changes from V2:
Removed recursive functions to schedule children to be shutdown before
parents, since existing device_shutdown loop will already do this
Changes from V1:
Rewritten using kernel async code (suggested by Lukas Wunner)
Stuart Hayes (4):
driver core: don't always lock parent in shutdown driver core: separate function to shutdown one device driver core: shut down devices asynchronously
nvme-pci: Make driver prefer asynchronous shutdown
drivers/base/base.h | 3 + drivers/base/bus.c | 47 +++++++++ drivers/base/core.c | 185 +++++++++++++++++++++++++++++----- drivers/nvme/host/pci.c | 1 + include/linux/device/driver.h | 8 ++ 5 files changed, 218 insertions(+), 26 deletions(-)
Signature · 1 line
2.39.3
PATCH v6 1/4 driver core: don't always lock parent in shutdown
Don't lock a parent device unless it is needed in device_shutdown. This is in preparation for making device shutdown asynchronous, when it will be needed to allow children of a common parent to shut down simultaneously.
--- drivers/base/core.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/base/core.c +2 -2
@@ -4856,7 +4856,7 @@ void device_shutdown(void)spin_unlock (&devices_kset->list_lock);/* hold lock to avoid race with probe/release */ -if (parent) +if (parent && dev->bus && dev->bus->need_parent_lock)device_lock (parent);device_lock (dev);
@@ -4880,7 +4880,7 @@ void device_shutdown(void) }device_unlock (dev); -if (parent) +if (parent && dev->bus && dev->bus->need_parent_lock)device_unlock (parent);put_device (dev);
Signature · 1 line
2.39.3
PATCH v6 2/4 driver core: separate function to shutdown one device
In reply to Stuart HayesMake a separate function for the part of device_shutdown() that does the shutown for a single device. This is in preparation for making device shutdown asynchronous.
--- drivers/base/core.c | 66 ++++++++++++++++++++++++--------------------- 1 file changed, 36 insertions(+), 30 deletions(-)
drivers/base/core.c +36 -30
@@ -4819,6 +4819,41 @@ int device_change_owner(struct device *dev, kuid_t kuid, kgid_t kgid) }EXPORT_SYMBOL_GPL (device_change_owner); +static void shutdown_one_device (struct device *dev) +{ +/* hold lock to avoid race with probe/release */ +if (dev->parent && dev->bus && dev->bus->need_parent_lock) +device_lock (dev->parent); +device_lock (dev); + +/* Don't allow any more runtime suspends */ +pm_runtime_get_noresume (dev); +pm_runtime_barrier (dev); + +if (dev->class && dev->class->shutdown_pre) { +if (initcall_debug) +dev_info (dev,"shutdown_pre\n" ); + dev->class->shutdown_pre (dev); + } +if (dev->bus && dev->bus->shutdown) { +if (initcall_debug) +dev_info (dev,"shutdown\n" ); + dev->bus->shutdown (dev); + }else if (dev->driver && dev->driver->shutdown) { +if (initcall_debug) +dev_info (dev,"shutdown\n" ); + dev->driver->shutdown (dev); + } + +device_unlock (dev); +if (dev->parent && dev->bus && dev->bus->need_parent_lock) +device_unlock (dev->parent); + +put_device (dev); +if (dev->parent) +put_device (dev->parent); +} +/** * device_shutdown - call ->shutdown() on each device to shutdown. */
@@ -4855,36 +4890,7 @@ void device_shutdown(void)list_del_init (&dev->kobj.entry);spin_unlock (&devices_kset->list_lock); -/* hold lock to avoid race with probe/release */ -if (parent && dev->bus && dev->bus->need_parent_lock) -device_lock (parent); -device_lock (dev); - -/* Don't allow any more runtime suspends */ -pm_runtime_get_noresume (dev); -pm_runtime_barrier (dev); - -if (dev->class && dev->class->shutdown_pre) { -if (initcall_debug) -dev_info (dev,"shutdown_pre\n" ); - dev->class->shutdown_pre (dev); - } -if (dev->bus && dev->bus->shutdown) { -if (initcall_debug) -dev_info (dev,"shutdown\n" ); - dev->bus->shutdown (dev); - }else if (dev->driver && dev->driver->shutdown) { -if (initcall_debug) -dev_info (dev,"shutdown\n" ); - dev->driver->shutdown (dev); - } - -device_unlock (dev); -if (parent && dev->bus && dev->bus->need_parent_lock) -device_unlock (parent); - -put_device (dev); -put_device (parent); +shutdown_one_device (dev);spin_lock (&devices_kset->list_lock); }
Signature · 1 line
2.39.3
PATCH v6 3/4 driver core: shut down devices asynchronously
In reply to Stuart HayesAdd code to shut down devices asynchronously, while ensuring that each device is shut down before its parents & suppliers, and allowing devices that share a driver to be shutdown one at a time if necessary.
Add /sys/kernel/async_shutdown to allow user control of this feature:
safe: shut down all devices synchronously, unless driver prefers async
shutdown (driver opt-in) (default)
on: shut down all devices asynchronously, unless disabled by the driver
(driver opt-out)
off: shut down all devices synchronously
Add async_shutdown to struct device_driver, and expose it via sysfs. This will be used to view or change driver opt-in/opt-out of asynchronous shutdown, if it is globally enabled.
async: driver opt-in to async device shutdown (devices will be shut down
asynchronously if async_shutdown is "on" or "safe")
sync: driver opt-out of async device shutdown (devices will always be
shut down synchronously)
default: devices will be shutdown asynchronously if async_shutdown is "on"
This can dramatically reduce system shutdown/reboot time on systems that have multiple devices that take many seconds to shut down (like certain NVMe drives). On one system tested, the shutdown time went from 11 minutes without this patch to 55 seconds with the patch.
--- drivers/base/base.h | 3 + drivers/base/bus.c | 47 +++++++++++++ drivers/base/core.c | 129 +++++++++++++++++++++++++++++++++- include/linux/device/driver.h | 8 +++ 4 files changed, 186 insertions(+), 1 deletion(-)
drivers/base/base.h +3 -0
@@ -10,6 +10,7 @@* shared outside of the drivers/base/ directory. * */ +#include <linux/async.h> #include <linux/notifier.h> /**
@@ -97,6 +98,7 @@ struct driver_private {* the device; typically because it depends on another driver getting * probed first. * @async_driver - pointer to device driver awaiting probe via async_probe +* @shutdown_after - used during async shutdown to ensure correct shutdown ordering. * @device - pointer back to the struct device that this structure is * associated with. * @dead - This device is currently either in the process of or has been
@@ -114,6 +116,7 @@ struct device_private {struct list_head deferred_probe;struct device_driver *async_driver;char *deferred_probe_reason; +async_cookie_t shutdown_after;struct device *device; u8 dead:1; };
drivers/base/bus.c +47 -0
65 lines not on this page · raw message
drivers/base/core.c +128 -1
175 lines not on this page · raw message
include/linux/device/driver.h +8 -0
@@ -48,6 +48,12 @@ enum probe_type { PROBE_FORCE_SYNCHRONOUS, }; +enum shutdown_type { +SHUTDOWN_DEFAULT_STRATEGY , +SHUTDOWN_PREFER_ASYNCHRONOUS , +SHUTDOWN_FORCE_SYNCHRONOUS , +}; +/** * struct device_driver - The basic device driver structure * @name: Name of the device driver.
@@ -56,6 +62,7 @@ enum probe_type {* @mod_name: Used for built-in modules. * @suppress_bind_attrs: Disables bind/unbind via sysfs. * @probe_type: Type of the probe (synchronous or asynchronous) to use. +* @shutdown_type: Type of the shutdown (synchronous or asynchronous) to use. * @of_match_table: The open firmware table. * @acpi_match_table: The ACPI match table. * @probe: Called to query the existence of a specific device,
@@ -102,6 +109,7 @@ struct device_driver {bool suppress_bind_attrs;/* disables bind/unbind via sysfs */ enum probe_type probe_type; +enum shutdown_type shutdown_type;const struct of_device_id *of_match_table;const struct acpi_device_id *acpi_match_table;
Signature · 1 line
2.39.3
Christoph Hellwig
maintainerRe: [PATCH v6 3/4] driver core: shut down devices asynchronously
Quoted 7 lines · Stuart Hayes
Add
/sys/kernel/async_shutdownto allow user control of this feature:safe: shut down all devices synchronously, unless driver prefers async
shutdown (driver opt-in) (default)
on: shut down all devices asynchronously, unless disabled by the driver
(driver opt-out)
off: shut down all devices synchronously
The on option seems very odd. IMHO safe is the only really sensible option, and maybe we have to support off as a bandaid due to userspace behavior dependent on synchronous shutdown, but I'd rather try even without that first.
Re: [PATCH v6 3/4] driver core: shut down devices asynchronously
Quoted 9 lines · Christoph Hellwig
Quoted 7 lines · Stuart Hayes
Add
/sys/kernel/async_shutdownto allow user control of this feature:safe: shut down all devices synchronously, unless driver prefers async
shutdown (driver opt-in) (default)
on: shut down all devices asynchronously, unless disabled by the driver
(driver opt-out)
off: shut down all devices synchronously
Quoted 5 lines
The on option seems very odd. IMHO safe is the only really sensible option, and maybe we have to support off as a bandaid due to userspace behavior dependent on synchronous shutdown, but I'd rather try even without that first.
I added the option because of comments from Greg K-H on the v4 submission of this patch--see
I thought it would be nice to have the option for testing, even if it gets removed later, but I'll certainly remove it now if necessary.
Greg Kroah-Hartman
maintainerRe: [PATCH v6 3/4] driver core: shut down devices asynchronously
Quoted 18 lines · stuart hayes
Quoted 9 lines · Christoph Hellwig
Quoted 7 lines · Stuart Hayes
Add
/sys/kernel/async_shutdownto allow user control of this feature:safe: shut down all devices synchronously, unless driver prefers async
shutdown (driver opt-in) (default)
on: shut down all devices asynchronously, unless disabled by the driver
(driver opt-out)
off: shut down all devices synchronouslyQuoted 5 lines
The on option seems very odd. IMHO safe is the only really sensible option, and maybe we have to support off as a bandaid due to userspace behavior dependent on synchronous shutdown, but I'd rather try even without that first.
Quoted 7 lines
I added the option because of comments from Greg K-H on the v4 submission of this patch--see
I thought it would be nice to have the option for testing, even if it gets removed later, but I'll certainly remove it now if necessary.
Opt-in is the requirement here, that's all I asked for. The "on" can probably be removed, and by doing that, you can make this option simpler as well.
thanks,
greg k-h
Keith Busch
maintainerRe: [PATCH v6 3/4] driver core: shut down devices asynchronously
In reply to Stuart HayesQuoted 22 lines · Stuart Hayes
Add code to shut down devices asynchronously, while ensuring that each device is shut down before its parents & suppliers, and allowing devices that share a driver to be shutdown one at a time if necessary.
Add
/sys/kernel/async_shutdownto allow user control of this feature:safe: shut down all devices synchronously, unless driver prefers async
shutdown (driver opt-in) (default)
on: shut down all devices asynchronously, unless disabled by the driver
(driver opt-out)
off: shut down all devices synchronouslyAdd
async_shutdownto structdevice_driver, and expose it via sysfs. This will be used to view or change driver opt-in/opt-out of asynchronous shutdown, if it is globally enabled.async: driver opt-in to async device shutdown (devices will be shut down
asynchronously ifasync_shutdownis "on" or "safe")
sync: driver opt-out of async device shutdown (devices will always be
shut down synchronously)
default: devices will be shutdown asynchronously ifasync_shutdownis "on"
Quoted 4 lines
This can dramatically reduce system shutdown/reboot time on systems that have multiple devices that take many seconds to shut down (like certain NVMe drives). On one system tested, the shutdown time went from 11 minutes without this patch to 55 seconds with the patch.
I've successfully tested this out on a few systems, and noticing a very decent shutdown time on my nvme systems. I also like the current solution here, as the two-pass method was harder to follow.
So I think just remove the extra options that Christoph mentioned and always use the driver's preferred shutdown method, then this would all look good to me.
Re: [PATCH v6 3/4] driver core: shut down devices asynchronously
Quoted 28 lines · Keith Busch
Quoted 22 lines · Stuart Hayes
Add code to shut down devices asynchronously, while ensuring that each device is shut down before its parents & suppliers, and allowing devices that share a driver to be shutdown one at a time if necessary.
Add
/sys/kernel/async_shutdownto allow user control of this feature:safe: shut down all devices synchronously, unless driver prefers async
shutdown (driver opt-in) (default)
on: shut down all devices asynchronously, unless disabled by the driver
(driver opt-out)
off: shut down all devices synchronouslyAdd
async_shutdownto structdevice_driver, and expose it via sysfs. This will be used to view or change driver opt-in/opt-out of asynchronous shutdown, if it is globally enabled.async: driver opt-in to async device shutdown (devices will be shut down
asynchronously ifasync_shutdownis "on" or "safe")
sync: driver opt-out of async device shutdown (devices will always be
shut down synchronously)
default: devices will be shutdown asynchronously ifasync_shutdownis "on"Quoted 4 lines
This can dramatically reduce system shutdown/reboot time on systems that have multiple devices that take many seconds to shut down (like certain NVMe drives). On one system tested, the shutdown time went from 11 minutes without this patch to 55 seconds with the patch.
Quoted 7 lines
I've successfully tested this out on a few systems, and noticing a very decent shutdown time on my nvme systems. I also like the current solution here, as the two-pass method was harder to follow.
So I think just remove the extra options that Christoph mentioned and always use the driver's preferred shutdown method, then this would all look good to me.
Yes, I have tested this patch on my systems and am greatly in favour of this instead of the two-pass version I was trying to make work. It is easy to understand and fixes the problem for my NVME issue.
Jeremy.
PATCH v6 4/4 nvme-pci: Make driver prefer asynchronous shutdown
In reply to Stuart HayesSet the driver default to prefer asynchronous shutdown.
--- drivers/nvme/host/pci.c | 1 + 1 file changed, 1 insertion(+)
drivers/nvme/host/pci.c +1 -0
@@ -3530,6 +3530,7 @@ static struct pci_driver nvme_driver = { .shutdown = nvme_shutdown, .driver = { .probe_type = PROBE_PREFER_ASYNCHRONOUS, + .shutdown_type = SHUTDOWN_PREFER_ASYNCHRONOUS,#ifdef CONFIG_PM_SLEEP .pm = &nvme_dev_pm_ops,#endif
Signature · 1 line
2.39.3