Skip to contentSkip to conversation

driver core: shut down devices asynchronously

PATCH Superseded Stuart Hayes 2 participants 2 messages Nov 28, 2023 – Nov 29, 2023 Driver core

v5 of 10 Superseded Latest is v10 →

lore
Share & tools
lore
#0 lore raw

PATCH v5 1/1 driver core: shut down devices asynchronously

Add code to shut down devices asynchronously, while ensuring that each device is shut down before its parent, and allowing devices that share a driver to be shutdown one at a time if necessary.

Add /sys/kernel/async_shutdown to allow user control of this feature:

off: shut down all devices synchronously (default)
on: shut down all devices asynchronously, unless disabled by the driver
(driver opt-out)
safe: shut down all devices synchronously, unless driver prefers async

	shutdown, or device has no driver or an unregistered driver (driver
	opt-in)

Add an async_shutdown sysfs attribute to drivers to allow users to see and change driver opt-in/opt-out, with values:

async: driver opt-in to async device shutdown (devices will be shut down
asynchronously if async_shutdown is "on" or "safe")
sync: driver opt-out of async device shutdown (devices will always be
shut down synchronously)
default: devices will be shutdown asynchronously if async_shutdown is "on"

This can dramatically reduce system shutdown/reboot time on systems that have multiple devices that take many seconds to shut down (like certain NVMe drives). On one system tested, the shutdown time went from 11 minutes without this patch to 55 seconds with the patch.

---
v1->v2: rewritten using kernel async code (suggested by Lukas Wunner) v2->v3: removed recursive functions to schedule children to be shutdown
before parents, since existing device_shutdown loop will already do this
v3->v4: bug fix (used "parent" not "dev->parent", in device_shutdown) v4->v5: change code to use cookies for synchronization rather than async
domains
allow async shutdown to be disabled via sysfs, and allow driver
opt-in or opt-out of async shutdown (when not disabled), with ability to control driver opt-in/opt-out via sysfs
---

drivers/base/base.h +12 -0
index eb4c0ace9242..8b87947a79b5 100644
@@ -11,6 +11,7 @@
  *
  */
 #include <linux/notifier.h>
+#include <linux/async.h>
 
 /**
  * struct subsys_private - structure to hold the private to the driver core portions of the bus_type/class structure.
@@ -75,12 +76,21 @@ static inline void subsys_put(struct subsys_private *sp)
 
 struct subsys_private *class_to_subsys(const struct class *class);
 
+/**
+ * struct driver_private - structure to hold the private to the driver core portions of the driver
+ * structure.
+ *
+ * @shutdown_cookie - cookie used to synchronize shutdown of devices used by this driver, used when
+ *                    async device shutdown is enabled system-wide, but not allowed for devices
+ *                    using this driver
+ */
 struct driver_private {
 	struct kobject kobj;
 	struct klist klist_devices;
 	struct klist_node knode_bus;
 	struct module_kobject *mkobj;
 	struct device_driver *driver;
+	async_cookie_t shutdown_cookie;
 };
 #define to_driver(obj) container_of(obj, struct driver_private, kobj)
 
@@ -97,6 +107,7 @@ struct driver_private {
  *	the device; typically because it depends on another driver getting
  *	probed first.
  * @async_driver - pointer to device driver awaiting probe via async_probe
+ * @shutdown_cookie - cookie used during async shutdown to ensure correct shutdown ordering.
  * @device - pointer back to the struct device that this structure is
  * associated with.
  * @dead - This device is currently either in the process of or has been
@@ -114,6 +125,7 @@ struct device_private {
 	struct list_head deferred_probe;
 	struct device_driver *async_driver;
 	char *deferred_probe_reason;
+	async_cookie_t shutdown_cookie;
 	struct device *device;
 	u8 dead:1;
 };
drivers/base/bus.c +47 -0

71 lines not on this page · raw message

drivers/base/core.c +180 -36

270 lines not on this page · raw message

include/linux/device/driver.h +8 -0
index 7738f458995f..f414c8a6f814 100644
@@ -48,6 +48,12 @@ enum probe_type {
 	PROBE_FORCE_SYNCHRONOUS,
 };
 
+enum shutdown_type {
+	SHUTDOWN_DEFAULT_STRATEGY,
+	SHUTDOWN_PREFER_ASYNCHRONOUS,
+	SHUTDOWN_FORCE_SYNCHRONOUS,
+};
+
 /**
  * struct device_driver - The basic device driver structure
  * @name:	Name of the device driver.
@@ -56,6 +62,7 @@ enum probe_type {
  * @mod_name:	Used for built-in modules.
  * @suppress_bind_attrs: Disables bind/unbind via sysfs.
  * @probe_type:	Type of the probe (synchronous or asynchronous) to use.
+ * @shutdown_type: Type of the shutdown (synchronous or asynchronous) to use.
  * @of_match_table: The open firmware table.
  * @acpi_match_table: The ACPI match table.
  * @probe:	Called to query the existence of a specific device,
@@ -102,6 +109,7 @@ struct device_driver {
 
 	bool suppress_bind_attrs;	/* disables bind/unbind via sysfs */
 	enum probe_type probe_type;
+	enum shutdown_type shutdown_type;
 
 	const struct of_device_id	*of_match_table;
 	const struct acpi_device_id	*acpi_match_table;
Signature · 1 line
2.39.3
#1 lore raw
kernel test robot noticed the following build warnings:

Hi Stuart,

kernel test robot noticed the following build warnings:

[auto build test WARNING on driver-core/driver-core-testing]
[also build test WARNING on driver-core/driver-core-next driver-core/driver-core-linus linus/master v6.7-rc3 next-20231128] [If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Stuart-Hayes/driver-core-shut-down-devices-asynchronously/20231129-055044
base:   driver-core/driver-core-testing
patch link:    https://lore.kernel.org/r/20231128210436.506510-1-stuart.w.hayes%40gmail.com
patch subject: [PATCH v5] driver core: shut down devices asynchronously
config: i386-randconfig-061-20231129 (https://download.01.org/0day-ci/archive/20231129/[email protected]/config)
compiler: gcc-7 (Ubuntu 7.5.0-6ubuntu2) 7.5.0
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20231129/[email protected]/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/

sparse warnings: (new ones prefixed by >>)

>> drivers/base/core.c:4782:1: sparse: sparse: symbol 'sd_domain' was not declared. Should it be static?
>> drivers/base/core.c:4783:16: sparse: sparse: symbol 'big_shutdown_cookie' was not declared. Should it be static?

drivers/base/core.c: note: in included file (through include/linux/mmzone.h, include/linux/gfp.h, include/linux/slab.h, ...):
include/linux/page-flags.h:242:46: sparse: sparse: self-comparison always evaluates to false

vim +/sd_domain +4782 drivers/base/core.c

4781

> 4782	ASYNC_DOMAIN(sd_domain);
> 4783	async_cookie_t big_shutdown_cookie;

4784

Signature · 2 lines
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki