Skip to contentSkip to conversation

driver core: Add device link related sysfs files

PATCH Superseded Saravana Kannan 2 participants 7 messages May 20, 2020 – May 21, 2020 Driver core

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

v2 cover driver core: Add device link related sysfs files

With fw_devlink and with sync_state() callback features, there's a lot of device/device link related information that's not available in sysfs.

Exposing these details to user space can be very useful in understanding suspend/resume issues, runtime pm issues, probing issues, figuring out the modules that'd be needed for first stage init, etc. In fact, an earlier verion of this series was very helpful in debugging and validating the recent memory leak fix[1].

This series combines combines a bunch of patches I've sent before.

I'm aware that I haven't added documentation for patch 1/2. I'm waiting on review to make sure the file location, name and values don't change before I add the documentation.

This series is based on driver-core-next and [1] cherry-picked on top of it.

[1] - https://lore.kernel.org/lkml/[email protected]/

v1->v2:
Patch 1/4
- New patch
Patch 2/4
- Fixed the warnings I saw before that were related to incorrect sysfs removal code when a device link is deleted.
- Fixed error handling in device_link_add()
- Split up flags into more meaningful files.
- Added status file.
Patch 3/4
- Fixed error handling that Greg pointed out before. Patch 4/4
- New patch

Saravana Kannan (4):
driver core: Remove unnecessary is_fwnode_dev variable in device_add() driver core: Expose device link details in sysfs
driver core: Add state_synced sysfs file for devices that support it driver core: Add waiting_for_supplier sysfs file for devices

 .../ABI/testing/sysfs-devices-state_synced    |  24 ++
 .../sysfs-devices-waiting_for_supplier        |  17 ++
 drivers/base/core.c                           | 249 ++++++++++++++++--
 drivers/base/dd.c                             |  22 ++
 include/linux/device.h                        |  58 ++--
 5 files changed, 326 insertions(+), 44 deletions(-)
 create mode 100644 Documentation/ABI/testing/sysfs-devices-state_synced
 create mode 100644 Documentation/ABI/testing/sysfs-devices-waiting_for_supplier
Signature · 1 line
2.26.2.761.g0e0b3e54be-goog
#1 lore raw

PATCH v2 1/4 driver core: Remove unnecessary is_fwnode_dev variable in device_add()

That variable is no longer necessary. Remove it and also fix a minor typo in comments.

---
 drivers/base/core.c | 12 ++++--------
 1 file changed, 4 insertions(+), 8 deletions(-)
drivers/base/core.c +4 -8
index f804e561e0a2..6dbee5885abb 100644
@@ -2543,7 +2543,6 @@ int device_add(struct device *dev)
 	struct class_interface *class_intf;
 	int error = -EINVAL;
 	struct kobject *glue_dir = NULL;
-	bool is_fwnode_dev = false;
 
 	dev = get_device(dev);
 	if (!dev)
@@ -2641,11 +2640,6 @@ int device_add(struct device *dev)
 
 	kobject_uevent(&dev->kobj, KOBJ_ADD);
 
-	if (dev->fwnode && !dev->fwnode->dev) {
-		dev->fwnode->dev = dev;
-		is_fwnode_dev = true;
-	}
-
 	/*
 	 * Check if any of the other devices (consumers) have been waiting for
 	 * this device (supplier) to be added so that they can create a device
@@ -2654,12 +2648,14 @@ int device_add(struct device *dev)
 	 * This needs to happen after device_pm_add() because device_link_add()
 	 * requires the supplier be registered before it's called.
 	 *
-	 * But this also needs to happe before bus_probe_device() to make sure
+	 * But this also needs to happen before bus_probe_device() to make sure
 	 * waiting consumers can link to it before the driver is bound to the
 	 * device and the driver sync_state callback is called for this device.
 	 */
-	if (is_fwnode_dev)
+	if (dev->fwnode && !dev->fwnode->dev) {
+		dev->fwnode->dev = dev;
 		fw_devlink_link_device(dev);
+	}
 
 	bus_probe_device(dev);
 	if (parent)
Signature · 1 line
2.26.2.761.g0e0b3e54be-goog
#2 lore raw

PATCH v2 2/4 driver core: Expose device link details in sysfs

In reply to Saravana Kannan

It's helpful to be able to look at device link details from sysfs. So, expose it in sysfs.

Say device-A is supplier of device-B. These are the additional files this patch would create:

/sys/class/devlink/device-A:device-B/

	auto_remove_on
	consumer/ -> .../device-B/
	runtime_pm
	status
	supplier/ -> .../device-A/
	sync_state_only

/sys/devices/.../device-A/

	consumer:device-B/ -> /sys/class/devlink/device-A:device-B/

/sys/devices/.../device-B/

	supplier:device-A/ -> /sys/class/devlink/device-A:device-B/

That way:
To get a list of all the device link in the system:
ls /sys/class/devlink/

To get the consumer names and links of a device:
ls -d /sys/devices/.../device-X/consumer:*

To get the supplier names and links of a device:
ls -d /sys/devices/.../device-X/supplier:*

---
 drivers/base/core.c    | 211 +++++++++++++++++++++++++++++++++++++++--
 include/linux/device.h |  58 +++++------
 2 files changed, 233 insertions(+), 36 deletions(-)
drivers/base/core.c +203 -8
index 6dbee5885abb..3304ea1a2604 100644
@@ -235,6 +235,186 @@ void device_pm_move_to_tail(struct device *dev)
 	device_links_read_unlock(idx);
 }
 
+#define to_devlink(dev)	container_of((dev), struct device_link, link_dev)
+
+static ssize_t status_show(struct device *dev,
+			  struct device_attribute *attr, char *buf)
+{
+	char *status;
+
+	switch (to_devlink(dev)->status) {
+	case DL_STATE_NONE:
+		status = "not tracked"; break;
+	case DL_STATE_DORMANT:
+		status = "dormant"; break;
+	case DL_STATE_AVAILABLE:
+		status = "available"; break;
+	case DL_STATE_CONSUMER_PROBE:
+		status = "consumer probing"; break;
+	case DL_STATE_ACTIVE:
+		status = "active"; break;
+	case DL_STATE_SUPPLIER_UNBIND:
+		status = "supplier unbinding"; break;
+	default:
+		status = "unknown"; break;
+	}
+	return sprintf(buf, "%s\n", status);
+}
+static DEVICE_ATTR_RO(status);
+
+static ssize_t auto_remove_on_show(struct device *dev,
+				   struct device_attribute *attr, char *buf)
+{
+	struct device_link *link = to_devlink(dev);
+	char *str;
+
+	if (link->flags & DL_FLAG_AUTOREMOVE_SUPPLIER)
+		str = "supplier unbind";
+	else if (link->flags & DL_FLAG_AUTOREMOVE_CONSUMER)
+		str = "consumer unbind";
+	else
+		str = "never";
+
+	return sprintf(buf, "%s\n", str);
+}
+static DEVICE_ATTR_RO(auto_remove_on);
+
+static ssize_t runtime_pm_show(struct device *dev,
+			       struct device_attribute *attr, char *buf)
+{
+	struct device_link *link = to_devlink(dev);
+
+	return sprintf(buf, "%d\n", !!(link->flags & DL_FLAG_PM_RUNTIME));
+}
+static DEVICE_ATTR_RO(runtime_pm);
+
+static ssize_t sync_state_only_show(struct device *dev,
+				    struct device_attribute *attr, char *buf)
+{
+	struct device_link *link = to_devlink(dev);
+
+	return sprintf(buf, "%d\n", !!(link->flags & DL_FLAG_SYNC_STATE_ONLY));
+}
+static DEVICE_ATTR_RO(sync_state_only);
+
+static struct attribute *devlink_attrs[] = {
+	&dev_attr_status.attr,
+	&dev_attr_auto_remove_on.attr,
+	&dev_attr_runtime_pm.attr,
+	&dev_attr_sync_state_only.attr,
+	NULL,
+};
+ATTRIBUTE_GROUPS(devlink);
+
+static void devlink_dev_release(struct device *dev)
+{
+	kfree(to_devlink(dev));
+}
+
+static struct class devlink_class = {
+	.name = "devlink",
+	.owner = THIS_MODULE,
+	.dev_groups = devlink_groups,
+	.dev_release = devlink_dev_release,
+};
+
+static int devlink_add_symlinks(struct device *dev,
+				struct class_interface *class_intf)
+{
+	int ret;
+	size_t len;
+	struct device_link *link = to_devlink(dev);
+	struct device *sup = link->supplier;
+	struct device *con = link->consumer;
+	char *buf;
+
+	len = max(strlen(dev_name(sup)), strlen(dev_name(con)));
+	len += strlen("supplier:") + 1;
+	buf = kzalloc(len, GFP_KERNEL);
+	if (!buf)
+		return -ENOMEM;
+
+	ret = sysfs_create_link(&link->link_dev.kobj, &sup->kobj, "supplier");
+	if (ret)
+		goto out;
+
+	ret = sysfs_create_link(&link->link_dev.kobj, &con->kobj, "consumer");
+	if (ret)
+		goto err_con;
+
+	snprintf(buf, len, "consumer:%s", dev_name(con));
+	ret = sysfs_create_link(&sup->kobj, &link->link_dev.kobj, buf);
+	if (ret)
+		goto err_con_dev;
+
+	snprintf(buf, len, "supplier:%s", dev_name(sup));
+	ret = sysfs_create_link(&con->kobj, &link->link_dev.kobj, buf);
+	if (ret)
+		goto err_sup_dev;
+
+	goto out;
+
+err_sup_dev:
+	snprintf(buf, len, "consumer:%s", dev_name(con));
+	sysfs_remove_link(&sup->kobj, buf);
+err_con_dev:
+	sysfs_remove_link(&link->link_dev.kobj, "consumer");
+err_con:
+	sysfs_remove_link(&link->link_dev.kobj, "supplier");
+out:
+	kfree(buf);
+	return ret;
+}
+
+static void devlink_remove_symlinks(struct device *dev,
+				   struct class_interface *class_intf)
+{
+	struct device_link *link = to_devlink(dev);
+	size_t len;
+	struct device *sup = link->supplier;
+	struct device *con = link->consumer;
+	char *buf;
+
+	sysfs_remove_link(&link->link_dev.kobj, "consumer");
+	sysfs_remove_link(&link->link_dev.kobj, "supplier");
+
+	len = max(strlen(dev_name(sup)), strlen(dev_name(con)));
+	len += strlen("supplier:") + 1;
+	buf = kzalloc(len, GFP_KERNEL);
+	if (!buf) {
+		WARN(1, "Unable to properly free device link symlinks!\n");
+		return;
+	}
+
+	snprintf(buf, len, "supplier:%s", dev_name(sup));
+	sysfs_remove_link(&con->kobj, buf);
+	snprintf(buf, len, "consumer:%s", dev_name(con));
+	sysfs_remove_link(&sup->kobj, buf);
+	kfree(buf);
+}
+
+static struct class_interface devlink_class_intf = {
+	.class = &devlink_class,
+	.add_dev = devlink_add_symlinks,
+	.remove_dev = devlink_remove_symlinks,
+};
+
+static int __init devlink_class_init(void)
+{
+	int ret;
+
+	ret = class_register(&devlink_class);
+	if (ret)
+		return ret;
+
+	ret = class_interface_register(&devlink_class_intf);
+	if (ret)
+		class_unregister(&devlink_class);
+
+	return ret;
+}
+postcore_initcall(devlink_class_init);
+
 #define DL_MANAGED_LINK_FLAGS (DL_FLAG_AUTOREMOVE_CONSUMER | \
 			       DL_FLAG_AUTOREMOVE_SUPPLIER | \
 			       DL_FLAG_AUTOPROBE_CONSUMER  | \
@@ -405,13 +585,6 @@ struct device_link *device_link_add(struct device *consumer,
 
 	refcount_set(&link->rpm_active, 1);
 
-	if (flags & DL_FLAG_PM_RUNTIME) {
-		if (flags & DL_FLAG_RPM_ACTIVE)
-			refcount_inc(&link->rpm_active);
-
-		pm_runtime_new_link(consumer);
-	}
-
 	get_device(supplier);
 	link->supplier = supplier;
 	INIT_LIST_HEAD(&link->s_node);
@@ -421,6 +594,25 @@ struct device_link *device_link_add(struct device *consumer,
 	link->flags = flags;
 	kref_init(&link->kref);
 
+	link->link_dev.class = &devlink_class;
+	device_set_pm_not_required(&link->link_dev);
+	dev_set_name(&link->link_dev, "%s:%s",
+		     dev_name(supplier), dev_name(consumer));
+	if (device_register(&link->link_dev)) {
+		put_device(consumer);
+		put_device(supplier);
+		kfree(link);
+		link = NULL;
+		goto out;
+	}
+
+	if (flags & DL_FLAG_PM_RUNTIME) {
+		if (flags & DL_FLAG_RPM_ACTIVE)
+			refcount_inc(&link->rpm_active);
+
+		pm_runtime_new_link(consumer);
+	}
+
 	/* Determine the initial link state. */
 	if (flags & DL_FLAG_STATELESS)
 		link->status = DL_STATE_NONE;
@@ -543,7 +735,7 @@ static void device_link_free(struct device_link *link)
 
 	put_device(link->consumer);
 	put_device(link->supplier);
-	kfree(link);
+	device_unregister(&link->link_dev);
 }
 
 #ifdef CONFIG_SRCU
@@ -1139,6 +1331,9 @@ static void device_links_purge(struct device *dev)
 {
 	struct device_link *link, *ln;
 
+	if (dev->class == &devlink_class)
+		return;
+
 	mutex_lock(&wfs_lock);
 	list_del(&dev->links.needs_suppliers);
 	mutex_unlock(&wfs_lock);
include/linux/device.h +30 -28

70 lines not on this page · raw message

Signature · 1 line
2.26.2.761.g0e0b3e54be-goog
#3 lore raw

PATCH v2 3/4 driver core: Add state_synced sysfs file for devices that support it

In reply to Saravana Kannan

This can be used to check if a device supports sync_state() callbacks and therefore keeps resources left on by the bootloader enabled till all its consumers have probed.

This can also be used to check if sync_state() has been called for a device or whether it is still trying to keep resources enabled because they were left enabled by the bootloader and all its consumers haven't probed yet.

---
 .../ABI/testing/sysfs-devices-state_synced    | 24 +++++++++++++++++++
 drivers/base/dd.c                             | 22 +++++++++++++++++
 2 files changed, 46 insertions(+)
 create mode 100644 Documentation/ABI/testing/sysfs-devices-state_synced
Documentation/ABI/testing/sysfs-devices-state_synced new file +24 -0
new file mode 100644
index 000000000000..0c922d7d02fc
@@ -0,0 +1,24 @@
+What:		/sys/devices/.../state_synced
+Date:		May 2020
+Contact:	Saravana Kannan <[email protected]>
+Description:
+		The /sys/devices/.../state_synced attribute is only present for
+		devices whose bus types or driver provides the .sync_state()
+		callback. The number read from it (0 or 1) reflects the value
+		of the device's 'state_synced' field. A value of 0 means the
+		.sync_state() callback hasn't been called yet. A value of 1
+		means the .sync_state() callback has been called.
+
+		Generally, if a device has sync_state() support and has some of
+		the resources it provides enabled at the time the kernel starts
+		(Eg: enabled by hardware reset or bootloader or anything that
+		run before the kernel starts), then it'll keep those resources
+		enabled and in a state that's compatible with the state they
+		were in at the start of the kernel. The device will stop doing
+		this only when the sync_state() callback has been called --
+		which happens only when all its consumer devices are registered
+		and have probed successfully. Resources that were left disabled
+		at the time the kernel starts are not affected or limited in
+		any way by sync_state() callbacks.
+
+
drivers/base/dd.c +22 -0
index 9a1d940342ac..7c04b88daac3 100644
@@ -463,6 +463,18 @@ static void driver_deferred_probe_add_trigger(struct device *dev,
 		driver_deferred_probe_trigger();
 }
 
+static ssize_t state_synced_show(struct device *dev,
+				 struct device_attribute *attr, char *buf)
+{
+	bool val;
+
+	device_lock(dev);
+	val = dev->state_synced;
+	device_unlock(dev);
+	return sprintf(buf, "%u\n", val);
+}
+static DEVICE_ATTR_RO(state_synced);
+
 static int really_probe(struct device *dev, struct device_driver *drv)
 {
 	int ret = -EPROBE_DEFER;
@@ -536,9 +548,16 @@ static int really_probe(struct device *dev, struct device_driver *drv)
 		goto dev_groups_failed;
 	}
 
+	if (dev_has_sync_state(dev) &&
+	    device_create_file(dev, &dev_attr_state_synced)) {
+		dev_err(dev, "state_synced sysfs add failed\n");
+		goto dev_sysfs_state_synced_failed;
+	}
+
 	if (test_remove) {
 		test_remove = false;
 
+		device_remove_file(dev, &dev_attr_state_synced);
 		device_remove_groups(dev, drv->dev_groups);
 
 		if (dev->bus->remove)
@@ -568,6 +587,8 @@ static int really_probe(struct device *dev, struct device_driver *drv)
 		 drv->bus->name, __func__, dev_name(dev), drv->name);
 	goto done;
 
+dev_sysfs_state_synced_failed:
+	device_remove_groups(dev, drv->dev_groups);
 dev_groups_failed:
 	if (dev->bus->remove)
 		dev->bus->remove(dev);
@@ -1105,6 +1126,7 @@ static void __device_release_driver(struct device *dev, struct device *parent)
 
 		pm_runtime_put_sync(dev);
 
+		device_remove_file(dev, &dev_attr_state_synced);
 		device_remove_groups(dev, drv->dev_groups);
 
 		if (dev->bus && dev->bus->remove)
Signature · 1 line
2.26.2.761.g0e0b3e54be-goog
#4 lore raw

PATCH v2 4/4 driver core: Add waiting_for_supplier sysfs file for devices

In reply to Saravana Kannan

This would be useful to check if a device is not probing because it's waiting for a supplier to be added and then linked to before it can probe.

To reduce sysfs clutter, this file is added only if it can ever be 1. So, if fw_devlink is disabled or set to permissive, this file is not added. Also, this file is removed once the device probes as it's no longer relevant.

---
 .../sysfs-devices-waiting_for_supplier        | 17 ++++++++++++
 drivers/base/core.c                           | 26 +++++++++++++++++++
 2 files changed, 43 insertions(+)
 create mode 100644 Documentation/ABI/testing/sysfs-devices-waiting_for_supplier
Documentation/ABI/testing/sysfs-devices-waiting_for_supplier new file +17 -0
new file mode 100644
index 000000000000..59d073d20db6
@@ -0,0 +1,17 @@
+What:		/sys/devices/.../waiting_for_supplier
+Date:		May 2020
+Contact:	Saravana Kannan <[email protected]>
+Description:
+		The /sys/devices/.../waiting_for_supplier attribute is only
+		present when fw_devlink kernel command line option is enabled
+		and is set to something stricter than "permissive".  It is
+		removed once a device probes successfully (because the
+		information is no longer relevant). The number read from it (0
+		or 1) reflects whether the device is waiting for one or more
+		suppliers to be added and then linked to using device links
+		before the device can probe.
+
+		A value of 0 means the device is not waiting for any suppliers
+		to be added before it can probe.  A value of 1 means the device
+		is waiting for one or more suppliers to be added before it can
+		probe.
drivers/base/core.c +26 -0
index 3304ea1a2604..83a3e0b62ce3 100644
@@ -1031,6 +1031,22 @@ static void device_link_drop_managed(struct device_link *link)
 	kref_put(&link->kref, __device_link_del);
 }
 
+static ssize_t waiting_for_supplier_show(struct device *dev,
+					 struct device_attribute *attr,
+					 char *buf)
+{
+	bool val;
+
+	device_lock(dev);
+	mutex_lock(&wfs_lock);
+	val = !list_empty(&dev->links.needs_suppliers)
+	      && dev->links.need_for_probe;
+	mutex_unlock(&wfs_lock);
+	device_unlock(dev);
+	return sprintf(buf, "%u\n", val);
+}
+static DEVICE_ATTR_RO(waiting_for_supplier);
+
 /**
  * device_links_driver_bound - Update device links after probing its driver.
  * @dev: Device to update the links for.
@@ -1055,6 +1071,7 @@ void device_links_driver_bound(struct device *dev)
 	mutex_lock(&wfs_lock);
 	list_del_init(&dev->links.needs_suppliers);
 	mutex_unlock(&wfs_lock);
+	device_remove_file(dev, &dev_attr_waiting_for_supplier);
 
 	device_links_write_lock();
 
@@ -2124,8 +2141,16 @@ static int device_add_attrs(struct device *dev)
 			goto err_remove_dev_groups;
 	}
 
+	if (fw_devlink_flags && !fw_devlink_is_permissive()) {
+		error = device_create_file(dev, &dev_attr_waiting_for_supplier);
+		if (error)
+			goto err_remove_dev_online;
+	}
+
 	return 0;
 
+ err_remove_dev_online:
+	device_remove_file(dev, &dev_attr_online);
  err_remove_dev_groups:
 	device_remove_groups(dev, dev->groups);
  err_remove_type_groups:
@@ -2143,6 +2168,7 @@ static void device_remove_attrs(struct device *dev)
 	struct class *class = dev->class;
 	const struct device_type *type = dev->type;
 
+	device_remove_file(dev, &dev_attr_waiting_for_supplier);
 	device_remove_file(dev, &dev_attr_online);
 	device_remove_groups(dev, dev->groups);
 
Signature · 1 line
2.26.2.761.g0e0b3e54be-goog
#5 lore raw
In reply to Saravana Kannan
Quoted 9 lines · Saravana Kannan

With fw_devlink and with sync_state() callback features, there's a lot of device/device link related information that's not available in sysfs.

Exposing these details to user space can be very useful in understanding suspend/resume issues, runtime pm issues, probing issues, figuring out the modules that'd be needed for first stage init, etc. In fact, an earlier verion of this series was very helpful in debugging and validating the recent memory leak fix[1].

Quoted 5 lines

This series combines combines a bunch of patches I've sent before.

I'm aware that I haven't added documentation for patch 1/2. I'm waiting on review to make sure the file location, name and values don't change before I add the documentation.

You mean patch 2/4, right?

Quoted 2 lines

This series is based on driver-core-next and [1] cherry-picked on top of it.

I've taken patch 1 now, as it was easy to review :)

I'll wait for the next series with the documentation update before going further.

thanks,

greg k-h

#6 lore raw
Quoted 11 lines · Greg Kroah-Hartman
Quoted 9 lines · Saravana Kannan

With fw_devlink and with sync_state() callback features, there's a lot of device/device link related information that's not available in sysfs.

Exposing these details to user space can be very useful in understanding suspend/resume issues, runtime pm issues, probing issues, figuring out the modules that'd be needed for first stage init, etc. In fact, an earlier verion of this series was very helpful in debugging and validating the recent memory leak fix[1].

Quoted 7 lines
Quoted 5 lines

This series combines combines a bunch of patches I've sent before.

I'm aware that I haven't added documentation for patch 1/2. I'm waiting on review to make sure the file location, name and values don't change before I add the documentation.

You mean patch 2/4, right?

Yes.

Quoted 7 lines
Quoted 2 lines

This series is based on driver-core-next and [1] cherry-picked on top of it.

I've taken patch 1 now, as it was easy to review :)

I'll wait for the next series with the documentation update before going further.

:( I was hoping I'll get an "okay" on where I expose the files, their names and content, etc before I document it. I'll add documentation now.

-Saravana