Skip to contentSkip to conversation

*** Support BPF traversal of wakeup sources ***

PATCH Superseded Samuel Wu 4 participants 8 messages Mar 20 – Mar 23

Subsystems (5)

v1 of 4 Superseded Latest is v4 →

lore
Share & tools
lore

Samuel Wu

author
#0 lore raw

cover *** Support BPF traversal of wakeup sources ***

This patchset adds requisite kfuncs for BPF programs to safely traverse wakeup_sources, and puts a config flag around the sysfs interface.

Currently, a traversal of wakeup sources require going through /sys/class/wakeup/* or /d/wakeup_sources/*. The repeated syscalls to query sysfs is inefficient, as there can be hundreds of wakeup_sources, with each wakeup source also having multiple attributes. debugfs is unstable and insecure.

Adding kfuncs to lock/unlock wakeup sources allows BPF program to safely traverse the wakeup sources list. The head address of wakeup_sources can safely be resolved through BPF helper functions or variable attributes.

On a quiescent Pixel 6 traversing 150 wakeup_sources, I am seeing ~34x speedup (sampled 75 times in table below). For a device under load, the speedup is greater.

+-------+----+----------+----------+
|       | n  | AVG (ms) | STD (ms) |
+-------+----+----------+----------+
| sysfs | 75 | 44.9     | 12.6     |
+-------+----+----------+----------+
| BPF   | 75 | 1.3      | 0.7      |
+-------+----+----------+----------+

On the memory side, between kernfs, dentry, and kmalloc, each wakeup source removed from sysfs saves at least 10kB.

The initial attempts for BPF traversal of wakeup_sources was with BPF iterators [1]. However, BPF already allows for traversing of a simple list with bpf_for(), and this current patchset has the added benefit of being ~2-3x more performant than BPF iterators.

[1]: https://lore.kernel.org/all/[email protected]/

Samuel Wu (2):
PM: wakeup: Add kfuncs to lock/unlock wakeup_sources
PM: Add config flag to gate sysfs wakeup_sources

 drivers/base/power/Makefile |  3 +-
 drivers/base/power/power.h  | 14 +++++++++
 drivers/base/power/wakeup.c | 63 +++++++++++++++++++++++++++++++++++--
 kernel/power/Kconfig        | 13 ++++++++
 4 files changed, 90 insertions(+), 3 deletions(-)
Signature · 1 line
2.53.0.959.g497ff81fa9-goog
author
#1 lore raw

PATCH 1/2 PM: wakeup: Add kfuncs to lock/unlock wakeup_sources

Add kfuncs to lock/unlock for safe traversal of wakeup sources.

Currently, a traversal of wakeup sources require going through /sys/class/wakeup/* or /d/wakeup_sources/*. The repeated syscalls to query sysfs is inefficient, as there can be hundreds of wakeup_sources, with each wakeup source also having multiple attributes. debugfs is unstable and insecure.

Adding kfuncs to lock/unlock wakeup sources allows BPF program to safely traverse the wakeup sources list. A new structure, bpf_ws_lock, acts as an opaque wrapper for the SRCU index. The head address of wakeup_sources can be safely resolved through BPF helper functions or variable attributes.

Doing the traversal in BPF is significantly more performant, and has an output in a format that the user specifies; this solves all the drawbacks of current interfaces.

---
 drivers/base/power/wakeup.c | 63 +++++++++++++++++++++++++++++++++++--
 1 file changed, 61 insertions(+), 2 deletions(-)
drivers/base/power/wakeup.c +61 -2
index b8e48a023bf0..7fc12ce125bc 100644
@@ -1168,11 +1168,70 @@ static const struct file_operations wakeup_sources_stats_fops = {
 	.release = seq_release_private,
 };
 
-static int __init wakeup_sources_debugfs_init(void)
+#ifdef CONFIG_BPF_SYSCALL
+#include <linux/btf.h>
+
+struct bpf_ws_lock { };
+
+__bpf_kfunc_start_defs();
+
+/**
+ * bpf_wakeup_sources_read_lock - Acquire the SRCU lock for wakeup sources
+ *
+ * The underlying SRCU lock returns an integer index. However, the BPF verifier
+ * requires a pointer (PTR_TO_BTF_ID) to strictly track the state of acquired
+ * resources using KF_ACQUIRE and KF_RELEASE semantics. We use an opaque
+ * structure pointer (struct bpf_ws_lock *) to satisfy the verifier while
+ * safely encoding the integer index within the pointer address itself.
+ *
+ * Return: An opaque pointer encoding the SRCU lock index + 1 (to avoid NULL).
+ */
+__bpf_kfunc struct bpf_ws_lock *bpf_wakeup_sources_read_lock(void)
+{
+	return (struct bpf_ws_lock *)(long)(wakeup_sources_read_lock() + 1);
+}
+
+/**
+ * bpf_wakeup_sources_read_unlock - Release the SRCU lock for wakeup sources
+ * @lock: The opaque pointer returned by bpf_wakeup_sources_read_lock()
+ *
+ * The BPF verifier guarantees that @lock is a valid, unreleased pointer from
+ * the acquire function. We decode the pointer back into the integer SRCU index
+ * by subtracting 1 and release the lock.
+ */
+__bpf_kfunc void bpf_wakeup_sources_read_unlock(struct bpf_ws_lock *lock)
+{
+	wakeup_sources_read_unlock((int)(long)lock - 1);
+}
+
+__bpf_kfunc_end_defs();
+
+BTF_KFUNCS_START(wakeup_source_kfunc_ids)
+BTF_ID_FLAGS(func, bpf_wakeup_sources_read_lock, KF_ACQUIRE)
+BTF_ID_FLAGS(func, bpf_wakeup_sources_read_unlock, KF_RELEASE)
+BTF_KFUNCS_END(wakeup_source_kfunc_ids)
+
+static const struct btf_kfunc_id_set wakeup_source_kfunc_set = {
+	.owner = THIS_MODULE,
+	.set   = &wakeup_source_kfunc_ids,
+};
+
+static void __init wakeup_sources_bpf_init(void)
+{
+	if (register_btf_kfunc_id_set(BPF_PROG_TYPE_SYSCALL, &wakeup_source_kfunc_set))
+		pm_pr_dbg("Wakeup: failed to register BTF kfuncs\n");
+}
+#else
+static inline void wakeup_sources_bpf_init(void) {}
+#endif /* CONFIG_BPF_SYSCALL */
+
+static int __init wakeup_sources_init(void)
 {
 	debugfs_create_file("wakeup_sources", 0444, NULL, NULL,
 			    &wakeup_sources_stats_fops);
+	wakeup_sources_bpf_init();
+
 	return 0;
 }
 
-postcore_initcall(wakeup_sources_debugfs_init);
+postcore_initcall(wakeup_sources_init);
Signature · 1 line
2.53.0.959.g497ff81fa9-goog
#2 lore raw

Re: [PATCH v1 1/2] PM: wakeup: Add kfuncs to lock/unlock wakeup_sources

kernel test robot noticed the following build warnings:

Hi Samuel,

kernel test robot noticed the following build warnings:

[auto build test WARNING on rafael-pm/linux-next]
[also build test WARNING on rafael-pm/bleeding-edge linus/master v7.0-rc5 next-20260320] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Samuel-Wu/PM-wakeup-Add-kfuncs-to-lock-unlock-wakeup_sources/20260323-064540
base:   https://git.kernel.org/pub/scm/linux/kernel/git/rafael/linux-pm.git linux-next
patch link:    https://lore.kernel.org/r/20260320160055.4114055-2-wusamuel%40google.com
patch subject: [PATCH v1 1/2] PM: wakeup: Add kfuncs to lock/unlock wakeup_sources
config: x86_64-randconfig-121-20260323 (https://download.01.org/0day-ci/archive/20260323/[email protected]/config)
compiler: gcc-12 (Debian 12.4.0-5) 12.4.0
sparse: v0.6.5-rc1
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20260323/[email protected]/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/

sparse warnings: (new ones prefixed by >>)

>> drivers/base/power/wakeup.c:1189:32: sparse: sparse: symbol 'bpf_wakeup_sources_read_lock' was not declared. Should it be static?
>> drivers/base/power/wakeup.c:1202:18: sparse: sparse: symbol 'bpf_wakeup_sources_read_unlock' was not declared. Should it be static?

vim +/bpf_wakeup_sources_read_lock +1189 drivers/base/power/wakeup.c

1177
1178 /**
1179 * bpf_wakeup_sources_read_lock - Acquire the SRCU lock for wakeup sources
1180 *
1181 * The underlying SRCU lock returns an integer index. However, the BPF verifier
1182 * requires a pointer (PTR_TO_BTF_ID) to strictly track the state of acquired
1183 * resources using KF_ACQUIRE and KF_RELEASE semantics. We use an opaque
1184 * structure pointer (struct bpf_ws_lock *) to satisfy the verifier while
1185 * safely encoding the integer index within the pointer address itself.
1186 *
1187 * Return: An opaque pointer encoding the SRCU lock index + 1 (to avoid NULL).
1188 */

> 1189	__bpf_kfunc struct bpf_ws_lock *bpf_wakeup_sources_read_lock(void)
  1190	{
  1191		return (struct bpf_ws_lock *)(long)(wakeup_sources_read_lock() + 1);
  1192	}
  1193	
  1194	/**
  1195	 * bpf_wakeup_sources_read_unlock - Release the SRCU lock for wakeup sources
  1196	 * @lock: The opaque pointer returned by bpf_wakeup_sources_read_lock()
  1197	 *
  1198	 * The BPF verifier guarantees that @lock is a valid, unreleased pointer from
  1199	 * the acquire function. We decode the pointer back into the integer SRCU index
  1200	 * by subtracting 1 and release the lock.
  1201	 */
> 1202	__bpf_kfunc void bpf_wakeup_sources_read_unlock(struct bpf_ws_lock *lock)

1203 {
1204 wakeup_sources_read_unlock((int)(long)lock - 1);
1205 }
1206

Signature · 2 lines
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
author
#3 lore raw

PATCH 2/2 PM: Add config flag to gate sysfs wakeup_sources

In reply to Samuel Wu

Add a config flag that gates the creation of /sys/class/wakeup_sources/*. This has the benefit of eliminating the work needed to create the nodes and corresponding attributes; and between kernfs, dentry, and malloc, there are some memory savings depending on the number of wakeup sources.

---
 drivers/base/power/Makefile |  3 ++-
 drivers/base/power/power.h  | 14 ++++++++++++++
 kernel/power/Kconfig        | 13 +++++++++++++
 3 files changed, 29 insertions(+), 1 deletion(-)
drivers/base/power/Makefile +2 -1
index 2989e42d0161..5933dadc0dd9 100644
@@ -1,6 +1,7 @@
 # SPDX-License-Identifier: GPL-2.0
 obj-$(CONFIG_PM)	+= sysfs.o generic_ops.o common.o qos.o runtime.o wakeirq.o
-obj-$(CONFIG_PM_SLEEP)	+= main.o wakeup.o wakeup_stats.o
+obj-$(CONFIG_PM_SLEEP)	+= main.o wakeup.o
+obj-$(CONFIG_PM_WAKEUP_STATS_SYSFS)	+= wakeup_stats.o
 obj-$(CONFIG_PM_TRACE_RTC)	+= trace.o
 obj-$(CONFIG_HAVE_CLK)	+= clock_ops.o
 obj-$(CONFIG_PM_QOS_KUNIT_TEST) += qos-test.o
drivers/base/power/power.h +14 -0
index 922ed457db19..364ca5512b4f 100644
@@ -125,11 +125,25 @@ static inline bool device_pm_initialized(struct device *dev)
 }
 
 /* drivers/base/power/wakeup_stats.c */
+#ifdef CONFIG_PM_WAKEUP_STATS_SYSFS
 extern int wakeup_source_sysfs_add(struct device *parent,
 				   struct wakeup_source *ws);
 extern void wakeup_source_sysfs_remove(struct wakeup_source *ws);
 
 extern int pm_wakeup_source_sysfs_add(struct device *parent);
+#else /* !CONFIG_PM_WAKEUP_STATS_SYSFS */
+static inline int wakeup_source_sysfs_add(struct device *parent,
+					  struct wakeup_source *ws)
+{
+	return 0;
+}
+static inline void wakeup_source_sysfs_remove(struct wakeup_source *ws) {}
+
+static inline int pm_wakeup_source_sysfs_add(struct device *parent)
+{
+	return 0;
+}
+#endif /* !CONFIG_PM_WAKEUP_STATS_SYSFS */
 
 #else /* !CONFIG_PM_SLEEP */
 
kernel/power/Kconfig +13 -0
index 05337f437cca..6945083ab053 100644
@@ -202,6 +202,19 @@ config PM_WAKELOCKS_GC
 	depends on PM_WAKELOCKS
 	default y
 
+config PM_WAKEUP_STATS_SYSFS
+	bool "Sysfs wakeup statistics"
+	depends on PM_SLEEP
+	default y
+	help
+	  Enable this for wakeup statistics in sysfs under /sys/class/wakeup/
+
+	  Disabling this option eliminates the work of creating the wakeup
+	  sources and each of their attributes in sysfs. Depending on the
+	  number of wakeup sources, this can also have a non-negligible memory
+	  impact. Regardless of this config option's value, wakeup statistics
+	  are still available via debugfs and BPF.
+
 config PM_QOS_CPU_SYSTEM_WAKEUP
 	bool "User space interface for CPU system wakeup QoS"
 	depends on CPU_IDLE
Signature · 1 line
2.53.0.959.g497ff81fa9-goog
#4 lore raw

Re: [PATCH v1 2/2] PM: Add config flag to gate sysfs wakeup_sources

Quoted 67 lines · Samuel Wu

Add a config flag that gates the creation of /sys/class/wakeup_sources/*. This has the benefit of eliminating the work needed to create the nodes and corresponding attributes; and between kernfs, dentry, and malloc, there are some memory savings depending on the number of wakeup sources.

---
 drivers/base/power/Makefile |  3 ++-
 drivers/base/power/power.h  | 14 ++++++++++++++
 kernel/power/Kconfig        | 13 +++++++++++++
 3 files changed, 29 insertions(+), 1 deletion(-)
drivers/base/power/Makefile +2 -1
index 2989e42d0161..5933dadc0dd9 100644
@@ -1,6 +1,7 @@
 # SPDX-License-Identifier: GPL-2.0
 obj-$(CONFIG_PM)       += sysfs.o generic_ops.o common.o qos.o runtime.o wakeirq.o
-obj-$(CONFIG_PM_SLEEP) += main.o wakeup.o wakeup_stats.o
+obj-$(CONFIG_PM_SLEEP) += main.o wakeup.o
+obj-$(CONFIG_PM_WAKEUP_STATS_SYSFS)    += wakeup_stats.o
 obj-$(CONFIG_PM_TRACE_RTC)     += trace.o
 obj-$(CONFIG_HAVE_CLK) += clock_ops.o
 obj-$(CONFIG_PM_QOS_KUNIT_TEST) += qos-test.o
drivers/base/power/power.h +14 -0
index 922ed457db19..364ca5512b4f 100644
@@ -125,11 +125,25 @@ static inline bool device_pm_initialized(struct device *dev)
 }
 
 /* drivers/base/power/wakeup_stats.c */
+#ifdef CONFIG_PM_WAKEUP_STATS_SYSFS
 extern int wakeup_source_sysfs_add(struct device *parent,
                                   struct wakeup_source *ws);
 extern void wakeup_source_sysfs_remove(struct wakeup_source *ws);
 
 extern int pm_wakeup_source_sysfs_add(struct device *parent);
+#else /* !CONFIG_PM_WAKEUP_STATS_SYSFS */
+static inline int wakeup_source_sysfs_add(struct device *parent,
+                                         struct wakeup_source *ws)
+{
+       return 0;
+}
+static inline void wakeup_source_sysfs_remove(struct wakeup_source *ws) {}
+
+static inline int pm_wakeup_source_sysfs_add(struct device *parent)
+{
+       return 0;
+}
+#endif /* !CONFIG_PM_WAKEUP_STATS_SYSFS */
 
 #else /* !CONFIG_PM_SLEEP */
 
kernel/power/Kconfig +2 -0
index 05337f437cca..6945083ab053 100644
@@ -202,6 +202,19 @@ config PM_WAKELOCKS_GC
        depends on PM_WAKELOCKS
        default y
 
+config PM_WAKEUP_STATS_SYSFS
+       bool "Sysfs wakeup statistics"

kernel/power/Kconfig config PM_WAKELOCKS_GC · line 216

Quoted 10 lines
kernel/power/Kconfig +10 -0
+       depends on PM_SLEEP
+       default y
+       help
+         Enable this for wakeup statistics in sysfs under /sys/class/wakeup/
+
+         Disabling this option eliminates the work of creating the wakeup
+         sources and each of their attributes in sysfs. Depending on the
+         number of wakeup sources, this can also have a non-negligible memory
+         impact. Regardless of this config option's value, wakeup statistics
+         are still available via debugfs and BPF.

Well, except that otherwise it may be hard to figure out which device the given wakeup source belongs to, so I'd rather not do that.

Quoted 5 lines
kernel/power/Kconfig +1 -1
+
 config PM_QOS_CPU_SYSTEM_WAKEUP
        bool "User space interface for CPU system wakeup QoS"
        depends on CPU_IDLE
--
author
#5 lore raw

Re: [PATCH v1 2/2] PM: Add config flag to gate sysfs wakeup_sources

Quoted 1 line · Rafael J. Wysocki

[ ... ]

Quoted 17 lines

kernel/power/Kconfig config PM_WAKELOCKS_GC · line 216

Quoted 13 lines
kernel/power/Kconfig +12 -0
+config PM_WAKEUP_STATS_SYSFS
+       bool "Sysfs wakeup statistics"
+       depends on PM_SLEEP
+       default y
+       help
+         Enable this for wakeup statistics in sysfs under /sys/class/wakeup/
+
+         Disabling this option eliminates the work of creating the wakeup
+         sources and each of their attributes in sysfs. Depending on the
+         number of wakeup sources, this can also have a non-negligible memory
+         impact. Regardless of this config option's value, wakeup statistics
+         are still available via debugfs and BPF.

Well, except that otherwise it may be hard to figure out which device the given wakeup source belongs to, so I'd rather not do that.

Since the config flag defaults to yes, this wouldn't affect current users right? And then the idea is that other users who don't need this sysfs node can turn the flag off to save some cycles and memory.

[ ... ]

#6 lore raw
In reply to Samuel Wu
Quoted 25 lines · Samuel Wu

This patchset adds requisite kfuncs for BPF programs to safely traverse wakeup_sources, and puts a config flag around the sysfs interface.

Currently, a traversal of wakeup sources require going through /sys/class/wakeup/* or /d/wakeup_sources/*. The repeated syscalls to query sysfs is inefficient, as there can be hundreds of wakeup_sources, with each wakeup source also having multiple attributes. debugfs is unstable and insecure.

Adding kfuncs to lock/unlock wakeup sources allows BPF program to safely traverse the wakeup sources list. The head address of wakeup_sources can safely be resolved through BPF helper functions or variable attributes.

On a quiescent Pixel 6 traversing 150 wakeup_sources, I am seeing ~34x speedup (sampled 75 times in table below). For a device under load, the speedup is greater.

+-------+----+----------+----------+
|       | n  | AVG (ms) | STD (ms) |
+-------+----+----------+----------+
| sysfs | 75 | 44.9     | 12.6     |
+-------+----+----------+----------+
| BPF   | 75 | 1.3      | 0.7      |
+-------+----+----------+----------+
Quoted 7 lines

On the memory side, between kernfs, dentry, and kmalloc, each wakeup source removed from sysfs saves at least 10kB.

The initial attempts for BPF traversal of wakeup_sources was with BPF iterators [1]. However, BPF already allows for traversing of a simple list with bpf_for(), and this current patchset has the added benefit of being ~2-3x more performant than BPF iterators.

See, it ended up being faster ;-).

That said, I didn't understand why you dropped the test. We still need unit tests that ensure the verifier causes the bpf_ws_lock to be released before exit, and an example program (like the one you benchmarked with) to demonstrate usage. Both are mandatory. See various *_fail.c files in progs/ and RUN_TESTS() macro for running negative tests easily. I doubt we will have any wakeup_sources in the CI environment, but the example has value nonetheless.

pw-bot: cr

Quoted 16 lines

[1]: https://lore.kernel.org/all/[email protected]/

Samuel Wu (2):
PM: wakeup: Add kfuncs to lock/unlock wakeup_sources
PM: Add config flag to gate sysfs wakeup_sources

 drivers/base/power/Makefile |  3 +-
 drivers/base/power/power.h  | 14 +++++++++
 drivers/base/power/wakeup.c | 63 +++++++++++++++++++++++++++++++++++--
 kernel/power/Kconfig        | 13 ++++++++
 4 files changed, 90 insertions(+), 3 deletions(-)

--
2.53.0.959.g497ff81fa9-goog

author
#7 lore raw
Quoted 1 line · Kumar Kartikeya Dwivedi

[ ... ]

Quoted 7 lines
Quoted 5 lines

The initial attempts for BPF traversal of wakeup_sources was with BPF iterators [1]. However, BPF already allows for traversing of a simple list with bpf_for(), and this current patchset has the added benefit of being ~2-3x more performant than BPF iterators.

See, it ended up being faster ;-).

Yup, thanks for the suggestion!

Quoted 11 lines

That said, I didn't understand why you dropped the test. We still need unit tests that ensure the verifier causes the bpf_ws_lock to be released before exit, and an example program (like the one you benchmarked with) to demonstrate usage. Both are mandatory. See various *_fail.c files in progs/ and RUN_TESTS() macro for running negative tests easily. I doubt we will have any wakeup_sources in the CI environment, but the example has value nonetheless.

pw-bot: cr

Sounds good, in v2 I can add a test for the lock/unlock kfunc. And thanks for the thorough feedback, let me take a look at the *_fail.c files and test macros.