kfence: allow use of a deferrable timer
Subsystems (5)
Marco Elver
maintainerPATCH v2 1/1 kfence: allow use of a deferrable timer
Allow the use of a deferrable timer, which does not force CPU wake-ups when the system is idle. A consequence is that the sample interval becomes very unpredictable, to the point that it is not guaranteed that the KFENCE KUnit test still passes.
Nevertheless, on power-constrained systems this may be preferable, so let's give the user the option should they accept the above trade-off.
---
v2:
* Add more documentation.
* Remove 'if EXPERT' from Kconfig option since it's configurable via kernel boot param anyway.
--- Documentation/dev-tools/kfence.rst | 12 ++++++++++++ lib/Kconfig.kfence | 12 ++++++++++++ mm/kfence/core.c | 15 +++++++++++++-- 3 files changed, 37 insertions(+), 2 deletions(-)
Documentation/dev-tools/kfence.rst +12 -0
@@ -41,6 +41,18 @@ guarded by KFENCE. The default is configurable via the Kconfig option ``CONFIG_KFENCE_SAMPLE_INTERVAL``. Setting ``kfence.sample_interval=0`` disables KFENCE. +The sample interval controls a timer that sets up KFENCE allocations. By +default, to keep the real sample interval predictable, the normal timer also +causes CPU wake-ups when the system is completely idle. This may be undesirable +on power-constrained systems. The boot parameter ``kfence.deferrable=1`` +instead switches to a "deferrable" timer which does not force CPU wake-ups on +idle systems, at the risk of unpredictable sample intervals. The default is +configurable via the Kconfig option ``CONFIG_KFENCE_DEFERRABLE``. + +.. warning:: + The KUnit test suite is very likely to fail when using a deferrable timer + since it currently causes very unpredictable sample intervals. + The KFENCE memory pool is of fixed size, and if the pool is exhausted, no further KFENCE allocations occur. With ``CONFIG_KFENCE_NUM_OBJECTS`` (default 255), the number of available guarded objects can be controlled. Each object
lib/Kconfig.kfence +12 -0
@@ -45,6 +45,18 @@ config KFENCE_NUM_OBJECTS pages are required; with one containing the object and two adjacent ones used as guard pages. +config KFENCE_DEFERRABLE + bool "Use a deferrable timer to trigger allocations" + help + Use a deferrable timer to trigger allocations. This avoids forcing + CPU wake-ups if the system is idle, at the risk of a less predictable + sample interval. + + Warning: The KUnit test suite fails with this option enabled - due to + the unpredictability of the sample interval! + + Say N if you are unsure. + config KFENCE_STATIC_KEYS bool "Use static keys to set up allocations" if EXPERT depends on JUMP_LABEL
mm/kfence/core.c +13 -2
@@ -95,6 +95,10 @@ module_param_cb(sample_interval, &sample_interval_param_ops, &kfence_sample_intestatic unsigned long kfence_skip_covered_thresh__read_mostly =75 ;module_param_named (skip_covered_thresh, kfence_skip_covered_thresh, ulong,0644 ); +/* If true, use a deferrable timer. */ +static bool kfence_deferrable__read_mostly =IS_ENABLED (CONFIG_KFENCE_DEFERRABLE ); +module_param_named (deferrable, kfence_deferrable, bool,0444 ); +/* The pool of pages used for guard pages and objects. */ char *__kfence_pool__read_mostly ;EXPORT_SYMBOL (__kfence_pool);/* Export for test modules. */
@@ -740,6 +744,8 @@ late_initcall(kfence_debugfs_init);/* === Allocation Gate Timer ================================================ */ +static struct delayed_work kfence_timer; +#ifdef CONFIG_KFENCE_STATIC_KEYS /* Wait queue to wake up allocation-gate timer task. */ static DECLARE_WAIT_QUEUE_HEAD (allocation_wait);
@@ -762,7 +768,6 @@ static DEFINE_IRQ_WORK(wake_up_kfence_timer_work, wake_up_kfence_timer);* avoids IPIs, at the cost of not immediately capturing allocations if the * instructions remain cached. */ -static struct delayed_work kfence_timer;static void toggle_allocation_gate (struct work_struct *work) {if (!READ_ONCE (kfence_enabled))
@@ -790,7 +795,6 @@ static void toggle_allocation_gate(struct work_struct *work)queue_delayed_work (system_unbound_wq, &kfence_timer,msecs_to_jiffies (kfence_sample_interval)); } -static DECLARE_DELAYED_WORK(kfence_timer, toggle_allocation_gate);/* === Public interface ===================================================== */
@@ -809,8 +813,15 @@ static void kfence_init_enable(void) {if (!IS_ENABLED (CONFIG_KFENCE_STATIC_KEYS ))static_branch_enable (&kfence_allocation_key); + +if (kfence_deferrable) +INIT_DEFERRABLE_WORK (&kfence_timer, toggle_allocation_gate); +else +INIT_DELAYED_WORK (&kfence_timer, toggle_allocation_gate); +WRITE_ONCE (kfence_enabled, true);queue_delayed_work (system_unbound_wq, &kfence_timer,0 ); +pr_info ("initialized - using %lu bytes for %d objects at 0x%p-0x%p\n" ,KFENCE_POOL_SIZE ,CONFIG_KFENCE_NUM_OBJECTS , (void *)__kfence_pool, (void *)(__kfence_pool +KFENCE_POOL_SIZE ));
Signature · 1 line
2.35.1.616.g0bdcbb4464-goog
Quoted 10 lines · Marco Elver
Allow the use of a deferrable timer, which does not force CPU wake-ups when the system is idle. A consequence is that the sample interval becomes very unpredictable, to the point that it is not guaranteed that the KFENCE KUnit test still passes.
Nevertheless, on power-constrained systems this may be preferable, so let's give the user the option should they accept the above trade-off.
Signed-off-by Marco Elver <[email protected]>
Quoted 140 lines
---
v2:
* Add more documentation.
* Remove 'if EXPERT' fromKconfigoption since it's configurable via kernel boot param anyway.--- Documentation/dev-tools/kfence.rst | 12 ++++++++++++ lib/Kconfig.kfence | 12 ++++++++++++ mm/kfence/core.c | 15 +++++++++++++-- 3 files changed, 37 insertions(+), 2 deletions(-)Documentation/dev-tools/kfence.rst +0
-0@@ -41,6 +41,18 @@ guarded by KFENCE. The default is configurable via the
Kconfigoption
``CONFIG_KFENCE_SAMPLE_INTERVAL``. Setting ``kfence.sample_interval=0`` disables KFENCE.Documentation/dev-tools/kfence.rst +2
-0+The sample interval controls a timer that sets up KFENCE allocations. By +default, to keep the real sample interval predictable, the normal timeralso
+causes CPU wake-ups when the system is completely idle. This may be undesirableDocumentation/dev-tools/kfence.rst +2
-0+on power-constrained systems. The boot parameter ``kfence.deferrable=1`` +instead switches to a "deferrable" timer which does not force CPUwake-ups on
+idle systems, at the risk of unpredictable sample intervals. The default isDocumentation/dev-tools/kfence.rst +4
-0+configurable via the Kconfig option ``CONFIG_KFENCE_DEFERRABLE``. + +.. warning:: + The KUnit test suite is very likely to fail when using a deferrabletimer
Documentation/dev-tools/kfence.rst +2
-0+ since it currently causes very unpredictable sample intervals. + The KFENCE memory pool is of fixed size, and if the pool is exhausted, no further KFENCE allocations occur. With ``CONFIG_KFENCE_NUM_OBJECTS``(default
255), the number of available guarded objects can be controlled. Each
objectlib/Kconfig.kfence +0
-0@@ -45,6 +45,18 @@ config KFENCE_NUM_OBJECTS pages are required; with one containing the object and twoadjacent
ones used as guard pages.lib/Kconfig.kfence +4
-0+config KFENCE_DEFERRABLE + bool "Use a deferrable timer to trigger allocations" + help + Use a deferrable timer to trigger allocations. This avoidsforcing
+ CPU wake-ups if the system is idle, at the risk of a less
predictablelib/Kconfig.kfence +3
-0+ sample interval. + + Warning: The KUnit test suite fails with this option enabled -due to
lib/Kconfig.kfence +4
-0+ the unpredictability of the sample interval! + + Say N if you are unsure. + config KFENCE_STATIC_KEYS bool "Use static keys to set up allocations" if EXPERT depends on JUMP_LABELmm/kfence/core.c +0
-0@@ -95,6 +95,10 @@ module_param_cb(sample_interval,
&sample_interval_param_ops,&kfence_sample_inte
static unsigned longkfence_skip_covered_thresh__read_mostly= 75; module_param_named(skip_covered_thresh,kfence_skip_covered_thresh,
ulong, 0644);mm/kfence/core.c +2
-0+/* If true, use a deferrable timer. */ +static bool kfence_deferrable __read_mostly =
IS_ENABLED(CONFIG_KFENCE_DEFERRABLE);mm/kfence/core.c +2
-0+module_param_named (deferrable, kfence_deferrable, bool,0444 ); +/* The pool of pages used for guard pages and objects. */ char *__kfence_pool__read_mostly ;EXPORT_SYMBOL (__kfence_pool);/* Export for test modules. */ @@ -740,6 +744,8 @@ late_initcall(kfence_debugfs_init);/* === Allocation Gate Timer ================================================ */
mm/kfence/core.c +2
-0+static struct delayed_work kfence_timer; +#ifdef CONFIG_KFENCE_STATIC_KEYS /* Wait queue to wake up allocation-gate timer task. */ static DECLARE_WAIT_QUEUE_HEAD (allocation_wait);@@ -762,7 +768,6 @@ static DEFINE_IRQ_WORK(wake_up_kfence_timer_work, wake_up_kfence_timer); * avoids IPIs, at the cost of not immediately capturing allocations ifthe
* instructions remain cached.
*/mm/kfence/core.c +0
-2-static struct delayed_work kfence_timer;static void toggle_allocation_gate (struct work_struct *work) {if (!READ_ONCE (kfence_enabled))@@ -790,7 +795,6 @@ static void toggle_allocation_gate(struct work_struct *work)queue_delayed_work (system_unbound_wq, &kfence_timer,msecs_to_jiffies (kfence_sample_interval)); } -static DECLARE_DELAYED_WORK(kfence_timer, toggle_allocation_gate);/* === Public interface ===================================================== */
mm/kfence/core.c +7
-0@@ -809,8 +813,15 @@ static void kfence_init_enable(void) { if (!IS_ENABLED(CONFIG_KFENCE_STATIC_KEYS)) static_branch_enable(&kfence_allocation_key); + + if (kfence_deferrable) + INIT_DEFERRABLE_WORK(&kfence_timer, toggle_allocation_gate); + else + INIT_DELAYED_WORK(&kfence_timer, toggle_allocation_gate); + WRITE_ONCE(kfence_enabled, true); queue_delayed_work(system_unbound_wq, &kfence_timer, 0); + pr_info("initialized - using %lu bytes for %d objects at 0x%p-0x%p\n", KFENCE_POOL_SIZE,
CONFIG_KFENCE_NUM_OBJECTS, (void *)__kfence_pool, (void *)(__kfence_pool +KFENCE_POOL_SIZE));
--
2.35.1.616.g0bdcbb4464-goog
Signature · 20 lines
Alexander Potapenko Software Engineer Google Germany GmbH Erika-Mann-Straße, 33 80636 München Geschäftsführer: Paul Manicle, Liana Sebastian Registergericht und -nummer: Hamburg, HRB 86891 Sitz der Gesellschaft: Hamburg Diese E-Mail ist vertraulich. Falls Sie diese fälschlicherweise erhalten haben sollten, leiten Sie diese bitte nicht an jemand anderes weiter, löschen Sie alle Kopien und Anhänge davon und lassen Sie mich bitte wissen, dass die E-Mail an die falsche Person gesendet wurde. This e-mail is confidential. If you received this communication by mistake, please don't forward it to anyone else, please erase all copies and attachments, and please let me know that it has gone to the wrong person.