kernel/isolation: Assert that a housekeeping CPU comes up at boot time
PATCH v2 1/1 kernel/isolation: Assert that a housekeeping CPU comes up at boot time
With the change to allow the boot CPU0 to be isolated, it is possible to specify command line options that result in no housekeeping CPU online at boot.
An 8 CPU system booted with "nohz_full=0-6 maxcpus=4", for example.
It is not easily possible at housekeeping init time to know all the various SMP options that will result in an invalid configuration, so this patch adds a sanity check after SMP init, to ensure that a housekeeping CPU has been onlined.
The panic is undesirable, but it's better than the alternative of an obscure non deterministic failure. The panic will reliably happen when advanced parameters are used incorrectly.
---
v2: Fix a NULL pointer dereference when not overriding housekeeping,
noticed by kernel test robot and Qais, who fixed it and verified the fix (thanks!)
kernel/sched/isolation.c | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+)
kernel/sched/isolation.c +26 -0
@@ -63,6 +63,32 @@ void __init housekeeping_init(void)WARN_ON_ONCE (cpumask_empty (housekeeping_mask)); } +static int __init housekeeping_verify_smp (void ) +{ +int cpu; + +if (!housekeeping_flags) +return 0 ; + +/* +* Early housekeeping setup is done before CPUs come up, and there are +* a range of options scattered around that can restrict which CPUs +* come up. It is possible to pass in a combination of housekeeping +* and SMP arguments that result in housekeeping assigned to an +* offline CPU. +* +* Check that condition here after SMP comes up, and give a useful +* error message rather than an obscure non deterministic crash or +* hang later. +*/ +for_each_online_cpu (cpu) { +if (cpumask_test_cpu (cpu, housekeeping_mask)) +return 0 ; + } +panic ("Housekeeping: nohz_full= or isolcpus= resulted in no online CPUs for housekeeping.\n" ); +} +core_initcall (housekeeping_verify_smp); +static int __init housekeeping_setup (char *str,enum hk_flags flags) {cpumask_var_t non_housekeeping_mask;
Signature · 1 line
2.20.1
Quoted 13 lines · Nicholas Piggin
kernel/sched/isolation.c +13
-0+static int __init housekeeping_verify_smp (void ) +{ +int cpu; + +if (!housekeeping_flags) +return 0 ; + +/* +* Early housekeeping setup is done before CPUs come up, and there are +* a range of options scattered around that can restrict which CPUs +* come up. It is possible to pass in a combination of housekeeping +* and SMP arguments that result in housekeeping assigned to an +* offline CPU.
kernel/sched/isolation.c void __init housekeeping_init(void) · line 88
Quoted 10 lines
kernel/sched/isolation.c +10
-0+* +* Check that condition here after SMP comes up, and give a useful +* error message rather than an obscure non deterministic crash or +* hang later. +*/ +for_each_online_cpu (cpu) { +if (cpumask_test_cpu (cpu, housekeeping_mask)) +return 0 ; + } +panic ("Housekeeping: nohz_full= or isolcpus= resulted in no online CPUs for housekeeping.\n" );
Ok let's keep the panic. But let's simplify and spare long iterations off boot load:
if (!cpumask_intersects(cpu_online_mask, housekeeping_mask))
panic(...);
Thanks.
Quoted 15 lines · Nicholas Piggin
With the change to allow the boot CPU0 to be isolated, it is possible to specify command line options that result in no housekeeping CPU online at boot.
An 8 CPU system booted with "nohz_full=0-6 maxcpus=4", for example.
It is not easily possible at housekeeping init time to know all the various SMP options that will result in an invalid configuration, so this patch adds a sanity check after SMP init, to ensure that a housekeeping CPU has been onlined.
The panic is undesirable, but it's better than the alternative of an obscure non deterministic failure. The panic will reliably happen when advanced parameters are used incorrectly.
Quoted 5 lines
Signed-off-by Nicholas Piggin <[email protected]>---
v2: Fix a NULL pointer dereference when not overriding housekeeping,
noticed by kernel test robot and Qais, who fixed it and verified the fix (thanks!)
Glad I could help. But for the record my problem wasn't a NULL pointer dereference and simply the loop didn't hit the condition to 'return 0' so I hit the panic.
I tested that the fix does indeed skip the verification if no nohz_full nor isolcpus is passed. But I didn't do the reverse check, although from the code this flag is only set in housekeeping_setup() so it should continue to work as intended.
Thanks!
--
Qais Yousef