Skip to contentSkip to conversation

Commit 911af505 introduced a bootmem warning.

DISCUSSION Robin Holt 2 participants 4 messages May 15, 2013

lore
Share & tools
lore

Robin Holt

author
#0 lore raw

Paul,

When we boot Linus' current kernel we get the following warning early in boot:

----------------------------------------------------------------------

[    0.000000] Memory: 63081268k/99598336k available (4832k kernel code, 34651396k absent, 1865672k reserved, 6269k data,
1672k init)
[    0.000000] Hierarchical RCU implementation.
[    0.000000]  RCU dyntick-idle grace-period acceleration is enabled.
[    0.000000]  RCU restricting CPUs from NR_CPUS=4096 to nr_cpu_ids=32.
[    0.000000] ------------[ cut here ]------------
[    0.000000] WARNING: at mm/nobootmem.c:215 ___alloc_bootmem_nopanic+0x79/0x82()
[    0.000000] Modules linked in:
[    0.000000] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 3.10.0-rc1-uv-hz100-rja+ #3
[    0.000000] Hardware name: SGI UV2000/ROMLEY, BIOS SGI UV 2000/3000 series BIOS 01/15/2013
[    0.000000]  00000000000000d7 ffffffff81a01e38 ffffffff814a873c ffffffff81a01e78
[    0.000000]  ffffffff81045a7b ffffffff81ced5a2 0000000000000000 ffffffffffffffff
[    0.000000]  0000000000000040 0000000000000200 0000000067da2bfc ffffffff81a01e88
[    0.000000] Call Trace:
[    0.000000]  [<ffffffff814a873c>] dump_stack+0x19/0x1d
[    0.000000]  [<ffffffff81045a7b>] warn_slowpath_common+0x6b/0xa0
[    0.000000]  [<ffffffff81045ac5>] warn_slowpath_null+0x15/0x20
[    0.000000]  [<ffffffff81b1793a>] ___alloc_bootmem_nopanic+0x79/0x82
[    0.000000]  [<ffffffff81b17a59>] ___alloc_bootmem+0x11/0x3c
[    0.000000]  [<ffffffff81b17aa4>] __alloc_bootmem+0x10/0x12
[    0.000000]  [<ffffffff81b233a7>] alloc_bootmem_cpumask_var+0x1d/0x27
[    0.000000]  [<ffffffff81b1119b>] rcu_bootup_announce_oddness+0xd0/0x153
[    0.000000]  [<ffffffff81b11735>] rcu_init+0x1e/0x1e6
[    0.000000]  [<ffffffff81aedf2f>] start_kernel+0x1e6/0x43c
[    0.000000]  [<ffffffff81aedb3b>] ? repair_env_string+0x58/0x58
[    0.000000]  [<ffffffff81aed4d1>] x86_64_start_reservations+0x1b/0x32
[    0.000000]  [<ffffffff81aed612>] x86_64_start_kernel+0x12a/0x131
[    0.000000] ---[ end trace c8b13488e92fad65 ]---
[    0.000000]  Experimental no-CBs for all CPUs
[    0.000000]  Experimental no-CBs CPUs: 0-31.
[    0.000000] NO_HZ: Full dynticks CPUs: 1-31.
$ grep RCU .config
# RCU Subsystem
CONFIG_TREE_RCU=y
# CONFIG_PREEMPT_RCU is not set
CONFIG_RCU_STALL_COMMON=y
CONFIG_RCU_USER_QS=y
CONFIG_RCU_FANOUT=64
CONFIG_RCU_FANOUT_LEAF=16
# CONFIG_RCU_FANOUT_EXACT is not set
CONFIG_RCU_FAST_NO_HZ=y
# CONFIG_TREE_RCU_TRACE is not set
CONFIG_RCU_NOCB_CPU=y
CONFIG_RCU_NOCB_CPU_ALL=y
# RCU Debugging
# CONFIG_SPARSE_RCU_POINTER is not set
# CONFIG_RCU_TORTURE_TEST is not set
CONFIG_RCU_CPU_STALL_TIMEOUT=60
# CONFIG_RCU_CPU_STALL_INFO is not set
# CONFIG_RCU_TRACE is not set
$ git rev-parse HEAD
1f638766ffcd9f08209afcabb3e2df961552fe18
cca6f3931 (Paul E. McKenney    2012-05-08 21:00:28 -0700   86)  if (nr_cpu_ids != NR_CPUS)
cca6f3931 (Paul E. McKenney    2012-05-08 21:00:28 -0700   87)          printk(KERN_INFO "\tRCU restricting CPUs from NR_C
PUS=%d to nr_cpu_ids=%d.\n", NR_CPUS, nr_cpu_ids);
3fbfbf7a3 (Paul E. McKenney    2012-08-19 21:35:53 -0700   88) #ifdef CONFIG_RCU_NOCB_CPU
911af505e (Paul E. McKenney    2013-02-11 10:23:27 -0800   89) #ifndef CONFIG_RCU_NOCB_CPU_NONE
911af505e (Paul E. McKenney    2013-02-11 10:23:27 -0800   90)  if (!have_rcu_nocb_mask) {
911af505e (Paul E. McKenney    2013-02-11 10:23:27 -0800   91)          alloc_bootmem_cpumask_var(&rcu_nocb_mask);
911af505e (Paul E. McKenney    2013-02-11 10:23:27 -0800   92)          have_rcu_nocb_mask = true;
911af505e (Paul E. McKenney    2013-02-11 10:23:27 -0800   93)  }
911af505e (Paul E. McKenney    2013-02-11 10:23:27 -0800   94) #ifdef CONFIG_RCU_NOCB_CPU_ZERO
911af505e (Paul E. McKenney    2013-02-11 10:23:27 -0800   95)  pr_info("\tExperimental no-CBs CPU 0\n");
911af505e (Paul E. McKenney    2013-02-11 10:23:27 -0800   96)  cpumask_set_cpu(0, rcu_nocb_mask);
911af505e (Paul E. McKenney    2013-02-11 10:23:27 -0800   97) #endif /* #ifdef CONFIG_RCU_NOCB_CPU_ZERO */
911af505e (Paul E. McKenney    2013-02-11 10:23:27 -0800   98) #ifdef CONFIG_RCU_NOCB_CPU_ALL
911af505e (Paul E. McKenney    2013-02-11 10:23:27 -0800   99)  pr_info("\tExperimental no-CBs for all CPUs\n");
911af505e (Paul E. McKenney    2013-02-11 10:23:27 -0800  100)  cpumask_setall(rcu_nocb_mask);
911af505e (Paul E. McKenney    2013-02-11 10:23:27 -0800  101) #endif /* #ifdef CONFIG_RCU_NOCB_CPU_ALL */
911af505e (Paul E. McKenney    2013-02-11 10:23:27 -0800  102) #endif /* #ifndef CONFIG_RCU_NOCB_CPU_NONE */

Thanks,
Robin Holt

Reviewed-by Josh Triplett
#1 lore raw
Quoted 29 lines · Robin Holt

Paul,

When we boot Linus' current kernel we get the following warning early in boot:

----------------------------------------------------------------------

[    0.000000] Memory: 63081268k/99598336k available (4832k kernel code, 34651396k absent, 1865672k reserved, 6269k data,
1672k init)
[    0.000000] Hierarchical RCU implementation.
[    0.000000]  RCU dyntick-idle grace-period acceleration is enabled.
[    0.000000]  RCU restricting CPUs from NR_CPUS=4096 to nr_cpu_ids=32.
[    0.000000] ------------[ cut here ]------------
[    0.000000] WARNING: at mm/nobootmem.c:215 ___alloc_bootmem_nopanic+0x79/0x82()
[    0.000000] Modules linked in:
[    0.000000] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 3.10.0-rc1-uv-hz100-rja+ #3
[    0.000000] Hardware name: SGI UV2000/ROMLEY, BIOS SGI UV 2000/3000 series BIOS 01/15/2013
[    0.000000]  00000000000000d7 ffffffff81a01e38 ffffffff814a873c ffffffff81a01e78
[    0.000000]  ffffffff81045a7b ffffffff81ced5a2 0000000000000000 ffffffffffffffff
[    0.000000]  0000000000000040 0000000000000200 0000000067da2bfc ffffffff81a01e88
[    0.000000] Call Trace:
[    0.000000]  [<ffffffff814a873c>] dump_stack+0x19/0x1d
[    0.000000]  [<ffffffff81045a7b>] warn_slowpath_common+0x6b/0xa0
[    0.000000]  [<ffffffff81045ac5>] warn_slowpath_null+0x15/0x20
[    0.000000]  [<ffffffff81b1793a>] ___alloc_bootmem_nopanic+0x79/0x82
[    0.000000]  [<ffffffff81b17a59>] ___alloc_bootmem+0x11/0x3c
[    0.000000]  [<ffffffff81b17aa4>] __alloc_bootmem+0x10/0x12
[    0.000000]  [<ffffffff81b233a7>] alloc_bootmem_cpumask_var+0x1d/0x27
[    0.000000]  [<ffffffff81b1119b>] rcu_bootup_announce_oddness+0xd0/0x153
[    0.000000]  [<ffffffff81b11735>] rcu_init+0x1e/0x1e6
Quoted 8 lines
[    0.000000]  [<ffffffff81aedf2f>] start_kernel+0x1e6/0x43c
[    0.000000]  [<ffffffff81aedb3b>] ? repair_env_string+0x58/0x58
[    0.000000]  [<ffffffff81aed4d1>] x86_64_start_reservations+0x1b/0x32
[    0.000000]  [<ffffffff81aed612>] x86_64_start_kernel+0x12a/0x131
[    0.000000] ---[ end trace c8b13488e92fad65 ]---
[    0.000000]  Experimental no-CBs for all CPUs
[    0.000000]  Experimental no-CBs CPUs: 0-31.
[    0.000000] NO_HZ: Full dynticks CPUs: 1-31.

Could you please try the following patch and let me know if it helps?

Thanx, Paul

------------------------------------------------------------------------

rcu: Don't allocate bootmem from rcu_init()

When rcu_init() is called we already have slab working, allocating bootmem at that point results in warnings and an allocation from slab. This commit therefore changes alloc_bootmem_cpumask_var() to alloc_cpumask_var() in rcu_bootup_announce_oddness(), which is called from rcu_init().

[paulmck: convert to zalloc_cpumask_var(), as suggested by Yinghai Lu.]

kernel/rcutree_plugin.h +1 -1
index a8c39c7..207844e 100644
@@ -87,7 +87,7 @@ static void __init rcu_bootup_announce_oddness(void)
 #ifdef CONFIG_RCU_NOCB_CPU
 #ifndef CONFIG_RCU_NOCB_CPU_NONE
 	if (!have_rcu_nocb_mask) {
-		alloc_bootmem_cpumask_var(&rcu_nocb_mask);
+		zalloc_cpumask_var(&rcu_nocb_mask, GFP_KERNEL);
 		have_rcu_nocb_mask = true;
 	}
 #ifdef CONFIG_RCU_NOCB_CPU_ZERO
author Tested-by
#2 lore raw
Quoted 53 lines · Paul E. McKenney
Quoted 29 lines · Robin Holt

Paul,

When we boot Linus' current kernel we get the following warning early in boot:

----------------------------------------------------------------------

[    0.000000] Memory: 63081268k/99598336k available (4832k kernel code, 34651396k absent, 1865672k reserved, 6269k data,
1672k init)
[    0.000000] Hierarchical RCU implementation.
[    0.000000]  RCU dyntick-idle grace-period acceleration is enabled.
[    0.000000]  RCU restricting CPUs from NR_CPUS=4096 to nr_cpu_ids=32.
[    0.000000] ------------[ cut here ]------------
[    0.000000] WARNING: at mm/nobootmem.c:215 ___alloc_bootmem_nopanic+0x79/0x82()
[    0.000000] Modules linked in:
[    0.000000] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 3.10.0-rc1-uv-hz100-rja+ #3
[    0.000000] Hardware name: SGI UV2000/ROMLEY, BIOS SGI UV 2000/3000 series BIOS 01/15/2013
[    0.000000]  00000000000000d7 ffffffff81a01e38 ffffffff814a873c ffffffff81a01e78
[    0.000000]  ffffffff81045a7b ffffffff81ced5a2 0000000000000000 ffffffffffffffff
[    0.000000]  0000000000000040 0000000000000200 0000000067da2bfc ffffffff81a01e88
[    0.000000] Call Trace:
[    0.000000]  [<ffffffff814a873c>] dump_stack+0x19/0x1d
[    0.000000]  [<ffffffff81045a7b>] warn_slowpath_common+0x6b/0xa0
[    0.000000]  [<ffffffff81045ac5>] warn_slowpath_null+0x15/0x20
[    0.000000]  [<ffffffff81b1793a>] ___alloc_bootmem_nopanic+0x79/0x82
[    0.000000]  [<ffffffff81b17a59>] ___alloc_bootmem+0x11/0x3c
[    0.000000]  [<ffffffff81b17aa4>] __alloc_bootmem+0x10/0x12
[    0.000000]  [<ffffffff81b233a7>] alloc_bootmem_cpumask_var+0x1d/0x27
[    0.000000]  [<ffffffff81b1119b>] rcu_bootup_announce_oddness+0xd0/0x153
[    0.000000]  [<ffffffff81b11735>] rcu_init+0x1e/0x1e6
Quoted 8 lines
[    0.000000]  [<ffffffff81aedf2f>] start_kernel+0x1e6/0x43c
[    0.000000]  [<ffffffff81aedb3b>] ? repair_env_string+0x58/0x58
[    0.000000]  [<ffffffff81aed4d1>] x86_64_start_reservations+0x1b/0x32
[    0.000000]  [<ffffffff81aed612>] x86_64_start_kernel+0x12a/0x131
[    0.000000] ---[ end trace c8b13488e92fad65 ]---
[    0.000000]  Experimental no-CBs for all CPUs
[    0.000000]  Experimental no-CBs CPUs: 0-31.
[    0.000000] NO_HZ: Full dynticks CPUs: 1-31.

Could you please try the following patch and let me know if it helps?

Thanx, Paul

------------------------------------------------------------------------

rcu: Don't allocate bootmem from rcu_init()

When rcu_init() is called we already have slab working, allocating bootmem at that point results in warnings and an allocation from slab. This commit therefore changes alloc_bootmem_cpumask_var() to alloc_cpumask_var() in rcu_bootup_announce_oddness(), which is called from rcu_init().

Quoted 3 lines

Works great.

Robin

#3 lore raw
Quoted 54 lines · Robin Holt
Quoted 53 lines · Paul E. McKenney
Quoted 29 lines · Robin Holt

Paul,

When we boot Linus' current kernel we get the following warning early in boot:

----------------------------------------------------------------------

[    0.000000] Memory: 63081268k/99598336k available (4832k kernel code, 34651396k absent, 1865672k reserved, 6269k data,
1672k init)
[    0.000000] Hierarchical RCU implementation.
[    0.000000]  RCU dyntick-idle grace-period acceleration is enabled.
[    0.000000]  RCU restricting CPUs from NR_CPUS=4096 to nr_cpu_ids=32.
[    0.000000] ------------[ cut here ]------------
[    0.000000] WARNING: at mm/nobootmem.c:215 ___alloc_bootmem_nopanic+0x79/0x82()
[    0.000000] Modules linked in:
[    0.000000] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 3.10.0-rc1-uv-hz100-rja+ #3
[    0.000000] Hardware name: SGI UV2000/ROMLEY, BIOS SGI UV 2000/3000 series BIOS 01/15/2013
[    0.000000]  00000000000000d7 ffffffff81a01e38 ffffffff814a873c ffffffff81a01e78
[    0.000000]  ffffffff81045a7b ffffffff81ced5a2 0000000000000000 ffffffffffffffff
[    0.000000]  0000000000000040 0000000000000200 0000000067da2bfc ffffffff81a01e88
[    0.000000] Call Trace:
[    0.000000]  [<ffffffff814a873c>] dump_stack+0x19/0x1d
[    0.000000]  [<ffffffff81045a7b>] warn_slowpath_common+0x6b/0xa0
[    0.000000]  [<ffffffff81045ac5>] warn_slowpath_null+0x15/0x20
[    0.000000]  [<ffffffff81b1793a>] ___alloc_bootmem_nopanic+0x79/0x82
[    0.000000]  [<ffffffff81b17a59>] ___alloc_bootmem+0x11/0x3c
[    0.000000]  [<ffffffff81b17aa4>] __alloc_bootmem+0x10/0x12
[    0.000000]  [<ffffffff81b233a7>] alloc_bootmem_cpumask_var+0x1d/0x27
[    0.000000]  [<ffffffff81b1119b>] rcu_bootup_announce_oddness+0xd0/0x153
[    0.000000]  [<ffffffff81b11735>] rcu_init+0x1e/0x1e6
Quoted 8 lines
[    0.000000]  [<ffffffff81aedf2f>] start_kernel+0x1e6/0x43c
[    0.000000]  [<ffffffff81aedb3b>] ? repair_env_string+0x58/0x58
[    0.000000]  [<ffffffff81aed4d1>] x86_64_start_reservations+0x1b/0x32
[    0.000000]  [<ffffffff81aed612>] x86_64_start_kernel+0x12a/0x131
[    0.000000] ---[ end trace c8b13488e92fad65 ]---
[    0.000000]  Experimental no-CBs for all CPUs
[    0.000000]  Experimental no-CBs CPUs: 0-31.
[    0.000000] NO_HZ: Full dynticks CPUs: 1-31.

Could you please try the following patch and let me know if it helps?

Thanx, Paul

------------------------------------------------------------------------

rcu: Don't allocate bootmem from rcu_init()

When rcu_init() is called we already have slab working, allocating bootmem at that point results in warnings and an allocation from slab. This commit therefore changes alloc_bootmem_cpumask_var() to alloc_cpumask_var() in rcu_bootup_announce_oddness(), which is called from rcu_init().

Quoted 7 lines
Quoted 3 lines

Works great.

Thank you, queued for 3.10 as a regression.

Thanx, Paul