Skip to contentSkip to conversation

Add SYSTEM_BOOTING_KMALLOC_AVAIL system_state

PATCH In review Mike Kravetz 6 participants 13 messages May 9, 2006 – May 10, 2006

Subsystems (8)
lore
Share & tools
lore
#0 lore raw

There are a few places that check the system_state variable to determine if they should use the bootmem or kmalloc allocator. However, this is not accurate as system_state transitions from SYSTEM_BOOTING to SYSTEM_RUNNING well after the bootmem allocator is no longer usable. Introduce the SYSTEM_BOOTING_KMALLOC_AVAIL state which indicates the kmalloc allocator is available for use.

arch/arm/kernel/setup.c +1 -1
diff -Naupr linux-2.6.17-rc3-mm1/arch/arm/kernel/setup.c linux-2.6.17-rc3-mm1.work/arch/arm/kernel/setup.c
@@ -378,7 +378,7 @@ void cpu_init(void)
 		BUG();
 	}
 
-	if (system_state == SYSTEM_BOOTING)
+	if (system_state < SYSTEM_RUNNING)
 		dump_cpu_info(cpu);
 
 	/*
arch/powerpc/kernel/smp.c +1 -1
diff -Naupr linux-2.6.17-rc3-mm1/arch/powerpc/kernel/smp.c linux-2.6.17-rc3-mm1.work/arch/powerpc/kernel/smp.c
@@ -540,7 +540,7 @@ int __devinit start_secondary(void *unus
 	if (smp_ops->take_timebase)
 		smp_ops->take_timebase();
 
-	if (system_state > SYSTEM_BOOTING)
+	if (system_state >= SYSTEM_RUNNING)
 		snapshot_timebase();
 
 	spin_lock(&call_lock);
include/linux/kernel.h +1 -0
diff -Naupr linux-2.6.17-rc3-mm1/include/linux/kernel.h linux-2.6.17-rc3-mm1.work/include/linux/kernel.h
@@ -189,6 +189,7 @@ extern void add_taint(unsigned);
 /* Values used for system_state */
 extern enum system_states {
 	SYSTEM_BOOTING,
+	SYSTEM_BOOTING_KMALLOC_AVAIL,
 	SYSTEM_RUNNING,
 	SYSTEM_HALT,
 	SYSTEM_POWER_OFF,
init/main.c +1 -0
diff -Naupr linux-2.6.17-rc3-mm1/init/main.c linux-2.6.17-rc3-mm1.work/init/main.c
@@ -513,6 +513,7 @@ asmlinkage void __init start_kernel(void
 	cpuset_init_early();
 	mem_init();
 	kmem_cache_init();
+	system_state = SYSTEM_BOOTING_KMALLOC_AVAIL;
 	setup_per_cpu_pageset();
 	numa_policy_init();
 	if (late_time_init)
kernel/sched.c +1 -1
diff -Naupr linux-2.6.17-rc3-mm1/kernel/sched.c linux-2.6.17-rc3-mm1.work/kernel/sched.c
@@ -5917,7 +5917,7 @@ static void calibrate_migration_costs(co
 			-1
 #endif
 		);
-	if (system_state == SYSTEM_BOOTING) {
+	if (system_state < SYSTEM_RUNNING) {
 		printk("migration_cost=");
 		for (distance = 0; distance <= max_distance; distance++) {
 			if (distance)
kernel/sys.c +2 -2
diff -Naupr linux-2.6.17-rc3-mm1/kernel/sys.c linux-2.6.17-rc3-mm1.work/kernel/sys.c
@@ -259,7 +259,7 @@ int blocking_notifier_chain_register(str
 	 * not yet working and interrupts must remain disabled.  At
 	 * such times we must not call down_write().
 	 */
-	if (unlikely(system_state == SYSTEM_BOOTING))
+	if (unlikely(system_state < SYSTEM_RUNNING))
 		return notifier_chain_register(&nh->head, n);
 
 	down_write(&nh->rwsem);
@@ -290,7 +290,7 @@ int blocking_notifier_chain_unregister(s
 	 * not yet working and interrupts must remain disabled.  At
 	 * such times we must not call down_write().
 	 */
-	if (unlikely(system_state == SYSTEM_BOOTING))
+	if (unlikely(system_state < SYSTEM_RUNNING))
 		return notifier_chain_unregister(&nh->head, n);
 
 	down_write(&nh->rwsem);
mm/page_alloc.c +2 -2
diff -Naupr linux-2.6.17-rc3-mm1/mm/page_alloc.c linux-2.6.17-rc3-mm1.work/mm/page_alloc.c
@@ -1782,7 +1782,7 @@ static int __meminit __build_all_zonelis
 
 void __meminit build_all_zonelists(void)
 {
-	if (system_state == SYSTEM_BOOTING) {
+	if (system_state < SYSTEM_RUNNING) {
 		__build_all_zonelists(0);
 		cpuset_init_current_mems_allowed();
 	} else {
@@ -2136,7 +2136,7 @@ int zone_wait_table_init(struct zone *zo
 	alloc_size = zone->wait_table_hash_nr_entries
 					* sizeof(wait_queue_head_t);
 
- 	if (system_state == SYSTEM_BOOTING) {
+ 	if (system_state < SYSTEM_RUNNING) {
 		zone->wait_table = (wait_queue_head_t *)
 			alloc_bootmem_node(pgdat, alloc_size);
 	} else {
mm/sparse.c +1 -1
diff -Naupr linux-2.6.17-rc3-mm1/mm/sparse.c linux-2.6.17-rc3-mm1.work/mm/sparse.c
@@ -32,7 +32,7 @@ static struct mem_section *sparse_index_
 	unsigned long array_size = SECTIONS_PER_ROOT *
 				   sizeof(struct mem_section);
 
-	if (system_state == SYSTEM_RUNNING)
+	if (system_state >= SYSTEM_BOOTING_KMALLOC_AVAIL)
 		section = kmalloc_node(array_size, GFP_KERNEL, nid);
 	else
 		section = alloc_bootmem_node(NODE_DATA(nid), array_size);
mm/vmscan.c +1 -1
diff -Naupr linux-2.6.17-rc3-mm1/mm/vmscan.c linux-2.6.17-rc3-mm1.work/mm/vmscan.c
@@ -1477,7 +1477,7 @@ int kswapd_run(int nid)
 	pgdat->kswapd = kthread_run(kswapd, pgdat, "kswapd%d", nid);
 	if (IS_ERR(pgdat->kswapd)) {
 		/* failure at boot is fatal */
-		BUG_ON(system_state == SYSTEM_BOOTING);
+		BUG_ON(system_state < SYSTEM_RUNNING);
 		printk("Failed to start kswapd on node %d\n",nid);
 		ret = -1;
 	}
#1 lore raw
Quoted 7 lines · Mike Kravetz

There are a few places that check the system_state variable to
determine if they should use the bootmem or kmalloc allocator. However, this is not accurate as system_state transitions from SYSTEM_BOOTING to SYSTEM_RUNNING well after the bootmem allocator is no longer usable. Introduce the SYSTEM_BOOTING_KMALLOC_AVAIL state which indicates the kmalloc allocator is available for use.

Let's not do this - system_state is getting out of control.

How about some private boolean in slab.c, and some special allocation function like

void __init *alloc_memory_early(size_t size, gfp_t gfp_flags)

{
	if (slab_is_available)
		return kmalloc(size, gfp_flags);
	return alloc_bootmem(size);
}	

?

#2 lore raw
Quoted 9 lines · Andrew Morton
Quoted 3 lines

SYSTEM_BOOTING to SYSTEM_RUNNING well after the bootmem allocator is no longer usable. Introduce the SYSTEM_BOOTING_KMALLOC_AVAIL state which indicates the kmalloc allocator is available for use.

Let's not do this - system_state is getting out of control.

How about some private boolean in slab.c, and some special allocation function like

Quoted 6 lines

void __init *alloc_memory_early(size_t size, gfp_t gfp_flags)

{
	if (slab_is_available)
		return kmalloc(size, gfp_flags);
	return alloc_bootmem(size);
}	

You may use g_cpucache_up to check the state of the slab allocator.

#3 lore raw
In reply to Andrew Morton
Quoted 14 lines · Andrew Morton
Quoted 7 lines · Mike Kravetz

There are a few places that check the system_state variable to
determine if they should use the bootmem or kmalloc allocator. However, this is not accurate as system_state transitions from SYSTEM_BOOTING to SYSTEM_RUNNING well after the bootmem allocator is no longer usable. Introduce the SYSTEM_BOOTING_KMALLOC_AVAIL state which indicates the kmalloc allocator is available for use.

Let's not do this - system_state is getting out of control.

How about some private boolean in slab.c, and some special allocation function like

Quoted 6 lines

void __init *alloc_memory_early(size_t size, gfp_t gfp_flags)

{
	if (slab_is_available)
		return kmalloc(size, gfp_flags);
	return alloc_bootmem(size);
}	

One issue with that approach is that you can't use it for larger allocations (which we have a lot of at boot-time). Would it be OK to fall back to the raw page allocator for things where kmalloc() fails? Oh, and do we want to make it explicitly NUMA aware?

-- Dave

#4 lore raw
Quoted 12 lines · Dave Hansen
Quoted 10 lines · Andrew Morton

How about some private boolean in slab.c, and some special allocation function like

void __init *alloc_memory_early(size_t size, gfp_t gfp_flags)

{
	if (slab_is_available)
		return kmalloc(size, gfp_flags);
	return alloc_bootmem(size);
}	
Quoted 4 lines

One issue with that approach is that you can't use it for larger allocations (which we have a lot of at boot-time). Would it be OK to fall back to the raw page allocator for things where kmalloc() fails? Oh, and do we want to make it explicitly NUMA aware?

Well, I am making it NUMA aware as the first identified user of such a routine does want to make node specific allocations. I haven't thought about the 'large' allocations.

Any thoughts about also including free_memory_early() routines? It seems like a good idea. However, I'm somewhat afraid of these gaining widespread use and the potential mis-use. Using free in the same routine as the alloc (such as in error paths) is generally ok. But, trying to do a free at a later time requires the attention of the coder. Just don't want people to think these wrappers will be a substitute for that analysis.

Signature · 1 line
Mike
#5 lore raw

PATCH 1/1 alloc_memory_early() routines

In reply to Andrew Morton

Add alloc_memory_early() routines so that code needing to allocate space during boot need not be aware of which allocator is in use. Includes first use of such routine by the SPARSEMEM code.

I did not include support for 'large' allocations as suggested by Dave, or corresponding free_memory_early() routines. The only immediate need is for NUMA/node aware allocation. Others can be added as the needs arise.

include/linux/slab.h +6 -0
diff -Naupr linux-2.6.17-rc3-mm1/include/linux/slab.h linux-2.6.17-rc3-mm1.work3/include/linux/slab.h
@@ -150,10 +150,12 @@ static inline void *kcalloc(size_t n, si
 
 extern void kfree(const void *);
 extern unsigned int ksize(const void *);
+extern void *alloc_memory_early(size_t size, gfp_t flags);
 
 #ifdef CONFIG_NUMA
 extern void *kmem_cache_alloc_node(kmem_cache_t *, gfp_t flags, int node);
 extern void *kmalloc_node(size_t size, gfp_t flags, int node);
+extern void *alloc_memory_early_node(size_t size, gfp_t flags, int node);
 #else
 static inline void *kmem_cache_alloc_node(kmem_cache_t *cachep, gfp_t flags, int node)
 {
@@ -163,6 +165,10 @@ static inline void *kmalloc_node(size_t 
 {
 	return kmalloc(size, flags);
 }
+static inline void *alloc_memory_early_node(size_t size, gfp_t flags, int node)
+{
+	return alloc_memory_early(size, flags);
+}
 #endif
 
 extern int FASTCALL(kmem_cache_reap(int));
mm/slab.c +17 -0
diff -Naupr linux-2.6.17-rc3-mm1/mm/slab.c linux-2.6.17-rc3-mm1.work3/mm/slab.c
@@ -108,6 +108,7 @@
 #include	<linux/mempolicy.h>
 #include	<linux/mutex.h>
 #include	<linux/rtmutex.h>
+#include	<linux/bootmem.h>
 
 #include	<asm/uaccess.h>
 #include	<asm/cacheflush.h>
@@ -3266,8 +3267,24 @@ void *kmalloc_node(size_t size, gfp_t fl
 	return kmem_cache_alloc_node(cachep, flags, node);
 }
 EXPORT_SYMBOL(kmalloc_node);
+
+void * __init alloc_memory_early_node(size_t size, gfp_t flags, int node)
+{
+	if (g_cpucache_up == FULL)
+		return kmalloc_node(size, flags, node);
+	else
+		return alloc_bootmem_node(NODE_DATA(node), size);
+}
 #endif
 
+void * __init alloc_memory_early(size_t size, gfp_t flags)
+{
+	if (g_cpucache_up == FULL)
+		return kmalloc(size, flags);
+	else
+		return alloc_bootmem(size);
+}
+
 /**
  * kmalloc - allocate memory
  * @size: how many bytes of memory are required.
mm/sparse.c +1 -5
diff -Naupr linux-2.6.17-rc3-mm1/mm/sparse.c linux-2.6.17-rc3-mm1.work3/mm/sparse.c
@@ -32,11 +32,7 @@ static struct mem_section *sparse_index_
 	unsigned long array_size = SECTIONS_PER_ROOT *
 				   sizeof(struct mem_section);
 
-	if (system_state == SYSTEM_RUNNING)
-		section = kmalloc_node(array_size, GFP_KERNEL, nid);
-	else
-		section = alloc_bootmem_node(NODE_DATA(nid), array_size);
-
+	section = alloc_memory_early_node(array_size, GFP_KERNEL, nid);
 	if (section)
 		memset(section, 0, array_size);
 
#6 lore raw

Re: [PATCH] alloc_memory_early() routines

Hi Mike,

Quoted 3 lines · Mike Kravetz

diff -Naupr linux-2.6.17-rc3-mm1/mm/slab.c linux-2.6.17-rc3-mm1.work3/mm/slab.c

mm/slab.c 2006-05-09 21:38:23.000000000 +0000 renamed from mm/slab.c 2006-05-03 22:19:16.000000000 +0000 +0 -0

[snip]

mm/slab.c 2006-05-09 21:38:23.000000000 +0000

Quoted 7 lines
mm/slab.c 2006-05-09 21:38:23.000000000 +0000 renamed from mm/slab.c 2006-05-03 22:19:16.000000000 +0000 +7 -0
+void * __init alloc_memory_early_node(size_t size, gfp_t flags, int node)
+{
+       if (g_cpucache_up == FULL)
+               return kmalloc_node(size, flags, node);
+       else
+               return alloc_bootmem_node(NODE_DATA(node), size);
+}

I'd prefer you put this in mm/bootmem.c and added a

int slab_is_available(void)

{
      return g_cpucache_up == FULL;
}

to mm/slab.c instead.

Pekka

#7 lore raw

Re: [PATCH] alloc_memory_early() routines

Quoted 10 lines · Pekka Enberg

mm/slab.c void *kmalloc_node(size_t size, gfp_t fl · line 3277

Quoted 7 lines
mm/slab.c +7 -0
+void * __init alloc_memory_early_node(size_t size, gfp_t flags, int node)
+{
+       if (g_cpucache_up == FULL)
+               return kmalloc_node(size, flags, node);
+       else
+               return alloc_bootmem_node(NODE_DATA(node), size);
+}

I'd prefer you put this in mm/bootmem.c and added a

Quoted 6 lines

int slab_is_available(void)

{
      return g_cpucache_up == FULL;
}

to mm/slab.c instead.

Does slab not available mean that bootmem can be used?

#8 lore raw

Re: [PATCH] alloc_memory_early() routines

Quoted 16 lines · Pekka Enberg
Quoted 16 lines

mm/slab.c void *kmalloc_node(size_t size, gfp_t fl · line 3277

Quoted 7 lines
mm/slab.c +7 -0
+void * __init alloc_memory_early_node(size_t size, gfp_t flags, int node)
+{
+       if (g_cpucache_up == FULL)
+               return kmalloc_node(size, flags, node);
+       else
+               return alloc_bootmem_node(NODE_DATA(node), size);
+}

I'd prefer you put this in mm/bootmem.c and added a

int slab_is_available(void)

{
      return g_cpucache_up == FULL;
}

to mm/slab.c instead.

Quoted 1 line · Christoph Lameter

Does slab not available mean that bootmem can be used?

Yes.

Pekka

#12 lore raw

Re: [PATCH] alloc_memory_early() routines

In reply to Mike Kravetz
Quoted 4 lines · Mike Kravetz

I did not include support for 'large' allocations as suggested by Dave, or corresponding free_memory_early() routines. The only immediate need is for NUMA/node aware allocation. Others can be added as the needs arise.

Sorry if this was already discussed, but you're not supposed to free the memory allocated by alloc_memory_early() ever? If so, please add a kerneldoc stating that.

Pekka

#9 lore raw

Re: [PATCH] alloc_memory_early() routines

In reply to Pekka J Enberg
Quoted 16 lines · Pekka J Enberg
Quoted 15 lines · Pekka Enberg
Quoted 15 lines

mm/slab.c void *kmalloc_node(size_t size, gfp_t fl · line 3277

Quoted 7 lines
mm/slab.c +7 -0
+void * __init alloc_memory_early_node(size_t size, gfp_t flags, int node)
+{
+       if (g_cpucache_up == FULL)
+               return kmalloc_node(size, flags, node);
+       else
+               return alloc_bootmem_node(NODE_DATA(node), size);
+}

I'd prefer you put this in mm/bootmem.c and added a

int slab_is_available(void)

{
      return g_cpucache_up == FULL;
}
Quoted 6 lines
Quoted 1 line
Quoted 1 line

to mm/slab.c instead.

Quoted 1 line · Christoph Lameter

Does slab not available mean that bootmem can be used?

Yes.

Actually it doesn't - in early boot up there is a phase where even bootmem doesn't work yet.

-Andi

#10 lore raw

Re: [PATCH] alloc_memory_early() routines

In reply to Christoph Lameter
Quoted 10 lines · Christoph Lameter
Quoted 8 lines

I'd prefer you put this in mm/bootmem.c and added a

int slab_is_available(void)

{
      return g_cpucache_up == FULL;
}

to mm/slab.c instead.

Does slab not available mean that bootmem can be used?

I like the 'slab_is_available()' check. How about if we simply add this routine and let the people doing the allocation determine what allocator to use?

As has already been stated, slab not available does NOT imply that bootmem can be used. Heck, on POWER there is even an allocator used before bootmem. I doubt we could provide an 'intelligent' routine to works in all cases. So, for right now we could/should just provide the slab not available() check. There is only one piece of code in SPARSEMEM that cares about this.

Sound reasonable?

Signature · 1 line
Mike
#11 lore raw

Re: [PATCH] alloc_memory_early() routines

Quoted 3 lines · Mike Kravetz

I like the 'slab_is_available()' check. How about if we simply add this routine and let the people doing the allocation determine what allocator to use?

I'm fine with that.

Pekka