Skip to contentSkip to conversation

add x86-64 support for memory hot-add

PATCH In review Matt Tolentino 4 participants 5 messages Dec 9, 2005 – Dec 10, 2005

lore
Share & tools
lore
#0 lore raw

PATCH 3/3 add x86-64 support for memory hot-add

This patch provide the capability to hot-add memory on single node x86-64 systems. Stub functions for adding and onlining pages as well as updating the kernel page tables when memory is added are provided.

---

arch/x86_64/Kconfig +6 -1
diff -urNp linux-2.6.15-rc5/arch/x86_64/Kconfig linux-2.6.15-rc5-matt/arch/x86_64/Kconfig
@@ -283,7 +283,11 @@ config ARCH_DISCONTIGMEM_DEFAULT
 
 config ARCH_SPARSEMEM_ENABLE
 	def_bool y
-	depends on NUMA
+	depends on (NUMA || EXPERIMENTAL)
+
+config ARCH_MEMORY_PROBE
+	def_bool y
+	depends on MEMORY_HOTPLUG
 
 config ARCH_FLATMEM_ENABLE
 	def_bool y
@@ -293,6 +297,7 @@ source "mm/Kconfig"
 
 config HAVE_ARCH_EARLY_PFN_TO_NID
 	def_bool y
+	depends on NUMA
 
 config NR_CPUS
 	int "Maximum number of CPUs (2-256)"
arch/x86_64/mm/init.c +128 -32

253 lines not on this page · raw message

#1 lore raw
Quoted 12 lines · Matt Tolentino
arch/x86_64/mm/init.c +2 -0
@@ -23,6 +23,8 @@
 #include <linux/bootmem.h>
 #include <linux/proc_fs.h>
 #include <linux/pci.h>
+#include <linux/module.h>
+#include <linux/memory_hotplug.h>
 
 #include <asm/processor.h>
 #include <asm/system.h>

arch/x86_64/mm/init.c static struct temp_map { · line 179

Quoted 5 lines
arch/x86_64/mm/init.c +1 -1
@@ -174,13 +176,19 @@ static  struct temp_map { 
 	{}
 }; 
 
-static __init void *alloc_low_page(int *index, unsigned long *phys) 
+static __devinit void *alloc_low_page(int *index, unsigned long *phys) 

These should be all __cpuinit.

In general SRAT has a hotplug memory bit so it's possible to predict how much memory there will be in advance. Since the overhead of the kernel page tables should be very low I would prefer if you just used instead.

(i.e. instead of extending the kernel mapping preallocate the direct mapping and just clear the P bits)

That should be much simpler.

-Andi

#2 lore raw

Re: [discuss] Re: [patch 3/3] add x86-64 support for memory hot-add II

Quoted 9 lines

In general SRAT has a hotplug memory bit so it's possible to predict how much memory there will be in advance. Since the overhead of the kernel page tables should be very low I would prefer if you just used instead.

(i.e. instead of extending the kernel mapping preallocate the direct mapping and just clear the P bits)

That should be much simpler.

Looking at it again - accessing SRAT currently relies on the direct mapping already. Untangling that would be possible, but require an bt_ioremap which would also add lots of code.

Ok I retract that objection. I guess your way is better for now.

In addition to the __cpuinit comment

+if (after_bootmem) spin_unlock(&init_mm.page_table_lock);

Conditional locking is evil. spinlocking in the boot case should just work too I think.

The EXPORTs should be probably EXPORT_SYMBOL_GPL.

With these changes it would look ok for me.

-Andi

#3 lore raw
In reply to Andi Kleen
Quoted 6 lines

These should be all __cpuinit.

In general SRAT has a hotplug memory bit so it's possible to predict how much memory there will be in advance. Since the overhead of the kernel page tables should be very low I would prefer if you just used instead.

How much overhead would there be?

Quoted 2 lines

(i.e. instead of extending the kernel mapping preallocate the direct mapping and just clear the P bits)

On my box the SRAT for hot-add areas exposed are from the end installed memory to way out in outerspace.
SRAT: hot plug zone found 280000000 - 2300000000
I can't hot add that sort of range on my box but the bios didn't want to limit or is planing for really really big dimms.

I wouldn't want to waste resource for areas that will never be added.

Thanks,
Keith

#4 lore raw
Quoted 8 lines · Keith Mannthey
Quoted 6 lines

These should be all __cpuinit.

In general SRAT has a hotplug memory bit so it's possible to predict how much memory there will be in advance. Since the overhead of the kernel page tables should be very low I would prefer if you just used instead.

How much overhead would there be?

It's 2MB pages in 3levels, so roughly 3*8=24 bytes per 2MB or roughly 512 bytes per GB (rounded up always to the next page)

Quoted 9 lines
Quoted 3 lines

(i.e. instead of extending the kernel mapping preallocate the direct mapping and just clear the P bits)

On my box the SRAT for hot-add areas exposed are from the end installed memory to way out in outerspace.
SRAT: hot plug zone found 280000000 - 2300000000
I can't hot add that sort of range on my box but the bios didn't want to limit or is planing for really really big dimms.

You're just proving someone's (anyone want to volunteer their name? ;-] I think Linus pointed it out originally, so let's call it Linus') law - as soon as we use some BIOS feature we soon find a BIOS that will get it totally wrong.

Anyways, I retracted anyways because of some other issues so Matt's original approach should be ok.

-Andi