memory-hotplug : hot-remove physical memory(clear page table)

RFC PATCH Superseded Wen Congyang 2 participants 17 messages Jul 20, 2012 – Jul 23, 2012

Subsystems (10)

Memory management x86 PowerPC ACPI +6 more subsystems

lore

Share & tools

lore

Wen Congyang

author

2012-07-20 07:02 UTC #0 lore raw

cover memory-hotplug : hot-remove physical memory(clear page table)

This patch series aims to support physical memory hot-remove(clear page table).

This patch series base on ishimatsu's patch series. You can get it here:
http://www.spinics.net/lists/linux-acpi/msg36804.html

The patches can remove following things:
- page table of removed memory

If you find lack of function for physical memory hot-remove, please let me know.

Note:
* The patch "remove memory info from list before freeing it" is being disccussed in other thread. But for testing the patch series, the patch is needed. So I added the patch as [PATCH 0/8].
* You need to apply ishimatsu's patch series first before applying this patch series.

Wen Congyang (8):
memory-hotplug: store the node id in acpi_memory_device
memory-hotplug: offline memory only when it is onlined
memory-hotplug: call remove_memory() to cleanup when removing memory
device
memory-hotplug: export the function acpi_bus_remove()
memory-hotplug: call acpi_bus_remove() to remove memory device
memory-hotplug: introduce new function arch_remove_memory()
x86: make __split_large_page() generally avialable
memory-hotplug: implement arch_remove_memory()

 arch/ia64/mm/init.c                  |   16 ++++
 arch/powerpc/mm/mem.c                |   14 +++
 arch/s390/mm/init.c                  |    8 ++
 arch/sh/mm/init.c                    |   15 +++
 arch/tile/mm/init.c                  |    8 ++
 arch/x86/include/asm/pgtable_types.h |    1 +
 arch/x86/mm/init_32.c                |   10 ++
 arch/x86/mm/init_64.c                |  160 ++++++++++++++++++++++++++++++++++
 arch/x86/mm/pageattr.c               |   47 +++++-----
 drivers/acpi/acpi_memhotplug.c       |   24 ++++--
 drivers/acpi/scan.c                  |    3 +-
 include/acpi/acpi_bus.h              |    1 +
 include/linux/memory_hotplug.h       |    1 +
 mm/memory_hotplug.c                  |    2 +-
 14 files changed, 280 insertions(+), 30 deletions(-)

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Wen Congyang

author

2012-07-20 07:03 UTC #1 lore raw

cover remove memory info from list before freeing it

We free info, but we forget to remove it from the list. It will cause unexpected problem when we access the list next time.

Signed-off-by Wen Congyang <[email protected]>

---
 drivers/acpi/acpi_memhotplug.c |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

drivers/acpi/acpi_memhotplug.c +1 -0

index 8fe0e02..5cafd6b 100644
@@ -323,6 +323,7 @@ static int acpi_memory_disable_device(struct acpi_memory_device *mem_device)
 			if (result)
 				return result;
 		}
+		list_del(&info->list);
 		kfree(info);
 	}

Signature · 1 line

1.7.1

Wen Congyang

author

2012-07-20 07:04 UTC #2 lore raw

RFC PATCH 1/8 memory-hotplug: store the node id in acpi_memory_device

In reply to Wen Congyang

The memory device has only one node id. Store the node id when enabling the memory device, and we can reuse it when removing the memory device.

CC David Rientjes <[email protected]>

CC Jiang Liu <[email protected]>

CC Len Brown <[email protected]>

CC Benjamin Herrenschmidt <[email protected]>

CC Paul Mackerras <[email protected]>

CC Christoph Lameter <[email protected]>

Cc Minchan Kim <[email protected]>

CC Andrew Morton <[email protected]>

CC KOSAKI Motohiro <[email protected]>

CC Yasuaki Ishimatsu <[email protected]>

Signed-off-by Wen Congyang <[email protected]>

---
 drivers/acpi/acpi_memhotplug.c |    8 +++++---
 1 files changed, 5 insertions(+), 3 deletions(-)

drivers/acpi/acpi_memhotplug.c +5 -3

index 5cafd6b..db8de39 100644
@@ -84,6 +84,7 @@ struct acpi_memory_info {
 struct acpi_memory_device {
 	struct acpi_device * device;
 	unsigned int state;	/* State of the memory device */
+	int nid;
 	struct list_head res_list;
 };

@@ -257,6 +258,9 @@ static int acpi_memory_enable_device(struct acpi_memory_device *mem_device)
 		info->enabled = 1;
 		num_enabled++;
 	}
+
+	mem_device->nid = node;
+
 	if (!num_enabled) {
 		printk(KERN_ERR PREFIX "add_memory failed\n");
 		mem_device->state = MEMORY_INVALID_STATE;

@@ -463,7 +467,7 @@ static int acpi_memory_device_remove(struct acpi_device *device, int type)
 
 	mem_device = acpi_driver_data(device);
 
-	node = acpi_get_node(mem_device->device->handle);
+	node = mem_device->nid;
 	list_for_each_entry_safe(info, tmp, &mem_device->res_list, list) {
 		if (!info->enabled)
 			continue;

@@ -473,8 +477,6 @@ static int acpi_memory_device_remove(struct acpi_device *device, int type)
 			if (result)
 				return result;
 		}
-		if (node < 0)
-			node = memory_add_physaddr_to_nid(info->start_addr);
 
 		result = remove_memory(node, info->start_addr, info->length);
 		if (result)

Signature · 1 line

1.7.1

Yasuaki Ishimatsu

Reviewed-by on v1 1/8

2012-07-20 07:35 UTC #3 lore raw

Re: [RFC PATCH 1/8] memory-hotplug: store the node id in acpi_memory_device

Hi Wen,

Quoted 15 lines · Wen Congyang

The memory device has only one node id. Store the node id when enabling the memory device, and we can reuse it when removing the memory device.
CC David Rientjes <[email protected]>
CC Jiang Liu <[email protected]>
CC Len Brown <[email protected]>
CC Benjamin Herrenschmidt <[email protected]>
CC Paul Mackerras <[email protected]>
CC Christoph Lameter <[email protected]>
Cc Minchan Kim <[email protected]>
CC Andrew Morton <[email protected]>
CC KOSAKI Motohiro <[email protected]>
CC Yasuaki Ishimatsu <[email protected]>
Signed-off-by Wen Congyang <[email protected]>

Quoted 1 line

---

It looks to me.

Reviewed-by Yasuaki Ishimatsu <[email protected]>

Thanks,
Yasuaki Ishimatsu

Quoted 36 lines

  drivers/acpi/acpi_memhotplug.c |    8 +++++---
  1 files changed, 5 insertions(+), 3 deletions(-)

drivers/acpi/acpi_memhotplug.c +5 -1

index 5cafd6b..db8de39 100644
@@ -84,6 +84,7 @@ struct acpi_memory_info {
  struct acpi_memory_device {
  	struct acpi_device * device;
  	unsigned int state;	/* State of the memory device */
+	int nid;
  	struct list_head res_list;
  };

@@ -257,6 +258,9 @@ static int acpi_memory_enable_device(struct acpi_memory_device *mem_device)
  		info->enabled = 1;
  		num_enabled++;
  	}
+
+	mem_device->nid = node;
+
  	if (!num_enabled) {
  		printk(KERN_ERR PREFIX "add_memory failed\n");
  		mem_device->state = MEMORY_INVALID_STATE;

@@ -463,7 +467,7 @@ static int acpi_memory_device_remove(struct acpi_device *device, int type)
 
  	mem_device = acpi_driver_data(device);
 
-	node = acpi_get_node(mem_device->device->handle);
+	node = mem_device->nid;
  	list_for_each_entry_safe(info, tmp, &mem_device->res_list, list) {
  		if (!info->enabled)
  			continue;

drivers/acpi/acpi_memhotplug.c static int acpi_memory_device_remove(struct acpi_device *device, int type) · line 482

Quoted 8 lines

drivers/acpi/acpi_memhotplug.c +0 -2

@@ -473,8 +477,6 @@ static int acpi_memory_device_remove(struct acpi_device *device, int type)
  			if (result)
  				return result;
  		}
-		if (node < 0)
-			node = memory_add_physaddr_to_nid(info->start_addr);
 
  		result = remove_memory(node, info->start_addr, info->length);
  		if (result)

Wen Congyang

author

2012-07-20 07:05 UTC #4 lore raw

RFC PATCH 2/8 memory-hotplug: offline memory only when it is onlined

In reply to Wen Congyang

offline_memory() will fail if the memory is not onlined. So check whether the memory is onlined before calling offline_memory().

CC David Rientjes <[email protected]>

CC Jiang Liu <[email protected]>

CC Len Brown <[email protected]>

CC Benjamin Herrenschmidt <[email protected]>

CC Paul Mackerras <[email protected]>

CC Christoph Lameter <[email protected]>

Cc Minchan Kim <[email protected]>

CC Andrew Morton <[email protected]>

CC KOSAKI Motohiro <[email protected]>

CC Yasuaki Ishimatsu <[email protected]>

Signed-off-by Wen Congyang <[email protected]>

---
 drivers/acpi/acpi_memhotplug.c |   10 +++++++---
 1 files changed, 7 insertions(+), 3 deletions(-)

drivers/acpi/acpi_memhotplug.c +7 -3

index db8de39..712e767 100644
@@ -323,9 +323,13 @@ static int acpi_memory_disable_device(struct acpi_memory_device *mem_device)
 	 */
 	list_for_each_entry_safe(info, n, &mem_device->res_list, list) {
 		if (info->enabled) {
-			result = offline_memory(info->start_addr, info->length);
-			if (result)
-				return result;
+			if (!is_memblk_offline(info->start_addr,
+					       info->length)) {
+				result = offline_memory(info->start_addr,
+							info->length);
+				if (result)
+					return result;
+			}
 		}
 		list_del(&info->list);
 		kfree(info);

Signature · 1 line

1.7.1

Yasuaki Ishimatsu

Reviewed-by on v1 2/8

2012-07-20 07:46 UTC #5 lore raw

Re: [RFC PATCH 2/8] memory-hotplug: offline memory only when it is onlined

Hi Wen,

Quoted 14 lines · Wen Congyang

offline_memory() will fail if the memory is not onlined. So check whether the memory is onlined before calling offline_memory().
CC David Rientjes <[email protected]>
CC Jiang Liu <[email protected]>
CC Len Brown <[email protected]>
CC Benjamin Herrenschmidt <[email protected]>
CC Paul Mackerras <[email protected]>
CC Christoph Lameter <[email protected]>
Cc Minchan Kim <[email protected]>
CC Andrew Morton <[email protected]>
CC KOSAKI Motohiro <[email protected]>
CC Yasuaki Ishimatsu <[email protected]>
Signed-off-by Wen Congyang <[email protected]>

Quoted 1 line

---

I have no comment.

Reviewed-by Yasuaki Ishimatsu <[email protected]>

Thanks,
Yasuaki Ishimatsu

Quoted 15 lines

  drivers/acpi/acpi_memhotplug.c |   10 +++++++---
  1 files changed, 7 insertions(+), 3 deletions(-)

drivers/acpi/acpi_memhotplug.c +0 -3

index db8de39..712e767 100644
@@ -323,9 +323,13 @@ static int acpi_memory_disable_device(struct acpi_memory_device *mem_device)
  	 */
  	list_for_each_entry_safe(info, n, &mem_device->res_list, list) {
  		if (info->enabled) {
-			result = offline_memory(info->start_addr, info->length);
-			if (result)
-				return result;

drivers/acpi/acpi_memhotplug.c static int acpi_memory_disable_device(struct acpi_memory_device *mem_device) · line 335

Quoted 10 lines

drivers/acpi/acpi_memhotplug.c +7 -0

+			if (!is_memblk_offline(info->start_addr,
+					       info->length)) {
+				result = offline_memory(info->start_addr,
+							info->length);
+				if (result)
+					return result;
+			}
  		}
  		list_del(&info->list);
  		kfree(info);

Wen Congyang

author

2012-07-20 07:05 UTC #6 lore raw

RFC PATCH 3/8 memory-hotplug: call remove_memory() to cleanup when removing memory device

In reply to Wen Congyang

We should remove the following things when removing the memory device:
1. memmap and related sysfs files
2. iomem_resource
3. mem_section and related sysfs files
4. node and related sysfs files

The function remove_memory() can do this. So call it after the memory device is offlined.

CC David Rientjes <[email protected]>

CC Jiang Liu <[email protected]>

CC Len Brown <[email protected]>

CC Benjamin Herrenschmidt <[email protected]>

CC Paul Mackerras <[email protected]>

CC Christoph Lameter <[email protected]>

Cc Minchan Kim <[email protected]>

CC Andrew Morton <[email protected]>

CC KOSAKI Motohiro <[email protected]>

CC Yasuaki Ishimatsu <[email protected]>

Signed-off-by Wen Congyang <[email protected]>

---
 drivers/acpi/acpi_memhotplug.c |    7 ++++++-
 1 files changed, 6 insertions(+), 1 deletions(-)

drivers/acpi/acpi_memhotplug.c +6 -1

index 712e767..58e4e63 100644
@@ -315,7 +315,7 @@ static int acpi_memory_disable_device(struct acpi_memory_device *mem_device)
 {
 	int result;
 	struct acpi_memory_info *info, *n;
-
+	int node = mem_device->nid;
 
 	/*
 	 * Ask the VM to offline this memory range.

@@ -330,6 +330,11 @@ static int acpi_memory_disable_device(struct acpi_memory_device *mem_device)
 				if (result)
 					return result;
 			}
+
+			result = remove_memory(node, info->start_addr,
+					       info->length);
+			if (result)
+				return result;
 		}
 		list_del(&info->list);
 		kfree(info);

Signature · 7 lines

1.7.1

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Yasuaki Ishimatsu

Reviewed-by on v1 3/8

2012-07-20 07:49 UTC #7 lore raw

Re: [RFC PATCH 3/8] memory-hotplug: call remove_memory() to cleanup when removing memory device

Hi Wen,

Quoted 20 lines · Wen Congyang

We should remove the following things when removing the memory device:
1. memmap and related sysfs files
2. iomem_resource
3. mem_section and related sysfs files
4. node and related sysfs files
The function remove_memory() can do this. So call it after the memory device is offlined.
CC David Rientjes <[email protected]>
CC Jiang Liu <[email protected]>
CC Len Brown <[email protected]>
CC Benjamin Herrenschmidt <[email protected]>
CC Paul Mackerras <[email protected]>
CC Christoph Lameter <[email protected]>
Cc Minchan Kim <[email protected]>
CC Andrew Morton <[email protected]>
CC KOSAKI Motohiro <[email protected]>
CC Yasuaki Ishimatsu <[email protected]>
Signed-off-by Wen Congyang <[email protected]>

Quoted 1 line

---

I have no comment.

Reviewed-by Yasuaki Ishimatsu <[email protected]>

Thanks,
Yasuaki Ishimatsu

Quoted 19 lines

  drivers/acpi/acpi_memhotplug.c |    7 ++++++-
  1 files changed, 6 insertions(+), 1 deletions(-)

drivers/acpi/acpi_memhotplug.c +1 -1

index 712e767..58e4e63 100644
@@ -315,7 +315,7 @@ static int acpi_memory_disable_device(struct acpi_memory_device *mem_device)
  {
  	int result;
  	struct acpi_memory_info *info, *n;
-
+	int node = mem_device->nid;
 
  	/*
  	 * Ask the VM to offline this memory range.

@@ -330,6 +330,11 @@ static int acpi_memory_disable_device(struct acpi_memory_device *mem_device)
  				if (result)

drivers/acpi/acpi_memhotplug.c static int acpi_memory_disable_device(struct acpi_memory_device *mem_device) · line 340

Quoted 10 lines

drivers/acpi/acpi_memhotplug.c +5 -0

  					return result;
  			}
+
+			result = remove_memory(node, info->start_addr,
+					       info->length);
+			if (result)
+				return result;
  		}
  		list_del(&info->list);
  		kfree(info);

Wen Congyang

author

2012-07-20 07:06 UTC #8 lore raw

RFC PATCH 4/8 memory-hotplug: export the function acpi_bus_remove()

In reply to Wen Congyang

The function acpi_bus_remove() can remove a acpi device from acpi device. When a acpi device is removed, we need to call this function to remove the acpi device from acpi bus. So export this function.

CC David Rientjes <[email protected]>

CC Jiang Liu <[email protected]>

CC Len Brown <[email protected]>

CC Benjamin Herrenschmidt <[email protected]>

CC Paul Mackerras <[email protected]>

CC Christoph Lameter <[email protected]>

Cc Minchan Kim <[email protected]>

CC Andrew Morton <[email protected]>

CC KOSAKI Motohiro <[email protected]>

CC Yasuaki Ishimatsu <[email protected]>

Signed-off-by Wen Congyang <[email protected]>

---
 drivers/acpi/scan.c     |    3 ++-
 include/acpi/acpi_bus.h |    1 +
 2 files changed, 3 insertions(+), 1 deletions(-)

drivers/acpi/scan.c +2 -1

index c8a1f3b..d5e8d74 100644
@@ -1206,7 +1206,7 @@ static int acpi_device_set_context(struct acpi_device *device)
 	return -ENODEV;
 }
 
-static int acpi_bus_remove(struct acpi_device *dev, int rmdevice)
+int acpi_bus_remove(struct acpi_device *dev, int rmdevice)
 {
 	if (!dev)
 		return -EINVAL;

@@ -1228,6 +1228,7 @@ static int acpi_bus_remove(struct acpi_device *dev, int rmdevice)
 
 	return 0;
 }
+EXPORT_SYMBOL(acpi_bus_remove);
 
 static int acpi_add_single_object(struct acpi_device **child,
 				  acpi_handle handle, int type,

include/acpi/acpi_bus.h +1 -0

index 9e6e1c6..3eb0ba7 100644
@@ -356,6 +356,7 @@ bool acpi_bus_power_manageable(acpi_handle handle);
 bool acpi_bus_can_wakeup(acpi_handle handle);
 int acpi_power_resource_register_device(struct device *dev, acpi_handle handle);
 void acpi_power_resource_unregister_device(struct device *dev, acpi_handle handle);
+int acpi_bus_remove(struct acpi_device *dev, int rmdevice);
 #ifdef CONFIG_ACPI_PROC_EVENT
 int acpi_bus_generate_proc_event(struct acpi_device *device, u8 type, int data);
 int acpi_bus_generate_proc_event4(const char *class, const char *bid, u8 type, int data);

Signature · 7 lines

1.7.1

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Wen Congyang

author

2012-07-20 07:06 UTC #9 lore raw

RFC PATCH 5/8 memory-hotplug: call acpi_bus_remove() to remove memory device

In reply to Wen Congyang

The memory device has been ejected and powoffed, so we can call acpi_bus_remove() to remove the memory device from acpi bus.

CC David Rientjes <[email protected]>

CC Jiang Liu <[email protected]>

CC Len Brown <[email protected]>

CC Benjamin Herrenschmidt <[email protected]>

CC Paul Mackerras <[email protected]>

CC Christoph Lameter <[email protected]>

Cc Minchan Kim <[email protected]>

CC Andrew Morton <[email protected]>

CC KOSAKI Motohiro <[email protected]>

CC Yasuaki Ishimatsu <[email protected]>

Signed-off-by Wen Congyang <[email protected]>

---
 drivers/acpi/acpi_memhotplug.c |    3 ++-
 1 files changed, 2 insertions(+), 1 deletions(-)

drivers/acpi/acpi_memhotplug.c +2 -1

index 58e4e63..431a17c 100644
@@ -402,8 +402,9 @@ static void acpi_memory_device_notify(acpi_handle handle, u32 event, void *data)
 			printk(KERN_ERR PREFIX
 				    "Disable memory device\n");
 		/*
-		 * TBD: Invoke acpi_bus_remove to cleanup data structures
+		 * Invoke acpi_bus_remove() to remove memory device
 		 */
+		acpi_bus_remove(device, 1);
 		break;
 	default:
 		ACPI_DEBUG_PRINT((ACPI_DB_INFO,

Signature · 1 line

1.7.1

Wen Congyang

author

2012-07-20 07:07 UTC #10 lore raw

RFC PATCH 6/8 memory-hotplug: introduce new function arch_remove_memory()

In reply to Wen Congyang

We don't call __add_pages() directly in the function add_memory() because some other architecture related thins needs to be done before or after calling __add_pages(). So we should not call __remove_pages() directly in the function remove_memory. Introduce new function arch_remove_memory() to revert the things done in arch_add_memory().

Note: the function for x86_64 will be implemented later. And I don't know how to implement it for s390.

CC David Rientjes <[email protected]>

CC Jiang Liu <[email protected]>

CC Len Brown <[email protected]>

CC Benjamin Herrenschmidt <[email protected]>

CC Paul Mackerras <[email protected]>

CC Christoph Lameter <[email protected]>

Cc Minchan Kim <[email protected]>

CC Andrew Morton <[email protected]>

CC KOSAKI Motohiro <[email protected]>

CC Yasuaki Ishimatsu <[email protected]>

Signed-off-by Wen Congyang <[email protected]>

---
 arch/ia64/mm/init.c            |   16 ++++++++++++++++
 arch/powerpc/mm/mem.c          |   14 ++++++++++++++
 arch/s390/mm/init.c            |    8 ++++++++
 arch/sh/mm/init.c              |   15 +++++++++++++++
 arch/tile/mm/init.c            |    8 ++++++++
 arch/x86/mm/init_32.c          |   10 ++++++++++
 arch/x86/mm/init_64.c          |    7 +++++++
 include/linux/memory_hotplug.h |    1 +
 mm/memory_hotplug.c            |    2 +-
 9 files changed, 80 insertions(+), 1 deletions(-)

arch/ia64/mm/init.c +16 -0

index 0eab454..1e345ed 100644
@@ -688,6 +688,22 @@ int arch_add_memory(int nid, u64 start, u64 size)
 
 	return ret;
 }
+
+#ifdef CONFIG_MEMORY_HOTREMOVE
+int arch_remove_memory(u64 start, u64 size)
+{
+	unsigned long start_pfn = start >> PAGE_SHIFT;
+	unsigned long nr_pages = size >> PAGE_SHIFT;
+	int ret;
+
+	ret = __remove_pages(start_pfn, nr_pages);
+	if (ret)
+		pr_warn("%s: Problem encountered in __remove_pages() as"
+			" ret=%d\n", __func__,  ret);
+
+	return ret;
+}
+#endif
 #endif
 
 /*

arch/powerpc/mm/mem.c +14 -0

index baaafde..249cef4 100644
@@ -133,6 +133,20 @@ int arch_add_memory(int nid, u64 start, u64 size)
 
 	return __add_pages(nid, zone, start_pfn, nr_pages);
 }
+
+#ifdef CONFIG_MEMORY_HOTREMOVE
+int arch_remove_memory(u64 start, u64 size)
+{
+	unsigned long start_pfn = start >> PAGE_SHIFT;
+	unsigned long nr_pages = size >> PAGE_SHIFT;
+
+	start = (unsigned long)__va(start);
+	if (remove_section_mapping(start, start + size))
+		return -EINVAL;
+
+	return __remove_pages(start_pfn, nr_pages);
+}
+#endif
 #endif /* CONFIG_MEMORY_HOTPLUG */
 
 /*

arch/s390/mm/init.c +8 -0

index 2bea060..3de0d5b 100644
@@ -259,4 +259,12 @@ int arch_add_memory(int nid, u64 start, u64 size)
 		vmem_remove_mapping(start, size);
 	return rc;
 }
+
+#ifdef CONFIG_MEMORY_HOTREMOVE
+int arch_remove_memory(u64 start, u64 size)
+{
+	/* TODO */
+	return -EBUSY;
+}
+#endif
 #endif /* CONFIG_MEMORY_HOTPLUG */

arch/sh/mm/init.c +15 -0

index 82cc576..fc84491 100644
@@ -558,4 +558,19 @@ int memory_add_physaddr_to_nid(u64 addr)
 EXPORT_SYMBOL_GPL(memory_add_physaddr_to_nid);
 #endif
 
+#ifdef CONFIG_MEMORY_HOTREMOVE
+int arch_remove_memory(u64 start, u64 size)
+{
+	unsigned long start_pfn = start >> PAGE_SHIFT;
+	unsigned long nr_pages = size >> PAGE_SHIFT;
+	int ret;
+
+	ret = __remove_pages(start_pfn, nr_pages);
+	if (unlikely(ret))
+		pr_warn("%s: Failed, __remove_pages() == %d\n", __func__,
+			ret);
+
+	return ret;
+}
+#endif
 #endif /* CONFIG_MEMORY_HOTPLUG */

arch/tile/mm/init.c +8 -0

index 630dd2c..bdd8a99 100644
@@ -947,6 +947,14 @@ int remove_memory(u64 start, u64 size)
 {
 	return -EINVAL;
 }
+
+#ifdef CONFIG_MEMORY_HOTREMOVE
+int arch_remove_memory(u64 start, u64 size)
+{
+	/* TODO */
+	return -EBUSY;
+}
+#endif
 #endif
 
 struct kmem_cache *pgd_cache;

arch/x86/mm/init_32.c +10 -0

index 575d86f..a690153 100644
@@ -842,6 +842,16 @@ int arch_add_memory(int nid, u64 start, u64 size)
 
 	return __add_pages(nid, zone, start_pfn, nr_pages);
 }
+
+#ifdef CONFIG_MEMORY_HOTREMOVE
+int arch_remove_memory(unsigned long start, unsigned long size)
+{
+	unsigned long start_pfn = start >> PAGE_SHIFT;
+	unsigned long nr_pages = size >> PAGE_SHIFT;
+
+	return __remove_pages(start_pfn, nr_pages);
+}
+#endif
 #endif
 
 /*

arch/x86/mm/init_64.c +7 -0

index 9e635b3..78b94bc 100644
@@ -675,6 +675,13 @@ int arch_add_memory(int nid, u64 start, u64 size)
 }
 EXPORT_SYMBOL_GPL(arch_add_memory);
 
+#ifdef CONFIG_MEMORY_HOTREMOVE
+int arch_remove_memory(unsigned long start, unsigned long size)
+{
+	/* TODO */
+	return -EBUSY;
+}
+#endif
 #endif /* CONFIG_MEMORY_HOTPLUG */
 
 static struct kcore_list kcore_vsyscall;

include/linux/memory_hotplug.h +1 -0

index 2ba0a1a..8639799 100644
@@ -84,6 +84,7 @@ extern void __online_page_free(struct page *page);
 
 #ifdef CONFIG_MEMORY_HOTREMOVE
 extern bool is_pageblock_removable_nolock(struct page *page);
+extern int arch_remove_memory(unsigned long start, unsigned long size);
 #endif /* CONFIG_MEMORY_HOTREMOVE */
 
 /* reasonably generic interface to expand the physical pages in a zone  */

mm/memory_hotplug.c +1 -1

index dccdf71..cc2c8b9 100644
@@ -1055,7 +1055,7 @@ int __ref remove_memory(int nid, u64 start, u64 size)
 		unregister_one_node(nid);
 	}
 
-	__remove_pages(start >> PAGE_SHIFT, size >> PAGE_SHIFT);
+	arch_remove_memory(start, size);
 out:
 	unlock_memory_hotplug();
 	return ret;

Signature · 7 lines

1.7.1

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Yasuaki Ishimatsu

2012-07-20 08:21 UTC #11 lore raw

Re: [RFC PATCH 6/8] memory-hotplug: introduce new function arch_remove_memory()

Quoted 9 lines · Wen Congyang

We don't call __add_pages() directly in the function add_memory() because some other architecture related thins needs to be done before or after calling __add_pages(). So we should not call __remove_pages() directly in the function remove_memory. Introduce new function arch_remove_memory() to revert the things done in arch_add_memory().
Note: the function for x86_64 will be implemented later. And I don't know how to implement it for s390.

I think you need cc to other arch ML for reviewing the patch.

Quoted 161 lines

CC David Rientjes <[email protected]>

CC Jiang Liu <[email protected]>

CC Len Brown <[email protected]>

CC Benjamin Herrenschmidt <[email protected]>

CC Paul Mackerras <[email protected]>

CC Christoph Lameter <[email protected]>

Cc Minchan Kim <[email protected]>

CC Andrew Morton <[email protected]>

CC KOSAKI Motohiro <[email protected]>

CC Yasuaki Ishimatsu <[email protected]>

Signed-off-by Wen Congyang <[email protected]>

---
  arch/ia64/mm/init.c            |   16 ++++++++++++++++
  arch/powerpc/mm/mem.c          |   14 ++++++++++++++
  arch/s390/mm/init.c            |    8 ++++++++
  arch/sh/mm/init.c              |   15 +++++++++++++++
  arch/tile/mm/init.c            |    8 ++++++++
  arch/x86/mm/init_32.c          |   10 ++++++++++
  arch/x86/mm/init_64.c          |    7 +++++++
  include/linux/memory_hotplug.h |    1 +
  mm/memory_hotplug.c            |    2 +-
  9 files changed, 80 insertions(+), 1 deletions(-)

arch/ia64/mm/init.c +16 -0

index 0eab454..1e345ed 100644
@@ -688,6 +688,22 @@ int arch_add_memory(int nid, u64 start, u64 size)
 
  	return ret;
  }
+
+#ifdef CONFIG_MEMORY_HOTREMOVE
+int arch_remove_memory(u64 start, u64 size)
+{
+	unsigned long start_pfn = start >> PAGE_SHIFT;
+	unsigned long nr_pages = size >> PAGE_SHIFT;
+	int ret;
+
+	ret = __remove_pages(start_pfn, nr_pages);
+	if (ret)
+		pr_warn("%s: Problem encountered in __remove_pages() as"
+			" ret=%d\n", __func__,  ret);
+
+	return ret;
+}
+#endif
  #endif
 
  /*

arch/powerpc/mm/mem.c +14 -0

index baaafde..249cef4 100644
@@ -133,6 +133,20 @@ int arch_add_memory(int nid, u64 start, u64 size)
 
  	return __add_pages(nid, zone, start_pfn, nr_pages);
  }
+
+#ifdef CONFIG_MEMORY_HOTREMOVE
+int arch_remove_memory(u64 start, u64 size)
+{
+	unsigned long start_pfn = start >> PAGE_SHIFT;
+	unsigned long nr_pages = size >> PAGE_SHIFT;
+
+	start = (unsigned long)__va(start);
+	if (remove_section_mapping(start, start + size))
+		return -EINVAL;
+
+	return __remove_pages(start_pfn, nr_pages);
+}
+#endif
  #endif /* CONFIG_MEMORY_HOTPLUG */
 
  /*

arch/s390/mm/init.c +8 -0

index 2bea060..3de0d5b 100644
@@ -259,4 +259,12 @@ int arch_add_memory(int nid, u64 start, u64 size)
  		vmem_remove_mapping(start, size);
  	return rc;
  }
+
+#ifdef CONFIG_MEMORY_HOTREMOVE
+int arch_remove_memory(u64 start, u64 size)
+{
+	/* TODO */
+	return -EBUSY;
+}
+#endif
  #endif /* CONFIG_MEMORY_HOTPLUG */

arch/sh/mm/init.c +15 -0

index 82cc576..fc84491 100644
@@ -558,4 +558,19 @@ int memory_add_physaddr_to_nid(u64 addr)
  EXPORT_SYMBOL_GPL(memory_add_physaddr_to_nid);
  #endif
 
+#ifdef CONFIG_MEMORY_HOTREMOVE
+int arch_remove_memory(u64 start, u64 size)
+{
+	unsigned long start_pfn = start >> PAGE_SHIFT;
+	unsigned long nr_pages = size >> PAGE_SHIFT;
+	int ret;
+
+	ret = __remove_pages(start_pfn, nr_pages);
+	if (unlikely(ret))
+		pr_warn("%s: Failed, __remove_pages() == %d\n", __func__,
+			ret);
+
+	return ret;
+}
+#endif
  #endif /* CONFIG_MEMORY_HOTPLUG */

arch/tile/mm/init.c +8 -0

index 630dd2c..bdd8a99 100644
@@ -947,6 +947,14 @@ int remove_memory(u64 start, u64 size)
  {
  	return -EINVAL;
  }
+
+#ifdef CONFIG_MEMORY_HOTREMOVE
+int arch_remove_memory(u64 start, u64 size)
+{
+	/* TODO */
+	return -EBUSY;
+}
+#endif
  #endif
 
  struct kmem_cache *pgd_cache;

arch/x86/mm/init_32.c +10 -0

index 575d86f..a690153 100644
@@ -842,6 +842,16 @@ int arch_add_memory(int nid, u64 start, u64 size)
 
  	return __add_pages(nid, zone, start_pfn, nr_pages);
  }
+
+#ifdef CONFIG_MEMORY_HOTREMOVE
+int arch_remove_memory(unsigned long start, unsigned long size)
+{
+	unsigned long start_pfn = start >> PAGE_SHIFT;
+	unsigned long nr_pages = size >> PAGE_SHIFT;
+
+	return __remove_pages(start_pfn, nr_pages);
+}
+#endif
  #endif
 
  /*

arch/x86/mm/init_64.c +0 -0

index 9e635b3..78b94bc 100644

arch/x86/mm/init_64.c int arch_add_memory(int nid, u64 start, u64 size) · line 684

Quoted 10 lines

arch/x86/mm/init_64.c +7 -0

@@ -675,6 +675,13 @@ int arch_add_memory(int nid, u64 start, u64 size)
  }
  EXPORT_SYMBOL_GPL(arch_add_memory);
 
+#ifdef CONFIG_MEMORY_HOTREMOVE
+int arch_remove_memory(unsigned long start, unsigned long size)
+{
+	/* TODO */
+	return -EBUSY;
+}
+#endif

Why does not the function call __remove_pages()?

Thanks,
Yasuaki ishimatsu

Quoted 21 lines

arch/x86/mm/init_64.c +0 -0

  #endif /* CONFIG_MEMORY_HOTPLUG */
 
  static struct kcore_list kcore_vsyscall;

include/linux/memory_hotplug.h +1 -0

index 2ba0a1a..8639799 100644
@@ -84,6 +84,7 @@ extern void __online_page_free(struct page *page);
 
  #ifdef CONFIG_MEMORY_HOTREMOVE
  extern bool is_pageblock_removable_nolock(struct page *page);
+extern int arch_remove_memory(unsigned long start, unsigned long size);
  #endif /* CONFIG_MEMORY_HOTREMOVE */
 
  /* reasonably generic interface to expand the physical pages in a zone  */

mm/memory_hotplug.c +0 -0

index dccdf71..cc2c8b9 100644

mm/memory_hotplug.c int __ref remove_memory(int nid, u64 start, u64 size) · line 1061

Quoted 8 lines

mm/memory_hotplug.c +1 -1

@@ -1055,7 +1055,7 @@ int __ref remove_memory(int nid, u64 start, u64 size)
  		unregister_one_node(nid);
  	}
 
-	__remove_pages(start >> PAGE_SHIFT, size >> PAGE_SHIFT);
+	arch_remove_memory(start, size);
  out:
  	unlock_memory_hotplug();
  	return ret;

Wen Congyang

author

2012-07-20 08:34 UTC #12 lore raw

Re: [RFC PATCH 6/8] memory-hotplug: introduce new function arch_remove_memory()

At 07/20/2012 04:21 PM, Yasuaki Ishimatsu Wrote:

Quoted 8 lines

Quoted 7 lines · Wen Congyang
We don't call __add_pages() directly in the function add_memory() because some other architecture related thins needs to be done before or after calling __add_pages(). So we should not call __remove_pages() directly in the function remove_memory. Introduce new function arch_remove_memory() to revert the things done in arch_add_memory().

Quoted 4 lines

Quoted 2 lines
Note: the function for x86_64 will be implemented later. And I don't know how to implement it for s390.
I think you need cc to other arch ML for reviewing the patch.

I forgot to do it.

Quoted 166 lines

CC David Rientjes <[email protected]>

CC Jiang Liu <[email protected]>

CC Len Brown <[email protected]>

CC Benjamin Herrenschmidt <[email protected]>

CC Paul Mackerras <[email protected]>

CC Christoph Lameter <[email protected]>

Cc Minchan Kim <[email protected]>

CC Andrew Morton <[email protected]>

CC KOSAKI Motohiro <[email protected]>

CC Yasuaki Ishimatsu <[email protected]>

Signed-off-by Wen Congyang <[email protected]>

---
  arch/ia64/mm/init.c            |   16 ++++++++++++++++
  arch/powerpc/mm/mem.c          |   14 ++++++++++++++
  arch/s390/mm/init.c            |    8 ++++++++
  arch/sh/mm/init.c              |   15 +++++++++++++++
  arch/tile/mm/init.c            |    8 ++++++++
  arch/x86/mm/init_32.c          |   10 ++++++++++
  arch/x86/mm/init_64.c          |    7 +++++++
  include/linux/memory_hotplug.h |    1 +
  mm/memory_hotplug.c            |    2 +-
  9 files changed, 80 insertions(+), 1 deletions(-)

arch/ia64/mm/init.c +16 -0

index 0eab454..1e345ed 100644
@@ -688,6 +688,22 @@ int arch_add_memory(int nid, u64 start, u64 size)
 
      return ret;
  }
+
+#ifdef CONFIG_MEMORY_HOTREMOVE
+int arch_remove_memory(u64 start, u64 size)
+{
+    unsigned long start_pfn = start >> PAGE_SHIFT;
+    unsigned long nr_pages = size >> PAGE_SHIFT;
+    int ret;
+
+    ret = __remove_pages(start_pfn, nr_pages);
+    if (ret)
+        pr_warn("%s: Problem encountered in __remove_pages() as"
+            " ret=%d\n", __func__,  ret);
+
+    return ret;
+}
+#endif
  #endif
 
  /*

arch/powerpc/mm/mem.c +14 -0

index baaafde..249cef4 100644
@@ -133,6 +133,20 @@ int arch_add_memory(int nid, u64 start, u64 size)
 
      return __add_pages(nid, zone, start_pfn, nr_pages);
  }
+
+#ifdef CONFIG_MEMORY_HOTREMOVE
+int arch_remove_memory(u64 start, u64 size)
+{
+    unsigned long start_pfn = start >> PAGE_SHIFT;
+    unsigned long nr_pages = size >> PAGE_SHIFT;
+
+    start = (unsigned long)__va(start);
+    if (remove_section_mapping(start, start + size))
+        return -EINVAL;
+
+    return __remove_pages(start_pfn, nr_pages);
+}
+#endif
  #endif /* CONFIG_MEMORY_HOTPLUG */
 
  /*

arch/s390/mm/init.c +8 -0

index 2bea060..3de0d5b 100644
@@ -259,4 +259,12 @@ int arch_add_memory(int nid, u64 start, u64 size)
          vmem_remove_mapping(start, size);
      return rc;
  }
+
+#ifdef CONFIG_MEMORY_HOTREMOVE
+int arch_remove_memory(u64 start, u64 size)
+{
+    /* TODO */
+    return -EBUSY;
+}
+#endif
  #endif /* CONFIG_MEMORY_HOTPLUG */

arch/sh/mm/init.c +15 -0

index 82cc576..fc84491 100644
@@ -558,4 +558,19 @@ int memory_add_physaddr_to_nid(u64 addr)
  EXPORT_SYMBOL_GPL(memory_add_physaddr_to_nid);
  #endif
 
+#ifdef CONFIG_MEMORY_HOTREMOVE
+int arch_remove_memory(u64 start, u64 size)
+{
+    unsigned long start_pfn = start >> PAGE_SHIFT;
+    unsigned long nr_pages = size >> PAGE_SHIFT;
+    int ret;
+
+    ret = __remove_pages(start_pfn, nr_pages);
+    if (unlikely(ret))
+        pr_warn("%s: Failed, __remove_pages() == %d\n", __func__,
+            ret);
+
+    return ret;
+}
+#endif
  #endif /* CONFIG_MEMORY_HOTPLUG */

arch/tile/mm/init.c +8 -0

index 630dd2c..bdd8a99 100644
@@ -947,6 +947,14 @@ int remove_memory(u64 start, u64 size)
  {
      return -EINVAL;
  }
+
+#ifdef CONFIG_MEMORY_HOTREMOVE
+int arch_remove_memory(u64 start, u64 size)
+{
+    /* TODO */
+    return -EBUSY;
+}
+#endif
  #endif
 
  struct kmem_cache *pgd_cache;

arch/x86/mm/init_32.c +10 -0

index 575d86f..a690153 100644
@@ -842,6 +842,16 @@ int arch_add_memory(int nid, u64 start, u64 size)
 
      return __add_pages(nid, zone, start_pfn, nr_pages);
  }
+
+#ifdef CONFIG_MEMORY_HOTREMOVE
+int arch_remove_memory(unsigned long start, unsigned long size)
+{
+    unsigned long start_pfn = start >> PAGE_SHIFT;
+    unsigned long nr_pages = size >> PAGE_SHIFT;
+
+    return __remove_pages(start_pfn, nr_pages);
+}
+#endif
  #endif
 
  /*

arch/x86/mm/init_64.c +0 -0

index 9e635b3..78b94bc 100644
@@ -675,6 +675,13 @@ int arch_add_memory(int nid, u64 start, u64 size)
  }
  EXPORT_SYMBOL_GPL(arch_add_memory);

Quoted 9 lines

arch/x86/mm/init_64.c int arch_add_memory(int nid, u64 start, u64 size) · line 684
Quoted 7 lines
arch/x86/mm/init_64.c +7 -0
+#ifdef CONFIG_MEMORY_HOTREMOVE
+int arch_remove_memory(unsigned long start, unsigned long size)
+{
+    /* TODO */
+    return -EBUSY;
+}
+#endif
Why does not the function call __remove_pages()?

arch_remove_memory() should clear the page table after remove pages here. I implement this function in patch 8.

Thanks
Wen Congyang

Quoted 30 lines

Thanks,
Yasuaki ishimatsu

Quoted 26 lines

arch/x86/mm/init_64.c +0 -0

  #endif /* CONFIG_MEMORY_HOTPLUG */
 
  static struct kcore_list kcore_vsyscall;

include/linux/memory_hotplug.h +1 -0

index 2ba0a1a..8639799 100644
@@ -84,6 +84,7 @@ extern void __online_page_free(struct page *page);
 
  #ifdef CONFIG_MEMORY_HOTREMOVE
  extern bool is_pageblock_removable_nolock(struct page *page);
+extern int arch_remove_memory(unsigned long start, unsigned long size);
  #endif /* CONFIG_MEMORY_HOTREMOVE */
 
  /* reasonably generic interface to expand the physical pages in a

zone */

mm/memory_hotplug.c +0 -0

index dccdf71..cc2c8b9 100644
@@ -1055,7 +1055,7 @@ int __ref remove_memory(int nid, u64 start, u64
 size)
          unregister_one_node(nid);
      }

Quoted 9 lines

Quoted 6 lines

mm/memory_hotplug.c +1 -1

-    __remove_pages(start >> PAGE_SHIFT, size >> PAGE_SHIFT);
+    arch_remove_memory(start, size);
  out:
      unlock_memory_hotplug();

      return ret;

Wen Congyang

author

2012-07-20 07:08 UTC #13 lore raw

RFC PATCH 7/8 x86: make __split_large_page() generally avialable

In reply to Wen Congyang

We need to clear page table after calling __remove_pages(). We may need to clear part of pmd or pud's entry when clearing the page table. So we need to split it to small page. Make __split_large_page() generally avialable, and we call call this function to split large page.

CC David Rientjes <[email protected]>

CC Jiang Liu <[email protected]>

CC Len Brown <[email protected]>

CC Benjamin Herrenschmidt <[email protected]>

CC Paul Mackerras <[email protected]>

CC Christoph Lameter <[email protected]>

Cc Minchan Kim <[email protected]>

CC Andrew Morton <[email protected]>

CC KOSAKI Motohiro <[email protected]>

CC Yasuaki Ishimatsu <[email protected]>

Signed-off-by Wen Congyang <[email protected]>

---
 arch/x86/include/asm/pgtable_types.h |    1 +
 arch/x86/mm/pageattr.c               |   47 ++++++++++++++++++----------------
 2 files changed, 26 insertions(+), 22 deletions(-)

arch/x86/include/asm/pgtable_types.h +1 -0

index 013286a..b725af2 100644
@@ -334,6 +334,7 @@ static inline void update_page_count(int level, unsigned long pages) { }
  * as a pte too.
  */
 extern pte_t *lookup_address(unsigned long address, unsigned int *level);
+extern int __split_large_page(pte_t *kpte, unsigned long address, pte_t *pbase);
 
 #endif	/* !__ASSEMBLY__ */

arch/x86/mm/pageattr.c +25 ~~-22~~

73 lines not on this page · raw message

Signature · 7 lines

1.7.1

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Wen Congyang

author

2012-07-20 07:09 UTC #14 lore raw

RFC PATCH 8/8 memory-hotplug: implement arch_remove_memory()

In reply to Wen Congyang

Set the entry for the removed memory to 0. If the entry related meory is not whole removed, split it to smaller page, and clear it.

CC David Rientjes <[email protected]>

CC Jiang Liu <[email protected]>

CC Len Brown <[email protected]>

CC Benjamin Herrenschmidt <[email protected]>

CC Paul Mackerras <[email protected]>

CC Christoph Lameter <[email protected]>

Cc Minchan Kim <[email protected]>

CC Andrew Morton <[email protected]>

CC KOSAKI Motohiro <[email protected]>

CC Yasuaki Ishimatsu <[email protected]>

Signed-off-by Wen Congyang <[email protected]>

---
 arch/x86/mm/init_64.c |  159 ++++++++++++++++++++++++++++++++++++++++++++++++-
 1 files changed, 156 insertions(+), 3 deletions(-)

arch/x86/mm/init_64.c +156 -3

index 78b94bc..d78f352 100644
@@ -675,11 +675,164 @@ int arch_add_memory(int nid, u64 start, u64 size)
 }
 EXPORT_SYMBOL_GPL(arch_add_memory);
 
+static void __meminit
+phys_pte_remove(pte_t *pte_page, unsigned long addr, unsigned long end)
+{
+	unsigned pages = 0;
+	int i = pte_index(addr);
+
+	pte_t *pte = pte_page + pte_index(addr);
+
+	for (; i < PTRS_PER_PTE; i++, addr += PAGE_SIZE, pte++) {
+
+		if (addr >= end)
+			break;
+
+		if (!pte_present(*pte))
+			continue;
+
+		pages++;
+		set_pte(pte, __pte(0));
+	}
+
+	update_page_count(PG_LEVEL_4K, -pages);
+}
+
+static void __meminit
+phys_pmd_remove(pmd_t *pmd_page, unsigned long addr, unsigned long end)
+{
+	unsigned long pages = 0, next;
+	int i = pmd_index(addr);
+
+	for (; i < PTRS_PER_PMD; i++, addr = next) {
+		unsigned long pte_phys;
+		pmd_t *pmd = pmd_page + pmd_index(addr);
+		pte_t *pte;
+
+		if (addr >= end)
+			break;
+
+		next = (addr & PMD_MASK) + PMD_SIZE;
+
+		if (!pmd_present(*pmd))
+			continue;
+
+		if (pmd_large(*pmd)) {
+			if ((addr & ~PMD_MASK) == 0 && next <= end) {
+				set_pmd(pmd, __pmd(0));
+				pages++;
+				continue;
+			}
+
+			/*
+			 * We use 2M page, but we need to remove part of them,
+			 * so split 2M page to 4K page.
+			 */
+			pte = alloc_low_page(&pte_phys);
+			__split_large_page((pte_t *)pmd, addr, pte);
+
+			spin_lock(&init_mm.page_table_lock);
+			pmd_populate_kernel(&init_mm, pmd, __va(pte_phys));
+			spin_unlock(&init_mm.page_table_lock);
+		}
+
+		spin_lock(&init_mm.page_table_lock);
+		pte = map_low_page((pte_t *)pmd_page_vaddr(*pmd));
+		phys_pte_remove(pte, addr, end);
+		unmap_low_page(pte);
+		spin_unlock(&init_mm.page_table_lock);
+	}
+	update_page_count(PG_LEVEL_2M, -pages);
+}
+
+static void __meminit
+phys_pud_remove(pud_t *pud_page, unsigned long addr, unsigned long end)
+{
+	unsigned long pages = 0, next;
+	int i = pud_index(addr);
+
+	for (; i < PTRS_PER_PUD; i++, addr = next) {
+		unsigned long pmd_phys;
+		pud_t *pud = pud_page + pud_index(addr);
+		pmd_t *pmd;
+
+		if (addr >= end)
+			break;
+
+		next = (addr & PUD_MASK) + PUD_SIZE;
+
+		if (!pud_present(*pud))
+			continue;
+
+		if (pud_large(*pud)) {
+			if ((addr & ~PUD_MASK) == 0 && next <= end) {
+				set_pud(pud, __pud(0));
+				pages++;
+				continue;
+			}
+
+			/*
+			 * We use 1G page, but we need to remove part of them,
+			 * so split 1G page to 2M page.
+			 */
+			pmd = alloc_low_page(&pmd_phys);
+			__split_large_page((pte_t *)pud, addr, (pte_t *)pmd);
+
+			spin_lock(&init_mm.page_table_lock);
+			pud_populate(&init_mm, pud, __va(pmd_phys));
+			spin_unlock(&init_mm.page_table_lock);
+		}
+
+		pmd = map_low_page(pmd_offset(pud, 0));
+		phys_pmd_remove(pmd, addr, end);
+		unmap_low_page(pmd);
+		__flush_tlb_all();
+	}
+	__flush_tlb_all();
+
+	update_page_count(PG_LEVEL_1G, -pages);
+}
+
+void __meminit
+kernel_physical_mapping_remove(unsigned long start, unsigned long end)
+{
+	unsigned long next;
+
+	start = (unsigned long)__va(start);
+	end = (unsigned long)__va(end);
+
+	for (; start < end; start = next) {
+		pgd_t *pgd = pgd_offset_k(start);
+		pud_t *pud;
+
+		next = (start + PGDIR_SIZE) & PGDIR_MASK;
+		if (next > end)
+			next = end;
+
+		if (!pgd_present(*pgd))
+			continue;
+
+		pud = map_low_page((pud_t *)pgd_page_vaddr(*pgd));
+		phys_pud_remove(pud, __pa(start), __pa(end));
+		unmap_low_page(pud);
+	}
+
+	__flush_tlb_all();
+}
+
 #ifdef CONFIG_MEMORY_HOTREMOVE
-int arch_remove_memory(unsigned long start, unsigned long size)
+int __ref arch_remove_memory(unsigned long start, unsigned long size)
 {
-	/* TODO */
-	return -EBUSY;
+	unsigned long start_pfn = start >> PAGE_SHIFT;
+	unsigned long nr_pages = size >> PAGE_SHIFT;
+	int ret;
+
+	ret = __remove_pages(start_pfn, nr_pages);
+	WARN_ON_ONCE(ret);
+
+	kernel_physical_mapping_remove(start, start + size);
+
+	return ret;
 }
 #endif
 #endif /* CONFIG_MEMORY_HOTPLUG */

Signature · 7 lines

1.7.1

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Yasuaki Ishimatsu

2012-07-20 07:31 UTC #15 lore raw

In reply to Wen Congyang

[Hi Wen,

Good news!! I was waiting for this patch to come. Applying the patches, can we hot-remove physical memory completely?

Thanks,
Yasuaki Ishimatsu

Quoted 45 lines · Wen Congyang

This patch series aims to support physical memory hot-remove(clear page table).
This patch series base on ishimatsu's patch series. You can get it here:
http://www.spinics.net/lists/linux-acpi/msg36804.html
The patches can remove following things:
- page table of removed memory
If you find lack of function for physical memory hot-remove, please let me know.
Note:
* The patch "remove memory info from list before freeing it" is being disccussed in other thread. But for testing the patch series, the patch is needed. So I added the patch as [PATCH 0/8].
* You need to apply ishimatsu's patch series first before applying this patch series.
Wen Congyang (8):
memory-hotplug: store the node id in acpi_memory_device
memory-hotplug: offline memory only when it is onlined
memory-hotplug: call remove_memory() to cleanup when removing memory
device
memory-hotplug: export the function acpi_bus_remove()
memory-hotplug: call acpi_bus_remove() to remove memory device
memory-hotplug: introduce new function arch_remove_memory()
x86: make __split_large_page() generally avialable
memory-hotplug: implement arch_remove_memory()
  arch/ia64/mm/init.c                  |   16 ++++
  arch/powerpc/mm/mem.c                |   14 +++
  arch/s390/mm/init.c                  |    8 ++
  arch/sh/mm/init.c                    |   15 +++
  arch/tile/mm/init.c                  |    8 ++
  arch/x86/include/asm/pgtable_types.h |    1 +
  arch/x86/mm/init_32.c                |   10 ++
  arch/x86/mm/init_64.c                |  160 ++++++++++++++++++++++++++++++++++
  arch/x86/mm/pageattr.c               |   47 +++++-----
  drivers/acpi/acpi_memhotplug.c       |   24 ++++--
  drivers/acpi/scan.c                  |    3 +-
  include/acpi/acpi_bus.h              |    1 +
  include/linux/memory_hotplug.h       |    1 +
  mm/memory_hotplug.c                  |    2 +-
  14 files changed, 280 insertions(+), 30 deletions(-)

Wen Congyang

author

2012-07-23 09:06 UTC #16 lore raw

At 07/20/2012 03:31 PM, Yasuaki Ishimatsu Wrote:

Quoted 4 lines

[Hi Wen,
Good news!! I was waiting for this patch to come. Applying the patches, can we hot-remove physical memory completely?

If all functions success, I guess so.

Thanks
Wen Congyang

Quoted 59 lines

Thanks,
Yasuaki Ishimatsu
Quoted 51 lines · Wen Congyang
This patch series aims to support physical memory hot-remove(clear page table).
This patch series base on ishimatsu's patch series. You can get it here:
http://www.spinics.net/lists/linux-acpi/msg36804.html
The patches can remove following things:
- page table of removed memory
If you find lack of function for physical memory hot-remove, please let me
know.
Note:
* The patch "remove memory info from list before freeing it" is being disccussed
in other thread. But for testing the patch series, the patch is
needed.
So I added the patch as [PATCH 0/8].
* You need to apply ishimatsu's patch series first before applying this patch
series.
Wen Congyang (8):
memory-hotplug: store the node id in acpi_memory_device
memory-hotplug: offline memory only when it is onlined
memory-hotplug: call remove_memory() to cleanup when removing memory
device
memory-hotplug: export the function acpi_bus_remove()
memory-hotplug: call acpi_bus_remove() to remove memory device
memory-hotplug: introduce new function arch_remove_memory()
x86: make __split_large_page() generally avialable
memory-hotplug: implement arch_remove_memory()
  arch/ia64/mm/init.c                  |   16 ++++
  arch/powerpc/mm/mem.c                |   14 +++
  arch/s390/mm/init.c                  |    8 ++
  arch/sh/mm/init.c                    |   15 +++
  arch/tile/mm/init.c                  |    8 ++
  arch/x86/include/asm/pgtable_types.h |    1 +
  arch/x86/mm/init_32.c                |   10 ++
  arch/x86/mm/init_64.c                |  160
++++++++++++++++++++++++++++++++++
  arch/x86/mm/pageattr.c               |   47 +++++-----
  drivers/acpi/acpi_memhotplug.c       |   24 ++++--
  drivers/acpi/scan.c                  |    3 +-
  include/acpi/acpi_bus.h              |    1 +
  include/linux/memory_hotplug.h       |    1 +
  mm/memory_hotplug.c                  |    2 +-
  14 files changed, 280 insertions(+), 30 deletions(-)