Skip to contentSkip to conversation

memory-hotplug : hot-remove physical memory

RFC PATCH Superseded Yasuaki Ishimatsu 5 participants 51 messages Jul 9, 2012 – Jul 17, 2012

Subsystems (7)
Memory management x86 ACPI MM core +3 more subsystems

v3 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

v3 cover memory-hotplug : hot-remove physical memory

This patch series aims to support physical memory hot-remove.

[RFC PATCH v3 1/13] memory-hotplug : rename remove_memory to offline_memory
[RFC PATCH v3 2/13] memory-hotplug : add physical memory hotplug code to acpi_memory_device_remove [RFC PATCH v3 3/13] memory-hotplug : unify argument of firmware_map_add_early/hotplug [RFC PATCH v3 4/13] memory-hotplug : remove /sys/firmware/memmap/X sysfs
[RFC PATCH v3 5/13] memory-hotplug : does not release memory region in PAGES_PER_SECTION chunks [RFC PATCH v3 6/13] memory-hotplug : add memory_block_release
[RFC PATCH v3 7/13] memory-hotplug : remove_memory calls __remove_pages
[RFC PATCH v3 8/13] memory-hotplug : check page type in get_page_bootmem
[RFC PATCH v3 9/13] memory-hotplug : move register_page_bootmem_info_node and put_page_bootmem for
sparse-vmemmap
[RFC PATCH v3 10/13] memory-hotplug : implement register_page_bootmem_info_section of sparse-vmemmap [RFC PATCH v3 11/13] memory-hotplug : free memmap of sparse-vmemmap
[RFC PATCH v3 12/13] memory-hotplug : add node_device_release
[RFC PATCH v3 13/13] memory-hotplug : remove sysfs file of node

Even if you apply these patches, you cannot remove the physical memory completely since these patches are still under development. I want you to cooperate to improve the physical memory hot-remove. So please review these patches and give your comment/idea.

The patches can free/remove following things:

- acpi_memory_info : [RFC PATCH 2/13]
- /sys/firmware/memmap/X/{end, start, type} : [RFC PATCH 4/13]

  - iomem_resource                            : [RFC PATCH 5/13]
  - mem_section and related sysfs files       : [RFC PATCH 6-11/13]
  - node and related sysfs files              : [RFC PATCH 12-13/13]

The patches cannot do following things yet:

- page table of removed memory

If you find lack of function for physical memory hot-remove, please let me know.

change log of v3:
* rebase to 3.5.0-rc6

[RFC PATCH v2 2/13]
* remove extra kobject_put()

* The patch was commented by Wen. Wen's comment is "acpi_memory_device_remove() should ignore a return value of remove_memory() since caller does not care the return value". But I did not change it since I think caller should care the return value. And I am trying to fix it as follow:

https://lkml.org/lkml/2012/7/5/624

[RFC PATCH v2 4/13]
* remove a firmware_memmap_entry allocated by kzmalloc()

change log of v2:
[RFC PATCH v2 2/13]
* check whether memory block is offline or not before calling offline_memory()
* check whether section is valid or not in is_memblk_offline()
* call kobject_put() for each memory_block in is_memblk_offline()

[RFC PATCH v2 3/13]
* unify the end argument of firmware_map_add_early/hotplug

[RFC PATCH v2 4/13]
* add release_firmware_map_entry() for freeing firmware_map_entry

[RFC PATCH v2 6/13]
* add release_memory_block() for freeing memory_block

[RFC PATCH v2 11/13]
* fix wrong arguments of free_pages()

---
 arch/powerpc/platforms/pseries/hotplug-memory.c |   16 +-
 arch/x86/mm/init_64.c                           |  144 ++++++++++++++++++++++++
 drivers/acpi/acpi_memhotplug.c                  |   28 ++++
 drivers/base/memory.c                           |   54 ++++++++-
 drivers/base/node.c                             |    7 +
 drivers/firmware/memmap.c                       |   78 ++++++++++++-
 include/linux/firmware-map.h                    |    6 +
 include/linux/memory.h                          |    5
 include/linux/memory_hotplug.h                  |   17 --
 include/linux/mm.h                              |    5
 mm/memory_hotplug.c                             |   98 ++++++++++++----
 mm/sparse.c                                     |    5
 12 files changed, 414 insertions(+), 49 deletions(-)
#1 lore raw

RFC PATCH v3 1/13 memory-hotplug : rename remove_memory to offline_memory

remove_memory() does not remove memory but just offlines memory. The patch changes name of it to offline_memory().

CC David Rientjes <[email protected]>
CC Jiang Liu <[email protected]>
CC Len Brown <[email protected]>
CC Benjamin Herrenschmidt <[email protected]>
CC Paul Mackerras <[email protected]>
CC Christoph Lameter <[email protected]>
Cc Minchan Kim <[email protected]>
CC Andrew Morton <[email protected]>
CC KOSAKI Motohiro <[email protected]>
CC Wen Congyang <[email protected]>
---
 drivers/acpi/acpi_memhotplug.c |    2 +-
 drivers/base/memory.c          |    4 ++--
 include/linux/memory_hotplug.h |    2 +-
 mm/memory_hotplug.c            |    6 +++---
 4 files changed, 7 insertions(+), 7 deletions(-)
drivers/acpi/acpi_memhotplug.c +1 -1
Index: linux-3.5-rc4/drivers/acpi/acpi_memhotplug.c
===================================================================
@@ -318,7 +318,7 @@ static int acpi_memory_disable_device(st
 	 */
 	list_for_each_entry_safe(info, n, &mem_device->res_list, list) {
 		if (info->enabled) {
-			result = remove_memory(info->start_addr, info->length);
+			result = offline_memory(info->start_addr, info->length);
 			if (result)
 				return result;
 		}
drivers/base/memory.c +2 -2
Index: linux-3.5-rc4/drivers/base/memory.c
===================================================================
@@ -266,8 +266,8 @@ memory_block_action(unsigned long phys_i
 			break;
 		case MEM_OFFLINE:
 			start_paddr = page_to_pfn(first_page) << PAGE_SHIFT;
-			ret = remove_memory(start_paddr,
-					    nr_pages << PAGE_SHIFT);
+			ret = offline_memory(start_paddr,
+					     nr_pages << PAGE_SHIFT);
 			break;
 		default:
 			WARN(1, KERN_WARNING "%s(%ld, %ld) unknown action: "
mm/memory_hotplug.c +3 -3
Index: linux-3.5-rc4/mm/memory_hotplug.c
===================================================================
@@ -990,7 +990,7 @@ out:
 	return ret;
 }
 
-int remove_memory(u64 start, u64 size)
+int offline_memory(u64 start, u64 size)
 {
 	unsigned long start_pfn, end_pfn;
 
@@ -999,9 +999,9 @@ int remove_memory(u64 start, u64 size)
 	return offline_pages(start_pfn, end_pfn, 120 * HZ);
 }
 #else
-int remove_memory(u64 start, u64 size)
+int offline_memory(u64 start, u64 size)
 {
 	return -EINVAL;
 }
 #endif /* CONFIG_MEMORY_HOTREMOVE */
-EXPORT_SYMBOL_GPL(remove_memory);
+EXPORT_SYMBOL_GPL(offline_memory);
include/linux/memory_hotplug.h +1 -1
Index: linux-3.5-rc4/include/linux/memory_hotplug.h
===================================================================
@@ -233,7 +233,7 @@ static inline int is_mem_section_removab
 extern int mem_online_node(int nid);
 extern int add_memory(int nid, u64 start, u64 size);
 extern int arch_add_memory(int nid, u64 start, u64 size);
-extern int remove_memory(u64 start, u64 size);
+extern int offline_memory(u64 start, u64 size);
 extern int sparse_add_one_section(struct zone *zone, unsigned long start_pfn,
 								int nr_pages);
 extern void sparse_remove_one_section(struct zone *zone, struct mem_section *ms);
#2 lore raw

RFC PATCH v3 2/13 memory-hotplug : add physical memory hotplug code to acpi_memory_device_remove

In reply to Yasuaki Ishimatsu

acpi_memory_device_remove() has been prepared to remove physical memory. But, the function only frees acpi_memory_device currentlry.

The patch adds following functions into acpi_memory_device_remove():
- offline memory
- remove physical memory (only return -EBUSY)
- free acpi_memory_device

CC David Rientjes <[email protected]>
CC Jiang Liu <[email protected]>
CC Len Brown <[email protected]>
CC Benjamin Herrenschmidt <[email protected]>
CC Paul Mackerras <[email protected]>
CC Christoph Lameter <[email protected]>
Cc Minchan Kim <[email protected]>
CC Andrew Morton <[email protected]>
CC KOSAKI Motohiro <[email protected]>
CC Wen Congyang <[email protected]>
---
 drivers/acpi/acpi_memhotplug.c |   26 +++++++++++++++++++++++++-
 drivers/base/memory.c          |   39 +++++++++++++++++++++++++++++++++++++++
 include/linux/memory.h         |    5 +++++
 include/linux/memory_hotplug.h |    1 +
 mm/memory_hotplug.c            |    8 ++++++++
 5 files changed, 78 insertions(+), 1 deletion(-)
drivers/acpi/acpi_memhotplug.c +25 -1
Index: linux-3.5-rc6/drivers/acpi/acpi_memhotplug.c
===================================================================
@@ -29,6 +29,7 @@
 #include <linux/module.h>
 #include <linux/init.h>
 #include <linux/types.h>
+#include <linux/memory.h>
 #include <linux/memory_hotplug.h>
 #include <linux/slab.h>
 #include <acpi/acpi_drivers.h>
@@ -452,12 +453,35 @@ static int acpi_memory_device_add(struct
 static int acpi_memory_device_remove(struct acpi_device *device, int type)
 {
 	struct acpi_memory_device *mem_device = NULL;
-
+	struct acpi_memory_info *info, *tmp;
+	int result;
+	int node;
 
 	if (!device || !acpi_driver_data(device))
 		return -EINVAL;
 
 	mem_device = acpi_driver_data(device);
+
+	node = acpi_get_node(mem_device->device->handle);
+
+	list_for_each_entry_safe(info, tmp, &mem_device->res_list, list) {
+		if (!info->enabled)
+			continue;
+
+		if (!is_memblk_offline(info->start_addr, info->length)) {
+			result = offline_memory(info->start_addr, info->length);
+			if (result)
+				return result;
+		}
+
+		result = remove_memory(node, info->start_addr, info->length);
+		if (result)
+			return result;
+
+		list_del(&info->list);
+		kfree(info);
+	}
+
 	kfree(mem_device);
 
 	return 0;
include/linux/memory_hotplug.h +1 -0
Index: linux-3.5-rc6/include/linux/memory_hotplug.h
===================================================================
@@ -233,6 +233,7 @@ static inline int is_mem_section_removab
 extern int mem_online_node(int nid);
 extern int add_memory(int nid, u64 start, u64 size);
 extern int arch_add_memory(int nid, u64 start, u64 size);
+extern int remove_memory(int nid, u64 start, u64 size);
 extern int offline_memory(u64 start, u64 size);
 extern int sparse_add_one_section(struct zone *zone, unsigned long start_pfn,
 								int nr_pages);
mm/memory_hotplug.c +8 -0
Index: linux-3.5-rc6/mm/memory_hotplug.c
===================================================================
@@ -659,6 +659,14 @@ out:
 }
 EXPORT_SYMBOL_GPL(add_memory);
 
+int remove_memory(int nid, u64 start, u64 size)
+{
+	return -EBUSY;
+
+}
+EXPORT_SYMBOL_GPL(remove_memory);
+
+
 #ifdef CONFIG_MEMORY_HOTREMOVE
 /*
  * A free page on the buddy free lists (not the per-cpu lists) has PageBuddy
drivers/base/memory.c +39 -0
Index: linux-3.5-rc6/drivers/base/memory.c
===================================================================
@@ -70,6 +70,45 @@ void unregister_memory_isolate_notifier(
 }
 EXPORT_SYMBOL(unregister_memory_isolate_notifier);
 
+bool is_memblk_offline(unsigned long start, unsigned long size)
+{
+	struct memory_block *mem = NULL;
+	struct mem_section *section;
+	unsigned long start_pfn, end_pfn;
+	unsigned long pfn, section_nr;
+
+	start_pfn = PFN_DOWN(start);
+	end_pfn = start_pfn + PFN_DOWN(start);
+
+	for (pfn = start_pfn; pfn < end_pfn; pfn += PAGES_PER_SECTION) {
+		section_nr = pfn_to_section_nr(pfn);
+		if (!present_section_nr(section_nr));
+			continue;
+
+		section = __nr_to_section(section_nr);
+		/* same memblock? */
+		if (mem)
+			if((section_nr >= mem->start_section_nr) &&
+			   (section_nr <= mem->end_section_nr))
+				continue;
+
+		mem = find_memory_block_hinted(section, mem);
+		if (!mem)
+			continue;
+		if (mem->state == MEM_OFFLINE)
+			continue;
+
+		kobject_put(&mem->dev.kobj);
+		return false;
+	}
+
+	if (mem)
+		kobject_put(&mem->dev.kobj);
+
+	return true;
+}
+EXPORT_SYMBOL(is_memblk_offline);
+
 /*
  * register_memory - Setup a sysfs device for a memory block
  */
include/linux/memory.h +5 -0
Index: linux-3.5-rc6/include/linux/memory.h
===================================================================
@@ -106,6 +106,10 @@ static inline int memory_isolate_notify(
 {
 	return 0;
 }
+static inline bool is_memblk_offline(unsigned long start, unsigned long size)
+{
+	return false;
+}
 #else
 extern int register_memory_notifier(struct notifier_block *nb);
 extern void unregister_memory_notifier(struct notifier_block *nb);
@@ -120,6 +124,7 @@ extern int memory_isolate_notify(unsigne
 extern struct memory_block *find_memory_block_hinted(struct mem_section *,
 							struct memory_block *);
 extern struct memory_block *find_memory_block(struct mem_section *);
+extern bool is_memblk_offline(unsigned long start, unsigned long size);
 #define CONFIG_MEM_BLOCK_SIZE	(PAGES_PER_SECTION<<PAGE_SHIFT)
 enum mem_add_context { BOOT, HOTPLUG };
 #endif /* CONFIG_MEMORY_HOTPLUG_SPARSE */

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#3 lore raw

Re: [RFC PATCH v3 2/13] memory-hotplug : add physical memory hotplug code to acpi_memory_device_remove

At 07/09/2012 06:24 PM, Yasuaki Ishimatsu Wrote:

Quoted 82 lines

acpi_memory_device_remove() has been prepared to remove physical memory. But, the function only frees acpi_memory_device currentlry.

The patch adds following functions into acpi_memory_device_remove():
- offline memory
- remove physical memory (only return -EBUSY)
- free acpi_memory_device

CC David Rientjes <[email protected]>
CC Jiang Liu <[email protected]>
CC Len Brown <[email protected]>
CC Benjamin Herrenschmidt <[email protected]>
CC Paul Mackerras <[email protected]>
CC Christoph Lameter <[email protected]>
Cc Minchan Kim <[email protected]>
CC Andrew Morton <[email protected]>
CC KOSAKI Motohiro <[email protected]>
CC Wen Congyang <[email protected]>
---
 drivers/acpi/acpi_memhotplug.c |   26 +++++++++++++++++++++++++-
 drivers/base/memory.c          |   39 +++++++++++++++++++++++++++++++++++++++
 include/linux/memory.h         |    5 +++++
 include/linux/memory_hotplug.h |    1 +
 mm/memory_hotplug.c            |    8 ++++++++
 5 files changed, 78 insertions(+), 1 deletion(-)

Index: linux-3.5-rc6/drivers/acpi/acpi_memhotplug.c
===================================================================

drivers/acpi/acpi_memhotplug.c +25 -1
@@ -29,6 +29,7 @@
 #include <linux/module.h>
 #include <linux/init.h>
 #include <linux/types.h>
+#include <linux/memory.h>
 #include <linux/memory_hotplug.h>
 #include <linux/slab.h>
 #include <acpi/acpi_drivers.h>
@@ -452,12 +453,35 @@ static int acpi_memory_device_add(struct
 static int acpi_memory_device_remove(struct acpi_device *device, int type)
 {
 	struct acpi_memory_device *mem_device = NULL;
-
+	struct acpi_memory_info *info, *tmp;
+	int result;
+	int node;
 
 	if (!device || !acpi_driver_data(device))
 		return -EINVAL;
 
 	mem_device = acpi_driver_data(device);
+
+	node = acpi_get_node(mem_device->device->handle);
+
+	list_for_each_entry_safe(info, tmp, &mem_device->res_list, list) {
+		if (!info->enabled)
+			continue;
+
+		if (!is_memblk_offline(info->start_addr, info->length)) {
+			result = offline_memory(info->start_addr, info->length);
+			if (result)
+				return result;
+		}
+
+		result = remove_memory(node, info->start_addr, info->length);
+		if (result)
+			return result;
+
+		list_del(&info->list);
+		kfree(info);
+	}
+
 	kfree(mem_device);
 
 	return 0;

Index: linux-3.5-rc6/include/linux/memory_hotplug.h
===================================================================

include/linux/memory_hotplug.h static inline int is_mem_section_removab · line 236

Quoted 4 lines
include/linux/memory_hotplug.h +1 -0
@@ -233,6 +233,7 @@ static inline int is_mem_section_removab
 extern int mem_online_node(int nid);
 extern int add_memory(int nid, u64 start, u64 size);
 extern int arch_add_memory(int nid, u64 start, u64 size);
+extern int remove_memory(int nid, u64 start, u64 size);

Here should be:
#ifdef CONFIG_MEMORY_HOTREMOVE
extern int remove_memory(int nid, u64 start, u64 size); #else
static int inline remove_memory(int nid, u64 start, u64 size)

{
	return -EBUSY;
}

#endif

Quoted 8 lines
include/linux/memory_hotplug.h +0 -0
 extern int offline_memory(u64 start, u64 size);
 extern int sparse_add_one_section(struct zone *zone, unsigned long start_pfn,
 								int nr_pages);

Index: linux-3.5-rc6/mm/memory_hotplug.c
===================================================================

mm/memory_hotplug.c out: · line 667

Quoted 9 lines
mm/memory_hotplug.c +6 -0
@@ -659,6 +659,14 @@ out:
 }
 EXPORT_SYMBOL_GPL(add_memory);
 
+int remove_memory(int nid, u64 start, u64 size)
+{
+	return -EBUSY;
+
+}
+EXPORT_SYMBOL_GPL(remove_memory);

We only need to implement this function when CONFIG_MEMORY_HOTREMOVE is defined here.

Thanks
Wen Congyang

Quoted 80 lines
mm/memory_hotplug.c +2 -0
+
+
 #ifdef CONFIG_MEMORY_HOTREMOVE
 /*
  * A free page on the buddy free lists (not the per-cpu lists) has PageBuddy

Index: linux-3.5-rc6/drivers/base/memory.c
===================================================================

drivers/base/memory.c +39 -0
@@ -70,6 +70,45 @@ void unregister_memory_isolate_notifier(
 }
 EXPORT_SYMBOL(unregister_memory_isolate_notifier);
 
+bool is_memblk_offline(unsigned long start, unsigned long size)
+{
+	struct memory_block *mem = NULL;
+	struct mem_section *section;
+	unsigned long start_pfn, end_pfn;
+	unsigned long pfn, section_nr;
+
+	start_pfn = PFN_DOWN(start);
+	end_pfn = start_pfn + PFN_DOWN(start);
+
+	for (pfn = start_pfn; pfn < end_pfn; pfn += PAGES_PER_SECTION) {
+		section_nr = pfn_to_section_nr(pfn);
+		if (!present_section_nr(section_nr));
+			continue;
+
+		section = __nr_to_section(section_nr);
+		/* same memblock? */
+		if (mem)
+			if((section_nr >= mem->start_section_nr) &&
+			   (section_nr <= mem->end_section_nr))
+				continue;
+
+		mem = find_memory_block_hinted(section, mem);
+		if (!mem)
+			continue;
+		if (mem->state == MEM_OFFLINE)
+			continue;
+
+		kobject_put(&mem->dev.kobj);
+		return false;
+	}
+
+	if (mem)
+		kobject_put(&mem->dev.kobj);
+
+	return true;
+}
+EXPORT_SYMBOL(is_memblk_offline);
+
 /*
  * register_memory - Setup a sysfs device for a memory block
  */

Index: linux-3.5-rc6/include/linux/memory.h
===================================================================

include/linux/memory.h +5 -0
@@ -106,6 +106,10 @@ static inline int memory_isolate_notify(
 {
 	return 0;
 }
+static inline bool is_memblk_offline(unsigned long start, unsigned long size)
+{
+	return false;
+}
 #else
 extern int register_memory_notifier(struct notifier_block *nb);
 extern void unregister_memory_notifier(struct notifier_block *nb);
@@ -120,6 +124,7 @@ extern int memory_isolate_notify(unsigne
 extern struct memory_block *find_memory_block_hinted(struct mem_section *,
 							struct memory_block *);
 extern struct memory_block *find_memory_block(struct mem_section *);
+extern bool is_memblk_offline(unsigned long start, unsigned long size);
 #define CONFIG_MEM_BLOCK_SIZE	(PAGES_PER_SECTION<<PAGE_SHIFT)
 enum mem_add_context { BOOT, HOTPLUG };
 #endif /* CONFIG_MEMORY_HOTPLUG_SPARSE */
#4 lore raw

Re: [RFC PATCH v3 2/13] memory-hotplug : add physical memory hotplug code to acpi_memory_device_remove

Hi Wen,

Quoted 94 lines · Wen Congyang

At 07/09/2012 06:24 PM, Yasuaki Ishimatsu Wrote:

Quoted 82 lines

acpi_memory_device_remove() has been prepared to remove physical memory. But, the function only frees acpi_memory_device currentlry.

The patch adds following functions into acpi_memory_device_remove():
- offline memory
- remove physical memory (only return -EBUSY)
- free acpi_memory_device

CC David Rientjes <[email protected]>
CC Jiang Liu <[email protected]>
CC Len Brown <[email protected]>
CC Benjamin Herrenschmidt <[email protected]>
CC Paul Mackerras <[email protected]>
CC Christoph Lameter <[email protected]>
Cc Minchan Kim <[email protected]>
CC Andrew Morton <[email protected]>
CC KOSAKI Motohiro <[email protected]>
CC Wen Congyang <[email protected]>
---
  drivers/acpi/acpi_memhotplug.c |   26 +++++++++++++++++++++++++-
  drivers/base/memory.c          |   39 +++++++++++++++++++++++++++++++++++++++
  include/linux/memory.h         |    5 +++++
  include/linux/memory_hotplug.h |    1 +
  mm/memory_hotplug.c            |    8 ++++++++
  5 files changed, 78 insertions(+), 1 deletion(-)

Index: linux-3.5-rc6/drivers/acpi/acpi_memhotplug.c
===================================================================

drivers/acpi/acpi_memhotplug.c +25 -1
@@ -29,6 +29,7 @@
  #include <linux/module.h>
  #include <linux/init.h>
  #include <linux/types.h>
+#include <linux/memory.h>
  #include <linux/memory_hotplug.h>
  #include <linux/slab.h>
  #include <acpi/acpi_drivers.h>
@@ -452,12 +453,35 @@ static int acpi_memory_device_add(struct
  static int acpi_memory_device_remove(struct acpi_device *device, int type)
  {
  	struct acpi_memory_device *mem_device = NULL;
-
+	struct acpi_memory_info *info, *tmp;
+	int result;
+	int node;
 
  	if (!device || !acpi_driver_data(device))
  		return -EINVAL;
 
  	mem_device = acpi_driver_data(device);
+
+	node = acpi_get_node(mem_device->device->handle);
+
+	list_for_each_entry_safe(info, tmp, &mem_device->res_list, list) {
+		if (!info->enabled)
+			continue;
+
+		if (!is_memblk_offline(info->start_addr, info->length)) {
+			result = offline_memory(info->start_addr, info->length);
+			if (result)
+				return result;
+		}
+
+		result = remove_memory(node, info->start_addr, info->length);
+		if (result)
+			return result;
+
+		list_del(&info->list);
+		kfree(info);
+	}
+
  	kfree(mem_device);
 
  	return 0;

Index: linux-3.5-rc6/include/linux/memory_hotplug.h
===================================================================

include/linux/memory_hotplug.h static inline int is_mem_section_removab · line 236

Quoted 4 lines
include/linux/memory_hotplug.h +1 -0
@@ -233,6 +233,7 @@ static inline int is_mem_section_removab
  extern int mem_online_node(int nid);
  extern int add_memory(int nid, u64 start, u64 size);
  extern int arch_add_memory(int nid, u64 start, u64 size);
+extern int remove_memory(int nid, u64 start, u64 size);

Here should be:
#ifdef CONFIG_MEMORY_HOTREMOVE
extern int remove_memory(int nid, u64 start, u64 size); #else
static int inline remove_memory(int nid, u64 start, u64 size)

Quoted 4 lines
{
	return -EBUSY;
}

#endif

O.K. I'll update it.

Thanks,
Yasuaki Ishimatsu

Quoted 103 lines
Quoted 9 lines
include/linux/memory_hotplug.h +0 -0
  extern int offline_memory(u64 start, u64 size);
  extern int sparse_add_one_section(struct zone *zone, unsigned long start_pfn,
  								int nr_pages);

Index: linux-3.5-rc6/mm/memory_hotplug.c
===================================================================

mm/memory_hotplug.c out: · line 667

Quoted 9 lines
mm/memory_hotplug.c +6 -0
@@ -659,6 +659,14 @@ out:
  }
  EXPORT_SYMBOL_GPL(add_memory);
 
+int remove_memory(int nid, u64 start, u64 size)
+{
+	return -EBUSY;
+
+}
+EXPORT_SYMBOL_GPL(remove_memory);

We only need to implement this function when CONFIG_MEMORY_HOTREMOVE is defined here.

Thanks
Wen Congyang

Quoted 78 lines
mm/memory_hotplug.c +2 -0
+
+
  #ifdef CONFIG_MEMORY_HOTREMOVE
  /*
   * A free page on the buddy free lists (not the per-cpu lists) has PageBuddy

Index: linux-3.5-rc6/drivers/base/memory.c
===================================================================

drivers/base/memory.c +39 -0
@@ -70,6 +70,45 @@ void unregister_memory_isolate_notifier(
  }
  EXPORT_SYMBOL(unregister_memory_isolate_notifier);
 
+bool is_memblk_offline(unsigned long start, unsigned long size)
+{
+	struct memory_block *mem = NULL;
+	struct mem_section *section;
+	unsigned long start_pfn, end_pfn;
+	unsigned long pfn, section_nr;
+
+	start_pfn = PFN_DOWN(start);
+	end_pfn = start_pfn + PFN_DOWN(start);
+
+	for (pfn = start_pfn; pfn < end_pfn; pfn += PAGES_PER_SECTION) {
+		section_nr = pfn_to_section_nr(pfn);
+		if (!present_section_nr(section_nr));
+			continue;
+
+		section = __nr_to_section(section_nr);
+		/* same memblock? */
+		if (mem)
+			if((section_nr >= mem->start_section_nr) &&
+			   (section_nr <= mem->end_section_nr))
+				continue;
+
+		mem = find_memory_block_hinted(section, mem);
+		if (!mem)
+			continue;
+		if (mem->state == MEM_OFFLINE)
+			continue;
+
+		kobject_put(&mem->dev.kobj);
+		return false;
+	}
+
+	if (mem)
+		kobject_put(&mem->dev.kobj);
+
+	return true;
+}
+EXPORT_SYMBOL(is_memblk_offline);
+
  /*
   * register_memory - Setup a sysfs device for a memory block
   */

Index: linux-3.5-rc6/include/linux/memory.h
===================================================================

include/linux/memory.h +5 -0
@@ -106,6 +106,10 @@ static inline int memory_isolate_notify(
  {
  	return 0;
  }
+static inline bool is_memblk_offline(unsigned long start, unsigned long size)
+{
+	return false;
+}
  #else
  extern int register_memory_notifier(struct notifier_block *nb);
  extern void unregister_memory_notifier(struct notifier_block *nb);
@@ -120,6 +124,7 @@ extern int memory_isolate_notify(unsigne
  extern struct memory_block *find_memory_block_hinted(struct mem_section *,
  							struct memory_block *);
  extern struct memory_block *find_memory_block(struct mem_section *);
+extern bool is_memblk_offline(unsigned long start, unsigned long size);
  #define CONFIG_MEM_BLOCK_SIZE	(PAGES_PER_SECTION<<PAGE_SHIFT)
  enum mem_add_context { BOOT, HOTPLUG };
  #endif /* CONFIG_MEMORY_HOTPLUG_SPARSE */
Quoted 9 lines
Quoted 2 lines

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

e

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#5 lore raw

Re: [RFC PATCH v3 2/13] memory-hotplug : add physical memory hotplug code to acpi_memory_device_remove

In reply to Yasuaki Ishimatsu

At 07/09/2012 06:24 PM, Yasuaki Ishimatsu Wrote:

Quoted 57 lines

acpi_memory_device_remove() has been prepared to remove physical memory. But, the function only frees acpi_memory_device currentlry.

The patch adds following functions into acpi_memory_device_remove():
- offline memory
- remove physical memory (only return -EBUSY)
- free acpi_memory_device

CC David Rientjes <[email protected]>
CC Jiang Liu <[email protected]>
CC Len Brown <[email protected]>
CC Benjamin Herrenschmidt <[email protected]>
CC Paul Mackerras <[email protected]>
CC Christoph Lameter <[email protected]>
Cc Minchan Kim <[email protected]>
CC Andrew Morton <[email protected]>
CC KOSAKI Motohiro <[email protected]>
CC Wen Congyang <[email protected]>
---
 drivers/acpi/acpi_memhotplug.c |   26 +++++++++++++++++++++++++-
 drivers/base/memory.c          |   39 +++++++++++++++++++++++++++++++++++++++
 include/linux/memory.h         |    5 +++++
 include/linux/memory_hotplug.h |    1 +
 mm/memory_hotplug.c            |    8 ++++++++
 5 files changed, 78 insertions(+), 1 deletion(-)

Index: linux-3.5-rc6/drivers/acpi/acpi_memhotplug.c
===================================================================

drivers/acpi/acpi_memhotplug.c +8 -1
@@ -29,6 +29,7 @@
 #include <linux/module.h>
 #include <linux/init.h>
 #include <linux/types.h>
+#include <linux/memory.h>
 #include <linux/memory_hotplug.h>
 #include <linux/slab.h>
 #include <acpi/acpi_drivers.h>
@@ -452,12 +453,35 @@ static int acpi_memory_device_add(struct
 static int acpi_memory_device_remove(struct acpi_device *device, int type)
 {
 	struct acpi_memory_device *mem_device = NULL;
-
+	struct acpi_memory_info *info, *tmp;
+	int result;
+	int node;
 
 	if (!device || !acpi_driver_data(device))
 		return -EINVAL;
 
 	mem_device = acpi_driver_data(device);
+
+	node = acpi_get_node(mem_device->device->handle);
+
+	list_for_each_entry_safe(info, tmp, &mem_device->res_list, list) {

drivers/acpi/acpi_memhotplug.c static int acpi_memory_device_add(struct · line 477

Quoted 10 lines
drivers/acpi/acpi_memhotplug.c +10 -0
+		if (!info->enabled)
+			continue;
+
+		if (!is_memblk_offline(info->start_addr, info->length)) {
+			result = offline_memory(info->start_addr, info->length);
+			if (result)
+				return result;
+		}
+
+		result = remove_memory(node, info->start_addr, info->length);

The user may online the memory between offline_memory() and remove_memory(). So I think we should lock memory hotplug before check the memory's status and release it after remove_memory().

Thanks
Wen Congyang

Quoted 109 lines
drivers/acpi/acpi_memhotplug.c +7 -0
+		if (result)
+			return result;
+
+		list_del(&info->list);
+		kfree(info);
+	}
+
 	kfree(mem_device);
 
 	return 0;

Index: linux-3.5-rc6/include/linux/memory_hotplug.h
===================================================================

include/linux/memory_hotplug.h +1 -0
@@ -233,6 +233,7 @@ static inline int is_mem_section_removab
 extern int mem_online_node(int nid);
 extern int add_memory(int nid, u64 start, u64 size);
 extern int arch_add_memory(int nid, u64 start, u64 size);
+extern int remove_memory(int nid, u64 start, u64 size);
 extern int offline_memory(u64 start, u64 size);
 extern int sparse_add_one_section(struct zone *zone, unsigned long start_pfn,
 								int nr_pages);

Index: linux-3.5-rc6/mm/memory_hotplug.c
===================================================================

mm/memory_hotplug.c +8 -0
@@ -659,6 +659,14 @@ out:
 }
 EXPORT_SYMBOL_GPL(add_memory);
 
+int remove_memory(int nid, u64 start, u64 size)
+{
+	return -EBUSY;
+
+}
+EXPORT_SYMBOL_GPL(remove_memory);
+
+
 #ifdef CONFIG_MEMORY_HOTREMOVE
 /*
  * A free page on the buddy free lists (not the per-cpu lists) has PageBuddy

Index: linux-3.5-rc6/drivers/base/memory.c
===================================================================

drivers/base/memory.c +39 -0
@@ -70,6 +70,45 @@ void unregister_memory_isolate_notifier(
 }
 EXPORT_SYMBOL(unregister_memory_isolate_notifier);
 
+bool is_memblk_offline(unsigned long start, unsigned long size)
+{
+	struct memory_block *mem = NULL;
+	struct mem_section *section;
+	unsigned long start_pfn, end_pfn;
+	unsigned long pfn, section_nr;
+
+	start_pfn = PFN_DOWN(start);
+	end_pfn = start_pfn + PFN_DOWN(start);
+
+	for (pfn = start_pfn; pfn < end_pfn; pfn += PAGES_PER_SECTION) {
+		section_nr = pfn_to_section_nr(pfn);
+		if (!present_section_nr(section_nr));
+			continue;
+
+		section = __nr_to_section(section_nr);
+		/* same memblock? */
+		if (mem)
+			if((section_nr >= mem->start_section_nr) &&
+			   (section_nr <= mem->end_section_nr))
+				continue;
+
+		mem = find_memory_block_hinted(section, mem);
+		if (!mem)
+			continue;
+		if (mem->state == MEM_OFFLINE)
+			continue;
+
+		kobject_put(&mem->dev.kobj);
+		return false;
+	}
+
+	if (mem)
+		kobject_put(&mem->dev.kobj);
+
+	return true;
+}
+EXPORT_SYMBOL(is_memblk_offline);
+
 /*
  * register_memory - Setup a sysfs device for a memory block
  */

Index: linux-3.5-rc6/include/linux/memory.h
===================================================================

include/linux/memory.h +4 -0
@@ -106,6 +106,10 @@ static inline int memory_isolate_notify(
 {
 	return 0;
 }
+static inline bool is_memblk_offline(unsigned long start, unsigned long size)
+{
+	return false;
+}
 #else
 extern int register_memory_notifier(struct notifier_block *nb);
 extern void unregister_memory_notifier(struct notifier_block *nb);

include/linux/memory.h extern int memory_isolate_notify(unsigne · line 130

Quoted 7 lines
include/linux/memory.h +1 -0
@@ -120,6 +124,7 @@ extern int memory_isolate_notify(unsigne
 extern struct memory_block *find_memory_block_hinted(struct mem_section *,
 							struct memory_block *);
 extern struct memory_block *find_memory_block(struct mem_section *);
+extern bool is_memblk_offline(unsigned long start, unsigned long size);
 #define CONFIG_MEM_BLOCK_SIZE	(PAGES_PER_SECTION<<PAGE_SHIFT)
 enum mem_add_context { BOOT, HOTPLUG };
 #endif /* CONFIG_MEMORY_HOTPLUG_SPARSE */

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#6 lore raw

Re: [RFC PATCH v3 2/13] memory-hotplug : add physical memory hotplug code to acpi_memory_device_remove

Hi Wen,

Quoted 64 lines · Wen Congyang

At 07/09/2012 06:24 PM, Yasuaki Ishimatsu Wrote:

Quoted 63 lines

acpi_memory_device_remove() has been prepared to remove physical memory. But, the function only frees acpi_memory_device currentlry.

The patch adds following functions into acpi_memory_device_remove():
- offline memory
- remove physical memory (only return -EBUSY)
- free acpi_memory_device

CC David Rientjes <[email protected]>
CC Jiang Liu <[email protected]>
CC Len Brown <[email protected]>
CC Benjamin Herrenschmidt <[email protected]>
CC Paul Mackerras <[email protected]>
CC Christoph Lameter <[email protected]>
Cc Minchan Kim <[email protected]>
CC Andrew Morton <[email protected]>
CC KOSAKI Motohiro <[email protected]>
CC Wen Congyang <[email protected]>
---
  drivers/acpi/acpi_memhotplug.c |   26 +++++++++++++++++++++++++-
  drivers/base/memory.c          |   39 +++++++++++++++++++++++++++++++++++++++
  include/linux/memory.h         |    5 +++++
  include/linux/memory_hotplug.h |    1 +
  mm/memory_hotplug.c            |    8 ++++++++
  5 files changed, 78 insertions(+), 1 deletion(-)

Index: linux-3.5-rc6/drivers/acpi/acpi_memhotplug.c
===================================================================

drivers/acpi/acpi_memhotplug.c +14 -1
@@ -29,6 +29,7 @@
  #include <linux/module.h>
  #include <linux/init.h>
  #include <linux/types.h>
+#include <linux/memory.h>
  #include <linux/memory_hotplug.h>
  #include <linux/slab.h>
  #include <acpi/acpi_drivers.h>
@@ -452,12 +453,35 @@ static int acpi_memory_device_add(struct
  static int acpi_memory_device_remove(struct acpi_device *device, int type)
  {
  	struct acpi_memory_device *mem_device = NULL;
-
+	struct acpi_memory_info *info, *tmp;
+	int result;
+	int node;
 
  	if (!device || !acpi_driver_data(device))
  		return -EINVAL;
 
  	mem_device = acpi_driver_data(device);
+
+	node = acpi_get_node(mem_device->device->handle);
+
+	list_for_each_entry_safe(info, tmp, &mem_device->res_list, list) {
+		if (!info->enabled)
+			continue;
+
+		if (!is_memblk_offline(info->start_addr, info->length)) {
+			result = offline_memory(info->start_addr, info->length);
+			if (result)
Quoted 8 lines

drivers/acpi/acpi_memhotplug.c static int acpi_memory_device_add(struct · line 477

Quoted 4 lines
drivers/acpi/acpi_memhotplug.c +4 -0
+				return result;
+		}
+
+		result = remove_memory(node, info->start_addr, info->length);

The user may online the memory between offline_memory() and remove_memory(). So I think we should lock memory hotplug before check the memory's status and release it after remove_memory().

How about get "mem_block->state_mutex" of removed memory? When offlining memory, we need to change "memory_block->state" into "MEM_OFFLINE". In this case, we get mem_block->state_mutex. So I think the mutex lock is beneficial.

Thanks,
Yasuaki Ishimatsu

Quoted 118 lines

Thanks
Wen Congyang

Quoted 114 lines
drivers/acpi/acpi_memhotplug.c +7 -0
+		if (result)
+			return result;
+
+		list_del(&info->list);
+		kfree(info);
+	}
+
  	kfree(mem_device);
 
  	return 0;

Index: linux-3.5-rc6/include/linux/memory_hotplug.h
===================================================================

include/linux/memory_hotplug.h +1 -0
@@ -233,6 +233,7 @@ static inline int is_mem_section_removab
  extern int mem_online_node(int nid);
  extern int add_memory(int nid, u64 start, u64 size);
  extern int arch_add_memory(int nid, u64 start, u64 size);
+extern int remove_memory(int nid, u64 start, u64 size);
  extern int offline_memory(u64 start, u64 size);
  extern int sparse_add_one_section(struct zone *zone, unsigned long start_pfn,
  								int nr_pages);

Index: linux-3.5-rc6/mm/memory_hotplug.c
===================================================================

mm/memory_hotplug.c +8 -0
@@ -659,6 +659,14 @@ out:
  }
  EXPORT_SYMBOL_GPL(add_memory);
 
+int remove_memory(int nid, u64 start, u64 size)
+{
+	return -EBUSY;
+
+}
+EXPORT_SYMBOL_GPL(remove_memory);
+
+
  #ifdef CONFIG_MEMORY_HOTREMOVE
  /*
   * A free page on the buddy free lists (not the per-cpu lists) has PageBuddy

Index: linux-3.5-rc6/drivers/base/memory.c
===================================================================

drivers/base/memory.c +39 -0
@@ -70,6 +70,45 @@ void unregister_memory_isolate_notifier(
  }
  EXPORT_SYMBOL(unregister_memory_isolate_notifier);
 
+bool is_memblk_offline(unsigned long start, unsigned long size)
+{
+	struct memory_block *mem = NULL;
+	struct mem_section *section;
+	unsigned long start_pfn, end_pfn;
+	unsigned long pfn, section_nr;
+
+	start_pfn = PFN_DOWN(start);
+	end_pfn = start_pfn + PFN_DOWN(start);
+
+	for (pfn = start_pfn; pfn < end_pfn; pfn += PAGES_PER_SECTION) {
+		section_nr = pfn_to_section_nr(pfn);
+		if (!present_section_nr(section_nr));
+			continue;
+
+		section = __nr_to_section(section_nr);
+		/* same memblock? */
+		if (mem)
+			if((section_nr >= mem->start_section_nr) &&
+			   (section_nr <= mem->end_section_nr))
+				continue;
+
+		mem = find_memory_block_hinted(section, mem);
+		if (!mem)
+			continue;
+		if (mem->state == MEM_OFFLINE)
+			continue;
+
+		kobject_put(&mem->dev.kobj);
+		return false;
+	}
+
+	if (mem)
+		kobject_put(&mem->dev.kobj);
+
+	return true;
+}
+EXPORT_SYMBOL(is_memblk_offline);
+
  /*
   * register_memory - Setup a sysfs device for a memory block
   */

Index: linux-3.5-rc6/include/linux/memory.h
===================================================================

include/linux/memory.h +5 -0
@@ -106,6 +106,10 @@ static inline int memory_isolate_notify(
  {
  	return 0;
  }
+static inline bool is_memblk_offline(unsigned long start, unsigned long size)
+{
+	return false;
+}
  #else
  extern int register_memory_notifier(struct notifier_block *nb);
  extern void unregister_memory_notifier(struct notifier_block *nb);
@@ -120,6 +124,7 @@ extern int memory_isolate_notify(unsigne
  extern struct memory_block *find_memory_block_hinted(struct mem_section *,
  							struct memory_block *);
  extern struct memory_block *find_memory_block(struct mem_section *);
+extern bool is_memblk_offline(unsigned long start, unsigned long size);
  #define CONFIG_MEM_BLOCK_SIZE	(PAGES_PER_SECTION<<PAGE_SHIFT)
  enum mem_add_context { BOOT, HOTPLUG };
  #endif /* CONFIG_MEMORY_HOTPLUG_SPARSE */
Quoted 9 lines
Quoted 2 lines

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#7 lore raw

Re: [RFC PATCH v3 2/13] memory-hotplug : add physical memory hotplug code to acpi_memory_device_remove

Hi Wen,

Quoted 76 lines · Yasuaki Ishimatsu

Hi Wen,

Quoted 64 lines · Wen Congyang

At 07/09/2012 06:24 PM, Yasuaki Ishimatsu Wrote:

Quoted 63 lines

acpi_memory_device_remove() has been prepared to remove physical memory. But, the function only frees acpi_memory_device currentlry.

The patch adds following functions into acpi_memory_device_remove():
- offline memory
- remove physical memory (only return -EBUSY)
- free acpi_memory_device

CC David Rientjes <[email protected]>
CC Jiang Liu <[email protected]>
CC Len Brown <[email protected]>
CC Benjamin Herrenschmidt <[email protected]>
CC Paul Mackerras <[email protected]>
CC Christoph Lameter <[email protected]>
Cc Minchan Kim <[email protected]>
CC Andrew Morton <[email protected]>
CC KOSAKI Motohiro <[email protected]>
CC Wen Congyang <[email protected]>
---
   drivers/acpi/acpi_memhotplug.c |   26 +++++++++++++++++++++++++-
   drivers/base/memory.c          |   39 +++++++++++++++++++++++++++++++++++++++
   include/linux/memory.h         |    5 +++++
   include/linux/memory_hotplug.h |    1 +
   mm/memory_hotplug.c            |    8 ++++++++
   5 files changed, 78 insertions(+), 1 deletion(-)

Index: linux-3.5-rc6/drivers/acpi/acpi_memhotplug.c
===================================================================

drivers/acpi/acpi_memhotplug.c +14 -1
@@ -29,6 +29,7 @@
   #include <linux/module.h>
   #include <linux/init.h>
   #include <linux/types.h>
+#include <linux/memory.h>
   #include <linux/memory_hotplug.h>
   #include <linux/slab.h>
   #include <acpi/acpi_drivers.h>
@@ -452,12 +453,35 @@ static int acpi_memory_device_add(struct
   static int acpi_memory_device_remove(struct acpi_device *device, int type)
   {
   	struct acpi_memory_device *mem_device = NULL;
-
+	struct acpi_memory_info *info, *tmp;
+	int result;
+	int node;
 
   	if (!device || !acpi_driver_data(device))
   		return -EINVAL;
 
   	mem_device = acpi_driver_data(device);
+
+	node = acpi_get_node(mem_device->device->handle);
+
+	list_for_each_entry_safe(info, tmp, &mem_device->res_list, list) {
+		if (!info->enabled)
+			continue;
+
+		if (!is_memblk_offline(info->start_addr, info->length)) {
+			result = offline_memory(info->start_addr, info->length);
+			if (result)
Quoted 8 lines

drivers/acpi/acpi_memhotplug.c static int acpi_memory_device_add(struct · line 477

Quoted 4 lines
drivers/acpi/acpi_memhotplug.c +4 -0
+				return result;
+		}
+
+		result = remove_memory(node, info->start_addr, info->length);

The user may online the memory between offline_memory() and remove_memory(). So I think we should lock memory hotplug before check the memory's status and release it after remove_memory().

Quoted 4 lines

How about get "mem_block->state_mutex" of removed memory? When offlining memory, we need to change "memory_block->state" into "MEM_OFFLINE". In this case, we get mem_block->state_mutex. So I think the mutex lock is beneficial.

It is not good idea since remove_memory frees mem_block structure... Do you have any ideas?

Thanks,
Yasuaki Ishimatsu

Quoted 133 lines

Thanks,
Yasuaki Ishimatsu

Quoted 118 lines

Thanks
Wen Congyang

Quoted 114 lines
drivers/acpi/acpi_memhotplug.c +7 -0
+		if (result)
+			return result;
+
+		list_del(&info->list);
+		kfree(info);
+	}
+
   	kfree(mem_device);
 
   	return 0;

Index: linux-3.5-rc6/include/linux/memory_hotplug.h
===================================================================

include/linux/memory_hotplug.h +1 -0
@@ -233,6 +233,7 @@ static inline int is_mem_section_removab
   extern int mem_online_node(int nid);
   extern int add_memory(int nid, u64 start, u64 size);
   extern int arch_add_memory(int nid, u64 start, u64 size);
+extern int remove_memory(int nid, u64 start, u64 size);
   extern int offline_memory(u64 start, u64 size);
   extern int sparse_add_one_section(struct zone *zone, unsigned long start_pfn,
   								int nr_pages);

Index: linux-3.5-rc6/mm/memory_hotplug.c
===================================================================

mm/memory_hotplug.c +8 -0
@@ -659,6 +659,14 @@ out:
   }
   EXPORT_SYMBOL_GPL(add_memory);
 
+int remove_memory(int nid, u64 start, u64 size)
+{
+	return -EBUSY;
+
+}
+EXPORT_SYMBOL_GPL(remove_memory);
+
+
   #ifdef CONFIG_MEMORY_HOTREMOVE
   /*
    * A free page on the buddy free lists (not the per-cpu lists) has PageBuddy

Index: linux-3.5-rc6/drivers/base/memory.c
===================================================================

drivers/base/memory.c +39 -0
@@ -70,6 +70,45 @@ void unregister_memory_isolate_notifier(
   }
   EXPORT_SYMBOL(unregister_memory_isolate_notifier);
 
+bool is_memblk_offline(unsigned long start, unsigned long size)
+{
+	struct memory_block *mem = NULL;
+	struct mem_section *section;
+	unsigned long start_pfn, end_pfn;
+	unsigned long pfn, section_nr;
+
+	start_pfn = PFN_DOWN(start);
+	end_pfn = start_pfn + PFN_DOWN(start);
+
+	for (pfn = start_pfn; pfn < end_pfn; pfn += PAGES_PER_SECTION) {
+		section_nr = pfn_to_section_nr(pfn);
+		if (!present_section_nr(section_nr));
+			continue;
+
+		section = __nr_to_section(section_nr);
+		/* same memblock? */
+		if (mem)
+			if((section_nr >= mem->start_section_nr) &&
+			   (section_nr <= mem->end_section_nr))
+				continue;
+
+		mem = find_memory_block_hinted(section, mem);
+		if (!mem)
+			continue;
+		if (mem->state == MEM_OFFLINE)
+			continue;
+
+		kobject_put(&mem->dev.kobj);
+		return false;
+	}
+
+	if (mem)
+		kobject_put(&mem->dev.kobj);
+
+	return true;
+}
+EXPORT_SYMBOL(is_memblk_offline);
+
   /*
    * register_memory - Setup a sysfs device for a memory block
    */

Index: linux-3.5-rc6/include/linux/memory.h
===================================================================

include/linux/memory.h +5 -0
@@ -106,6 +106,10 @@ static inline int memory_isolate_notify(
   {
   	return 0;
   }
+static inline bool is_memblk_offline(unsigned long start, unsigned long size)
+{
+	return false;
+}
   #else
   extern int register_memory_notifier(struct notifier_block *nb);
   extern void unregister_memory_notifier(struct notifier_block *nb);
@@ -120,6 +124,7 @@ extern int memory_isolate_notify(unsigne
   extern struct memory_block *find_memory_block_hinted(struct mem_section *,
   							struct memory_block *);
   extern struct memory_block *find_memory_block(struct mem_section *);
+extern bool is_memblk_offline(unsigned long start, unsigned long size);
   #define CONFIG_MEM_BLOCK_SIZE	(PAGES_PER_SECTION<<PAGE_SHIFT)
   enum mem_add_context { BOOT, HOTPLUG };
   #endif /* CONFIG_MEMORY_HOTPLUG_SPARSE */
Quoted 9 lines
Quoted 2 lines

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Quoted 5 lines

--
To unsubscribe from this list: send the line "unsubscribe linux-acpi" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#8 lore raw

Re: [RFC PATCH v3 2/13] memory-hotplug : add physical memory hotplug code to acpi_memory_device_remove

At 07/17/2012 09:54 AM, Yasuaki Ishimatsu Wrote:

Quoted 79 lines

Hi Wen,

Quoted 76 lines · Yasuaki Ishimatsu

Hi Wen,

Quoted 72 lines · Wen Congyang

At 07/09/2012 06:24 PM, Yasuaki Ishimatsu Wrote:

Quoted 57 lines

acpi_memory_device_remove() has been prepared to remove physical memory. But, the function only frees acpi_memory_device currentlry.

The patch adds following functions into acpi_memory_device_remove():
- offline memory
- remove physical memory (only return -EBUSY)
- free acpi_memory_device

CC David Rientjes <[email protected]>
CC Jiang Liu <[email protected]>
CC Len Brown <[email protected]>
CC Benjamin Herrenschmidt <[email protected]>
CC Paul Mackerras <[email protected]>
CC Christoph Lameter <[email protected]>
Cc Minchan Kim <[email protected]>
CC Andrew Morton <[email protected]>
CC KOSAKI Motohiro <[email protected]>
CC Wen Congyang <[email protected]>
---
   drivers/acpi/acpi_memhotplug.c |   26 +++++++++++++++++++++++++-
   drivers/base/memory.c          |   39 +++++++++++++++++++++++++++++++++++++++
   include/linux/memory.h         |    5 +++++
   include/linux/memory_hotplug.h |    1 +
   mm/memory_hotplug.c            |    8 ++++++++
   5 files changed, 78 insertions(+), 1 deletion(-)

Index: linux-3.5-rc6/drivers/acpi/acpi_memhotplug.c
===================================================================

drivers/acpi/acpi_memhotplug.c +8 -1
@@ -29,6 +29,7 @@
   #include <linux/module.h>
   #include <linux/init.h>
   #include <linux/types.h>
+#include <linux/memory.h>
   #include <linux/memory_hotplug.h>
   #include <linux/slab.h>
   #include <acpi/acpi_drivers.h>
@@ -452,12 +453,35 @@ static int acpi_memory_device_add(struct
   static int acpi_memory_device_remove(struct acpi_device *device, int type)
   {
   	struct acpi_memory_device *mem_device = NULL;
-
+	struct acpi_memory_info *info, *tmp;
+	int result;
+	int node;
 
   	if (!device || !acpi_driver_data(device))
   		return -EINVAL;
 
   	mem_device = acpi_driver_data(device);
+
+	node = acpi_get_node(mem_device->device->handle);
+
+	list_for_each_entry_safe(info, tmp, &mem_device->res_list, list) {

drivers/acpi/acpi_memhotplug.c static int acpi_memory_device_add(struct · line 477

Quoted 10 lines
drivers/acpi/acpi_memhotplug.c +10 -0
+		if (!info->enabled)
+			continue;
+
+		if (!is_memblk_offline(info->start_addr, info->length)) {
+			result = offline_memory(info->start_addr, info->length);
+			if (result)
+				return result;
+		}
+
+		result = remove_memory(node, info->start_addr, info->length);

The user may online the memory between offline_memory() and remove_memory(). So I think we should lock memory hotplug before check the memory's status and release it after remove_memory().

Quoted 7 lines
Quoted 4 lines

How about get "mem_block->state_mutex" of removed memory? When offlining memory, we need to change "memory_block->state" into "MEM_OFFLINE". In this case, we get mem_block->state_mutex. So I think the mutex lock is beneficial.

It is not good idea since remove_memory frees mem_block structure... Do you have any ideas?

Hmm, split offline_memory() to 2 functions: offline_pages() and __offline_pages()

offline_pages()

	lock_memory_hotplug();
	__offline_pages();
	unlock_memory_hotplug();

and implement remove_memory() like this:
remove_memory()

	lock_memory_hotplug()
	if (!is_memblk_offline()) {
		__offline_pages();
	}
	// cleanup
	unlock_memory_hotplug();

What about this?

Thanks
Wen Congyang

Quoted 150 lines

Thanks,
Yasuaki Ishimatsu

Quoted 133 lines

Thanks,
Yasuaki Ishimatsu

Quoted 118 lines

Thanks
Wen Congyang

Quoted 114 lines
drivers/acpi/acpi_memhotplug.c +7 -0
+		if (result)
+			return result;
+
+		list_del(&info->list);
+		kfree(info);
+	}
+
   	kfree(mem_device);
 
   	return 0;

Index: linux-3.5-rc6/include/linux/memory_hotplug.h
===================================================================

include/linux/memory_hotplug.h +1 -0
@@ -233,6 +233,7 @@ static inline int is_mem_section_removab
   extern int mem_online_node(int nid);
   extern int add_memory(int nid, u64 start, u64 size);
   extern int arch_add_memory(int nid, u64 start, u64 size);
+extern int remove_memory(int nid, u64 start, u64 size);
   extern int offline_memory(u64 start, u64 size);
   extern int sparse_add_one_section(struct zone *zone, unsigned long start_pfn,
   								int nr_pages);

Index: linux-3.5-rc6/mm/memory_hotplug.c
===================================================================

mm/memory_hotplug.c +8 -0
@@ -659,6 +659,14 @@ out:
   }
   EXPORT_SYMBOL_GPL(add_memory);
 
+int remove_memory(int nid, u64 start, u64 size)
+{
+	return -EBUSY;
+
+}
+EXPORT_SYMBOL_GPL(remove_memory);
+
+
   #ifdef CONFIG_MEMORY_HOTREMOVE
   /*
    * A free page on the buddy free lists (not the per-cpu lists) has PageBuddy

Index: linux-3.5-rc6/drivers/base/memory.c
===================================================================

drivers/base/memory.c +39 -0
@@ -70,6 +70,45 @@ void unregister_memory_isolate_notifier(
   }
   EXPORT_SYMBOL(unregister_memory_isolate_notifier);
 
+bool is_memblk_offline(unsigned long start, unsigned long size)
+{
+	struct memory_block *mem = NULL;
+	struct mem_section *section;
+	unsigned long start_pfn, end_pfn;
+	unsigned long pfn, section_nr;
+
+	start_pfn = PFN_DOWN(start);
+	end_pfn = start_pfn + PFN_DOWN(start);
+
+	for (pfn = start_pfn; pfn < end_pfn; pfn += PAGES_PER_SECTION) {
+		section_nr = pfn_to_section_nr(pfn);
+		if (!present_section_nr(section_nr));
+			continue;
+
+		section = __nr_to_section(section_nr);
+		/* same memblock? */
+		if (mem)
+			if((section_nr >= mem->start_section_nr) &&
+			   (section_nr <= mem->end_section_nr))
+				continue;
+
+		mem = find_memory_block_hinted(section, mem);
+		if (!mem)
+			continue;
+		if (mem->state == MEM_OFFLINE)
+			continue;
+
+		kobject_put(&mem->dev.kobj);
+		return false;
+	}
+
+	if (mem)
+		kobject_put(&mem->dev.kobj);
+
+	return true;
+}
+EXPORT_SYMBOL(is_memblk_offline);
+
   /*
    * register_memory - Setup a sysfs device for a memory block
    */

Index: linux-3.5-rc6/include/linux/memory.h
===================================================================

include/linux/memory.h +5 -0
@@ -106,6 +106,10 @@ static inline int memory_isolate_notify(
   {
   	return 0;
   }
+static inline bool is_memblk_offline(unsigned long start, unsigned long size)
+{
+	return false;
+}
   #else
   extern int register_memory_notifier(struct notifier_block *nb);
   extern void unregister_memory_notifier(struct notifier_block *nb);
@@ -120,6 +124,7 @@ extern int memory_isolate_notify(unsigne
   extern struct memory_block *find_memory_block_hinted(struct mem_section *,
   							struct memory_block *);
   extern struct memory_block *find_memory_block(struct mem_section *);
+extern bool is_memblk_offline(unsigned long start, unsigned long size);
   #define CONFIG_MEM_BLOCK_SIZE	(PAGES_PER_SECTION<<PAGE_SHIFT)
   enum mem_add_context { BOOT, HOTPLUG };
   #endif /* CONFIG_MEMORY_HOTPLUG_SPARSE */
Quoted 9 lines
Quoted 2 lines

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Quoted 5 lines

--
To unsubscribe from this list: send the line "unsubscribe linux-acpi" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-acpi" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#9 lore raw

Re: [RFC PATCH v3 2/13] memory-hotplug : add physical memory hotplug code to acpi_memory_device_remove

Hi Wen,

Quoted 97 lines · Wen Congyang

At 07/17/2012 09:54 AM, Yasuaki Ishimatsu Wrote:

Quoted 79 lines

Hi Wen,

Quoted 76 lines · Yasuaki Ishimatsu

Hi Wen,

Quoted 72 lines · Wen Congyang

At 07/09/2012 06:24 PM, Yasuaki Ishimatsu Wrote:

Quoted 57 lines

acpi_memory_device_remove() has been prepared to remove physical memory. But, the function only frees acpi_memory_device currentlry.

The patch adds following functions into acpi_memory_device_remove():
- offline memory
- remove physical memory (only return -EBUSY)
- free acpi_memory_device

CC David Rientjes <[email protected]>
CC Jiang Liu <[email protected]>
CC Len Brown <[email protected]>
CC Benjamin Herrenschmidt <[email protected]>
CC Paul Mackerras <[email protected]>
CC Christoph Lameter <[email protected]>
Cc Minchan Kim <[email protected]>
CC Andrew Morton <[email protected]>
CC KOSAKI Motohiro <[email protected]>
CC Wen Congyang <[email protected]>
---
    drivers/acpi/acpi_memhotplug.c |   26 +++++++++++++++++++++++++-
    drivers/base/memory.c          |   39 +++++++++++++++++++++++++++++++++++++++
    include/linux/memory.h         |    5 +++++
    include/linux/memory_hotplug.h |    1 +
    mm/memory_hotplug.c            |    8 ++++++++
    5 files changed, 78 insertions(+), 1 deletion(-)

Index: linux-3.5-rc6/drivers/acpi/acpi_memhotplug.c
===================================================================

drivers/acpi/acpi_memhotplug.c +8 -1
@@ -29,6 +29,7 @@
    #include <linux/module.h>
    #include <linux/init.h>
    #include <linux/types.h>
+#include <linux/memory.h>
    #include <linux/memory_hotplug.h>
    #include <linux/slab.h>
    #include <acpi/acpi_drivers.h>
@@ -452,12 +453,35 @@ static int acpi_memory_device_add(struct
    static int acpi_memory_device_remove(struct acpi_device *device, int type)
    {
    	struct acpi_memory_device *mem_device = NULL;
-
+	struct acpi_memory_info *info, *tmp;
+	int result;
+	int node;
 
    	if (!device || !acpi_driver_data(device))
    		return -EINVAL;
 
    	mem_device = acpi_driver_data(device);
+
+	node = acpi_get_node(mem_device->device->handle);
+
+	list_for_each_entry_safe(info, tmp, &mem_device->res_list, list) {

drivers/acpi/acpi_memhotplug.c static int acpi_memory_device_add(struct · line 477

Quoted 10 lines
drivers/acpi/acpi_memhotplug.c +10 -0
+		if (!info->enabled)
+			continue;
+
+		if (!is_memblk_offline(info->start_addr, info->length)) {
+			result = offline_memory(info->start_addr, info->length);
+			if (result)
+				return result;
+		}
+
+		result = remove_memory(node, info->start_addr, info->length);

The user may online the memory between offline_memory() and remove_memory(). So I think we should lock memory hotplug before check the memory's status and release it after remove_memory().

Quoted 7 lines
Quoted 4 lines

How about get "mem_block->state_mutex" of removed memory? When offlining memory, we need to change "memory_block->state" into "MEM_OFFLINE". In this case, we get mem_block->state_mutex. So I think the mutex lock is beneficial.

It is not good idea since remove_memory frees mem_block structure... Do you have any ideas?

Hmm, split offline_memory() to 2 functions: offline_pages() and __offline_pages()

offline_pages()

	lock_memory_hotplug();
	__offline_pages();
	unlock_memory_hotplug();

and implement remove_memory() like this:
remove_memory()

Quoted 8 lines
	lock_memory_hotplug()
	if (!is_memblk_offline()) {
		__offline_pages();
	}
	// cleanup
	unlock_memory_hotplug();

What about this?

I also thought about it once. But a problem remains. Current offilne_pages() cannot realize the memory has been removed by remove_memory(). So even if protecting the race by lock_memory_hotplug(), offline_pages() can offline the removed memory. offline_pages() should have the means to know the memory was removed. But I don't have good idea.

Thanks,
Yasuaki Ishimatsu

Quoted 154 lines

Thanks
Wen Congyang

Quoted 150 lines

Thanks,
Yasuaki Ishimatsu

Quoted 133 lines

Thanks,
Yasuaki Ishimatsu

Quoted 118 lines

Thanks
Wen Congyang

Quoted 114 lines
drivers/acpi/acpi_memhotplug.c +7 -0
+		if (result)
+			return result;
+
+		list_del(&info->list);
+		kfree(info);
+	}
+
    	kfree(mem_device);
 
    	return 0;

Index: linux-3.5-rc6/include/linux/memory_hotplug.h
===================================================================

include/linux/memory_hotplug.h +1 -0
@@ -233,6 +233,7 @@ static inline int is_mem_section_removab
    extern int mem_online_node(int nid);
    extern int add_memory(int nid, u64 start, u64 size);
    extern int arch_add_memory(int nid, u64 start, u64 size);
+extern int remove_memory(int nid, u64 start, u64 size);
    extern int offline_memory(u64 start, u64 size);
    extern int sparse_add_one_section(struct zone *zone, unsigned long start_pfn,
    								int nr_pages);

Index: linux-3.5-rc6/mm/memory_hotplug.c
===================================================================

mm/memory_hotplug.c +8 -0
@@ -659,6 +659,14 @@ out:
    }
    EXPORT_SYMBOL_GPL(add_memory);
 
+int remove_memory(int nid, u64 start, u64 size)
+{
+	return -EBUSY;
+
+}
+EXPORT_SYMBOL_GPL(remove_memory);
+
+
    #ifdef CONFIG_MEMORY_HOTREMOVE
    /*
     * A free page on the buddy free lists (not the per-cpu lists) has PageBuddy

Index: linux-3.5-rc6/drivers/base/memory.c
===================================================================

drivers/base/memory.c +39 -0
@@ -70,6 +70,45 @@ void unregister_memory_isolate_notifier(
    }
    EXPORT_SYMBOL(unregister_memory_isolate_notifier);
 
+bool is_memblk_offline(unsigned long start, unsigned long size)
+{
+	struct memory_block *mem = NULL;
+	struct mem_section *section;
+	unsigned long start_pfn, end_pfn;
+	unsigned long pfn, section_nr;
+
+	start_pfn = PFN_DOWN(start);
+	end_pfn = start_pfn + PFN_DOWN(start);
+
+	for (pfn = start_pfn; pfn < end_pfn; pfn += PAGES_PER_SECTION) {
+		section_nr = pfn_to_section_nr(pfn);
+		if (!present_section_nr(section_nr));
+			continue;
+
+		section = __nr_to_section(section_nr);
+		/* same memblock? */
+		if (mem)
+			if((section_nr >= mem->start_section_nr) &&
+			   (section_nr <= mem->end_section_nr))
+				continue;
+
+		mem = find_memory_block_hinted(section, mem);
+		if (!mem)
+			continue;
+		if (mem->state == MEM_OFFLINE)
+			continue;
+
+		kobject_put(&mem->dev.kobj);
+		return false;
+	}
+
+	if (mem)
+		kobject_put(&mem->dev.kobj);
+
+	return true;
+}
+EXPORT_SYMBOL(is_memblk_offline);
+
    /*
     * register_memory - Setup a sysfs device for a memory block
     */

Index: linux-3.5-rc6/include/linux/memory.h
===================================================================

include/linux/memory.h +5 -0
@@ -106,6 +106,10 @@ static inline int memory_isolate_notify(
    {
    	return 0;
    }
+static inline bool is_memblk_offline(unsigned long start, unsigned long size)
+{
+	return false;
+}
    #else
    extern int register_memory_notifier(struct notifier_block *nb);
    extern void unregister_memory_notifier(struct notifier_block *nb);
@@ -120,6 +124,7 @@ extern int memory_isolate_notify(unsigne
    extern struct memory_block *find_memory_block_hinted(struct mem_section *,
    							struct memory_block *);
    extern struct memory_block *find_memory_block(struct mem_section *);
+extern bool is_memblk_offline(unsigned long start, unsigned long size);
    #define CONFIG_MEM_BLOCK_SIZE	(PAGES_PER_SECTION<<PAGE_SHIFT)
    enum mem_add_context { BOOT, HOTPLUG };
    #endif /* CONFIG_MEMORY_HOTPLUG_SPARSE */
Quoted 9 lines
Quoted 2 lines

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Quoted 5 lines

--
To unsubscribe from this list: send the line "unsubscribe linux-acpi" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-acpi" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#10 lore raw

Re: [RFC PATCH v3 2/13] memory-hotplug : add physical memory hotplug code to acpi_memory_device_remove

At 07/17/2012 11:08 AM, Yasuaki Ishimatsu Wrote:

Quoted 107 lines

Hi Wen,

Quoted 104 lines · Wen Congyang

At 07/17/2012 09:54 AM, Yasuaki Ishimatsu Wrote:

Quoted 79 lines

Hi Wen,

Quoted 76 lines · Yasuaki Ishimatsu

Hi Wen,

Quoted 72 lines · Wen Congyang

At 07/09/2012 06:24 PM, Yasuaki Ishimatsu Wrote:

Quoted 57 lines

acpi_memory_device_remove() has been prepared to remove physical memory. But, the function only frees acpi_memory_device currentlry.

The patch adds following functions into acpi_memory_device_remove():
- offline memory
- remove physical memory (only return -EBUSY)
- free acpi_memory_device

CC David Rientjes <[email protected]>
CC Jiang Liu <[email protected]>
CC Len Brown <[email protected]>
CC Benjamin Herrenschmidt <[email protected]>
CC Paul Mackerras <[email protected]>
CC Christoph Lameter <[email protected]>
Cc Minchan Kim <[email protected]>
CC Andrew Morton <[email protected]>
CC KOSAKI Motohiro <[email protected]>
CC Wen Congyang <[email protected]>
---
    drivers/acpi/acpi_memhotplug.c |   26 +++++++++++++++++++++++++-
    drivers/base/memory.c          |   39 +++++++++++++++++++++++++++++++++++++++
    include/linux/memory.h         |    5 +++++
    include/linux/memory_hotplug.h |    1 +
    mm/memory_hotplug.c            |    8 ++++++++
    5 files changed, 78 insertions(+), 1 deletion(-)

Index: linux-3.5-rc6/drivers/acpi/acpi_memhotplug.c
===================================================================

drivers/acpi/acpi_memhotplug.c +8 -1
@@ -29,6 +29,7 @@
    #include <linux/module.h>
    #include <linux/init.h>
    #include <linux/types.h>
+#include <linux/memory.h>
    #include <linux/memory_hotplug.h>
    #include <linux/slab.h>
    #include <acpi/acpi_drivers.h>
@@ -452,12 +453,35 @@ static int acpi_memory_device_add(struct
    static int acpi_memory_device_remove(struct acpi_device *device, int type)
    {
    	struct acpi_memory_device *mem_device = NULL;
-
+	struct acpi_memory_info *info, *tmp;
+	int result;
+	int node;
 
    	if (!device || !acpi_driver_data(device))
    		return -EINVAL;
 
    	mem_device = acpi_driver_data(device);
+
+	node = acpi_get_node(mem_device->device->handle);
+
+	list_for_each_entry_safe(info, tmp, &mem_device->res_list, list) {

drivers/acpi/acpi_memhotplug.c static int acpi_memory_device_add(struct · line 477

Quoted 10 lines
drivers/acpi/acpi_memhotplug.c +10 -0
+		if (!info->enabled)
+			continue;
+
+		if (!is_memblk_offline(info->start_addr, info->length)) {
+			result = offline_memory(info->start_addr, info->length);
+			if (result)
+				return result;
+		}
+
+		result = remove_memory(node, info->start_addr, info->length);

The user may online the memory between offline_memory() and remove_memory(). So I think we should lock memory hotplug before check the memory's status and release it after remove_memory().

Quoted 7 lines
Quoted 4 lines

How about get "mem_block->state_mutex" of removed memory? When offlining memory, we need to change "memory_block->state" into "MEM_OFFLINE". In this case, we get mem_block->state_mutex. So I think the mutex lock is beneficial.

It is not good idea since remove_memory frees mem_block structure... Do you have any ideas?

Hmm, split offline_memory() to 2 functions: offline_pages() and __offline_pages()

offline_pages()

	lock_memory_hotplug();
	__offline_pages();
	unlock_memory_hotplug();

and implement remove_memory() like this:
remove_memory()

	lock_memory_hotplug()
	if (!is_memblk_offline()) {
		__offline_pages();
	}
	// cleanup
	unlock_memory_hotplug();
Quoted 7 lines
Quoted 1 line

What about this?

I also thought about it once. But a problem remains. Current offilne_pages() cannot realize the memory has been removed by remove_memory(). So even if protecting the race by lock_memory_hotplug(), offline_pages() can offline the removed memory. offline_pages() should have the means to know the memory was removed. But I don't have good idea.

We can not online/offline part of memory block, so what about this?

remove_memory()

	lock_memory_hotplug()
	for each memory block:
		if (!is_memblk_offline()) {
			__offline_pages();
		}
	// cleanup
	unlock_memory_hotplug();

Thanks
Wen Congyang

Quoted 162 lines

Thanks,
Yasuaki Ishimatsu

Quoted 154 lines

Thanks
Wen Congyang

Quoted 150 lines

Thanks,
Yasuaki Ishimatsu

Quoted 133 lines

Thanks,
Yasuaki Ishimatsu

Quoted 118 lines

Thanks
Wen Congyang

Quoted 114 lines
drivers/acpi/acpi_memhotplug.c +7 -0
+		if (result)
+			return result;
+
+		list_del(&info->list);
+		kfree(info);
+	}
+
    	kfree(mem_device);
 
    	return 0;

Index: linux-3.5-rc6/include/linux/memory_hotplug.h
===================================================================

include/linux/memory_hotplug.h +1 -0
@@ -233,6 +233,7 @@ static inline int is_mem_section_removab
    extern int mem_online_node(int nid);
    extern int add_memory(int nid, u64 start, u64 size);
    extern int arch_add_memory(int nid, u64 start, u64 size);
+extern int remove_memory(int nid, u64 start, u64 size);
    extern int offline_memory(u64 start, u64 size);
    extern int sparse_add_one_section(struct zone *zone, unsigned long start_pfn,
    								int nr_pages);

Index: linux-3.5-rc6/mm/memory_hotplug.c
===================================================================

mm/memory_hotplug.c +8 -0
@@ -659,6 +659,14 @@ out:
    }
    EXPORT_SYMBOL_GPL(add_memory);
 
+int remove_memory(int nid, u64 start, u64 size)
+{
+	return -EBUSY;
+
+}
+EXPORT_SYMBOL_GPL(remove_memory);
+
+
    #ifdef CONFIG_MEMORY_HOTREMOVE
    /*
     * A free page on the buddy free lists (not the per-cpu lists) has PageBuddy

Index: linux-3.5-rc6/drivers/base/memory.c
===================================================================

drivers/base/memory.c +39 -0
@@ -70,6 +70,45 @@ void unregister_memory_isolate_notifier(
    }
    EXPORT_SYMBOL(unregister_memory_isolate_notifier);
 
+bool is_memblk_offline(unsigned long start, unsigned long size)
+{
+	struct memory_block *mem = NULL;
+	struct mem_section *section;
+	unsigned long start_pfn, end_pfn;
+	unsigned long pfn, section_nr;
+
+	start_pfn = PFN_DOWN(start);
+	end_pfn = start_pfn + PFN_DOWN(start);
+
+	for (pfn = start_pfn; pfn < end_pfn; pfn += PAGES_PER_SECTION) {
+		section_nr = pfn_to_section_nr(pfn);
+		if (!present_section_nr(section_nr));
+			continue;
+
+		section = __nr_to_section(section_nr);
+		/* same memblock? */
+		if (mem)
+			if((section_nr >= mem->start_section_nr) &&
+			   (section_nr <= mem->end_section_nr))
+				continue;
+
+		mem = find_memory_block_hinted(section, mem);
+		if (!mem)
+			continue;
+		if (mem->state == MEM_OFFLINE)
+			continue;
+
+		kobject_put(&mem->dev.kobj);
+		return false;
+	}
+
+	if (mem)
+		kobject_put(&mem->dev.kobj);
+
+	return true;
+}
+EXPORT_SYMBOL(is_memblk_offline);
+
    /*
     * register_memory - Setup a sysfs device for a memory block
     */

Index: linux-3.5-rc6/include/linux/memory.h
===================================================================

include/linux/memory.h +5 -0
@@ -106,6 +106,10 @@ static inline int memory_isolate_notify(
    {
    	return 0;
    }
+static inline bool is_memblk_offline(unsigned long start, unsigned long size)
+{
+	return false;
+}
    #else
    extern int register_memory_notifier(struct notifier_block *nb);
    extern void unregister_memory_notifier(struct notifier_block *nb);
@@ -120,6 +124,7 @@ extern int memory_isolate_notify(unsigne
    extern struct memory_block *find_memory_block_hinted(struct mem_section *,
    							struct memory_block *);
    extern struct memory_block *find_memory_block(struct mem_section *);
+extern bool is_memblk_offline(unsigned long start, unsigned long size);
    #define CONFIG_MEM_BLOCK_SIZE	(PAGES_PER_SECTION<<PAGE_SHIFT)
    enum mem_add_context { BOOT, HOTPLUG };
    #endif /* CONFIG_MEMORY_HOTPLUG_SPARSE */
Quoted 9 lines
Quoted 2 lines

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Quoted 5 lines

--
To unsubscribe from this list: send the line "unsubscribe linux-acpi" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-acpi" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#11 lore raw

Re: [RFC PATCH v3 2/13] memory-hotplug : add physical memory hotplug code to acpi_memory_device_remove

Hi Wen,

Quoted 110 lines · Wen Congyang

At 07/17/2012 11:08 AM, Yasuaki Ishimatsu Wrote:

Quoted 109 lines

Hi Wen,

Quoted 105 lines · Wen Congyang

At 07/17/2012 09:54 AM, Yasuaki Ishimatsu Wrote:

Quoted 79 lines

Hi Wen,

Quoted 76 lines · Yasuaki Ishimatsu

Hi Wen,

Quoted 72 lines · Wen Congyang

At 07/09/2012 06:24 PM, Yasuaki Ishimatsu Wrote:

Quoted 57 lines

acpi_memory_device_remove() has been prepared to remove physical memory. But, the function only frees acpi_memory_device currentlry.

The patch adds following functions into acpi_memory_device_remove():
- offline memory
- remove physical memory (only return -EBUSY)
- free acpi_memory_device

CC David Rientjes <[email protected]>
CC Jiang Liu <[email protected]>
CC Len Brown <[email protected]>
CC Benjamin Herrenschmidt <[email protected]>
CC Paul Mackerras <[email protected]>
CC Christoph Lameter <[email protected]>
Cc Minchan Kim <[email protected]>
CC Andrew Morton <[email protected]>
CC KOSAKI Motohiro <[email protected]>
CC Wen Congyang <[email protected]>
---
     drivers/acpi/acpi_memhotplug.c |   26 +++++++++++++++++++++++++-
     drivers/base/memory.c          |   39 +++++++++++++++++++++++++++++++++++++++
     include/linux/memory.h         |    5 +++++
     include/linux/memory_hotplug.h |    1 +
     mm/memory_hotplug.c            |    8 ++++++++
     5 files changed, 78 insertions(+), 1 deletion(-)

Index: linux-3.5-rc6/drivers/acpi/acpi_memhotplug.c
===================================================================

drivers/acpi/acpi_memhotplug.c +8 -1
@@ -29,6 +29,7 @@
     #include <linux/module.h>
     #include <linux/init.h>
     #include <linux/types.h>
+#include <linux/memory.h>
     #include <linux/memory_hotplug.h>
     #include <linux/slab.h>
     #include <acpi/acpi_drivers.h>
@@ -452,12 +453,35 @@ static int acpi_memory_device_add(struct
     static int acpi_memory_device_remove(struct acpi_device *device, int type)
     {
     	struct acpi_memory_device *mem_device = NULL;
-
+	struct acpi_memory_info *info, *tmp;
+	int result;
+	int node;
 
     	if (!device || !acpi_driver_data(device))
     		return -EINVAL;
 
     	mem_device = acpi_driver_data(device);
+
+	node = acpi_get_node(mem_device->device->handle);
+
+	list_for_each_entry_safe(info, tmp, &mem_device->res_list, list) {

drivers/acpi/acpi_memhotplug.c static int acpi_memory_device_add(struct · line 477

Quoted 10 lines
drivers/acpi/acpi_memhotplug.c +10 -0
+		if (!info->enabled)
+			continue;
+
+		if (!is_memblk_offline(info->start_addr, info->length)) {
+			result = offline_memory(info->start_addr, info->length);
+			if (result)
+				return result;
+		}
+
+		result = remove_memory(node, info->start_addr, info->length);

The user may online the memory between offline_memory() and remove_memory(). So I think we should lock memory hotplug before check the memory's status and release it after remove_memory().

Quoted 7 lines
Quoted 4 lines

How about get "mem_block->state_mutex" of removed memory? When offlining memory, we need to change "memory_block->state" into "MEM_OFFLINE". In this case, we get mem_block->state_mutex. So I think the mutex lock is beneficial.

It is not good idea since remove_memory frees mem_block structure... Do you have any ideas?

Hmm, split offline_memory() to 2 functions: offline_pages() and __offline_pages()

offline_pages()

	lock_memory_hotplug();
	__offline_pages();
	unlock_memory_hotplug();

and implement remove_memory() like this:
remove_memory()

	lock_memory_hotplug()
	if (!is_memblk_offline()) {
		__offline_pages();
	}
	// cleanup
	unlock_memory_hotplug();

What about this?

Quoted 7 lines
Quoted 5 lines

I also thought about it once. But a problem remains. Current offilne_pages() cannot realize the memory has been removed by remove_memory(). So even if protecting the race by lock_memory_hotplug(), offline_pages() can offline the removed memory. offline_pages() should have the means to know the memory was removed. But I don't have good idea.

We can not online/offline part of memory block, so what about this?

It seems you do not understand my concern.
When memory_remove() and offline_pages() run to same memory simultaneously, offline_pages runs to removed memory.

memory_remove() | offline_pages()
-----------------------------------------------------------

lock_memory_hotplug()        |
                             | wait at lock_memory_hotplug()
remove memory                |
unlock_memory_hotplug()      |
                             | wake up and start offline_pages()
			     | offline page
			     | => but the memory has already removed
			     |    by memory_remove()

In this case, offline_page() may access removed memory.

Thanks,
Yasuaki Ishimatsu

Quoted 175 lines

remove_memory()

	lock_memory_hotplug()
	for each memory block:
		if (!is_memblk_offline()) {
			__offline_pages();
		}
	// cleanup
	unlock_memory_hotplug();

Thanks
Wen Congyang

Quoted 162 lines

Thanks,
Yasuaki Ishimatsu

Quoted 154 lines

Thanks
Wen Congyang

Quoted 150 lines

Thanks,
Yasuaki Ishimatsu

Quoted 133 lines

Thanks,
Yasuaki Ishimatsu

Quoted 118 lines

Thanks
Wen Congyang

Quoted 114 lines
drivers/acpi/acpi_memhotplug.c +7 -0
+		if (result)
+			return result;
+
+		list_del(&info->list);
+		kfree(info);
+	}
+
     	kfree(mem_device);
 
     	return 0;

Index: linux-3.5-rc6/include/linux/memory_hotplug.h
===================================================================

include/linux/memory_hotplug.h +1 -0
@@ -233,6 +233,7 @@ static inline int is_mem_section_removab
     extern int mem_online_node(int nid);
     extern int add_memory(int nid, u64 start, u64 size);
     extern int arch_add_memory(int nid, u64 start, u64 size);
+extern int remove_memory(int nid, u64 start, u64 size);
     extern int offline_memory(u64 start, u64 size);
     extern int sparse_add_one_section(struct zone *zone, unsigned long start_pfn,
     								int nr_pages);

Index: linux-3.5-rc6/mm/memory_hotplug.c
===================================================================

mm/memory_hotplug.c +8 -0
@@ -659,6 +659,14 @@ out:
     }
     EXPORT_SYMBOL_GPL(add_memory);
 
+int remove_memory(int nid, u64 start, u64 size)
+{
+	return -EBUSY;
+
+}
+EXPORT_SYMBOL_GPL(remove_memory);
+
+
     #ifdef CONFIG_MEMORY_HOTREMOVE
     /*
      * A free page on the buddy free lists (not the per-cpu lists) has PageBuddy

Index: linux-3.5-rc6/drivers/base/memory.c
===================================================================

drivers/base/memory.c +39 -0
@@ -70,6 +70,45 @@ void unregister_memory_isolate_notifier(
     }
     EXPORT_SYMBOL(unregister_memory_isolate_notifier);
 
+bool is_memblk_offline(unsigned long start, unsigned long size)
+{
+	struct memory_block *mem = NULL;
+	struct mem_section *section;
+	unsigned long start_pfn, end_pfn;
+	unsigned long pfn, section_nr;
+
+	start_pfn = PFN_DOWN(start);
+	end_pfn = start_pfn + PFN_DOWN(start);
+
+	for (pfn = start_pfn; pfn < end_pfn; pfn += PAGES_PER_SECTION) {
+		section_nr = pfn_to_section_nr(pfn);
+		if (!present_section_nr(section_nr));
+			continue;
+
+		section = __nr_to_section(section_nr);
+		/* same memblock? */
+		if (mem)
+			if((section_nr >= mem->start_section_nr) &&
+			   (section_nr <= mem->end_section_nr))
+				continue;
+
+		mem = find_memory_block_hinted(section, mem);
+		if (!mem)
+			continue;
+		if (mem->state == MEM_OFFLINE)
+			continue;
+
+		kobject_put(&mem->dev.kobj);
+		return false;
+	}
+
+	if (mem)
+		kobject_put(&mem->dev.kobj);
+
+	return true;
+}
+EXPORT_SYMBOL(is_memblk_offline);
+
     /*
      * register_memory - Setup a sysfs device for a memory block
      */

Index: linux-3.5-rc6/include/linux/memory.h
===================================================================

include/linux/memory.h +5 -0
@@ -106,6 +106,10 @@ static inline int memory_isolate_notify(
     {
     	return 0;
     }
+static inline bool is_memblk_offline(unsigned long start, unsigned long size)
+{
+	return false;
+}
     #else
     extern int register_memory_notifier(struct notifier_block *nb);
     extern void unregister_memory_notifier(struct notifier_block *nb);
@@ -120,6 +124,7 @@ extern int memory_isolate_notify(unsigne
     extern struct memory_block *find_memory_block_hinted(struct mem_section *,
     							struct memory_block *);
     extern struct memory_block *find_memory_block(struct mem_section *);
+extern bool is_memblk_offline(unsigned long start, unsigned long size);
     #define CONFIG_MEM_BLOCK_SIZE	(PAGES_PER_SECTION<<PAGE_SHIFT)
     enum mem_add_context { BOOT, HOTPLUG };
     #endif /* CONFIG_MEMORY_HOTPLUG_SPARSE */
Quoted 9 lines
Quoted 2 lines

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Quoted 5 lines

--
To unsubscribe from this list: send the line "unsubscribe linux-acpi" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-acpi" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Quoted 6 lines

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#12 lore raw

Re: [RFC PATCH v3 2/13] memory-hotplug : add physical memory hotplug code to acpi_memory_device_remove

At 07/17/2012 12:51 PM, Yasuaki Ishimatsu Wrote:

Quoted 136 lines

Hi Wen,

Quoted 110 lines · Wen Congyang

At 07/17/2012 11:08 AM, Yasuaki Ishimatsu Wrote:

Quoted 109 lines

Hi Wen,

Quoted 105 lines · Wen Congyang

At 07/17/2012 09:54 AM, Yasuaki Ishimatsu Wrote:

Quoted 79 lines

Hi Wen,

Quoted 76 lines · Yasuaki Ishimatsu

Hi Wen,

Quoted 72 lines · Wen Congyang

At 07/09/2012 06:24 PM, Yasuaki Ishimatsu Wrote:

Quoted 57 lines

acpi_memory_device_remove() has been prepared to remove physical memory. But, the function only frees acpi_memory_device currentlry.

The patch adds following functions into acpi_memory_device_remove():
- offline memory
- remove physical memory (only return -EBUSY)
- free acpi_memory_device

CC David Rientjes <[email protected]>
CC Jiang Liu <[email protected]>
CC Len Brown <[email protected]>
CC Benjamin Herrenschmidt <[email protected]>
CC Paul Mackerras <[email protected]>
CC Christoph Lameter <[email protected]>
Cc Minchan Kim <[email protected]>
CC Andrew Morton <[email protected]>
CC KOSAKI Motohiro <[email protected]>
CC Wen Congyang <[email protected]>
---
     drivers/acpi/acpi_memhotplug.c |   26 +++++++++++++++++++++++++-
     drivers/base/memory.c          |   39 +++++++++++++++++++++++++++++++++++++++
     include/linux/memory.h         |    5 +++++
     include/linux/memory_hotplug.h |    1 +
     mm/memory_hotplug.c            |    8 ++++++++
     5 files changed, 78 insertions(+), 1 deletion(-)

Index: linux-3.5-rc6/drivers/acpi/acpi_memhotplug.c
===================================================================

drivers/acpi/acpi_memhotplug.c +8 -1
@@ -29,6 +29,7 @@
     #include <linux/module.h>
     #include <linux/init.h>
     #include <linux/types.h>
+#include <linux/memory.h>
     #include <linux/memory_hotplug.h>
     #include <linux/slab.h>
     #include <acpi/acpi_drivers.h>
@@ -452,12 +453,35 @@ static int acpi_memory_device_add(struct
     static int acpi_memory_device_remove(struct acpi_device *device, int type)
     {
     	struct acpi_memory_device *mem_device = NULL;
-
+	struct acpi_memory_info *info, *tmp;
+	int result;
+	int node;
 
     	if (!device || !acpi_driver_data(device))
     		return -EINVAL;
 
     	mem_device = acpi_driver_data(device);
+
+	node = acpi_get_node(mem_device->device->handle);
+
+	list_for_each_entry_safe(info, tmp, &mem_device->res_list, list) {

drivers/acpi/acpi_memhotplug.c static int acpi_memory_device_add(struct · line 477

Quoted 10 lines
drivers/acpi/acpi_memhotplug.c +10 -0
+		if (!info->enabled)
+			continue;
+
+		if (!is_memblk_offline(info->start_addr, info->length)) {
+			result = offline_memory(info->start_addr, info->length);
+			if (result)
+				return result;
+		}
+
+		result = remove_memory(node, info->start_addr, info->length);

The user may online the memory between offline_memory() and remove_memory(). So I think we should lock memory hotplug before check the memory's status and release it after remove_memory().

Quoted 7 lines
Quoted 4 lines

How about get "mem_block->state_mutex" of removed memory? When offlining memory, we need to change "memory_block->state" into "MEM_OFFLINE". In this case, we get mem_block->state_mutex. So I think the mutex lock is beneficial.

It is not good idea since remove_memory frees mem_block structure... Do you have any ideas?

Hmm, split offline_memory() to 2 functions: offline_pages() and __offline_pages()

offline_pages()

	lock_memory_hotplug();
	__offline_pages();
	unlock_memory_hotplug();

and implement remove_memory() like this:
remove_memory()

	lock_memory_hotplug()
	if (!is_memblk_offline()) {
		__offline_pages();
	}
	// cleanup
	unlock_memory_hotplug();

What about this?

Quoted 7 lines
Quoted 5 lines

I also thought about it once. But a problem remains. Current offilne_pages() cannot realize the memory has been removed by remove_memory(). So even if protecting the race by lock_memory_hotplug(), offline_pages() can offline the removed memory. offline_pages() should have the means to know the memory was removed. But I don't have good idea.

We can not online/offline part of memory block, so what about this?

It seems you do not understand my concern.
When memory_remove() and offline_pages() run to same memory simultaneously, offline_pages runs to removed memory.

memory_remove() | offline_pages()
-----------------------------------------------------------

lock_memory_hotplug()        |
                             | wait at lock_memory_hotplug()
remove memory                |
unlock_memory_hotplug()      |
                             | wake up and start offline_pages()
			     | offline page
			     | => but the memory has already removed
			     |    by memory_remove()
Quoted 1 line

In this case, offline_page() may access removed memory.

Yes, in this case, the kernel may panic.

I think we can call pfn_present() in online_pages()/offline_pages() to check whether the memory is removed.

Thanks
Wen Congyang

Quoted 188 lines

Thanks,
Yasuaki Ishimatsu

Quoted 175 lines

remove_memory()

	lock_memory_hotplug()
	for each memory block:
		if (!is_memblk_offline()) {
			__offline_pages();
		}
	// cleanup
	unlock_memory_hotplug();

Thanks
Wen Congyang

Quoted 162 lines

Thanks,
Yasuaki Ishimatsu

Quoted 154 lines

Thanks
Wen Congyang

Quoted 150 lines

Thanks,
Yasuaki Ishimatsu

Quoted 133 lines

Thanks,
Yasuaki Ishimatsu

Quoted 118 lines

Thanks
Wen Congyang

Quoted 114 lines
drivers/acpi/acpi_memhotplug.c +7 -0
+		if (result)
+			return result;
+
+		list_del(&info->list);
+		kfree(info);
+	}
+
     	kfree(mem_device);
 
     	return 0;

Index: linux-3.5-rc6/include/linux/memory_hotplug.h
===================================================================

include/linux/memory_hotplug.h +1 -0
@@ -233,6 +233,7 @@ static inline int is_mem_section_removab
     extern int mem_online_node(int nid);
     extern int add_memory(int nid, u64 start, u64 size);
     extern int arch_add_memory(int nid, u64 start, u64 size);
+extern int remove_memory(int nid, u64 start, u64 size);
     extern int offline_memory(u64 start, u64 size);
     extern int sparse_add_one_section(struct zone *zone, unsigned long start_pfn,
     								int nr_pages);

Index: linux-3.5-rc6/mm/memory_hotplug.c
===================================================================

mm/memory_hotplug.c +8 -0
@@ -659,6 +659,14 @@ out:
     }
     EXPORT_SYMBOL_GPL(add_memory);
 
+int remove_memory(int nid, u64 start, u64 size)
+{
+	return -EBUSY;
+
+}
+EXPORT_SYMBOL_GPL(remove_memory);
+
+
     #ifdef CONFIG_MEMORY_HOTREMOVE
     /*
      * A free page on the buddy free lists (not the per-cpu lists) has PageBuddy

Index: linux-3.5-rc6/drivers/base/memory.c
===================================================================

drivers/base/memory.c +39 -0
@@ -70,6 +70,45 @@ void unregister_memory_isolate_notifier(
     }
     EXPORT_SYMBOL(unregister_memory_isolate_notifier);
 
+bool is_memblk_offline(unsigned long start, unsigned long size)
+{
+	struct memory_block *mem = NULL;
+	struct mem_section *section;
+	unsigned long start_pfn, end_pfn;
+	unsigned long pfn, section_nr;
+
+	start_pfn = PFN_DOWN(start);
+	end_pfn = start_pfn + PFN_DOWN(start);
+
+	for (pfn = start_pfn; pfn < end_pfn; pfn += PAGES_PER_SECTION) {
+		section_nr = pfn_to_section_nr(pfn);
+		if (!present_section_nr(section_nr));
+			continue;
+
+		section = __nr_to_section(section_nr);
+		/* same memblock? */
+		if (mem)
+			if((section_nr >= mem->start_section_nr) &&
+			   (section_nr <= mem->end_section_nr))
+				continue;
+
+		mem = find_memory_block_hinted(section, mem);
+		if (!mem)
+			continue;
+		if (mem->state == MEM_OFFLINE)
+			continue;
+
+		kobject_put(&mem->dev.kobj);
+		return false;
+	}
+
+	if (mem)
+		kobject_put(&mem->dev.kobj);
+
+	return true;
+}
+EXPORT_SYMBOL(is_memblk_offline);
+
     /*
      * register_memory - Setup a sysfs device for a memory block
      */

Index: linux-3.5-rc6/include/linux/memory.h
===================================================================

include/linux/memory.h +5 -0
@@ -106,6 +106,10 @@ static inline int memory_isolate_notify(
     {
     	return 0;
     }
+static inline bool is_memblk_offline(unsigned long start, unsigned long size)
+{
+	return false;
+}
     #else
     extern int register_memory_notifier(struct notifier_block *nb);
     extern void unregister_memory_notifier(struct notifier_block *nb);
@@ -120,6 +124,7 @@ extern int memory_isolate_notify(unsigne
     extern struct memory_block *find_memory_block_hinted(struct mem_section *,
     							struct memory_block *);
     extern struct memory_block *find_memory_block(struct mem_section *);
+extern bool is_memblk_offline(unsigned long start, unsigned long size);
     #define CONFIG_MEM_BLOCK_SIZE	(PAGES_PER_SECTION<<PAGE_SHIFT)
     enum mem_add_context { BOOT, HOTPLUG };
     #endif /* CONFIG_MEMORY_HOTPLUG_SPARSE */
Quoted 9 lines
Quoted 2 lines

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Quoted 5 lines

--
To unsubscribe from this list: send the line "unsubscribe linux-acpi" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-acpi" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Quoted 6 lines

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Quoted 6 lines

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#13 lore raw

Re: [RFC PATCH v3 2/13] memory-hotplug : add physical memory hotplug code to acpi_memory_device_remove

Hi Wen,

Quoted 137 lines · Wen Congyang

At 07/17/2012 12:51 PM, Yasuaki Ishimatsu Wrote:

Quoted 136 lines

Hi Wen,

Quoted 110 lines · Wen Congyang

At 07/17/2012 11:08 AM, Yasuaki Ishimatsu Wrote:

Quoted 109 lines

Hi Wen,

Quoted 105 lines · Wen Congyang

At 07/17/2012 09:54 AM, Yasuaki Ishimatsu Wrote:

Quoted 79 lines

Hi Wen,

Quoted 76 lines · Yasuaki Ishimatsu

Hi Wen,

Quoted 72 lines · Wen Congyang

At 07/09/2012 06:24 PM, Yasuaki Ishimatsu Wrote:

Quoted 57 lines

acpi_memory_device_remove() has been prepared to remove physical memory. But, the function only frees acpi_memory_device currentlry.

The patch adds following functions into acpi_memory_device_remove():
- offline memory
- remove physical memory (only return -EBUSY)
- free acpi_memory_device

CC David Rientjes <[email protected]>
CC Jiang Liu <[email protected]>
CC Len Brown <[email protected]>
CC Benjamin Herrenschmidt <[email protected]>
CC Paul Mackerras <[email protected]>
CC Christoph Lameter <[email protected]>
Cc Minchan Kim <[email protected]>
CC Andrew Morton <[email protected]>
CC KOSAKI Motohiro <[email protected]>
CC Wen Congyang <[email protected]>
---
      drivers/acpi/acpi_memhotplug.c |   26 +++++++++++++++++++++++++-
      drivers/base/memory.c          |   39 +++++++++++++++++++++++++++++++++++++++
      include/linux/memory.h         |    5 +++++
      include/linux/memory_hotplug.h |    1 +
      mm/memory_hotplug.c            |    8 ++++++++
      5 files changed, 78 insertions(+), 1 deletion(-)

Index: linux-3.5-rc6/drivers/acpi/acpi_memhotplug.c
===================================================================

drivers/acpi/acpi_memhotplug.c +8 -1
@@ -29,6 +29,7 @@
      #include <linux/module.h>
      #include <linux/init.h>
      #include <linux/types.h>
+#include <linux/memory.h>
      #include <linux/memory_hotplug.h>
      #include <linux/slab.h>
      #include <acpi/acpi_drivers.h>
@@ -452,12 +453,35 @@ static int acpi_memory_device_add(struct
      static int acpi_memory_device_remove(struct acpi_device *device, int type)
      {
      	struct acpi_memory_device *mem_device = NULL;
-
+	struct acpi_memory_info *info, *tmp;
+	int result;
+	int node;
 
      	if (!device || !acpi_driver_data(device))
      		return -EINVAL;
 
      	mem_device = acpi_driver_data(device);
+
+	node = acpi_get_node(mem_device->device->handle);
+
+	list_for_each_entry_safe(info, tmp, &mem_device->res_list, list) {

drivers/acpi/acpi_memhotplug.c static int acpi_memory_device_add(struct · line 477

Quoted 10 lines
drivers/acpi/acpi_memhotplug.c +10 -0
+		if (!info->enabled)
+			continue;
+
+		if (!is_memblk_offline(info->start_addr, info->length)) {
+			result = offline_memory(info->start_addr, info->length);
+			if (result)
+				return result;
+		}
+
+		result = remove_memory(node, info->start_addr, info->length);

The user may online the memory between offline_memory() and remove_memory(). So I think we should lock memory hotplug before check the memory's status and release it after remove_memory().

Quoted 7 lines
Quoted 4 lines

How about get "mem_block->state_mutex" of removed memory? When offlining memory, we need to change "memory_block->state" into "MEM_OFFLINE". In this case, we get mem_block->state_mutex. So I think the mutex lock is beneficial.

It is not good idea since remove_memory frees mem_block structure... Do you have any ideas?

Hmm, split offline_memory() to 2 functions: offline_pages() and __offline_pages()

offline_pages()

	lock_memory_hotplug();
	__offline_pages();
	unlock_memory_hotplug();

and implement remove_memory() like this:
remove_memory()

	lock_memory_hotplug()
	if (!is_memblk_offline()) {
		__offline_pages();
	}
	// cleanup
	unlock_memory_hotplug();

What about this?

Quoted 7 lines
Quoted 5 lines

I also thought about it once. But a problem remains. Current offilne_pages() cannot realize the memory has been removed by remove_memory(). So even if protecting the race by lock_memory_hotplug(), offline_pages() can offline the removed memory. offline_pages() should have the means to know the memory was removed. But I don't have good idea.

We can not online/offline part of memory block, so what about this?

It seems you do not understand my concern.
When memory_remove() and offline_pages() run to same memory simultaneously, offline_pages runs to removed memory.

memory_remove() | offline_pages()
-----------------------------------------------------------
lock_memory_hotplug() |
| wait at lock_memory_hotplug()
remove memory |
unlock_memory_hotplug() |
| wake up and start offline_pages()
| offline page
| => but the memory has already removed
| by memory_remove()

Quoted 6 lines
Quoted 1 line

In this case, offline_page() may access removed memory.

Yes, in this case, the kernel may panic.

I think we can call pfn_present() in online_pages()/offline_pages() to check whether the memory is removed.

Thank you for good idea. I'll add it.

Thanks,
Yasuaki Ishimatsu

Quoted 198 lines

Thanks
Wen Congyang

Quoted 194 lines

Thanks,
Yasuaki Ishimatsu

Quoted 175 lines

remove_memory()

	lock_memory_hotplug()
	for each memory block:
		if (!is_memblk_offline()) {
			__offline_pages();
		}
	// cleanup
	unlock_memory_hotplug();

Thanks
Wen Congyang

Quoted 162 lines

Thanks,
Yasuaki Ishimatsu

Quoted 154 lines

Thanks
Wen Congyang

Quoted 150 lines

Thanks,
Yasuaki Ishimatsu

Quoted 133 lines

Thanks,
Yasuaki Ishimatsu

Quoted 118 lines

Thanks
Wen Congyang

Quoted 114 lines
drivers/acpi/acpi_memhotplug.c +7 -0
+		if (result)
+			return result;
+
+		list_del(&info->list);
+		kfree(info);
+	}
+
      	kfree(mem_device);
 
      	return 0;

Index: linux-3.5-rc6/include/linux/memory_hotplug.h
===================================================================

include/linux/memory_hotplug.h +1 -0
@@ -233,6 +233,7 @@ static inline int is_mem_section_removab
      extern int mem_online_node(int nid);
      extern int add_memory(int nid, u64 start, u64 size);
      extern int arch_add_memory(int nid, u64 start, u64 size);
+extern int remove_memory(int nid, u64 start, u64 size);
      extern int offline_memory(u64 start, u64 size);
      extern int sparse_add_one_section(struct zone *zone, unsigned long start_pfn,
      								int nr_pages);

Index: linux-3.5-rc6/mm/memory_hotplug.c
===================================================================

mm/memory_hotplug.c +8 -0
@@ -659,6 +659,14 @@ out:
      }
      EXPORT_SYMBOL_GPL(add_memory);
 
+int remove_memory(int nid, u64 start, u64 size)
+{
+	return -EBUSY;
+
+}
+EXPORT_SYMBOL_GPL(remove_memory);
+
+
      #ifdef CONFIG_MEMORY_HOTREMOVE
      /*
       * A free page on the buddy free lists (not the per-cpu lists) has PageBuddy

Index: linux-3.5-rc6/drivers/base/memory.c
===================================================================

drivers/base/memory.c +39 -0
@@ -70,6 +70,45 @@ void unregister_memory_isolate_notifier(
      }
      EXPORT_SYMBOL(unregister_memory_isolate_notifier);
 
+bool is_memblk_offline(unsigned long start, unsigned long size)
+{
+	struct memory_block *mem = NULL;
+	struct mem_section *section;
+	unsigned long start_pfn, end_pfn;
+	unsigned long pfn, section_nr;
+
+	start_pfn = PFN_DOWN(start);
+	end_pfn = start_pfn + PFN_DOWN(start);
+
+	for (pfn = start_pfn; pfn < end_pfn; pfn += PAGES_PER_SECTION) {
+		section_nr = pfn_to_section_nr(pfn);
+		if (!present_section_nr(section_nr));
+			continue;
+
+		section = __nr_to_section(section_nr);
+		/* same memblock? */
+		if (mem)
+			if((section_nr >= mem->start_section_nr) &&
+			   (section_nr <= mem->end_section_nr))
+				continue;
+
+		mem = find_memory_block_hinted(section, mem);
+		if (!mem)
+			continue;
+		if (mem->state == MEM_OFFLINE)
+			continue;
+
+		kobject_put(&mem->dev.kobj);
+		return false;
+	}
+
+	if (mem)
+		kobject_put(&mem->dev.kobj);
+
+	return true;
+}
+EXPORT_SYMBOL(is_memblk_offline);
+
      /*
       * register_memory - Setup a sysfs device for a memory block
       */

Index: linux-3.5-rc6/include/linux/memory.h
===================================================================

include/linux/memory.h +5 -0
@@ -106,6 +106,10 @@ static inline int memory_isolate_notify(
      {
      	return 0;
      }
+static inline bool is_memblk_offline(unsigned long start, unsigned long size)
+{
+	return false;
+}
      #else
      extern int register_memory_notifier(struct notifier_block *nb);
      extern void unregister_memory_notifier(struct notifier_block *nb);
@@ -120,6 +124,7 @@ extern int memory_isolate_notify(unsigne
      extern struct memory_block *find_memory_block_hinted(struct mem_section *,
      							struct memory_block *);
      extern struct memory_block *find_memory_block(struct mem_section *);
+extern bool is_memblk_offline(unsigned long start, unsigned long size);
      #define CONFIG_MEM_BLOCK_SIZE	(PAGES_PER_SECTION<<PAGE_SHIFT)
      enum mem_add_context { BOOT, HOTPLUG };
      #endif /* CONFIG_MEMORY_HOTPLUG_SPARSE */
Quoted 9 lines
Quoted 2 lines

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Quoted 5 lines

--
To unsubscribe from this list: send the line "unsubscribe linux-acpi" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-acpi" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Quoted 6 lines

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

#14 lore raw

Re: [RFC PATCH v3 2/13] memory-hotplug : add physical memory hotplug code to acpi_memory_device_remove

In reply to Yasuaki Ishimatsu

At 07/09/2012 06:24 PM, Yasuaki Ishimatsu Wrote:

Quoted 45 lines

acpi_memory_device_remove() has been prepared to remove physical memory. But, the function only frees acpi_memory_device currentlry.

The patch adds following functions into acpi_memory_device_remove():
- offline memory
- remove physical memory (only return -EBUSY)
- free acpi_memory_device

CC David Rientjes <[email protected]>
CC Jiang Liu <[email protected]>
CC Len Brown <[email protected]>
CC Benjamin Herrenschmidt <[email protected]>
CC Paul Mackerras <[email protected]>
CC Christoph Lameter <[email protected]>
Cc Minchan Kim <[email protected]>
CC Andrew Morton <[email protected]>
CC KOSAKI Motohiro <[email protected]>
CC Wen Congyang <[email protected]>
---
 drivers/acpi/acpi_memhotplug.c |   26 +++++++++++++++++++++++++-
 drivers/base/memory.c          |   39 +++++++++++++++++++++++++++++++++++++++
 include/linux/memory.h         |    5 +++++
 include/linux/memory_hotplug.h |    1 +
 mm/memory_hotplug.c            |    8 ++++++++
 5 files changed, 78 insertions(+), 1 deletion(-)

Index: linux-3.5-rc6/drivers/acpi/acpi_memhotplug.c
===================================================================

drivers/acpi/acpi_memhotplug.c +1 -1
@@ -29,6 +29,7 @@
 #include <linux/module.h>
 #include <linux/init.h>
 #include <linux/types.h>
+#include <linux/memory.h>
 #include <linux/memory_hotplug.h>
 #include <linux/slab.h>
 #include <acpi/acpi_drivers.h>
@@ -452,12 +453,35 @@ static int acpi_memory_device_add(struct
 static int acpi_memory_device_remove(struct acpi_device *device, int type)
 {
 	struct acpi_memory_device *mem_device = NULL;
-

drivers/acpi/acpi_memhotplug.c static int acpi_memory_device_add(struct · line 465

Quoted 10 lines
drivers/acpi/acpi_memhotplug.c +5 -0
+	struct acpi_memory_info *info, *tmp;
+	int result;
+	int node;
 
 	if (!device || !acpi_driver_data(device))
 		return -EINVAL;
 
 	mem_device = acpi_driver_data(device);
+
+	node = acpi_get_node(mem_device->device->handle);

acpi_get_node() may return -1, and you should call memory_add_physaddr_to_nid() to get the node id.

Thanks
Wen Congyang

Quoted 121 lines
drivers/acpi/acpi_memhotplug.c +19 -0
+
+	list_for_each_entry_safe(info, tmp, &mem_device->res_list, list) {
+		if (!info->enabled)
+			continue;
+
+		if (!is_memblk_offline(info->start_addr, info->length)) {
+			result = offline_memory(info->start_addr, info->length);
+			if (result)
+				return result;
+		}
+
+		result = remove_memory(node, info->start_addr, info->length);
+		if (result)
+			return result;
+
+		list_del(&info->list);
+		kfree(info);
+	}
+
 	kfree(mem_device);
 
 	return 0;

Index: linux-3.5-rc6/include/linux/memory_hotplug.h
===================================================================

include/linux/memory_hotplug.h +1 -0
@@ -233,6 +233,7 @@ static inline int is_mem_section_removab
 extern int mem_online_node(int nid);
 extern int add_memory(int nid, u64 start, u64 size);
 extern int arch_add_memory(int nid, u64 start, u64 size);
+extern int remove_memory(int nid, u64 start, u64 size);
 extern int offline_memory(u64 start, u64 size);
 extern int sparse_add_one_section(struct zone *zone, unsigned long start_pfn,
 								int nr_pages);

Index: linux-3.5-rc6/mm/memory_hotplug.c
===================================================================

mm/memory_hotplug.c +8 -0
@@ -659,6 +659,14 @@ out:
 }
 EXPORT_SYMBOL_GPL(add_memory);
 
+int remove_memory(int nid, u64 start, u64 size)
+{
+	return -EBUSY;
+
+}
+EXPORT_SYMBOL_GPL(remove_memory);
+
+
 #ifdef CONFIG_MEMORY_HOTREMOVE
 /*
  * A free page on the buddy free lists (not the per-cpu lists) has PageBuddy

Index: linux-3.5-rc6/drivers/base/memory.c
===================================================================

drivers/base/memory.c +39 -0
@@ -70,6 +70,45 @@ void unregister_memory_isolate_notifier(
 }
 EXPORT_SYMBOL(unregister_memory_isolate_notifier);
 
+bool is_memblk_offline(unsigned long start, unsigned long size)
+{
+	struct memory_block *mem = NULL;
+	struct mem_section *section;
+	unsigned long start_pfn, end_pfn;
+	unsigned long pfn, section_nr;
+
+	start_pfn = PFN_DOWN(start);
+	end_pfn = start_pfn + PFN_DOWN(start);
+
+	for (pfn = start_pfn; pfn < end_pfn; pfn += PAGES_PER_SECTION) {
+		section_nr = pfn_to_section_nr(pfn);
+		if (!present_section_nr(section_nr));
+			continue;
+
+		section = __nr_to_section(section_nr);
+		/* same memblock? */
+		if (mem)
+			if((section_nr >= mem->start_section_nr) &&
+			   (section_nr <= mem->end_section_nr))
+				continue;
+
+		mem = find_memory_block_hinted(section, mem);
+		if (!mem)
+			continue;
+		if (mem->state == MEM_OFFLINE)
+			continue;
+
+		kobject_put(&mem->dev.kobj);
+		return false;
+	}
+
+	if (mem)
+		kobject_put(&mem->dev.kobj);
+
+	return true;
+}
+EXPORT_SYMBOL(is_memblk_offline);
+
 /*
  * register_memory - Setup a sysfs device for a memory block
  */

Index: linux-3.5-rc6/include/linux/memory.h
===================================================================

include/linux/memory.h +4 -0
@@ -106,6 +106,10 @@ static inline int memory_isolate_notify(
 {
 	return 0;
 }
+static inline bool is_memblk_offline(unsigned long start, unsigned long size)
+{
+	return false;
+}
 #else
 extern int register_memory_notifier(struct notifier_block *nb);
 extern void unregister_memory_notifier(struct notifier_block *nb);

include/linux/memory.h extern int memory_isolate_notify(unsigne · line 130

Quoted 7 lines
include/linux/memory.h +1 -0
@@ -120,6 +124,7 @@ extern int memory_isolate_notify(unsigne
 extern struct memory_block *find_memory_block_hinted(struct mem_section *,
 							struct memory_block *);
 extern struct memory_block *find_memory_block(struct mem_section *);
+extern bool is_memblk_offline(unsigned long start, unsigned long size);
 #define CONFIG_MEM_BLOCK_SIZE	(PAGES_PER_SECTION<<PAGE_SHIFT)
 enum mem_add_context { BOOT, HOTPLUG };
 #endif /* CONFIG_MEMORY_HOTPLUG_SPARSE */

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#15 lore raw

Re: [RFC PATCH v3 2/13] memory-hotplug : add physical memory hotplug code to acpi_memory_device_remove

Hi Wen,

Quoted 53 lines · Wen Congyang

At 07/09/2012 06:24 PM, Yasuaki Ishimatsu Wrote:

Quoted 52 lines

acpi_memory_device_remove() has been prepared to remove physical memory. But, the function only frees acpi_memory_device currentlry.

The patch adds following functions into acpi_memory_device_remove():
- offline memory
- remove physical memory (only return -EBUSY)
- free acpi_memory_device

CC David Rientjes <[email protected]>
CC Jiang Liu <[email protected]>
CC Len Brown <[email protected]>
CC Benjamin Herrenschmidt <[email protected]>
CC Paul Mackerras <[email protected]>
CC Christoph Lameter <[email protected]>
Cc Minchan Kim <[email protected]>
CC Andrew Morton <[email protected]>
CC KOSAKI Motohiro <[email protected]>
CC Wen Congyang <[email protected]>
---
  drivers/acpi/acpi_memhotplug.c |   26 +++++++++++++++++++++++++-
  drivers/base/memory.c          |   39 +++++++++++++++++++++++++++++++++++++++
  include/linux/memory.h         |    5 +++++
  include/linux/memory_hotplug.h |    1 +
  mm/memory_hotplug.c            |    8 ++++++++
  5 files changed, 78 insertions(+), 1 deletion(-)

Index: linux-3.5-rc6/drivers/acpi/acpi_memhotplug.c
===================================================================

drivers/acpi/acpi_memhotplug.c +4 -1
@@ -29,6 +29,7 @@
  #include <linux/module.h>
  #include <linux/init.h>
  #include <linux/types.h>
+#include <linux/memory.h>
  #include <linux/memory_hotplug.h>
  #include <linux/slab.h>
  #include <acpi/acpi_drivers.h>
@@ -452,12 +453,35 @@ static int acpi_memory_device_add(struct
  static int acpi_memory_device_remove(struct acpi_device *device, int type)
  {
  	struct acpi_memory_device *mem_device = NULL;
-
+	struct acpi_memory_info *info, *tmp;
+	int result;
+	int node;
 
  	if (!device || !acpi_driver_data(device))
  		return -EINVAL;
 
Quoted 6 lines

drivers/acpi/acpi_memhotplug.c static int acpi_memory_device_add(struct · line 465

Quoted 3 lines
drivers/acpi/acpi_memhotplug.c +2 -0
  	mem_device = acpi_driver_data(device);
+
+	node = acpi_get_node(mem_device->device->handle);

acpi_get_node() may return -1, and you should call memory_add_physaddr_to_nid() to get the node id.

O.K. I'll update it.

Thanks,
Yasuaki Ishimatsu

Quoted 139 lines

Thanks
Wen Congyang

Quoted 121 lines
drivers/acpi/acpi_memhotplug.c +19 -0
+
+	list_for_each_entry_safe(info, tmp, &mem_device->res_list, list) {
+		if (!info->enabled)
+			continue;
+
+		if (!is_memblk_offline(info->start_addr, info->length)) {
+			result = offline_memory(info->start_addr, info->length);
+			if (result)
+				return result;
+		}
+
+		result = remove_memory(node, info->start_addr, info->length);
+		if (result)
+			return result;
+
+		list_del(&info->list);
+		kfree(info);
+	}
+
  	kfree(mem_device);
 
  	return 0;

Index: linux-3.5-rc6/include/linux/memory_hotplug.h
===================================================================

include/linux/memory_hotplug.h +1 -0
@@ -233,6 +233,7 @@ static inline int is_mem_section_removab
  extern int mem_online_node(int nid);
  extern int add_memory(int nid, u64 start, u64 size);
  extern int arch_add_memory(int nid, u64 start, u64 size);
+extern int remove_memory(int nid, u64 start, u64 size);
  extern int offline_memory(u64 start, u64 size);
  extern int sparse_add_one_section(struct zone *zone, unsigned long start_pfn,
  								int nr_pages);

Index: linux-3.5-rc6/mm/memory_hotplug.c
===================================================================

mm/memory_hotplug.c +8 -0
@@ -659,6 +659,14 @@ out:
  }
  EXPORT_SYMBOL_GPL(add_memory);
 
+int remove_memory(int nid, u64 start, u64 size)
+{
+	return -EBUSY;
+
+}
+EXPORT_SYMBOL_GPL(remove_memory);
+
+
  #ifdef CONFIG_MEMORY_HOTREMOVE
  /*
   * A free page on the buddy free lists (not the per-cpu lists) has PageBuddy

Index: linux-3.5-rc6/drivers/base/memory.c
===================================================================

drivers/base/memory.c +39 -0
@@ -70,6 +70,45 @@ void unregister_memory_isolate_notifier(
  }
  EXPORT_SYMBOL(unregister_memory_isolate_notifier);
 
+bool is_memblk_offline(unsigned long start, unsigned long size)
+{
+	struct memory_block *mem = NULL;
+	struct mem_section *section;
+	unsigned long start_pfn, end_pfn;
+	unsigned long pfn, section_nr;
+
+	start_pfn = PFN_DOWN(start);
+	end_pfn = start_pfn + PFN_DOWN(start);
+
+	for (pfn = start_pfn; pfn < end_pfn; pfn += PAGES_PER_SECTION) {
+		section_nr = pfn_to_section_nr(pfn);
+		if (!present_section_nr(section_nr));
+			continue;
+
+		section = __nr_to_section(section_nr);
+		/* same memblock? */
+		if (mem)
+			if((section_nr >= mem->start_section_nr) &&
+			   (section_nr <= mem->end_section_nr))
+				continue;
+
+		mem = find_memory_block_hinted(section, mem);
+		if (!mem)
+			continue;
+		if (mem->state == MEM_OFFLINE)
+			continue;
+
+		kobject_put(&mem->dev.kobj);
+		return false;
+	}
+
+	if (mem)
+		kobject_put(&mem->dev.kobj);
+
+	return true;
+}
+EXPORT_SYMBOL(is_memblk_offline);
+
  /*
   * register_memory - Setup a sysfs device for a memory block
   */

Index: linux-3.5-rc6/include/linux/memory.h
===================================================================

include/linux/memory.h +4 -0
@@ -106,6 +106,10 @@ static inline int memory_isolate_notify(
  {
  	return 0;
  }
+static inline bool is_memblk_offline(unsigned long start, unsigned long size)
+{
+	return false;
+}
  #else
  extern int register_memory_notifier(struct notifier_block *nb);
  extern void unregister_memory_notifier(struct notifier_block *nb);

include/linux/memory.h extern int memory_isolate_notify(unsigne · line 130

Quoted 7 lines
include/linux/memory.h +1 -0
@@ -120,6 +124,7 @@ extern int memory_isolate_notify(unsigne
  extern struct memory_block *find_memory_block_hinted(struct mem_section *,
  							struct memory_block *);
  extern struct memory_block *find_memory_block(struct mem_section *);
+extern bool is_memblk_offline(unsigned long start, unsigned long size);
  #define CONFIG_MEM_BLOCK_SIZE	(PAGES_PER_SECTION<<PAGE_SHIFT)
  enum mem_add_context { BOOT, HOTPLUG };
  #endif /* CONFIG_MEMORY_HOTPLUG_SPARSE */

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#16 lore raw

RFC PATCH v3 3/13 memory-hotplug : unify argument of firmware_map_add_early/hotplug

In reply to Yasuaki Ishimatsu

There are two ways to create /sys/firmware/memmap/X sysfs:

- firmware_map_add_early
When the system starts, it is calledd from e820_reserve_resources()
- firmware_map_add_hotplug
When the memory is hot plugged, it is called from add_memory()

But these functions are called without unifying value of end argument as below:

- end argument of firmware_map_add_early() : start + size - 1
- end argument of firmware_map_add_hogplug() : start + size

The patch unifies them to "start + size - 1".

CC David Rientjes <[email protected]>
CC Jiang Liu <[email protected]>
CC Len Brown <[email protected]>
CC Benjamin Herrenschmidt <[email protected]>
CC Paul Mackerras <[email protected]>
CC Christoph Lameter <[email protected]>
Cc Minchan Kim <[email protected]>
CC Andrew Morton <[email protected]>
CC KOSAKI Motohiro <[email protected]>
CC Wen Congyang <[email protected]>
---
 mm/memory_hotplug.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
mm/memory_hotplug.c +1 -1
Index: linux-3.5-rc6/mm/memory_hotplug.c
===================================================================
@@ -642,7 +642,7 @@ int __ref add_memory(int nid, u64 start,
 	}
 
 	/* create new memmap entry */
-	firmware_map_add_hotplug(start, start + size, "System RAM");
+	firmware_map_add_hotplug(start, start + size - 1, "System RAM");
 
 	goto out;
 
#17 lore raw

Re: [RFC PATCH v3 3/13] memory-hotplug : unify argument of firmware_map_add_early/hotplug

mm/memory_hotplug.c int __ref add_memory(int nid, u64 start, · line 645

Quoted 6 lines · Yasuaki Ishimatsu
mm/memory_hotplug.c +1 -1
@@ -642,7 +642,7 @@ int __ref add_memory(int nid, u64 start,
 	}
 
 	/* create new memmap entry */
-	firmware_map_add_hotplug(start, start + size, "System RAM");
+	firmware_map_add_hotplug(start, start + size - 1, "System RAM");

I know the firmware_map_*() calls use inclusive end addresses internally, but do we really need to expose them? Both of the callers you mentioned do:

	firmware_map_add_hotplug(start, start + size - 1, "System RAM");

or

                firmware_map_add_early(entry->addr,
                        entry->addr + entry->size - 1,
                        e820_type_to_string(entry->type));

So it seems a _bit_ silly to keep all of the callers doing this size-1 thing. I also noted that the new caller that you added does the same thing. Could we just change the external calling convention to be exclusive?

BTW, this patch should probably be first in your series. It's a real bugfix.

#18 lore raw

Re: [RFC PATCH v3 3/13] memory-hotplug : unify argument of firmware_map_add_early/hotplug

Hi Dave,

Quoted 16 lines · Dave Hansen

Review int __ref add_memory(int nid, u64 start, · line 645

Quoted 6 lines · Yasuaki Ishimatsu
(quoted diff) +1 -1
@@ -642,7 +642,7 @@ int __ref add_memory(int nid, u64 start,
  	}
 
  	/* create new memmap entry */
-	firmware_map_add_hotplug(start, start + size, "System RAM");
+	firmware_map_add_hotplug(start, start + size - 1, "System RAM");

I know the firmware_map_*() calls use inclusive end addresses internally, but do we really need to expose them? Both of the callers you mentioned do:

	firmware_map_add_hotplug(start, start + size - 1, "System RAM");

or

Quoted 8 lines
                 firmware_map_add_early(entry->addr,
                         entry->addr + entry->size - 1,
                         e820_type_to_string(entry->type));

So it seems a _bit_ silly to keep all of the callers doing this size-1 thing. I also noted that the new caller that you added does the same thing. Could we just change the external calling convention to be exclusive?

Thank you for your comment.

Does the following patch include your comment? If O.K., I will separate the patch from the series and send it for bug fix.

---
 arch/x86/kernel/e820.c    |    2 +-
 drivers/firmware/memmap.c |    8 ++++----
 2 files changed, 5 insertions(+), 5 deletions(-)
arch/x86/kernel/e820.c +1 -1
Index: linux-next/arch/x86/kernel/e820.c
===================================================================
@@ -944,7 +944,7 @@
 	for (i = 0; i < e820_saved.nr_map; i++) {
 		struct e820entry *entry = &e820_saved.map[i];
 		firmware_map_add_early(entry->addr,
-			entry->addr + entry->size - 1,
+			entry->addr + entry->size,
 			e820_type_to_string(entry->type));
 	}
 }
drivers/firmware/memmap.c +4 -4
Index: linux-next/drivers/firmware/memmap.c
===================================================================
@@ -98,7 +98,7 @@
 /**
  * firmware_map_add_entry() - Does the real work to add a firmware memmap entry.
  * @start: Start of the memory range.
- * @end:   End of the memory range (inclusive).
+ * @end:   End of the memory range.
  * @type:  Type of the memory range.
  * @entry: Pre-allocated (either kmalloc() or bootmem allocator), uninitialised
  *         entry.
@@ -113,7 +113,7 @@
 	BUG_ON(start > end);
 
 	entry->start = start;
-	entry->end = end;
+	entry->end = end - 1;
 	entry->type = type;
 	INIT_LIST_HEAD(&entry->list);
 	kobject_init(&entry->kobj, &memmap_ktype);
@@ -148,7 +148,7 @@
  * firmware_map_add_hotplug() - Adds a firmware mapping entry when we do
  * memory hotplug.
  * @start: Start of the memory range.
- * @end:   End of the memory range (inclusive).
+ * @end:   End of the memory range.
  * @type:  Type of the memory range.
  *
  * Adds a firmware mapping entry. This function is for memory hotplug, it is
@@ -175,7 +175,7 @@
 /**
  * firmware_map_add_early() - Adds a firmware mapping entry.
  * @start: Start of the memory range.
- * @end:   End of the memory range (inclusive).
+ * @end:   End of the memory range.
  * @type:  Type of the memory range.
  *
  * Adds a firmware mapping entry. This function uses the bootmem allocator
#19 lore raw

Re: [RFC PATCH v3 3/13] memory-hotplug : unify argument of firmware_map_add_early/hotplug

Quoted 2 lines · Yasuaki Ishimatsu

Does the following patch include your comment? If O.K., I will separate the patch from the series and send it for bug fix.

Looks sane to me. It does now mean that the calling conventions for some of the other firmware_map*() functions are different, but I think that's OK since they're only used internally to memmap.c.

#20 lore raw

Re: [RFC PATCH v3 3/13] memory-hotplug : unify argument of firmware_map_add_early/hotplug

Hi Dave,

Quoted 7 lines · Dave Hansen
Quoted 2 lines · Yasuaki Ishimatsu

Does the following patch include your comment? If O.K., I will separate the patch from the series and send it for bug fix.

Looks sane to me. It does now mean that the calling conventions for some of the other firmware_map*() functions are different, but I think that's OK since they're only used internally to memmap.c.

Thank you for reviewing my patch.
I'll send the patch.

Thanks,
Yasuaki Ishimatsu

Quoted 6 lines

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#21 lore raw

Re: [RFC PATCH v3 3/13] memory-hotplug : unify argument of firmware_map_add_early/hotplug

In reply to Dave Hansen

Hi Dave,

Quoted 7 lines · Dave Hansen
Quoted 2 lines · Yasuaki Ishimatsu

Does the following patch include your comment? If O.K., I will separate the patch from the series and send it for bug fix.

Looks sane to me. It does now mean that the calling conventions for some of the other firmware_map*() functions are different, but I think that's OK since they're only used internally to memmap.c.

Can I add "Reviewed-by: Dave Hansen" to the patch?

Thanks,
Yasuaki Ishimatsu

Quoted 6 lines

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#22 lore raw

RFC PATCH v3 4/13 memory-hotplug : remove /sys/firmware/memmap/X sysfs

In reply to Yasuaki Ishimatsu

When (hot)adding memory into system, /sys/firmware/memmap/X/{end, start, type} sysfs files are created. But there is no code to remove these files. The patch implements the function to remove them.

Note : The code does not free firmware_map_entry since there is no way to free
memory which is allocated by bootmem.

CC David Rientjes <[email protected]>
CC Jiang Liu <[email protected]>
CC Len Brown <[email protected]>
CC Benjamin Herrenschmidt <[email protected]>
CC Paul Mackerras <[email protected]>
CC Christoph Lameter <[email protected]>
Cc Minchan Kim <[email protected]>
CC Andrew Morton <[email protected]>
CC KOSAKI Motohiro <[email protected]>
CC Wen Congyang <[email protected]>
---
 drivers/firmware/memmap.c    |   78 ++++++++++++++++++++++++++++++++++++++++++-
 include/linux/firmware-map.h |    6 +++
 mm/memory_hotplug.c          |    6 ++-
 3 files changed, 88 insertions(+), 2 deletions(-)
mm/memory_hotplug.c +5 -1
Index: linux-3.5-rc6/mm/memory_hotplug.c
===================================================================
@@ -661,7 +661,11 @@ EXPORT_SYMBOL_GPL(add_memory);
 
 int remove_memory(int nid, u64 start, u64 size)
 {
-	return -EBUSY;
+	lock_memory_hotplug();
+	/* remove memmap entry */
+	firmware_map_remove(start, start + size - 1, "System RAM");
+	unlock_memory_hotplug();
+	return 0;
 
 }
 EXPORT_SYMBOL_GPL(remove_memory);
include/linux/firmware-map.h +6 -0
Index: linux-3.5-rc6/include/linux/firmware-map.h
===================================================================
@@ -25,6 +25,7 @@
 
 int firmware_map_add_early(u64 start, u64 end, const char *type);
 int firmware_map_add_hotplug(u64 start, u64 end, const char *type);
+int firmware_map_remove(u64 start, u64 end, const char *type);
 
 #else /* CONFIG_FIRMWARE_MEMMAP */
 
@@ -38,6 +39,11 @@ static inline int firmware_map_add_hotpl
 	return 0;
 }
 
+static inline int firmware_map_remove(u64 start, u64 end, const char *type)
+{
+	return 0;
+}
+
 #endif /* CONFIG_FIRMWARE_MEMMAP */
 
 #endif /* _LINUX_FIRMWARE_MAP_H */
drivers/firmware/memmap.c +77 -1

115 lines not on this page · raw message

#23 lore raw

Re: [RFC PATCH v3 4/13] memory-hotplug : remove /sys/firmware/memmap/X sysfs

At 07/09/2012 06:26 PM, Yasuaki Ishimatsu Wrote:

Quoted 30 lines

When (hot)adding memory into system, /sys/firmware/memmap/X/{end, start, type} sysfs files are created. But there is no code to remove these files. The patch implements the function to remove them.

Note : The code does not free firmware_map_entry since there is no way to free
memory which is allocated by bootmem.

CC David Rientjes <[email protected]>
CC Jiang Liu <[email protected]>
CC Len Brown <[email protected]>
CC Benjamin Herrenschmidt <[email protected]>
CC Paul Mackerras <[email protected]>
CC Christoph Lameter <[email protected]>
Cc Minchan Kim <[email protected]>
CC Andrew Morton <[email protected]>
CC KOSAKI Motohiro <[email protected]>
CC Wen Congyang <[email protected]>
---
 drivers/firmware/memmap.c    |   78 ++++++++++++++++++++++++++++++++++++++++++-
 include/linux/firmware-map.h |    6 +++
 mm/memory_hotplug.c          |    6 ++-
 3 files changed, 88 insertions(+), 2 deletions(-)

Index: linux-3.5-rc6/mm/memory_hotplug.c
===================================================================

mm/memory_hotplug.c EXPORT_SYMBOL_GPL(add_memory); · line 666

Quoted 7 lines
mm/memory_hotplug.c +3 -1
@@ -661,7 +661,11 @@ EXPORT_SYMBOL_GPL(add_memory);
 
 int remove_memory(int nid, u64 start, u64 size)
 {
-	return -EBUSY;
+	lock_memory_hotplug();
+	/* remove memmap entry */
+	firmware_map_remove(start, start + size - 1, "System RAM");

firmware_map_remove() is in meminit section, so remove_memory() should be in ref section.

Thanks
Wen Congyang

Quoted 149 lines
mm/memory_hotplug.c +2 -0
+	unlock_memory_hotplug();
+	return 0;
 
 }
 EXPORT_SYMBOL_GPL(remove_memory);

Index: linux-3.5-rc6/include/linux/firmware-map.h
===================================================================

include/linux/firmware-map.h +6 -0
@@ -25,6 +25,7 @@
 
 int firmware_map_add_early(u64 start, u64 end, const char *type);
 int firmware_map_add_hotplug(u64 start, u64 end, const char *type);
+int firmware_map_remove(u64 start, u64 end, const char *type);
 
 #else /* CONFIG_FIRMWARE_MEMMAP */
 
@@ -38,6 +39,11 @@ static inline int firmware_map_add_hotpl
 	return 0;
 }
 
+static inline int firmware_map_remove(u64 start, u64 end, const char *type)
+{
+	return 0;
+}
+
 #endif /* CONFIG_FIRMWARE_MEMMAP */
 
 #endif /* _LINUX_FIRMWARE_MAP_H */

Index: linux-3.5-rc6/drivers/firmware/memmap.c
===================================================================

drivers/firmware/memmap.c +77 -0
@@ -21,6 +21,7 @@
 #include <linux/types.h>
 #include <linux/bootmem.h>
 #include <linux/slab.h>
+#include <linux/mm.h>
 
 /*
  * Data types ------------------------------------------------------------------
@@ -79,7 +80,22 @@ static const struct sysfs_ops memmap_att
 	.show = memmap_attr_show,
 };
 
+#define to_memmap_entry(obj) container_of(obj, struct firmware_map_entry, kobj)
+
+static void release_firmware_map_entry(struct kobject *kobj)
+{
+	struct firmware_map_entry *entry = to_memmap_entry(kobj);
+	struct page *head_page;
+
+	head_page = virt_to_head_page(entry);
+	if (PageSlab(head_page))
+		kfree(entry);
+
+	/* There is no way to free memory allocated from bootmem*/
+}
+
 static struct kobj_type memmap_ktype = {
+	.release	= release_firmware_map_entry,
 	.sysfs_ops	= &memmap_attr_ops,
 	.default_attrs	= def_attrs,
 };
@@ -123,6 +139,16 @@ static int firmware_map_add_entry(u64 st
 	return 0;
 }
 
+/**
+ * firmware_map_remove_entry() - Does the real work to remove a firmware
+ * memmap entry.
+ * @entry: removed entry.
+ **/
+static inline void firmware_map_remove_entry(struct firmware_map_entry *entry)
+{
+	list_del(&entry->list);
+}
+
 /*
  * Add memmap entry on sysfs
  */
@@ -144,6 +170,31 @@ static int add_sysfs_fw_map_entry(struct
 	return 0;
 }
 
+/*
+ * Remove memmap entry on sysfs
+ */
+static inline void remove_sysfs_fw_map_entry(struct firmware_map_entry *entry)
+{
+	kobject_put(&entry->kobj);
+}
+
+/*
+ * Search memmap entry
+ */
+
+struct firmware_map_entry * __meminit
+find_firmware_map_entry(u64 start, u64 end, const char *type)
+{
+	struct firmware_map_entry *entry;
+
+	list_for_each_entry(entry, &map_entries, list)
+		if ((entry->start == start) && (entry->end == end) &&
+		    (!strcmp(entry->type, type)))
+			return entry;
+
+	return NULL;
+}
+
 /**
  * firmware_map_add_hotplug() - Adds a firmware mapping entry when we do
  * memory hotplug.
@@ -196,6 +247,32 @@ int __init firmware_map_add_early(u64 st
 	return firmware_map_add_entry(start, end, type, entry);
 }
 
+/**
+ * firmware_map_remove() - remove a firmware mapping entry
+ * @start: Start of the memory range.
+ * @end:   End of the memory range (inclusive).
+ * @type:  Type of the memory range.
+ *
+ * removes a firmware mapping entry.
+ *
+ * Returns 0 on success, or -EINVAL if no entry.
+ **/
+int __meminit firmware_map_remove(u64 start, u64 end, const char *type)
+{
+	struct firmware_map_entry *entry;
+
+	entry = find_firmware_map_entry(start, end, type);
+	if (!entry)
+		return -EINVAL;
+
+	/* remove the memmap entry */
+	remove_sysfs_fw_map_entry(entry);
+
+	firmware_map_remove_entry(entry);
+
+	return 0;
+}
+
 /*
  * Sysfs functions -------------------------------------------------------------
  */

drivers/firmware/memmap.c static ssize_t type_show(struct firmware · line 300

Quoted 7 lines
drivers/firmware/memmap.c +0 -1
@@ -218,7 +295,6 @@ static ssize_t type_show(struct firmware
 }
 
 #define to_memmap_attr(_attr) container_of(_attr, struct memmap_attribute, attr)
-#define to_memmap_entry(obj) container_of(obj, struct firmware_map_entry, kobj)
 
 static ssize_t memmap_attr_show(struct kobject *kobj,
 				struct attribute *attr, char *buf)

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#24 lore raw

Re: [RFC PATCH v3 4/13] memory-hotplug : remove /sys/firmware/memmap/X sysfs

Hi Wen,

Quoted 32 lines · Wen Congyang

At 07/09/2012 06:26 PM, Yasuaki Ishimatsu Wrote:

Quoted 31 lines

When (hot)adding memory into system, /sys/firmware/memmap/X/{end, start, type} sysfs files are created. But there is no code to remove these files. The patch implements the function to remove them.

Note : The code does not free firmware_map_entry since there is no way to free
memory which is allocated by bootmem.

CC David Rientjes <[email protected]>
CC Jiang Liu <[email protected]>
CC Len Brown <[email protected]>
CC Benjamin Herrenschmidt <[email protected]>
CC Paul Mackerras <[email protected]>
CC Christoph Lameter <[email protected]>
Cc Minchan Kim <[email protected]>
CC Andrew Morton <[email protected]>
CC KOSAKI Motohiro <[email protected]>
CC Wen Congyang <[email protected]>
---
  drivers/firmware/memmap.c    |   78 ++++++++++++++++++++++++++++++++++++++++++-
  include/linux/firmware-map.h |    6 +++
  mm/memory_hotplug.c          |    6 ++-
  3 files changed, 88 insertions(+), 2 deletions(-)

Index: linux-3.5-rc6/mm/memory_hotplug.c
===================================================================

mm/memory_hotplug.c +0 -0
@@ -661,7 +661,11 @@ EXPORT_SYMBOL_GPL(add_memory);
 
Quoted 9 lines

mm/memory_hotplug.c EXPORT_SYMBOL_GPL(add_memory); · line 666

Quoted 6 lines
mm/memory_hotplug.c +3 -1
  int remove_memory(int nid, u64 start, u64 size)
  {
-	return -EBUSY;
+	lock_memory_hotplug();
+	/* remove memmap entry */
+	firmware_map_remove(start, start + size - 1, "System RAM");

firmware_map_remove() is in meminit section, so remove_memory() should be in ref section.

I'll add it.

Thanks,
Yasuaki Ishimatsu

Quoted 156 lines

Thanks
Wen Congyang

Quoted 152 lines
mm/memory_hotplug.c +2 -0
+	unlock_memory_hotplug();
+	return 0;
 
  }
  EXPORT_SYMBOL_GPL(remove_memory);

Index: linux-3.5-rc6/include/linux/firmware-map.h
===================================================================

include/linux/firmware-map.h +6 -0
@@ -25,6 +25,7 @@
 
  int firmware_map_add_early(u64 start, u64 end, const char *type);
  int firmware_map_add_hotplug(u64 start, u64 end, const char *type);
+int firmware_map_remove(u64 start, u64 end, const char *type);
 
  #else /* CONFIG_FIRMWARE_MEMMAP */
 
@@ -38,6 +39,11 @@ static inline int firmware_map_add_hotpl
  	return 0;
  }
 
+static inline int firmware_map_remove(u64 start, u64 end, const char *type)
+{
+	return 0;
+}
+
  #endif /* CONFIG_FIRMWARE_MEMMAP */
 
  #endif /* _LINUX_FIRMWARE_MAP_H */

Index: linux-3.5-rc6/drivers/firmware/memmap.c
===================================================================

drivers/firmware/memmap.c +77 -1
@@ -21,6 +21,7 @@
  #include <linux/types.h>
  #include <linux/bootmem.h>
  #include <linux/slab.h>
+#include <linux/mm.h>
 
  /*
   * Data types ------------------------------------------------------------------
@@ -79,7 +80,22 @@ static const struct sysfs_ops memmap_att
  	.show = memmap_attr_show,
  };
 
+#define to_memmap_entry(obj) container_of(obj, struct firmware_map_entry, kobj)
+
+static void release_firmware_map_entry(struct kobject *kobj)
+{
+	struct firmware_map_entry *entry = to_memmap_entry(kobj);
+	struct page *head_page;
+
+	head_page = virt_to_head_page(entry);
+	if (PageSlab(head_page))
+		kfree(entry);
+
+	/* There is no way to free memory allocated from bootmem*/
+}
+
  static struct kobj_type memmap_ktype = {
+	.release	= release_firmware_map_entry,
  	.sysfs_ops	= &memmap_attr_ops,
  	.default_attrs	= def_attrs,
  };
@@ -123,6 +139,16 @@ static int firmware_map_add_entry(u64 st
  	return 0;
  }
 
+/**
+ * firmware_map_remove_entry() - Does the real work to remove a firmware
+ * memmap entry.
+ * @entry: removed entry.
+ **/
+static inline void firmware_map_remove_entry(struct firmware_map_entry *entry)
+{
+	list_del(&entry->list);
+}
+
  /*
   * Add memmap entry on sysfs
   */
@@ -144,6 +170,31 @@ static int add_sysfs_fw_map_entry(struct
  	return 0;
  }
 
+/*
+ * Remove memmap entry on sysfs
+ */
+static inline void remove_sysfs_fw_map_entry(struct firmware_map_entry *entry)
+{
+	kobject_put(&entry->kobj);
+}
+
+/*
+ * Search memmap entry
+ */
+
+struct firmware_map_entry * __meminit
+find_firmware_map_entry(u64 start, u64 end, const char *type)
+{
+	struct firmware_map_entry *entry;
+
+	list_for_each_entry(entry, &map_entries, list)
+		if ((entry->start == start) && (entry->end == end) &&
+		    (!strcmp(entry->type, type)))
+			return entry;
+
+	return NULL;
+}
+
  /**
   * firmware_map_add_hotplug() - Adds a firmware mapping entry when we do
   * memory hotplug.
@@ -196,6 +247,32 @@ int __init firmware_map_add_early(u64 st
  	return firmware_map_add_entry(start, end, type, entry);
  }
 
+/**
+ * firmware_map_remove() - remove a firmware mapping entry
+ * @start: Start of the memory range.
+ * @end:   End of the memory range (inclusive).
+ * @type:  Type of the memory range.
+ *
+ * removes a firmware mapping entry.
+ *
+ * Returns 0 on success, or -EINVAL if no entry.
+ **/
+int __meminit firmware_map_remove(u64 start, u64 end, const char *type)
+{
+	struct firmware_map_entry *entry;
+
+	entry = find_firmware_map_entry(start, end, type);
+	if (!entry)
+		return -EINVAL;
+
+	/* remove the memmap entry */
+	remove_sysfs_fw_map_entry(entry);
+
+	firmware_map_remove_entry(entry);
+
+	return 0;
+}
+
  /*
   * Sysfs functions -------------------------------------------------------------
   */
@@ -218,7 +295,6 @@ static ssize_t type_show(struct firmware
  }
 
  #define to_memmap_attr(_attr) container_of(_attr, struct memmap_attribute, attr)
-#define to_memmap_entry(obj) container_of(obj, struct firmware_map_entry, kobj)
 
Quoted 11 lines

drivers/firmware/memmap.c static ssize_t type_show(struct firmware · line 300

Quoted 4 lines
drivers/firmware/memmap.c +0 -0
  static ssize_t memmap_attr_show(struct kobject *kobj,
  				struct attribute *attr, char *buf)

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#25 lore raw

Re: [RFC PATCH v3 4/13] memory-hotplug : remove /sys/firmware/memmap/X sysfs

In reply to Yasuaki Ishimatsu

At 07/09/2012 06:26 PM, Yasuaki Ishimatsu Wrote:

Quoted 166 lines

When (hot)adding memory into system, /sys/firmware/memmap/X/{end, start, type} sysfs files are created. But there is no code to remove these files. The patch implements the function to remove them.

Note : The code does not free firmware_map_entry since there is no way to free
memory which is allocated by bootmem.

CC David Rientjes <[email protected]>
CC Jiang Liu <[email protected]>
CC Len Brown <[email protected]>
CC Benjamin Herrenschmidt <[email protected]>
CC Paul Mackerras <[email protected]>
CC Christoph Lameter <[email protected]>
Cc Minchan Kim <[email protected]>
CC Andrew Morton <[email protected]>
CC KOSAKI Motohiro <[email protected]>
CC Wen Congyang <[email protected]>
---
 drivers/firmware/memmap.c    |   78 ++++++++++++++++++++++++++++++++++++++++++-
 include/linux/firmware-map.h |    6 +++
 mm/memory_hotplug.c          |    6 ++-
 3 files changed, 88 insertions(+), 2 deletions(-)

Index: linux-3.5-rc6/mm/memory_hotplug.c
===================================================================

mm/memory_hotplug.c +5 -1
@@ -661,7 +661,11 @@ EXPORT_SYMBOL_GPL(add_memory);
 
 int remove_memory(int nid, u64 start, u64 size)
 {
-	return -EBUSY;
+	lock_memory_hotplug();
+	/* remove memmap entry */
+	firmware_map_remove(start, start + size - 1, "System RAM");
+	unlock_memory_hotplug();
+	return 0;
 
 }
 EXPORT_SYMBOL_GPL(remove_memory);

Index: linux-3.5-rc6/include/linux/firmware-map.h
===================================================================

include/linux/firmware-map.h +6 -0
@@ -25,6 +25,7 @@
 
 int firmware_map_add_early(u64 start, u64 end, const char *type);
 int firmware_map_add_hotplug(u64 start, u64 end, const char *type);
+int firmware_map_remove(u64 start, u64 end, const char *type);
 
 #else /* CONFIG_FIRMWARE_MEMMAP */
 
@@ -38,6 +39,11 @@ static inline int firmware_map_add_hotpl
 	return 0;
 }
 
+static inline int firmware_map_remove(u64 start, u64 end, const char *type)
+{
+	return 0;
+}
+
 #endif /* CONFIG_FIRMWARE_MEMMAP */
 
 #endif /* _LINUX_FIRMWARE_MAP_H */

Index: linux-3.5-rc6/drivers/firmware/memmap.c
===================================================================

drivers/firmware/memmap.c +63 -0
@@ -21,6 +21,7 @@
 #include <linux/types.h>
 #include <linux/bootmem.h>
 #include <linux/slab.h>
+#include <linux/mm.h>
 
 /*
  * Data types ------------------------------------------------------------------
@@ -79,7 +80,22 @@ static const struct sysfs_ops memmap_att
 	.show = memmap_attr_show,
 };
 
+#define to_memmap_entry(obj) container_of(obj, struct firmware_map_entry, kobj)
+
+static void release_firmware_map_entry(struct kobject *kobj)
+{
+	struct firmware_map_entry *entry = to_memmap_entry(kobj);
+	struct page *head_page;
+
+	head_page = virt_to_head_page(entry);
+	if (PageSlab(head_page))
+		kfree(entry);
+
+	/* There is no way to free memory allocated from bootmem*/
+}
+
 static struct kobj_type memmap_ktype = {
+	.release	= release_firmware_map_entry,
 	.sysfs_ops	= &memmap_attr_ops,
 	.default_attrs	= def_attrs,
 };
@@ -123,6 +139,16 @@ static int firmware_map_add_entry(u64 st
 	return 0;
 }
 
+/**
+ * firmware_map_remove_entry() - Does the real work to remove a firmware
+ * memmap entry.
+ * @entry: removed entry.
+ **/
+static inline void firmware_map_remove_entry(struct firmware_map_entry *entry)
+{
+	list_del(&entry->list);
+}
+
 /*
  * Add memmap entry on sysfs
  */
@@ -144,6 +170,31 @@ static int add_sysfs_fw_map_entry(struct
 	return 0;
 }
 
+/*
+ * Remove memmap entry on sysfs
+ */
+static inline void remove_sysfs_fw_map_entry(struct firmware_map_entry *entry)
+{
+	kobject_put(&entry->kobj);
+}
+
+/*
+ * Search memmap entry
+ */
+
+struct firmware_map_entry * __meminit
+find_firmware_map_entry(u64 start, u64 end, const char *type)
+{
+	struct firmware_map_entry *entry;
+
+	list_for_each_entry(entry, &map_entries, list)
+		if ((entry->start == start) && (entry->end == end) &&
+		    (!strcmp(entry->type, type)))
+			return entry;
+
+	return NULL;
+}
+
 /**
  * firmware_map_add_hotplug() - Adds a firmware mapping entry when we do
  * memory hotplug.
@@ -196,6 +247,32 @@ int __init firmware_map_add_early(u64 st
 	return firmware_map_add_entry(start, end, type, entry);
 }
 
+/**
+ * firmware_map_remove() - remove a firmware mapping entry
+ * @start: Start of the memory range.
+ * @end:   End of the memory range (inclusive).
+ * @type:  Type of the memory range.
+ *
+ * removes a firmware mapping entry.
+ *
+ * Returns 0 on success, or -EINVAL if no entry.
+ **/
+int __meminit firmware_map_remove(u64 start, u64 end, const char *type)
+{

drivers/firmware/memmap.c int __init firmware_map_add_early(u64 st · line 271

Quoted 10 lines
drivers/firmware/memmap.c +10 -0
+	struct firmware_map_entry *entry;
+
+	entry = find_firmware_map_entry(start, end, type);
+	if (!entry)
+		return -EINVAL;
+
+	/* remove the memmap entry */
+	remove_sysfs_fw_map_entry(entry);
+
+	firmware_map_remove_entry(entry);

You should call firmware_map_remove_entry() before remove_sysfs_fw_map_entry(), otherwise it will cause kernel panicked(entry may be freed in remove_sysfs_fw_map_entry()).

Thanks
Wen Congyang

Quoted 10 lines
drivers/firmware/memmap.c +4 -0
+
+	return 0;
+}
+
 /*
  * Sysfs functions -------------------------------------------------------------
  */

drivers/firmware/memmap.c static ssize_t type_show(struct firmware · line 300

Quoted 7 lines
drivers/firmware/memmap.c +0 -1
@@ -218,7 +295,6 @@ static ssize_t type_show(struct firmware
 }
 
 #define to_memmap_attr(_attr) container_of(_attr, struct memmap_attribute, attr)
-#define to_memmap_entry(obj) container_of(obj, struct firmware_map_entry, kobj)
 
 static ssize_t memmap_attr_show(struct kobject *kobj,
 				struct attribute *attr, char *buf)

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#26 lore raw

Re: [RFC PATCH v3 4/13] memory-hotplug : remove /sys/firmware/memmap/X sysfs

Hi Wen,

Quoted 172 lines · Wen Congyang

At 07/09/2012 06:26 PM, Yasuaki Ishimatsu Wrote:

Quoted 171 lines

When (hot)adding memory into system, /sys/firmware/memmap/X/{end, start, type} sysfs files are created. But there is no code to remove these files. The patch implements the function to remove them.

Note : The code does not free firmware_map_entry since there is no way to free
memory which is allocated by bootmem.

CC David Rientjes <[email protected]>
CC Jiang Liu <[email protected]>
CC Len Brown <[email protected]>
CC Benjamin Herrenschmidt <[email protected]>
CC Paul Mackerras <[email protected]>
CC Christoph Lameter <[email protected]>
Cc Minchan Kim <[email protected]>
CC Andrew Morton <[email protected]>
CC KOSAKI Motohiro <[email protected]>
CC Wen Congyang <[email protected]>
---
  drivers/firmware/memmap.c    |   78 ++++++++++++++++++++++++++++++++++++++++++-
  include/linux/firmware-map.h |    6 +++
  mm/memory_hotplug.c          |    6 ++-
  3 files changed, 88 insertions(+), 2 deletions(-)

Index: linux-3.5-rc6/mm/memory_hotplug.c
===================================================================

mm/memory_hotplug.c +5 -1
@@ -661,7 +661,11 @@ EXPORT_SYMBOL_GPL(add_memory);
 
  int remove_memory(int nid, u64 start, u64 size)
  {
-	return -EBUSY;
+	lock_memory_hotplug();
+	/* remove memmap entry */
+	firmware_map_remove(start, start + size - 1, "System RAM");
+	unlock_memory_hotplug();
+	return 0;
 
  }
  EXPORT_SYMBOL_GPL(remove_memory);

Index: linux-3.5-rc6/include/linux/firmware-map.h
===================================================================

include/linux/firmware-map.h +6 -0
@@ -25,6 +25,7 @@
 
  int firmware_map_add_early(u64 start, u64 end, const char *type);
  int firmware_map_add_hotplug(u64 start, u64 end, const char *type);
+int firmware_map_remove(u64 start, u64 end, const char *type);
 
  #else /* CONFIG_FIRMWARE_MEMMAP */
 
@@ -38,6 +39,11 @@ static inline int firmware_map_add_hotpl
  	return 0;
  }
 
+static inline int firmware_map_remove(u64 start, u64 end, const char *type)
+{
+	return 0;
+}
+
  #endif /* CONFIG_FIRMWARE_MEMMAP */
 
  #endif /* _LINUX_FIRMWARE_MAP_H */

Index: linux-3.5-rc6/drivers/firmware/memmap.c
===================================================================

drivers/firmware/memmap.c +68 -0
@@ -21,6 +21,7 @@
  #include <linux/types.h>
  #include <linux/bootmem.h>
  #include <linux/slab.h>
+#include <linux/mm.h>
 
  /*
   * Data types ------------------------------------------------------------------
@@ -79,7 +80,22 @@ static const struct sysfs_ops memmap_att
  	.show = memmap_attr_show,
  };
 
+#define to_memmap_entry(obj) container_of(obj, struct firmware_map_entry, kobj)
+
+static void release_firmware_map_entry(struct kobject *kobj)
+{
+	struct firmware_map_entry *entry = to_memmap_entry(kobj);
+	struct page *head_page;
+
+	head_page = virt_to_head_page(entry);
+	if (PageSlab(head_page))
+		kfree(entry);
+
+	/* There is no way to free memory allocated from bootmem*/
+}
+
  static struct kobj_type memmap_ktype = {
+	.release	= release_firmware_map_entry,
  	.sysfs_ops	= &memmap_attr_ops,
  	.default_attrs	= def_attrs,
  };
@@ -123,6 +139,16 @@ static int firmware_map_add_entry(u64 st
  	return 0;
  }
 
+/**
+ * firmware_map_remove_entry() - Does the real work to remove a firmware
+ * memmap entry.
+ * @entry: removed entry.
+ **/
+static inline void firmware_map_remove_entry(struct firmware_map_entry *entry)
+{
+	list_del(&entry->list);
+}
+
  /*
   * Add memmap entry on sysfs
   */
@@ -144,6 +170,31 @@ static int add_sysfs_fw_map_entry(struct
  	return 0;
  }
 
+/*
+ * Remove memmap entry on sysfs
+ */
+static inline void remove_sysfs_fw_map_entry(struct firmware_map_entry *entry)
+{
+	kobject_put(&entry->kobj);
+}
+
+/*
+ * Search memmap entry
+ */
+
+struct firmware_map_entry * __meminit
+find_firmware_map_entry(u64 start, u64 end, const char *type)
+{
+	struct firmware_map_entry *entry;
+
+	list_for_each_entry(entry, &map_entries, list)
+		if ((entry->start == start) && (entry->end == end) &&
+		    (!strcmp(entry->type, type)))
+			return entry;
+
+	return NULL;
+}
+
  /**
   * firmware_map_add_hotplug() - Adds a firmware mapping entry when we do
   * memory hotplug.
@@ -196,6 +247,32 @@ int __init firmware_map_add_early(u64 st
  	return firmware_map_add_entry(start, end, type, entry);
  }
 
+/**
+ * firmware_map_remove() - remove a firmware mapping entry
+ * @start: Start of the memory range.
+ * @end:   End of the memory range (inclusive).
+ * @type:  Type of the memory range.
+ *
+ * removes a firmware mapping entry.
+ *
+ * Returns 0 on success, or -EINVAL if no entry.
+ **/
+int __meminit firmware_map_remove(u64 start, u64 end, const char *type)
+{
+	struct firmware_map_entry *entry;
+
+	entry = find_firmware_map_entry(start, end, type);
+	if (!entry)
+		return -EINVAL;
Quoted 8 lines

drivers/firmware/memmap.c int __init firmware_map_add_early(u64 st · line 271

Quoted 5 lines
drivers/firmware/memmap.c +5 -0
+
+	/* remove the memmap entry */
+	remove_sysfs_fw_map_entry(entry);
+
+	firmware_map_remove_entry(entry);

You should call firmware_map_remove_entry() before remove_sysfs_fw_map_entry(), otherwise it will cause kernel panicked(entry may be freed in remove_sysfs_fw_map_entry()).

You are right. I'll update it.

Thanks,
Yasuaki Ishimatsu

Quoted 28 lines

Thanks
Wen Congyang

Quoted 10 lines
drivers/firmware/memmap.c +4 -0
+
+	return 0;
+}
+
  /*
   * Sysfs functions -------------------------------------------------------------
   */

drivers/firmware/memmap.c static ssize_t type_show(struct firmware · line 300

Quoted 7 lines
drivers/firmware/memmap.c +0 -1
@@ -218,7 +295,6 @@ static ssize_t type_show(struct firmware
  }
 
  #define to_memmap_attr(_attr) container_of(_attr, struct memmap_attribute, attr)
-#define to_memmap_entry(obj) container_of(obj, struct firmware_map_entry, kobj)
 
  static ssize_t memmap_attr_show(struct kobject *kobj,
  				struct attribute *attr, char *buf)

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#27 lore raw

RFC PATCH v3 5/13 memory-hotplug : does not release memory region in PAGES_PER_SECTION chunks

In reply to Yasuaki Ishimatsu

Since applying a patch(de7f0cba96786c), release_mem_region() has been changed as called in PAGES_PER_SECTION chunks because register_memory_resource() is called in PAGES_PER_SECTION chunks by add_memory(). But it seems firmware dependency. If CRS are written in the PAGES_PER_SECTION chunks in ACPI DSDT Table, register_memory_resource() is called in PAGES_PER_SECTION chunks. But if CRS are written in the DIMM unit in ACPI DSDT Table, register_memory_resource() is called in DIMM unit. So release_mem_region() should not be called in PAGES_PER_SECTION chunks. The patch fixes it.

CC David Rientjes <[email protected]>
CC Jiang Liu <[email protected]>
CC Len Brown <[email protected]>
CC Benjamin Herrenschmidt <[email protected]>
CC Paul Mackerras <[email protected]>
CC Christoph Lameter <[email protected]>
Cc Minchan Kim <[email protected]>
CC Andrew Morton <[email protected]>
CC KOSAKI Motohiro <[email protected]>
CC Wen Congyang <[email protected]>
---
 arch/powerpc/platforms/pseries/hotplug-memory.c |   13 +++++++++----
 mm/memory_hotplug.c                             |    4 ++--
 2 files changed, 11 insertions(+), 6 deletions(-)
mm/memory_hotplug.c +2 -2
Index: linux-3.5-rc4/mm/memory_hotplug.c
===================================================================
@@ -358,11 +358,11 @@ int __remove_pages(struct zone *zone, un
 	BUG_ON(phys_start_pfn & ~PAGE_SECTION_MASK);
 	BUG_ON(nr_pages % PAGES_PER_SECTION);
 
+	release_mem_region(phys_start_pfn << PAGE_SHIFT,  nr_pages * PAGE_SIZE);
+
 	sections_to_remove = nr_pages / PAGES_PER_SECTION;
 	for (i = 0; i < sections_to_remove; i++) {
 		unsigned long pfn = phys_start_pfn + i*PAGES_PER_SECTION;
-		release_mem_region(pfn << PAGE_SHIFT,
-				   PAGES_PER_SECTION << PAGE_SHIFT);
 		ret = __remove_section(zone, __pfn_to_section(pfn));
 		if (ret)
 			break;

Index: linux-3.5-rc4/arch/powerpc/platforms/pseries/hotplug-memory.c
===================================================================
--- linux-3.5-rc4.orig/arch/powerpc/platforms/pseries/hotplug-memory.c 2012-07-03 14:21:45.641422678
+0900
+++ linux-3.5-rc4/arch/powerpc/platforms/pseries/hotplug-memory.c 2012-07-03 14:22:05.920169437 +0900
@@ -77,7 +77,8 @@ static int pseries_remove_memblock(unsig
{
unsigned long start, start_pfn;
struct zone *zone;
- int ret;
+ int i, ret;
+ int sections_to_remove;

start_pfn = base >> PAGE_SHIFT;

@@ -97,9 +98,13 @@ static int pseries_remove_memblock(unsig
* to sysfs "state" file and we can't remove sysfs entries
* while writing to it. So we have to defer it to here. */
- ret = __remove_pages(zone, start_pfn, memblock_size >> PAGE_SHIFT);
- if (ret)
- return ret;
+ sections_to_remove = (memblock_size >> PAGE_SHIFT) / PAGES_PER_SECTION;
+ for (i = 0; i < sections_to_remove; i++) {
+ unsigned long pfn = start_pfn + i * PAGES_PER_SECTION;
+ ret = __remove_pages(zone, start_pfn, PAGES_PER_SECTION);
+ if (ret)
+ return ret;
+ }

/*
* Update memory regions for memory remove

#28 lore raw

Re: [RFC PATCH v3 5/13] memory-hotplug : does not release memory region in PAGES_PER_SECTION chunks

At 07/09/2012 06:26 PM, Yasuaki Ishimatsu Wrote:

Quoted 44 lines

Since applying a patch(de7f0cba96786c), release_mem_region() has been changed as called in PAGES_PER_SECTION chunks because register_memory_resource() is called in PAGES_PER_SECTION chunks by add_memory(). But it seems firmware dependency. If CRS are written in the PAGES_PER_SECTION chunks in ACPI DSDT Table, register_memory_resource() is called in PAGES_PER_SECTION chunks. But if CRS are written in the DIMM unit in ACPI DSDT Table, register_memory_resource() is called in DIMM unit. So release_mem_region() should not be called in PAGES_PER_SECTION chunks. The patch fixes it.

CC David Rientjes <[email protected]>
CC Jiang Liu <[email protected]>
CC Len Brown <[email protected]>
CC Benjamin Herrenschmidt <[email protected]>
CC Paul Mackerras <[email protected]>
CC Christoph Lameter <[email protected]>
Cc Minchan Kim <[email protected]>
CC Andrew Morton <[email protected]>
CC KOSAKI Motohiro <[email protected]>
CC Wen Congyang <[email protected]>
---
 arch/powerpc/platforms/pseries/hotplug-memory.c |   13 +++++++++----
 mm/memory_hotplug.c                             |    4 ++--
 2 files changed, 11 insertions(+), 6 deletions(-)

Index: linux-3.5-rc4/mm/memory_hotplug.c
===================================================================

mm/memory_hotplug.c +2 -2
@@ -358,11 +358,11 @@ int __remove_pages(struct zone *zone, un
 	BUG_ON(phys_start_pfn & ~PAGE_SECTION_MASK);
 	BUG_ON(nr_pages % PAGES_PER_SECTION);
 
+	release_mem_region(phys_start_pfn << PAGE_SHIFT,  nr_pages * PAGE_SIZE);
+
 	sections_to_remove = nr_pages / PAGES_PER_SECTION;
 	for (i = 0; i < sections_to_remove; i++) {
 		unsigned long pfn = phys_start_pfn + i*PAGES_PER_SECTION;
-		release_mem_region(pfn << PAGE_SHIFT,
-				   PAGES_PER_SECTION << PAGE_SHIFT);
 		ret = __remove_section(zone, __pfn_to_section(pfn));
 		if (ret)
 			break;
Quoted 4 lines

Index: linux-3.5-rc4/arch/powerpc/platforms/pseries/hotplug-memory.c
===================================================================
--- linux-3.5-rc4.orig/arch/powerpc/platforms/pseries/hotplug-memory.c 2012-07-03 14:21:45.641422678
+0900

Hmm, I think you should change your mail client's config.

Thanks
Wen Congyang

Quoted 30 lines

+++ linux-3.5-rc4/arch/powerpc/platforms/pseries/hotplug-memory.c 2012-07-03 14:22:05.920169437 +0900

mm/memory_hotplug.c +9 -5
@@ -77,7 +77,8 @@ static int pseries_remove_memblock(unsig
 {
 	unsigned long start, start_pfn;
 	struct zone *zone;
-	int ret;
+	int i, ret;
+	int sections_to_remove;
 
 	start_pfn = base >> PAGE_SHIFT;
 
@@ -97,9 +98,13 @@ static int pseries_remove_memblock(unsig
 	 * to sysfs "state" file and we can't remove sysfs entries
 	 * while writing to it. So we have to defer it to here.
 	 */
-	ret = __remove_pages(zone, start_pfn, memblock_size >> PAGE_SHIFT);
-	if (ret)
-		return ret;
+	sections_to_remove = (memblock_size >> PAGE_SHIFT) / PAGES_PER_SECTION;
+	for (i = 0; i < sections_to_remove; i++) {
+		unsigned long pfn = start_pfn + i * PAGES_PER_SECTION;
+		ret = __remove_pages(zone, start_pfn,  PAGES_PER_SECTION);
+		if (ret)
+			return ret;
+	}
 
 	/*
 	 * Update memory regions for memory remove
 
--
Quoted 5 lines

To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#29 lore raw

RFC PATCH v3 6/13 memory-hotplug : add memory_block_release

In reply to Yasuaki Ishimatsu

When calling remove_memory_block(), the function shows following message at device_release().

Device 'memory528' does not have a release() function, it is broken and must be fixed.

remove_memory_block() calls kfree(mem). I think it shouled be called from device_release(). So the patch implements memory_block_release()

CC David Rientjes <[email protected]>
CC Jiang Liu <[email protected]>
CC Len Brown <[email protected]>
CC Benjamin Herrenschmidt <[email protected]>
CC Paul Mackerras <[email protected]>
CC Christoph Lameter <[email protected]>
Cc Minchan Kim <[email protected]>
CC Andrew Morton <[email protected]>
CC KOSAKI Motohiro <[email protected]>
CC Wen Congyang <[email protected]>
---
 drivers/base/memory.c |   11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)
drivers/base/memory.c +10 -1
Index: linux-3.5-rc6/drivers/base/memory.c
===================================================================
@@ -109,6 +109,15 @@ bool is_memblk_offline(unsigned long sta
 }
 EXPORT_SYMBOL(is_memblk_offline);
 
+#define to_memory_block(device) container_of(device, struct memory_block, dev)
+
+static void release_memory_block(struct device *dev)
+{
+	struct memory_block *mem = to_memory_block(dev);
+
+	kfree(mem);
+}
+
 /*
  * register_memory - Setup a sysfs device for a memory block
  */
@@ -119,6 +128,7 @@ int register_memory(struct memory_block
 
 	memory->dev.bus = &memory_subsys;
 	memory->dev.id = memory->start_section_nr / sections_per_block;
+	memory->dev.release = release_memory_block;
 
 	error = device_register(&memory->dev);
 	return error;
@@ -669,7 +679,6 @@ int remove_memory_block(unsigned long no
 		mem_remove_simple_file(mem, phys_device);
 		mem_remove_simple_file(mem, removable);
 		unregister_memory(mem);
-		kfree(mem);
 	} else
 		kobject_put(&mem->dev.kobj);
 
#30 lore raw

RFC PATCH v3 7/13 memory-hotplug : remove_memory calls __remove_pages

In reply to Yasuaki Ishimatsu

The patch adds __remove_pages() to remove_memory(). Then the range of phys_start_pfn argument and nr_pages argument in __remove_pagse() may have different zone. So zone argument is removed from __remove_pages() and __remove_pages() caluculates zone in each section.

When CONFIG_SPARSEMEM_VMEMMAP is defined, there is no way to remove a memmap. So __remove_section only calls unregister_memory_section().

CC David Rientjes <[email protected]>
CC Jiang Liu <[email protected]>
CC Len Brown <[email protected]>
CC Benjamin Herrenschmidt <[email protected]>
CC Paul Mackerras <[email protected]>
CC Christoph Lameter <[email protected]>
Cc Minchan Kim <[email protected]>
CC Andrew Morton <[email protected]>
CC KOSAKI Motohiro <[email protected]>
CC Wen Congyang <[email protected]>
---
 arch/powerpc/platforms/pseries/hotplug-memory.c |    5 +----
 include/linux/memory_hotplug.h                  |    3 +--
 mm/memory_hotplug.c                             |   20 +++++++++++++-------
 3 files changed, 15 insertions(+), 13 deletions(-)
mm/memory_hotplug.c +13 -7
Index: linux-3.5-rc4/mm/memory_hotplug.c
===================================================================
@@ -275,11 +275,14 @@ static int __meminit __add_section(int n
 #ifdef CONFIG_SPARSEMEM_VMEMMAP
 static int __remove_section(struct zone *zone, struct mem_section *ms)
 {
-	/*
-	 * XXX: Freeing memmap with vmemmap is not implement yet.
-	 *      This should be removed later.
-	 */
-	return -EBUSY;
+	int ret;
+
+	if (!valid_section(ms))
+		return ret;
+
+	ret = unregister_memory_section(ms);
+
+	return ret;
 }
 #else
 static int __remove_section(struct zone *zone, struct mem_section *ms)
@@ -346,11 +349,11 @@ EXPORT_SYMBOL_GPL(__add_pages);
  * sure that pages are marked reserved and zones are adjust properly by
  * calling offline_pages().
  */
-int __remove_pages(struct zone *zone, unsigned long phys_start_pfn,
-		 unsigned long nr_pages)
+int __remove_pages(unsigned long phys_start_pfn, unsigned long nr_pages)
 {
 	unsigned long i, ret = 0;
 	int sections_to_remove;
+	struct zone *zone;
 
 	/*
 	 * We can only remove entire sections
@@ -363,6 +366,7 @@ int __remove_pages(struct zone *zone, un
 	sections_to_remove = nr_pages / PAGES_PER_SECTION;
 	for (i = 0; i < sections_to_remove; i++) {
 		unsigned long pfn = phys_start_pfn + i*PAGES_PER_SECTION;
+		zone = page_zone(pfn_to_page(pfn));
 		ret = __remove_section(zone, __pfn_to_section(pfn));
 		if (ret)
 			break;
@@ -664,6 +668,8 @@ int remove_memory(int nid, u64 start, u6
 	lock_memory_hotplug();
 	/* remove memmap entry */
 	firmware_map_remove(start, start + size - 1, "System RAM");
+
+	__remove_pages(start >> PAGE_SHIFT, size >> PAGE_SHIFT);
 	unlock_memory_hotplug();
 	return 0;
 
include/linux/memory_hotplug.h +1 -2
Index: linux-3.5-rc4/include/linux/memory_hotplug.h
===================================================================
@@ -89,8 +89,7 @@ extern bool is_pageblock_removable_noloc
 /* reasonably generic interface to expand the physical pages in a zone  */
 extern int __add_pages(int nid, struct zone *zone, unsigned long start_pfn,
 	unsigned long nr_pages);
-extern int __remove_pages(struct zone *zone, unsigned long start_pfn,
-	unsigned long nr_pages);
+extern int __remove_pages(unsigned long start_pfn, unsigned long nr_pages);
 
 #ifdef CONFIG_NUMA
 extern int memory_add_physaddr_to_nid(u64 start);

Index: linux-3.5-rc4/arch/powerpc/platforms/pseries/hotplug-memory.c
===================================================================
--- linux-3.5-rc4.orig/arch/powerpc/platforms/pseries/hotplug-memory.c 2012-07-03 14:22:05.920169437
+0900
+++ linux-3.5-rc4/arch/powerpc/platforms/pseries/hotplug-memory.c 2012-07-03 14:22:10.172116353 +0900
@@ -76,7 +76,6 @@ unsigned long memory_block_size_bytes(vo
static int pseries_remove_memblock(unsigned long base, unsigned int memblock_size) {
unsigned long start, start_pfn;
- struct zone *zone;
int i, ret;
int sections_to_remove;

@@ -87,8 +86,6 @@ static int pseries_remove_memblock(unsig
return 0;
}

- zone = page_zone(pfn_to_page(start_pfn));
-
/*
* Remove section mappings and sysfs entries for the
* section of the memory we are removing.
@@ -101,7 +98,7 @@ static int pseries_remove_memblock(unsig
sections_to_remove = (memblock_size >> PAGE_SHIFT) / PAGES_PER_SECTION; for (i = 0; i < sections_to_remove; i++) {
unsigned long pfn = start_pfn + i * PAGES_PER_SECTION;
- ret = __remove_pages(zone, start_pfn, PAGES_PER_SECTION);
+ ret = __remove_pages(start_pfn, PAGES_PER_SECTION);
if (ret)
return ret;
}

#31 lore raw

RFC PATCH v3 8/13 memory-hotplug : check page type in get_page_bootmem

In reply to Yasuaki Ishimatsu

There is a possibility that get_page_bootmem() is called to the same page many times. So when get_page_bootmem is called to the same page, the function only increments page->_count.

CC David Rientjes <[email protected]>
CC Jiang Liu <[email protected]>
CC Len Brown <[email protected]>
CC Benjamin Herrenschmidt <[email protected]>
CC Paul Mackerras <[email protected]>
CC Christoph Lameter <[email protected]>
Cc Minchan Kim <[email protected]>
CC Andrew Morton <[email protected]>
CC KOSAKI Motohiro <[email protected]>
CC Wen Congyang <[email protected]>
---
 mm/memory_hotplug.c |   15 +++++++++++----
 1 file changed, 11 insertions(+), 4 deletions(-)
mm/memory_hotplug.c +11 -4
Index: linux-3.5-rc4/mm/memory_hotplug.c
===================================================================
@@ -95,10 +95,17 @@ static void release_memory_resource(stru
 static void get_page_bootmem(unsigned long info,  struct page *page,
 			     unsigned long type)
 {
-	page->lru.next = (struct list_head *) type;
-	SetPagePrivate(page);
-	set_page_private(page, info);
-	atomic_inc(&page->_count);
+	unsigned long page_type;
+
+	page_type = (unsigned long) page->lru.next;
+	if (type < MEMORY_HOTPLUG_MIN_BOOTMEM_TYPE ||
+	    type > MEMORY_HOTPLUG_MAX_BOOTMEM_TYPE){
+		page->lru.next = (struct list_head *) type;
+		SetPagePrivate(page);
+		set_page_private(page, info);
+		atomic_inc(&page->_count);
+	} else
+		atomic_inc(&page->_count);
 }
 
 /* reference to __meminit __free_pages_bootmem is valid

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#32 lore raw

RFC PATCH v3 9/13 memory-hotplug : move register_page_bootmem_info_node and put_page_bootmem for sparse-vmemmap

In reply to Yasuaki Ishimatsu

For implementing register_page_bootmem_info_node of sparse-vmemmap, register_page_bootmem_info_node and put_page_bootmem are moved to memory_hotplug.c

CC David Rientjes <[email protected]>
CC Jiang Liu <[email protected]>
CC Len Brown <[email protected]>
CC Benjamin Herrenschmidt <[email protected]>
CC Paul Mackerras <[email protected]>
CC Christoph Lameter <[email protected]>
Cc Minchan Kim <[email protected]>
CC Andrew Morton <[email protected]>
CC KOSAKI Motohiro <[email protected]>
CC Wen Congyang <[email protected]>
---
 include/linux/memory_hotplug.h |    9 ---------
 mm/memory_hotplug.c            |    8 ++++++--
 2 files changed, 6 insertions(+), 11 deletions(-)
include/linux/memory_hotplug.h +0 -9
Index: linux-3.5-rc4/include/linux/memory_hotplug.h
===================================================================
@@ -160,17 +160,8 @@ static inline void arch_refresh_nodedata
 #endif /* CONFIG_NUMA */
 #endif /* CONFIG_HAVE_ARCH_NODEDATA_EXTENSION */
 
-#ifdef CONFIG_SPARSEMEM_VMEMMAP
-static inline void register_page_bootmem_info_node(struct pglist_data *pgdat)
-{
-}
-static inline void put_page_bootmem(struct page *page)
-{
-}
-#else
 extern void register_page_bootmem_info_node(struct pglist_data *pgdat);
 extern void put_page_bootmem(struct page *page);
-#endif
 
 /*
  * Lock for memory hotplug guarantees 1) all callbacks for memory hotplug
mm/memory_hotplug.c +6 -2
Index: linux-3.5-rc4/mm/memory_hotplug.c
===================================================================
@@ -91,7 +91,6 @@ static void release_memory_resource(stru
 }
 
 #ifdef CONFIG_MEMORY_HOTPLUG_SPARSE
-#ifndef CONFIG_SPARSEMEM_VMEMMAP
 static void get_page_bootmem(unsigned long info,  struct page *page,
 			     unsigned long type)
 {
@@ -127,6 +126,7 @@ void __ref put_page_bootmem(struct page
 
 }
 
+#ifndef CONFIG_SPARSEMEM_VMEMMAP
 static void register_page_bootmem_info_section(unsigned long start_pfn)
 {
 	unsigned long *usemap, mapsize, section_nr, i;
@@ -163,6 +163,11 @@ static void register_page_bootmem_info_s
 		get_page_bootmem(section_nr, page, MIX_SECTION_INFO);
 
 }
+#else
+static inline void register_page_bootmem_info_section(unsigned long start_pfn)
+{
+}
+#endif
 
 void register_page_bootmem_info_node(struct pglist_data *pgdat)
 {
@@ -198,7 +203,6 @@ void register_page_bootmem_info_node(str
 		register_page_bootmem_info_section(pfn);
 
 }
-#endif /* !CONFIG_SPARSEMEM_VMEMMAP */
 
 static void grow_zone_span(struct zone *zone, unsigned long start_pfn,
 			   unsigned long end_pfn)

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#33 lore raw

RFC PATCH v3 10/13 memory-hotplug : implement register_page_bootmem_info_section of sparse-vmemmap

In reply to Yasuaki Ishimatsu

For removing memmap region of sparse-vmemmap which is allocated bootmem, memmap region of sparse-vmemmap needs to be registered by get_page_bootmem(). So the patch searches pages of virtual mapping and registers the pages by get_page_bootmem().

CC David Rientjes <[email protected]>
CC Jiang Liu <[email protected]>
CC Len Brown <[email protected]>
CC Benjamin Herrenschmidt <[email protected]>
CC Paul Mackerras <[email protected]>
CC Christoph Lameter <[email protected]>
Cc Minchan Kim <[email protected]>
CC Andrew Morton <[email protected]>
CC KOSAKI Motohiro <[email protected]>
CC Wen Congyang <[email protected]>
---
 arch/x86/mm/init_64.c          |   53 +++++++++++++++++++++++++++++++++++++++++
 include/linux/memory_hotplug.h |    2 +
 include/linux/mm.h             |    3 +-
 mm/memory_hotplug.c            |   23 +++++++++++++++--
 4 files changed, 77 insertions(+), 4 deletions(-)
mm/memory_hotplug.c +20 -3
Index: linux-3.5-rc4/mm/memory_hotplug.c
===================================================================
@@ -91,8 +91,8 @@ static void release_memory_resource(stru
 }
 
 #ifdef CONFIG_MEMORY_HOTPLUG_SPARSE
-static void get_page_bootmem(unsigned long info,  struct page *page,
-			     unsigned long type)
+void get_page_bootmem(unsigned long info,  struct page *page,
+		      unsigned long type)
 {
 	unsigned long page_type;
 
@@ -164,8 +164,25 @@ static void register_page_bootmem_info_s
 
 }
 #else
-static inline void register_page_bootmem_info_section(unsigned long start_pfn)
+static void register_page_bootmem_info_section(unsigned long start_pfn)
 {
+	unsigned long mapsize, section_nr;
+	struct mem_section *ms;
+	struct page *page, *memmap;
+
+	if (!pfn_valid(start_pfn))
+		return;
+
+	section_nr = pfn_to_section_nr(start_pfn);
+	ms = __nr_to_section(section_nr);
+
+	memmap = sparse_decode_mem_map(ms->section_mem_map, section_nr);
+
+	page = virt_to_page(memmap);
+	mapsize = sizeof(struct page) * PAGES_PER_SECTION;
+	mapsize = PAGE_ALIGN(mapsize) >> PAGE_SHIFT;
+
+	register_page_bootmem_memmap(section_nr, memmap, PAGES_PER_SECTION);
 }
 #endif
 
include/linux/mm.h +2 -1
Index: linux-3.5-rc4/include/linux/mm.h
===================================================================
@@ -1586,7 +1586,8 @@ int vmemmap_populate_basepages(struct pa
 						unsigned long pages, int node);
 int vmemmap_populate(struct page *start_page, unsigned long pages, int node);
 void vmemmap_populate_print_last(void);
-
+void register_page_bootmem_memmap(unsigned long section_nr, struct page *map,
+				  unsigned long size);
 
 enum mf_flags {
 	MF_COUNT_INCREASED = 1 << 0,
arch/x86/mm/init_64.c +53 -0

59 lines not on this page · raw message

include/linux/memory_hotplug.h +2 -0
Index: linux-3.5-rc4/include/linux/memory_hotplug.h
===================================================================
@@ -162,6 +162,8 @@ static inline void arch_refresh_nodedata
 
 extern void register_page_bootmem_info_node(struct pglist_data *pgdat);
 extern void put_page_bootmem(struct page *page);
+extern void get_page_bootmem(unsigned long ingo, struct page *page,
+			     unsigned long type);
 
 /*
  * Lock for memory hotplug guarantees 1) all callbacks for memory hotplug
#34 lore raw

RFC PATCH v3 11/13 memory-hotplug : free memmap of sparse-vmemmap

In reply to Yasuaki Ishimatsu

I don't think that all pages of virtual mapping in removed memory can be freed, since page which type is MIX_SECTION_INFO is difficult to free. So, the patch only frees page which type is SECTION_INFO at first.

CC David Rientjes <[email protected]>
CC Jiang Liu <[email protected]>
CC Len Brown <[email protected]>
CC Benjamin Herrenschmidt <[email protected]>
CC Paul Mackerras <[email protected]>
CC Christoph Lameter <[email protected]>
Cc Minchan Kim <[email protected]>
CC Andrew Morton <[email protected]>
CC KOSAKI Motohiro <[email protected]>
CC Wen Congyang <[email protected]>
---
 arch/x86/mm/init_64.c |   91 ++++++++++++++++++++++++++++++++++++++++++++++++++
 include/linux/mm.h    |    2 +
 mm/memory_hotplug.c   |    5 ++
 mm/sparse.c           |    5 +-
 4 files changed, 101 insertions(+), 2 deletions(-)
include/linux/mm.h +2 -0
Index: linux-3.5-rc4/include/linux/mm.h
===================================================================
@@ -1588,6 +1588,8 @@ int vmemmap_populate(struct page *start_
 void vmemmap_populate_print_last(void);
 void register_page_bootmem_memmap(unsigned long section_nr, struct page *map,
 				  unsigned long size);
+void vmemmap_kfree(struct page *memmpa, unsigned long nr_pages);
+void vmemmap_free_bootmem(struct page *memmpa, unsigned long nr_pages);
 
 enum mf_flags {
 	MF_COUNT_INCREASED = 1 << 0,
mm/sparse.c +3 -2
Index: linux-3.5-rc4/mm/sparse.c
===================================================================
@@ -614,12 +614,13 @@ static inline struct page *kmalloc_secti
 	/* This will make the necessary allocations eventually. */
 	return sparse_mem_map_populate(pnum, nid);
 }
-static void __kfree_section_memmap(struct page *memmap, unsigned long nr_pages)
+static void __kfree_section_memmap(struct page *page, unsigned long nr_pages)
 {
-	return; /* XXX: Not implemented yet */
+	vmemmap_kfree(page, nr_pages);
 }
 static void free_map_bootmem(struct page *page, unsigned long nr_pages)
 {
+	vmemmap_free_bootmem(page, nr_pages);
 }
 #else
 static struct page *__kmalloc_section_memmap(unsigned long nr_pages)
arch/x86/mm/init_64.c +91 -0

97 lines not on this page · raw message

mm/memory_hotplug.c +5 -0
Index: linux-3.5-rc4/mm/memory_hotplug.c
===================================================================
@@ -303,6 +303,8 @@ static int __meminit __add_section(int n
 #ifdef CONFIG_SPARSEMEM_VMEMMAP
 static int __remove_section(struct zone *zone, struct mem_section *ms)
 {
+	unsigned long flags;
+	struct pglist_data *pgdat = zone->zone_pgdat;
 	int ret;
 
 	if (!valid_section(ms))
@@ -310,6 +312,9 @@ static int __remove_section(struct zone
 
 	ret = unregister_memory_section(ms);
 
+	pgdat_resize_lock(pgdat, &flags);
+	sparse_remove_one_section(zone, ms);
+	pgdat_resize_unlock(pgdat, &flags);
 	return ret;
 }
 #else
#35 lore raw

Re: [RFC PATCH v3 11/13] memory-hotplug : free memmap of sparse-vmemmap

At 07/09/2012 06:33 PM, Yasuaki Ishimatsu Wrote:

Quoted 41 lines

I don't think that all pages of virtual mapping in removed memory can be freed, since page which type is MIX_SECTION_INFO is difficult to free. So, the patch only frees page which type is SECTION_INFO at first.

CC David Rientjes <[email protected]>
CC Jiang Liu <[email protected]>
CC Len Brown <[email protected]>
CC Benjamin Herrenschmidt <[email protected]>
CC Paul Mackerras <[email protected]>
CC Christoph Lameter <[email protected]>
Cc Minchan Kim <[email protected]>
CC Andrew Morton <[email protected]>
CC KOSAKI Motohiro <[email protected]>
CC Wen Congyang <[email protected]>
---
 arch/x86/mm/init_64.c |   91 ++++++++++++++++++++++++++++++++++++++++++++++++++
 include/linux/mm.h    |    2 +
 mm/memory_hotplug.c   |    5 ++
 mm/sparse.c           |    5 +-
 4 files changed, 101 insertions(+), 2 deletions(-)

Index: linux-3.5-rc4/include/linux/mm.h
===================================================================

include/linux/mm.h +2 -0
@@ -1588,6 +1588,8 @@ int vmemmap_populate(struct page *start_
 void vmemmap_populate_print_last(void);
 void register_page_bootmem_memmap(unsigned long section_nr, struct page *map,
 				  unsigned long size);
+void vmemmap_kfree(struct page *memmpa, unsigned long nr_pages);
+void vmemmap_free_bootmem(struct page *memmpa, unsigned long nr_pages);
 
 enum mf_flags {
 	MF_COUNT_INCREASED = 1 << 0,

Index: linux-3.5-rc4/mm/sparse.c
===================================================================

mm/sparse.c static inline struct page *kmalloc_secti · line 619

Quoted 8 lines
mm/sparse.c +2 -2
@@ -614,12 +614,13 @@ static inline struct page *kmalloc_secti
 	/* This will make the necessary allocations eventually. */
 	return sparse_mem_map_populate(pnum, nid);
 }
-static void __kfree_section_memmap(struct page *memmap, unsigned long nr_pages)
+static void __kfree_section_memmap(struct page *page, unsigned long nr_pages)
 {
-	return; /* XXX: Not implemented yet */
+	vmemmap_kfree(page, nr_pages);

Hmm, I think you try to free the memory allocated in kmalloc_section_memmap().

mm/sparse.c static inline struct page *kmalloc_secti · line 624

Quoted 5 lines
mm/sparse.c +1 -0
 }
 static void free_map_bootmem(struct page *page, unsigned long nr_pages)
 {
+	vmemmap_free_bootmem(page, nr_pages);
 }

Hmm, which function is the memory you try to free allocated in?

Quoted 14 lines
mm/sparse.c +0 -0
 #else
 static struct page *__kmalloc_section_memmap(unsigned long nr_pages)

Index: linux-3.5-rc4/arch/x86/mm/init_64.c
===================================================================

arch/x86/mm/init_64.c +4 -0
@@ -978,6 +978,97 @@ vmemmap_populate(struct page *start_page
 	return 0;
 }
 
+unsigned long find_and_clear_pte_page(unsigned long addr, unsigned long end,
+				      struct page **pp)
+{
+	pgd_t *pgd;

arch/x86/mm/init_64.c vmemmap_populate(struct page *start_page · line 994

Quoted 10 lines
arch/x86/mm/init_64.c +10 -0
+	pud_t *pud;
+	pmd_t *pmd;
+	pte_t *pte;
+	unsigned long next;
+
+	*pp = NULL;
+
+	pgd = pgd_offset_k(addr);
+	if (pgd_none(*pgd))
+		return (addr + PAGE_SIZE) & PAGE_MASK;

Hmm, why not goto next pgd?

Quoted 7 lines
arch/x86/mm/init_64.c +7 -0
+
+	pud = pud_offset(pgd, addr);
+	if (pud_none(*pud))
+		return (addr + PAGE_SIZE) & PAGE_MASK;
+
+	if (!cpu_has_pse) {
+		next = (addr + PAGE_SIZE) & PAGE_MASK;

arch/x86/mm/init_64.c vmemmap_populate(struct page *start_page · line 1011

Quoted 10 lines
arch/x86/mm/init_64.c +10 -0
+		pmd = pmd_offset(pud, addr);
+		if (pmd_none(*pmd))
+			return next;
+
+		pte = pte_offset_kernel(pmd, addr);
+		if (pte_none(*pte))
+			return next;
+
+		*pp = pte_page(*pte);
+		pte_clear(&init_mm, addr, pte);

I think you should flush tlb here.

Quoted 21 lines
arch/x86/mm/init_64.c +21 -0
+	} else {
+		next = pmd_addr_end(addr, end);
+
+		pmd = pmd_offset(pud, addr);
+		if (pmd_none(*pmd))
+			return next;
+
+		*pp = pmd_page(*pmd);
+		pmd_clear(pmd);
+	}
+
+	return next;
+}
+
+void __meminit
+vmemmap_kfree(struct page *memmap, unsigned long nr_pages)
+{
+	unsigned long addr = (unsigned long)memmap;
+	unsigned long end = (unsigned long)(memmap + nr_pages);
+	unsigned long next;
+	unsigned int order;

arch/x86/mm/init_64.c vmemmap_populate(struct page *start_page · line 1042

Quoted 10 lines
arch/x86/mm/init_64.c +10 -0
+	struct page *page;
+
+	for (; addr < end; addr = next) {
+		page = NULL;
+		next = find_and_clear_pte_page(addr, end, &page);
+		if (!page)
+			continue;
+
+		if (is_vmalloc_addr(page_address(page)))
+			vfree(page_address(page));

Hmm, the memory is allocated in vmemmap_alloc_block(), and the address can not be vmalloc address.

arch/x86/mm/init_64.c vmemmap_populate(struct page *start_page · line 1046

Quoted 4 lines
arch/x86/mm/init_64.c +4 -0
+		else {
+			order = next - addr;
+			free_pages((unsigned long)page_address(page),
+				   get_order(order));

OOPS. I think we cannot free pages here.

sizeof(struct page) is less than PAGE_SIZE. We store more than one struct page in the same page. If you free it here while the other struct page is in use, it is very dangerous.

Quoted 33 lines
arch/x86/mm/init_64.c +25 -0
+		}
+	}
+}
+
+void __meminit
+vmemmap_free_bootmem(struct page *memmap, unsigned long nr_pages)
+{
+	unsigned long addr = (unsigned long)memmap;
+	unsigned long end = (unsigned long)(memmap + nr_pages);
+	unsigned long next;
+	struct page *page;
+	unsigned long magic;
+
+	for (; addr < end; addr = next) {
+		page = NULL;
+		next = find_and_clear_pte_page(addr, end, &page);
+		if (!page)
+			continue;
+
+		magic = (unsigned long) page->lru.next;
+		if (magic == SECTION_INFO)
+			put_page_bootmem(page);
+	}
+}
+
 void __meminit
 register_page_bootmem_memmap(unsigned long section_nr, struct page *start_page,
 			     unsigned long size)

Index: linux-3.5-rc4/mm/memory_hotplug.c
===================================================================

mm/memory_hotplug.c static int __meminit __add_section(int n · line 303

Quoted 1 line
mm/memory_hotplug.c +0 -0
@@ -303,6 +303,8 @@ static int __meminit __add_section(int n
 #ifdef CONFIG_SPARSEMEM_VMEMMAP

I think this line can be removed now.

Thanks
Wen Congyang

Quoted 10 lines
mm/memory_hotplug.c +2 -0
 static int __remove_section(struct zone *zone, struct mem_section *ms)
 {
+	unsigned long flags;
+	struct pglist_data *pgdat = zone->zone_pgdat;
 	int ret;
 
 	if (!valid_section(ms))

mm/memory_hotplug.c static int __remove_section(struct zone · line 320

Quoted 9 lines
mm/memory_hotplug.c +3 -0
@@ -310,6 +312,9 @@ static int __remove_section(struct zone
 
 	ret = unregister_memory_section(ms);
 
+	pgdat_resize_lock(pgdat, &flags);
+	sparse_remove_one_section(zone, ms);
+	pgdat_resize_unlock(pgdat, &flags);
 	return ret;
 }
 #else

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#36 lore raw

Re: [RFC PATCH v3 11/13] memory-hotplug : free memmap of sparse-vmemmap

2012/07/11 14:06, Wen Congyang wrote:
Hi Wen,

Quoted 44 lines

At 07/09/2012 06:33 PM, Yasuaki Ishimatsu Wrote:

Quoted 43 lines

I don't think that all pages of virtual mapping in removed memory can be freed, since page which type is MIX_SECTION_INFO is difficult to free. So, the patch only frees page which type is SECTION_INFO at first.

CC David Rientjes <[email protected]>
CC Jiang Liu <[email protected]>
CC Len Brown <[email protected]>
CC Benjamin Herrenschmidt <[email protected]>
CC Paul Mackerras <[email protected]>
CC Christoph Lameter <[email protected]>
Cc Minchan Kim <[email protected]>
CC Andrew Morton <[email protected]>
CC KOSAKI Motohiro <[email protected]>
CC Wen Congyang <[email protected]>
---
  arch/x86/mm/init_64.c |   91 ++++++++++++++++++++++++++++++++++++++++++++++++++
  include/linux/mm.h    |    2 +
  mm/memory_hotplug.c   |    5 ++
  mm/sparse.c           |    5 +-
  4 files changed, 101 insertions(+), 2 deletions(-)

Index: linux-3.5-rc4/include/linux/mm.h
===================================================================

include/linux/mm.h +2 -0
@@ -1588,6 +1588,8 @@ int vmemmap_populate(struct page *start_
  void vmemmap_populate_print_last(void);
  void register_page_bootmem_memmap(unsigned long section_nr, struct page *map,
  				  unsigned long size);
+void vmemmap_kfree(struct page *memmpa, unsigned long nr_pages);
+void vmemmap_free_bootmem(struct page *memmpa, unsigned long nr_pages);
 
  enum mf_flags {
  	MF_COUNT_INCREASED = 1 << 0,

Index: linux-3.5-rc4/mm/sparse.c
===================================================================

mm/sparse.c +0 -0
@@ -614,12 +614,13 @@ static inline struct page *kmalloc_secti
  	/* This will make the necessary allocations eventually. */
  	return sparse_mem_map_populate(pnum, nid);
Quoted 8 lines

mm/sparse.c static inline struct page *kmalloc_secti · line 619

Quoted 6 lines
mm/sparse.c +2 -2
  }
-static void __kfree_section_memmap(struct page *memmap, unsigned long nr_pages)
+static void __kfree_section_memmap(struct page *page, unsigned long nr_pages)
  {
-	return; /* XXX: Not implemented yet */
+	vmemmap_kfree(page, nr_pages);

Hmm, I think you try to free the memory allocated in kmalloc_section_memmap().

Yes.

Quoted 8 lines

mm/sparse.c static inline struct page *kmalloc_secti · line 624

Quoted 6 lines
mm/sparse.c +1 -0
  }
  static void free_map_bootmem(struct page *page, unsigned long nr_pages)
  {
+	vmemmap_free_bootmem(page, nr_pages);
  }

Hmm, which function is the memory you try to free allocated in?

The function try to free memory allocated from bootmem. The memory has been registered by get_page_bootmem(). So we can free the memory by put_page_bootmem().

Quoted 11 lines
Quoted 11 lines
mm/sparse.c +0 -0
  #else
  static struct page *__kmalloc_section_memmap(unsigned long nr_pages)

Index: linux-3.5-rc4/arch/x86/mm/init_64.c
===================================================================

arch/x86/mm/init_64.c +0 -0
@@ -978,6 +978,97 @@ vmemmap_populate(struct page *start_page
  	return 0;
  }
 
Quoted 16 lines
Quoted 4 lines
arch/x86/mm/init_64.c +4 -0
+unsigned long find_and_clear_pte_page(unsigned long addr, unsigned long end,
+				      struct page **pp)
+{
+	pgd_t *pgd;

arch/x86/mm/init_64.c vmemmap_populate(struct page *start_page · line 994

Quoted 10 lines
arch/x86/mm/init_64.c +10 -0
+	pud_t *pud;
+	pmd_t *pmd;
+	pte_t *pte;
+	unsigned long next;
+
+	*pp = NULL;
+
+	pgd = pgd_offset_k(addr);
+	if (pgd_none(*pgd))
+		return (addr + PAGE_SIZE) & PAGE_MASK;

Hmm, why not goto next pgd?

Does it mean "return (addr + PGDIR_SIZE) & PGDIR_MASK"?

Quoted 12 lines
Quoted 12 lines
arch/x86/mm/init_64.c +11 -0
+
+	pud = pud_offset(pgd, addr);
+	if (pud_none(*pud))
+		return (addr + PAGE_SIZE) & PAGE_MASK;
+
+	if (!cpu_has_pse) {
+		next = (addr + PAGE_SIZE) & PAGE_MASK;
+		pmd = pmd_offset(pud, addr);
+		if (pmd_none(*pmd))
+			return next;
+
Quoted 8 lines

arch/x86/mm/init_64.c vmemmap_populate(struct page *start_page · line 1011

Quoted 6 lines
arch/x86/mm/init_64.c +6 -0
+		pte = pte_offset_kernel(pmd, addr);
+		if (pte_none(*pte))
+			return next;
+
+		*pp = pte_page(*pte);
+		pte_clear(&init_mm, addr, pte);

I think you should flush tlb here.

Thanks, I'll update it.

Quoted 27 lines
Quoted 27 lines
arch/x86/mm/init_64.c +26 -0
+	} else {
+		next = pmd_addr_end(addr, end);
+
+		pmd = pmd_offset(pud, addr);
+		if (pmd_none(*pmd))
+			return next;
+
+		*pp = pmd_page(*pmd);
+		pmd_clear(pmd);
+	}
+
+	return next;
+}
+
+void __meminit
+vmemmap_kfree(struct page *memmap, unsigned long nr_pages)
+{
+	unsigned long addr = (unsigned long)memmap;
+	unsigned long end = (unsigned long)(memmap + nr_pages);
+	unsigned long next;
+	unsigned int order;
+	struct page *page;
+
+	for (; addr < end; addr = next) {
+		page = NULL;
+		next = find_and_clear_pte_page(addr, end, &page);
Quoted 8 lines

arch/x86/mm/init_64.c vmemmap_populate(struct page *start_page · line 1042

Quoted 5 lines
arch/x86/mm/init_64.c +5 -0
+		if (!page)
+			continue;
+
+		if (is_vmalloc_addr(page_address(page)))
+			vfree(page_address(page));

Hmm, the memory is allocated in vmemmap_alloc_block(), and the address can not be vmalloc address.

Does it mean the if sentence is unnecessary?

Quoted 11 lines

arch/x86/mm/init_64.c vmemmap_populate(struct page *start_page · line 1046

Quoted 5 lines
arch/x86/mm/init_64.c +4 -0
+		else {
+			order = next - addr;
+			free_pages((unsigned long)page_address(page),
+				   get_order(order));

OOPS. I think we cannot free pages here.

sizeof(struct page) is less than PAGE_SIZE. We store more than one struct page in the same page. If you free it here while the other struct page is in use, it is very dangerous.

The memory has page structures for hot-removed memory. So nobody is using these pages, since the hot-removed memory has been offlined.

Quoted 28 lines
Quoted 28 lines
arch/x86/mm/init_64.c +25 -0
+		}
+	}
+}
+
+void __meminit
+vmemmap_free_bootmem(struct page *memmap, unsigned long nr_pages)
+{
+	unsigned long addr = (unsigned long)memmap;
+	unsigned long end = (unsigned long)(memmap + nr_pages);
+	unsigned long next;
+	struct page *page;
+	unsigned long magic;
+
+	for (; addr < end; addr = next) {
+		page = NULL;
+		next = find_and_clear_pte_page(addr, end, &page);
+		if (!page)
+			continue;
+
+		magic = (unsigned long) page->lru.next;
+		if (magic == SECTION_INFO)
+			put_page_bootmem(page);
+	}
+}
+
  void __meminit
  register_page_bootmem_memmap(unsigned long section_nr, struct page *start_page,
  			     unsigned long size)
Quoted 8 lines

mm/memory_hotplug.c static int __meminit __add_section(int n · line 303

Quoted 6 lines

Index: linux-3.5-rc4/mm/memory_hotplug.c
===================================================================

mm/memory_hotplug.c +0 -0
@@ -303,6 +303,8 @@ static int __meminit __add_section(int n
  #ifdef CONFIG_SPARSEMEM_VMEMMAP

I think this line can be removed now.

I'll update it.

Thanks,
Yasuaki Ishimatsu

Quoted 15 lines

Thanks
Wen Congyang

Quoted 11 lines
mm/memory_hotplug.c +2 -0
  static int __remove_section(struct zone *zone, struct mem_section *ms)
  {
+	unsigned long flags;
+	struct pglist_data *pgdat = zone->zone_pgdat;
  	int ret;
 
  	if (!valid_section(ms))
@@ -310,6 +312,9 @@ static int __remove_section(struct zone
 
  	ret = unregister_memory_section(ms);
 
Quoted 9 lines

mm/memory_hotplug.c static int __remove_section(struct zone · line 320

Quoted 8 lines
mm/memory_hotplug.c +3 -0
+	pgdat_resize_lock(pgdat, &flags);
+	sparse_remove_one_section(zone, ms);
+	pgdat_resize_unlock(pgdat, &flags);
  	return ret;
  }
  #else

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#37 lore raw

Re: [RFC PATCH v3 11/13] memory-hotplug : free memmap of sparse-vmemmap

At 07/11/2012 01:52 PM, Yasuaki Ishimatsu Wrote:

Quoted 67 lines

2012/07/11 14:06, Wen Congyang wrote:
Hi Wen,

Quoted 44 lines

At 07/09/2012 06:33 PM, Yasuaki Ishimatsu Wrote:

Quoted 43 lines

I don't think that all pages of virtual mapping in removed memory can be freed, since page which type is MIX_SECTION_INFO is difficult to free. So, the patch only frees page which type is SECTION_INFO at first.

CC David Rientjes <[email protected]>
CC Jiang Liu <[email protected]>
CC Len Brown <[email protected]>
CC Benjamin Herrenschmidt <[email protected]>
CC Paul Mackerras <[email protected]>
CC Christoph Lameter <[email protected]>
Cc Minchan Kim <[email protected]>
CC Andrew Morton <[email protected]>
CC KOSAKI Motohiro <[email protected]>
CC Wen Congyang <[email protected]>
---
  arch/x86/mm/init_64.c |   91 ++++++++++++++++++++++++++++++++++++++++++++++++++
  include/linux/mm.h    |    2 +
  mm/memory_hotplug.c   |    5 ++
  mm/sparse.c           |    5 +-
  4 files changed, 101 insertions(+), 2 deletions(-)

Index: linux-3.5-rc4/include/linux/mm.h
===================================================================

include/linux/mm.h +2 -0
@@ -1588,6 +1588,8 @@ int vmemmap_populate(struct page *start_
  void vmemmap_populate_print_last(void);
  void register_page_bootmem_memmap(unsigned long section_nr, struct page *map,
  				  unsigned long size);
+void vmemmap_kfree(struct page *memmpa, unsigned long nr_pages);
+void vmemmap_free_bootmem(struct page *memmpa, unsigned long nr_pages);
 
  enum mf_flags {
  	MF_COUNT_INCREASED = 1 << 0,

Index: linux-3.5-rc4/mm/sparse.c
===================================================================

mm/sparse.c +0 -0
@@ -614,12 +614,13 @@ static inline struct page *kmalloc_secti
  	/* This will make the necessary allocations eventually. */
  	return sparse_mem_map_populate(pnum, nid);
Quoted 8 lines

mm/sparse.c static inline struct page *kmalloc_secti · line 619

Quoted 6 lines
mm/sparse.c +2 -2
  }
-static void __kfree_section_memmap(struct page *memmap, unsigned long nr_pages)
+static void __kfree_section_memmap(struct page *page, unsigned long nr_pages)
  {
-	return; /* XXX: Not implemented yet */
+	vmemmap_kfree(page, nr_pages);

Hmm, I think you try to free the memory allocated in kmalloc_section_memmap().

Yes.

Quoted 8 lines

mm/sparse.c static inline struct page *kmalloc_secti · line 624

Quoted 6 lines
mm/sparse.c +1 -0
  }
  static void free_map_bootmem(struct page *page, unsigned long nr_pages)
  {
+	vmemmap_free_bootmem(page, nr_pages);
  }

Hmm, which function is the memory you try to free allocated in?

Quoted 3 lines

The function try to free memory allocated from bootmem. The memory has been registered by get_page_bootmem(). So we can free the memory by put_page_bootmem().

OK, I will read these codes, and check it.

Quoted 100 lines
Quoted 12 lines
Quoted 12 lines
mm/sparse.c +0 -0
  #else
  static struct page *__kmalloc_section_memmap(unsigned long nr_pages)

Index: linux-3.5-rc4/arch/x86/mm/init_64.c
===================================================================

arch/x86/mm/init_64.c +0 -0
@@ -978,6 +978,97 @@ vmemmap_populate(struct page *start_page
  	return 0;
  }
 
Quoted 16 lines
Quoted 4 lines
arch/x86/mm/init_64.c +4 -0
+unsigned long find_and_clear_pte_page(unsigned long addr, unsigned long end,
+				      struct page **pp)
+{
+	pgd_t *pgd;

arch/x86/mm/init_64.c vmemmap_populate(struct page *start_page · line 994

Quoted 10 lines
arch/x86/mm/init_64.c +10 -0
+	pud_t *pud;
+	pmd_t *pmd;
+	pte_t *pte;
+	unsigned long next;
+
+	*pp = NULL;
+
+	pgd = pgd_offset_k(addr);
+	if (pgd_none(*pgd))
+		return (addr + PAGE_SIZE) & PAGE_MASK;

Hmm, why not goto next pgd?

Does it mean "return (addr + PGDIR_SIZE) & PGDIR_MASK"?

Quoted 12 lines
Quoted 12 lines
arch/x86/mm/init_64.c +11 -0
+
+	pud = pud_offset(pgd, addr);
+	if (pud_none(*pud))
+		return (addr + PAGE_SIZE) & PAGE_MASK;
+
+	if (!cpu_has_pse) {
+		next = (addr + PAGE_SIZE) & PAGE_MASK;
+		pmd = pmd_offset(pud, addr);
+		if (pmd_none(*pmd))
+			return next;
+
Quoted 8 lines

arch/x86/mm/init_64.c vmemmap_populate(struct page *start_page · line 1011

Quoted 6 lines
arch/x86/mm/init_64.c +6 -0
+		pte = pte_offset_kernel(pmd, addr);
+		if (pte_none(*pte))
+			return next;
+
+		*pp = pte_page(*pte);
+		pte_clear(&init_mm, addr, pte);

I think you should flush tlb here.

Thanks, I'll update it.

Quoted 27 lines
Quoted 27 lines
arch/x86/mm/init_64.c +26 -0
+	} else {
+		next = pmd_addr_end(addr, end);
+
+		pmd = pmd_offset(pud, addr);
+		if (pmd_none(*pmd))
+			return next;
+
+		*pp = pmd_page(*pmd);
+		pmd_clear(pmd);
+	}
+
+	return next;
+}
+
+void __meminit
+vmemmap_kfree(struct page *memmap, unsigned long nr_pages)
+{
+	unsigned long addr = (unsigned long)memmap;
+	unsigned long end = (unsigned long)(memmap + nr_pages);
+	unsigned long next;
+	unsigned int order;
+	struct page *page;
+
+	for (; addr < end; addr = next) {
+		page = NULL;
+		next = find_and_clear_pte_page(addr, end, &page);
Quoted 8 lines

arch/x86/mm/init_64.c vmemmap_populate(struct page *start_page · line 1042

Quoted 5 lines
arch/x86/mm/init_64.c +5 -0
+		if (!page)
+			continue;
+
+		if (is_vmalloc_addr(page_address(page)))
+			vfree(page_address(page));

Hmm, the memory is allocated in vmemmap_alloc_block(), and the address can not be vmalloc address.

Does it mean the if sentence is unnecessary?

Quoted 8 lines

arch/x86/mm/init_64.c vmemmap_populate(struct page *start_page · line 1046

Quoted 5 lines
arch/x86/mm/init_64.c +4 -0
+		else {
+			order = next - addr;
+			free_pages((unsigned long)page_address(page),
+				   get_order(order));

OOPS. I think we cannot free pages here.

Quoted 6 lines
Quoted 3 lines

sizeof(struct page) is less than PAGE_SIZE. We store more than one struct page in the same page. If you free it here while the other struct page is in use, it is very dangerous.

The memory has page structures for hot-removed memory. So nobody is using these pages, since the hot-removed memory has been offlined.

The memory has page structures for hot-removed memory, but it may contain page structures for the other hot-added memory.

IIUC, If we use sparse-vmemmap, all page structures is stored here.

Thanks
Wen Congyang

Quoted 38 lines
Quoted 29 lines
Quoted 29 lines
arch/x86/mm/init_64.c +25 -0
+		}
+	}
+}
+
+void __meminit
+vmemmap_free_bootmem(struct page *memmap, unsigned long nr_pages)
+{
+	unsigned long addr = (unsigned long)memmap;
+	unsigned long end = (unsigned long)(memmap + nr_pages);
+	unsigned long next;
+	struct page *page;
+	unsigned long magic;
+
+	for (; addr < end; addr = next) {
+		page = NULL;
+		next = find_and_clear_pte_page(addr, end, &page);
+		if (!page)
+			continue;
+
+		magic = (unsigned long) page->lru.next;
+		if (magic == SECTION_INFO)
+			put_page_bootmem(page);
+	}
+}
+
  void __meminit
  register_page_bootmem_memmap(unsigned long section_nr, struct page *start_page,
  			     unsigned long size)
Quoted 8 lines

mm/memory_hotplug.c static int __meminit __add_section(int n · line 303

Quoted 6 lines

Index: linux-3.5-rc4/mm/memory_hotplug.c
===================================================================

mm/memory_hotplug.c +0 -0
@@ -303,6 +303,8 @@ static int __meminit __add_section(int n
  #ifdef CONFIG_SPARSEMEM_VMEMMAP

I think this line can be removed now.

Quoted 33 lines

I'll update it.

Thanks,
Yasuaki Ishimatsu

Quoted 24 lines

Thanks
Wen Congyang

Quoted 19 lines
mm/memory_hotplug.c +5 -0
  static int __remove_section(struct zone *zone, struct mem_section *ms)
  {
+	unsigned long flags;
+	struct pglist_data *pgdat = zone->zone_pgdat;
  	int ret;
 
  	if (!valid_section(ms))
@@ -310,6 +312,9 @@ static int __remove_section(struct zone
 
  	ret = unregister_memory_section(ms);
 
+	pgdat_resize_lock(pgdat, &flags);
+	sparse_remove_one_section(zone, ms);
+	pgdat_resize_unlock(pgdat, &flags);
  	return ret;
  }
  #else

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#38 lore raw

Re: [RFC PATCH v3 11/13] memory-hotplug : free memmap of sparse-vmemmap

Hi Wen,

Quoted 178 lines · Wen Congyang

At 07/11/2012 01:52 PM, Yasuaki Ishimatsu Wrote:

Quoted 67 lines

2012/07/11 14:06, Wen Congyang wrote:
Hi Wen,

Quoted 44 lines

At 07/09/2012 06:33 PM, Yasuaki Ishimatsu Wrote:

Quoted 43 lines

I don't think that all pages of virtual mapping in removed memory can be freed, since page which type is MIX_SECTION_INFO is difficult to free. So, the patch only frees page which type is SECTION_INFO at first.

CC David Rientjes <[email protected]>
CC Jiang Liu <[email protected]>
CC Len Brown <[email protected]>
CC Benjamin Herrenschmidt <[email protected]>
CC Paul Mackerras <[email protected]>
CC Christoph Lameter <[email protected]>
Cc Minchan Kim <[email protected]>
CC Andrew Morton <[email protected]>
CC KOSAKI Motohiro <[email protected]>
CC Wen Congyang <[email protected]>
---
   arch/x86/mm/init_64.c |   91 ++++++++++++++++++++++++++++++++++++++++++++++++++
   include/linux/mm.h    |    2 +
   mm/memory_hotplug.c   |    5 ++
   mm/sparse.c           |    5 +-
   4 files changed, 101 insertions(+), 2 deletions(-)

Index: linux-3.5-rc4/include/linux/mm.h
===================================================================

include/linux/mm.h +2 -0
@@ -1588,6 +1588,8 @@ int vmemmap_populate(struct page *start_
   void vmemmap_populate_print_last(void);
   void register_page_bootmem_memmap(unsigned long section_nr, struct page *map,
   				  unsigned long size);
+void vmemmap_kfree(struct page *memmpa, unsigned long nr_pages);
+void vmemmap_free_bootmem(struct page *memmpa, unsigned long nr_pages);
 
   enum mf_flags {
   	MF_COUNT_INCREASED = 1 << 0,

Index: linux-3.5-rc4/mm/sparse.c
===================================================================

mm/sparse.c +0 -0
@@ -614,12 +614,13 @@ static inline struct page *kmalloc_secti
   	/* This will make the necessary allocations eventually. */
   	return sparse_mem_map_populate(pnum, nid);
Quoted 8 lines

mm/sparse.c static inline struct page *kmalloc_secti · line 619

Quoted 6 lines
mm/sparse.c +2 -2
   }
-static void __kfree_section_memmap(struct page *memmap, unsigned long nr_pages)
+static void __kfree_section_memmap(struct page *page, unsigned long nr_pages)
   {
-	return; /* XXX: Not implemented yet */
+	vmemmap_kfree(page, nr_pages);

Hmm, I think you try to free the memory allocated in kmalloc_section_memmap().

Yes.

Quoted 8 lines

mm/sparse.c static inline struct page *kmalloc_secti · line 624

Quoted 6 lines
mm/sparse.c +1 -0
   }
   static void free_map_bootmem(struct page *page, unsigned long nr_pages)
   {
+	vmemmap_free_bootmem(page, nr_pages);
   }

Hmm, which function is the memory you try to free allocated in?

Quoted 3 lines

The function try to free memory allocated from bootmem. The memory has been registered by get_page_bootmem(). So we can free the memory by put_page_bootmem().

OK, I will read these codes, and check it.

Quoted 104 lines
Quoted 12 lines
Quoted 12 lines
mm/sparse.c +0 -0
   #else
   static struct page *__kmalloc_section_memmap(unsigned long nr_pages)

Index: linux-3.5-rc4/arch/x86/mm/init_64.c
===================================================================

arch/x86/mm/init_64.c +0 -0
@@ -978,6 +978,97 @@ vmemmap_populate(struct page *start_page
   	return 0;
   }
 
Quoted 16 lines
Quoted 4 lines
arch/x86/mm/init_64.c +4 -0
+unsigned long find_and_clear_pte_page(unsigned long addr, unsigned long end,
+				      struct page **pp)
+{
+	pgd_t *pgd;

arch/x86/mm/init_64.c vmemmap_populate(struct page *start_page · line 994

Quoted 10 lines
arch/x86/mm/init_64.c +10 -0
+	pud_t *pud;
+	pmd_t *pmd;
+	pte_t *pte;
+	unsigned long next;
+
+	*pp = NULL;
+
+	pgd = pgd_offset_k(addr);
+	if (pgd_none(*pgd))
+		return (addr + PAGE_SIZE) & PAGE_MASK;

Hmm, why not goto next pgd?

Does it mean "return (addr + PGDIR_SIZE) & PGDIR_MASK"?

Quoted 12 lines
Quoted 12 lines
arch/x86/mm/init_64.c +11 -0
+
+	pud = pud_offset(pgd, addr);
+	if (pud_none(*pud))
+		return (addr + PAGE_SIZE) & PAGE_MASK;
+
+	if (!cpu_has_pse) {
+		next = (addr + PAGE_SIZE) & PAGE_MASK;
+		pmd = pmd_offset(pud, addr);
+		if (pmd_none(*pmd))
+			return next;
+
Quoted 8 lines

arch/x86/mm/init_64.c vmemmap_populate(struct page *start_page · line 1011

Quoted 6 lines
arch/x86/mm/init_64.c +6 -0
+		pte = pte_offset_kernel(pmd, addr);
+		if (pte_none(*pte))
+			return next;
+
+		*pp = pte_page(*pte);
+		pte_clear(&init_mm, addr, pte);

I think you should flush tlb here.

Thanks, I'll update it.

Quoted 27 lines
Quoted 27 lines
arch/x86/mm/init_64.c +26 -0
+	} else {
+		next = pmd_addr_end(addr, end);
+
+		pmd = pmd_offset(pud, addr);
+		if (pmd_none(*pmd))
+			return next;
+
+		*pp = pmd_page(*pmd);
+		pmd_clear(pmd);
+	}
+
+	return next;
+}
+
+void __meminit
+vmemmap_kfree(struct page *memmap, unsigned long nr_pages)
+{
+	unsigned long addr = (unsigned long)memmap;
+	unsigned long end = (unsigned long)(memmap + nr_pages);
+	unsigned long next;
+	unsigned int order;
+	struct page *page;
+
+	for (; addr < end; addr = next) {
+		page = NULL;
+		next = find_and_clear_pte_page(addr, end, &page);
Quoted 8 lines

arch/x86/mm/init_64.c vmemmap_populate(struct page *start_page · line 1042

Quoted 5 lines
arch/x86/mm/init_64.c +5 -0
+		if (!page)
+			continue;
+
+		if (is_vmalloc_addr(page_address(page)))
+			vfree(page_address(page));

Hmm, the memory is allocated in vmemmap_alloc_block(), and the address can not be vmalloc address.

Does it mean the if sentence is unnecessary?

Quoted 11 lines

arch/x86/mm/init_64.c vmemmap_populate(struct page *start_page · line 1046

Quoted 5 lines
arch/x86/mm/init_64.c +4 -0
+		else {
+			order = next - addr;
+			free_pages((unsigned long)page_address(page),
+				   get_order(order));

OOPS. I think we cannot free pages here.

sizeof(struct page) is less than PAGE_SIZE. We store more than one struct page in the same page. If you free it here while the other struct page is in use, it is very dangerous.

Quoted 5 lines
Quoted 2 lines

The memory has page structures for hot-removed memory. So nobody is using these pages, since the hot-removed memory has been offlined.

The memory has page structures for hot-removed memory, but it may contain page structures for the other hot-added memory.

Yes. There may be such corner case. But when does the corner case appear? When removed memory is not aligned to PMD_SIZE/PAGE_SIZE, does the corner case appear? Do you know it?

Thank,
Yasuaki Ishimatsu

Quoted 78 lines

IIUC, If we use sparse-vmemmap, all page structures is stored here.

Thanks
Wen Congyang

Quoted 71 lines
Quoted 29 lines
Quoted 29 lines
arch/x86/mm/init_64.c +25 -0
+		}
+	}
+}
+
+void __meminit
+vmemmap_free_bootmem(struct page *memmap, unsigned long nr_pages)
+{
+	unsigned long addr = (unsigned long)memmap;
+	unsigned long end = (unsigned long)(memmap + nr_pages);
+	unsigned long next;
+	struct page *page;
+	unsigned long magic;
+
+	for (; addr < end; addr = next) {
+		page = NULL;
+		next = find_and_clear_pte_page(addr, end, &page);
+		if (!page)
+			continue;
+
+		magic = (unsigned long) page->lru.next;
+		if (magic == SECTION_INFO)
+			put_page_bootmem(page);
+	}
+}
+
   void __meminit
   register_page_bootmem_memmap(unsigned long section_nr, struct page *start_page,
   			     unsigned long size)
Quoted 8 lines

mm/memory_hotplug.c static int __meminit __add_section(int n · line 303

Quoted 6 lines

Index: linux-3.5-rc4/mm/memory_hotplug.c
===================================================================

mm/memory_hotplug.c +0 -0
@@ -303,6 +303,8 @@ static int __meminit __add_section(int n
   #ifdef CONFIG_SPARSEMEM_VMEMMAP

I think this line can be removed now.

I'll update it.

Thanks,
Yasuaki Ishimatsu

Quoted 24 lines

Thanks
Wen Congyang

Quoted 19 lines
mm/memory_hotplug.c +5 -0
   static int __remove_section(struct zone *zone, struct mem_section *ms)
   {
+	unsigned long flags;
+	struct pglist_data *pgdat = zone->zone_pgdat;
   	int ret;
 
   	if (!valid_section(ms))
@@ -310,6 +312,9 @@ static int __remove_section(struct zone
 
   	ret = unregister_memory_section(ms);
 
+	pgdat_resize_lock(pgdat, &flags);
+	sparse_remove_one_section(zone, ms);
+	pgdat_resize_unlock(pgdat, &flags);
   	return ret;
   }
   #else

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#39 lore raw

Re: [RFC PATCH v3 11/13] memory-hotplug : free memmap of sparse-vmemmap

At 07/11/2012 02:48 PM, Yasuaki Ishimatsu Wrote:

Quoted 184 lines

Hi Wen,

Quoted 181 lines · Wen Congyang

At 07/11/2012 01:52 PM, Yasuaki Ishimatsu Wrote:

Quoted 67 lines

2012/07/11 14:06, Wen Congyang wrote:
Hi Wen,

Quoted 44 lines

At 07/09/2012 06:33 PM, Yasuaki Ishimatsu Wrote:

Quoted 43 lines

I don't think that all pages of virtual mapping in removed memory can be freed, since page which type is MIX_SECTION_INFO is difficult to free. So, the patch only frees page which type is SECTION_INFO at first.

CC David Rientjes <[email protected]>
CC Jiang Liu <[email protected]>
CC Len Brown <[email protected]>
CC Benjamin Herrenschmidt <[email protected]>
CC Paul Mackerras <[email protected]>
CC Christoph Lameter <[email protected]>
Cc Minchan Kim <[email protected]>
CC Andrew Morton <[email protected]>
CC KOSAKI Motohiro <[email protected]>
CC Wen Congyang <[email protected]>
---
   arch/x86/mm/init_64.c |   91 ++++++++++++++++++++++++++++++++++++++++++++++++++
   include/linux/mm.h    |    2 +
   mm/memory_hotplug.c   |    5 ++
   mm/sparse.c           |    5 +-
   4 files changed, 101 insertions(+), 2 deletions(-)

Index: linux-3.5-rc4/include/linux/mm.h
===================================================================

include/linux/mm.h +2 -0
@@ -1588,6 +1588,8 @@ int vmemmap_populate(struct page *start_
   void vmemmap_populate_print_last(void);
   void register_page_bootmem_memmap(unsigned long section_nr, struct page *map,
   				  unsigned long size);
+void vmemmap_kfree(struct page *memmpa, unsigned long nr_pages);
+void vmemmap_free_bootmem(struct page *memmpa, unsigned long nr_pages);
 
   enum mf_flags {
   	MF_COUNT_INCREASED = 1 << 0,

Index: linux-3.5-rc4/mm/sparse.c
===================================================================

mm/sparse.c +0 -0
@@ -614,12 +614,13 @@ static inline struct page *kmalloc_secti
   	/* This will make the necessary allocations eventually. */
   	return sparse_mem_map_populate(pnum, nid);
Quoted 8 lines

mm/sparse.c static inline struct page *kmalloc_secti · line 619

Quoted 6 lines
mm/sparse.c +2 -2
   }
-static void __kfree_section_memmap(struct page *memmap, unsigned long nr_pages)
+static void __kfree_section_memmap(struct page *page, unsigned long nr_pages)
   {
-	return; /* XXX: Not implemented yet */
+	vmemmap_kfree(page, nr_pages);

Hmm, I think you try to free the memory allocated in kmalloc_section_memmap().

Yes.

Quoted 8 lines

mm/sparse.c static inline struct page *kmalloc_secti · line 624

Quoted 6 lines
mm/sparse.c +1 -0
   }
   static void free_map_bootmem(struct page *page, unsigned long nr_pages)
   {
+	vmemmap_free_bootmem(page, nr_pages);
   }

Hmm, which function is the memory you try to free allocated in?

Quoted 3 lines

The function try to free memory allocated from bootmem. The memory has been registered by get_page_bootmem(). So we can free the memory by put_page_bootmem().

OK, I will read these codes, and check it.

Quoted 106 lines
Quoted 12 lines
Quoted 12 lines
mm/sparse.c +0 -0
   #else
   static struct page *__kmalloc_section_memmap(unsigned long nr_pages)

Index: linux-3.5-rc4/arch/x86/mm/init_64.c
===================================================================

arch/x86/mm/init_64.c +0 -0
@@ -978,6 +978,97 @@ vmemmap_populate(struct page *start_page
   	return 0;
   }
 
Quoted 16 lines
Quoted 4 lines
arch/x86/mm/init_64.c +4 -0
+unsigned long find_and_clear_pte_page(unsigned long addr, unsigned long end,
+				      struct page **pp)
+{
+	pgd_t *pgd;

arch/x86/mm/init_64.c vmemmap_populate(struct page *start_page · line 994

Quoted 10 lines
arch/x86/mm/init_64.c +10 -0
+	pud_t *pud;
+	pmd_t *pmd;
+	pte_t *pte;
+	unsigned long next;
+
+	*pp = NULL;
+
+	pgd = pgd_offset_k(addr);
+	if (pgd_none(*pgd))
+		return (addr + PAGE_SIZE) & PAGE_MASK;

Hmm, why not goto next pgd?

Does it mean "return (addr + PGDIR_SIZE) & PGDIR_MASK"?

Quoted 12 lines
Quoted 12 lines
arch/x86/mm/init_64.c +11 -0
+
+	pud = pud_offset(pgd, addr);
+	if (pud_none(*pud))
+		return (addr + PAGE_SIZE) & PAGE_MASK;
+
+	if (!cpu_has_pse) {
+		next = (addr + PAGE_SIZE) & PAGE_MASK;
+		pmd = pmd_offset(pud, addr);
+		if (pmd_none(*pmd))
+			return next;
+
Quoted 8 lines

arch/x86/mm/init_64.c vmemmap_populate(struct page *start_page · line 1011

Quoted 6 lines
arch/x86/mm/init_64.c +6 -0
+		pte = pte_offset_kernel(pmd, addr);
+		if (pte_none(*pte))
+			return next;
+
+		*pp = pte_page(*pte);
+		pte_clear(&init_mm, addr, pte);

I think you should flush tlb here.

Thanks, I'll update it.

Quoted 27 lines
Quoted 27 lines
arch/x86/mm/init_64.c +26 -0
+	} else {
+		next = pmd_addr_end(addr, end);
+
+		pmd = pmd_offset(pud, addr);
+		if (pmd_none(*pmd))
+			return next;
+
+		*pp = pmd_page(*pmd);
+		pmd_clear(pmd);
+	}
+
+	return next;
+}
+
+void __meminit
+vmemmap_kfree(struct page *memmap, unsigned long nr_pages)
+{
+	unsigned long addr = (unsigned long)memmap;
+	unsigned long end = (unsigned long)(memmap + nr_pages);
+	unsigned long next;
+	unsigned int order;
+	struct page *page;
+
+	for (; addr < end; addr = next) {
+		page = NULL;
+		next = find_and_clear_pte_page(addr, end, &page);
Quoted 8 lines

arch/x86/mm/init_64.c vmemmap_populate(struct page *start_page · line 1042

Quoted 5 lines
arch/x86/mm/init_64.c +5 -0
+		if (!page)
+			continue;
+
+		if (is_vmalloc_addr(page_address(page)))
+			vfree(page_address(page));

Hmm, the memory is allocated in vmemmap_alloc_block(), and the address can not be vmalloc address.

Does it mean the if sentence is unnecessary?

Quoted 11 lines

arch/x86/mm/init_64.c vmemmap_populate(struct page *start_page · line 1046

Quoted 5 lines
arch/x86/mm/init_64.c +4 -0
+		else {
+			order = next - addr;
+			free_pages((unsigned long)page_address(page),
+				   get_order(order));

OOPS. I think we cannot free pages here.

sizeof(struct page) is less than PAGE_SIZE. We store more than one struct page in the same page. If you free it here while the other struct page is in use, it is very dangerous.

The memory has page structures for hot-removed memory. So nobody is using these pages, since the hot-removed memory has been offlined.

Quoted 6 lines
Quoted 2 lines

The memory has page structures for hot-removed memory, but it may contain page structures for the other hot-added memory.

Yes. There may be such corner case. But when does the corner case appear? When removed memory is not aligned to PMD_SIZE/PAGE_SIZE, does the corner case appear? Do you know it?

It does not depend whether the removed memory is aligned to PMD_SIZE/PAGE_SIZE. If PAGE_SIZE % sizeof(struct page) != 0, this case will happen.

Thanks
Wen Congyang

Quoted 85 lines

Thank,
Yasuaki Ishimatsu

Quoted 78 lines

IIUC, If we use sparse-vmemmap, all page structures is stored here.

Thanks
Wen Congyang

Quoted 71 lines
Quoted 29 lines
Quoted 29 lines
arch/x86/mm/init_64.c +25 -0
+		}
+	}
+}
+
+void __meminit
+vmemmap_free_bootmem(struct page *memmap, unsigned long nr_pages)
+{
+	unsigned long addr = (unsigned long)memmap;
+	unsigned long end = (unsigned long)(memmap + nr_pages);
+	unsigned long next;
+	struct page *page;
+	unsigned long magic;
+
+	for (; addr < end; addr = next) {
+		page = NULL;
+		next = find_and_clear_pte_page(addr, end, &page);
+		if (!page)
+			continue;
+
+		magic = (unsigned long) page->lru.next;
+		if (magic == SECTION_INFO)
+			put_page_bootmem(page);
+	}
+}
+
   void __meminit
   register_page_bootmem_memmap(unsigned long section_nr, struct page *start_page,
   			     unsigned long size)
Quoted 8 lines

mm/memory_hotplug.c static int __meminit __add_section(int n · line 303

Quoted 6 lines

Index: linux-3.5-rc4/mm/memory_hotplug.c
===================================================================

mm/memory_hotplug.c +0 -0
@@ -303,6 +303,8 @@ static int __meminit __add_section(int n
   #ifdef CONFIG_SPARSEMEM_VMEMMAP

I think this line can be removed now.

I'll update it.

Thanks,
Yasuaki Ishimatsu

Quoted 24 lines

Thanks
Wen Congyang

Quoted 19 lines
mm/memory_hotplug.c +5 -0
   static int __remove_section(struct zone *zone, struct mem_section *ms)
   {
+	unsigned long flags;
+	struct pglist_data *pgdat = zone->zone_pgdat;
   	int ret;
 
   	if (!valid_section(ms))
@@ -310,6 +312,9 @@ static int __remove_section(struct zone
 
   	ret = unregister_memory_section(ms);
 
+	pgdat_resize_lock(pgdat, &flags);
+	sparse_remove_one_section(zone, ms);
+	pgdat_resize_unlock(pgdat, &flags);
   	return ret;
   }
   #else
Quoted 5 lines

--
To unsubscribe from this list: send the line "unsubscribe linux-acpi" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#40 lore raw

RFC PATCH v3 12/13 memory-hotplug : add node_device_release

In reply to Yasuaki Ishimatsu

When calling unregister_node(), the function shows following message at device_release().

Device 'node2' does not have a release() function, it is broken and must be fixed.

So the patch implements node_device_release()

CC David Rientjes <[email protected]>
CC Jiang Liu <[email protected]>
CC Len Brown <[email protected]>
CC Benjamin Herrenschmidt <[email protected]>
CC Paul Mackerras <[email protected]>
CC Christoph Lameter <[email protected]>
Cc Minchan Kim <[email protected]>
CC Andrew Morton <[email protected]>
CC KOSAKI Motohiro <[email protected]>
CC Wen Congyang <[email protected]>
---
 drivers/base/node.c |    7 +++++++
 1 file changed, 7 insertions(+)
drivers/base/node.c +7 -0
Index: linux-3.5-rc4/drivers/base/node.c
===================================================================
@@ -252,6 +252,12 @@ static inline void hugetlb_register_node
 static inline void hugetlb_unregister_node(struct node *node) {}
 #endif
 
+static void node_device_release(struct device *dev)
+{
+	struct node *node_dev = to_node(dev);
+
+	memset(node_dev, 0, sizeof(struct node));
+}
 
 /*
  * register_node - Setup a sysfs device for a node.
@@ -265,6 +271,7 @@ int register_node(struct node *node, int
 
 	node->dev.id = num;
 	node->dev.bus = &node_subsys;
+	node->dev.release = node_device_release;
 	error = device_register(&node->dev);
 
 	if (!error){
#41 lore raw

RFC PATCH v3 13/13 memory-hotplug : remove sysfs file of node

In reply to Yasuaki Ishimatsu

The patch adds node_set_offline() and unregister_one_node() to remove_memory() for removing sysfs file of node.

CC David Rientjes <[email protected]>
CC Jiang Liu <[email protected]>
CC Len Brown <[email protected]>
CC Benjamin Herrenschmidt <[email protected]>
CC Paul Mackerras <[email protected]>
CC Christoph Lameter <[email protected]>
Cc Minchan Kim <[email protected]>
CC Andrew Morton <[email protected]>
CC KOSAKI Motohiro <[email protected]>
CC Wen Congyang <[email protected]>
---
 mm/memory_hotplug.c |    5 +++++
 1 file changed, 5 insertions(+)
mm/memory_hotplug.c +5 -0
Index: linux-3.5-rc4/mm/memory_hotplug.c
===================================================================
@@ -702,6 +702,11 @@ int remove_memory(int nid, u64 start, u6
 	/* remove memmap entry */
 	firmware_map_remove(start, start + size - 1, "System RAM");
 
+	if (!node_present_pages(nid)) {
+		node_set_offline(nid);
+		unregister_one_node(nid);
+	}
+
 	__remove_pages(start >> PAGE_SHIFT, size >> PAGE_SHIFT);
 	unlock_memory_hotplug();
 	return 0;
#42 lore raw
In reply to Yasuaki Ishimatsu
Quoted 4 lines · Yasuaki Ishimatsu

Even if you apply these patches, you cannot remove the physical memory completely since these patches are still under development. I want you to cooperate to improve the physical memory hot-remove. So please review these patches and give your comment/idea.

Could you at least give a method on how you want to do physical memory removal? You would have to remove all objects from the range you want to physically remove. That is only possible under special circumstances and with a limited set of objects. Even if you exclusively use ZONE_MOVEABLE you still may get cases where pages are pinned for a long time.

I am not sure that these patches are useful unless we know where you are going with this. If we end up with a situation where we still cannot remove physical memory then this patchset is not helpful.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#43 lore raw

Hi Christoph,

Quoted 10 lines · Christoph Lameter
Quoted 4 lines · Yasuaki Ishimatsu

Even if you apply these patches, you cannot remove the physical memory completely since these patches are still under development. I want you to cooperate to improve the physical memory hot-remove. So please review these patches and give your comment/idea.

Could you at least give a method on how you want to do physical memory removal?

We plan to release a dynamic hardware partitionable system. It will be able to hot remove/add a system board which included memory and cpu. But as you know, Linux does not support memory hot-remove on x86 box. So I try to develop it.

Current plan to hot remove system board is to use container driver. Thus I define the system board in ACPI DSDT table as a container device. It have supported hot-add a container device. And if container device has _EJ0 ACPI method, "eject" file to remove the container device is prepared as follow:

# ls -l /sys/bus/acpi/devices/ACPI0004\:01/eject
--w-------. 1 root root 4096 Jul 10 18:19 /sys/bus/acpi/devices/ACPI0004:01/eject

When I hot-remove the container device, I echo 1 to the file as follow:

#echo 1 > /sys/bus/acpi/devices/ACPI0004\:02/eject

Then acpi_bus_trim() is called. And it calls acpi_memory_device_remove() for removing memory device. But the code does not do nothing. So I developed the continuation of the function.

Quoted 4 lines

You would have to remove all objects from the range you want to physically remove. That is only possible under special circumstances and with a limited set of objects. Even if you exclusively use ZONE_MOVEABLE you still may get cases where pages are pinned for a long time.

I know it. So my memory hot-remove plan is as follows:

1. hot-added a system board
All memory which included the system board is offline.

2. online the memory as removable page
The function has not supported yet. It is being developed by Lai as follow:
http://lkml.indiana.edu/hypermail/linux/kernel/1207.0/01478.html
If it is supported, I will be able to create movable memory.

3. hot-remove the memory by container device's eject file

Thanks,
Yasuaki Ishimatsu

Quoted 4 lines

I am not sure that these patches are useful unless we know where you are going with this. If we end up with a situation where we still cannot remove physical memory then this patchset is not helpful.

#44 lore raw
Quoted 46 lines · Yasuaki Ishimatsu

Hi Christoph,

Quoted 10 lines · Christoph Lameter
Quoted 4 lines · Yasuaki Ishimatsu

Even if you apply these patches, you cannot remove the physical memory completely since these patches are still under development. I want you to cooperate to improve the physical memory hot-remove. So please review these patches and give your comment/idea.

Could you at least give a method on how you want to do physical memory removal?

We plan to release a dynamic hardware partitionable system. It will be able to hot remove/add a system board which included memory and cpu. But as you know, Linux does not support memory hot-remove on x86 box. So I try to develop it.

Current plan to hot remove system board is to use container driver. Thus I define the system board in ACPI DSDT table as a container device. It have supported hot-add a container device. And if container device has _EJ0 ACPI method, "eject" file to remove the container device is prepared as follow:

# ls -l /sys/bus/acpi/devices/ACPI0004\:01/eject
--w-------. 1 root root 4096 Jul 10 18:19 /sys/bus/acpi/devices/ACPI0004:01/eject

When I hot-remove the container device, I echo 1 to the file as follow:

#echo 1 > /sys/bus/acpi/devices/ACPI0004\:02/eject

Then acpi_bus_trim() is called. And it calls acpi_memory_device_remove() for removing memory device. But the code does not do nothing. So I developed the continuation of the function.

Quoted 4 lines

You would have to remove all objects from the range you want to physically remove. That is only possible under special circumstances and with a limited set of objects. Even if you exclusively use ZONE_MOVEABLE you still may get cases where pages are pinned for a long time.

I know it. So my memory hot-remove plan is as follows:

1. hot-added a system board
All memory which included the system board is offline.

Quoted 6 lines

2. online the memory as removable page
The function has not supported yet. It is being developed by Lai as follow:
http://lkml.indiana.edu/hypermail/linux/kernel/1207.0/01478.html
If it is supported, I will be able to create movable memory.

3. hot-remove the memory by container device's eject file

We have implemented a prototype to do physical node (mem + CPU + IOH) hotplug for Itanium and is now porting it to x86. But with currently solution, memory hotplug functionality may cause 10-20% performance decrease because we concentrate all DMA/Normal memory to the first NUMA node, and all other NUMA nodes only hosts ZONE_MOVABLE. We are working on solution to minimize the performance drop now.

Quoted 11 lines

Thanks,
Yasuaki Ishimatsu

Quoted 4 lines

I am not sure that these patches are useful unless we know where you are going with this. If we end up with a situation where we still cannot remove physical memory then this patchset is not helpful.

#45 lore raw

Hi Jiang,

Quoted 52 lines · Jiang Liu
Quoted 51 lines · Yasuaki Ishimatsu

Hi Christoph,

Quoted 10 lines · Christoph Lameter
Quoted 4 lines · Yasuaki Ishimatsu

Even if you apply these patches, you cannot remove the physical memory completely since these patches are still under development. I want you to cooperate to improve the physical memory hot-remove. So please review these patches and give your comment/idea.

Could you at least give a method on how you want to do physical memory removal?

We plan to release a dynamic hardware partitionable system. It will be able to hot remove/add a system board which included memory and cpu. But as you know, Linux does not support memory hot-remove on x86 box. So I try to develop it.

Current plan to hot remove system board is to use container driver. Thus I define the system board in ACPI DSDT table as a container device. It have supported hot-add a container device. And if container device has _EJ0 ACPI method, "eject" file to remove the container device is prepared as follow:

# ls -l /sys/bus/acpi/devices/ACPI0004\:01/eject
--w-------. 1 root root 4096 Jul 10 18:19 /sys/bus/acpi/devices/ACPI0004:01/eject

When I hot-remove the container device, I echo 1 to the file as follow:

#echo 1 > /sys/bus/acpi/devices/ACPI0004\:02/eject

Then acpi_bus_trim() is called. And it calls acpi_memory_device_remove() for removing memory device. But the code does not do nothing. So I developed the continuation of the function.

Quoted 4 lines

You would have to remove all objects from the range you want to physically remove. That is only possible under special circumstances and with a limited set of objects. Even if you exclusively use ZONE_MOVEABLE you still may get cases where pages are pinned for a long time.

I know it. So my memory hot-remove plan is as follows:

1. hot-added a system board
All memory which included the system board is offline.

2. online the memory as removable page
The function has not supported yet. It is being developed by Lai as follow:
http://lkml.indiana.edu/hypermail/linux/kernel/1207.0/01478.html
If it is supported, I will be able to create movable memory.

Quoted 7 lines
Quoted 1 line

3. hot-remove the memory by container device's eject file

We have implemented a prototype to do physical node (mem + CPU + IOH) hotplug for Itanium and is now porting it to x86. But with currently solution, memory hotplug functionality may cause 10-20% performance decrease because we concentrate all DMA/Normal memory to the first NUMA node, and all other NUMA nodes only hosts ZONE_MOVABLE. We are working on solution to minimize the performance drop now.

Thank you for your interesting response.

I have a question. How do you move all other NUMA nodes to ZONE_MOVABLE? To use ZONE_MOVABLE, we need to use boot options like kernelcore or movablecore. But it is not enough, since the requested amount is spread evenly throughout all nodes in the system. So I think we do not have way to move all other NUMA node to ZONE_MOVABLE.

Thanks,
Yasuaki Ishimatsu

Quoted 14 lines
Quoted 12 lines

Thanks,
Yasuaki Ishimatsu

Quoted 4 lines

I am not sure that these patches are useful unless we know where you are going with this. If we end up with a situation where we still cannot remove physical memory then this patchset is not helpful.

#46 lore raw
Quoted 63 lines · Yasuaki Ishimatsu

Hi Jiang,

Quoted 52 lines · Jiang Liu
Quoted 51 lines · Yasuaki Ishimatsu

Hi Christoph,

Quoted 10 lines · Christoph Lameter
Quoted 4 lines · Yasuaki Ishimatsu

Even if you apply these patches, you cannot remove the physical memory completely since these patches are still under development. I want you to cooperate to improve the physical memory hot-remove. So please review these patches and give your comment/idea.

Could you at least give a method on how you want to do physical memory removal?

We plan to release a dynamic hardware partitionable system. It will be able to hot remove/add a system board which included memory and cpu. But as you know, Linux does not support memory hot-remove on x86 box. So I try to develop it.

Current plan to hot remove system board is to use container driver. Thus I define the system board in ACPI DSDT table as a container device. It have supported hot-add a container device. And if container device has _EJ0 ACPI method, "eject" file to remove the container device is prepared as follow:

# ls -l /sys/bus/acpi/devices/ACPI0004\:01/eject
--w-------. 1 root root 4096 Jul 10 18:19 /sys/bus/acpi/devices/ACPI0004:01/eject

When I hot-remove the container device, I echo 1 to the file as follow:

#echo 1 > /sys/bus/acpi/devices/ACPI0004\:02/eject

Then acpi_bus_trim() is called. And it calls acpi_memory_device_remove() for removing memory device. But the code does not do nothing. So I developed the continuation of the function.

Quoted 4 lines

You would have to remove all objects from the range you want to physically remove. That is only possible under special circumstances and with a limited set of objects. Even if you exclusively use ZONE_MOVEABLE you still may get cases where pages are pinned for a long time.

I know it. So my memory hot-remove plan is as follows:

1. hot-added a system board
All memory which included the system board is offline.

2. online the memory as removable page
The function has not supported yet. It is being developed by Lai as follow:
http://lkml.indiana.edu/hypermail/linux/kernel/1207.0/01478.html
If it is supported, I will be able to create movable memory.

Quoted 7 lines
Quoted 1 line

3. hot-remove the memory by container device's eject file

We have implemented a prototype to do physical node (mem + CPU + IOH) hotplug for Itanium and is now porting it to x86. But with currently solution, memory hotplug functionality may cause 10-20% performance decrease because we concentrate all DMA/Normal memory to the first NUMA node, and all other NUMA nodes only hosts ZONE_MOVABLE. We are working on solution to minimize the performance drop now.

Quoted 7 lines

Thank you for your interesting response.

I have a question. How do you move all other NUMA nodes to ZONE_MOVABLE? To use ZONE_MOVABLE, we need to use boot options like kernelcore or movablecore. But it is not enough, since the requested amount is spread evenly throughout all nodes in the system. So I think we do not have way to move all other NUMA node to ZONE_MOVABLE.

We have modified the ZONE_MOVABLE spreading and bootmem allocation. If the kernelcore or movablecore kernel parameters are present, we follow current behavior. If those parameter are absent and the platform supports physical hotplug, we will concentrate DMA/NORMAL memory to specific nodes.

Quoted 21 lines

Thanks,
Yasuaki Ishimatsu

Quoted 14 lines
Quoted 12 lines

Thanks,
Yasuaki Ishimatsu

Quoted 4 lines

I am not sure that these patches are useful unless we know where you are going with this. If we end up with a situation where we still cannot remove physical memory then this patchset is not helpful.

#47 lore raw

Hi Jiang,

Quoted 71 lines · Jiang Liu
Quoted 63 lines · Yasuaki Ishimatsu

Hi Jiang,

Quoted 52 lines · Jiang Liu
Quoted 51 lines · Yasuaki Ishimatsu

Hi Christoph,

Quoted 10 lines · Christoph Lameter
Quoted 4 lines · Yasuaki Ishimatsu

Even if you apply these patches, you cannot remove the physical memory completely since these patches are still under development. I want you to cooperate to improve the physical memory hot-remove. So please review these patches and give your comment/idea.

Could you at least give a method on how you want to do physical memory removal?

We plan to release a dynamic hardware partitionable system. It will be able to hot remove/add a system board which included memory and cpu. But as you know, Linux does not support memory hot-remove on x86 box. So I try to develop it.

Current plan to hot remove system board is to use container driver. Thus I define the system board in ACPI DSDT table as a container device. It have supported hot-add a container device. And if container device has _EJ0 ACPI method, "eject" file to remove the container device is prepared as follow:

# ls -l /sys/bus/acpi/devices/ACPI0004\:01/eject
--w-------. 1 root root 4096 Jul 10 18:19 /sys/bus/acpi/devices/ACPI0004:01/eject

When I hot-remove the container device, I echo 1 to the file as follow:

#echo 1 > /sys/bus/acpi/devices/ACPI0004\:02/eject

Then acpi_bus_trim() is called. And it calls acpi_memory_device_remove() for removing memory device. But the code does not do nothing. So I developed the continuation of the function.

Quoted 4 lines

You would have to remove all objects from the range you want to physically remove. That is only possible under special circumstances and with a limited set of objects. Even if you exclusively use ZONE_MOVEABLE you still may get cases where pages are pinned for a long time.

I know it. So my memory hot-remove plan is as follows:

1. hot-added a system board
All memory which included the system board is offline.

2. online the memory as removable page
The function has not supported yet. It is being developed by Lai as follow:
http://lkml.indiana.edu/hypermail/linux/kernel/1207.0/01478.html
If it is supported, I will be able to create movable memory.

Quoted 7 lines
Quoted 1 line

3. hot-remove the memory by container device's eject file

We have implemented a prototype to do physical node (mem + CPU + IOH) hotplug for Itanium and is now porting it to x86. But with currently solution, memory hotplug functionality may cause 10-20% performance decrease because we concentrate all DMA/Normal memory to the first NUMA node, and all other NUMA nodes only hosts ZONE_MOVABLE. We are working on solution to minimize the performance drop now.

Quoted 7 lines

Thank you for your interesting response.

I have a question. How do you move all other NUMA nodes to ZONE_MOVABLE? To use ZONE_MOVABLE, we need to use boot options like kernelcore or movablecore. But it is not enough, since the requested amount is spread evenly throughout all nodes in the system. So I think we do not have way to move all other NUMA node to ZONE_MOVABLE.

Quoted 4 lines

We have modified the ZONE_MOVABLE spreading and bootmem allocation. If the kernelcore or movablecore kernel parameters are present, we follow current behavior. If those parameter are absent and the platform supports physical hotplug, we will concentrate DMA/NORMAL memory to specific nodes.

That's interesting. I want to know more details, if you do not mind. Current kernel doesn't do the behavior, does it? So I think you have some patches for changing the behavior. Will you merge these patches into community kernel?

Thanks,
Yasuaki Ishimatsu

Quoted 11 lines
Quoted 11 lines

Thanks,
Yasuaki Ishimatsu

Quoted 6 lines
Quoted 6 lines

Thanks,
Yasuaki Ishimatsu

Quoted 1 line
Quoted 13 lines
Quoted 11 lines
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines

I am not sure that these patches are useful unless we know where you are going with this. If we end up with a situation where we still cannot remove physical memory then this patchset is not helpful.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#48 lore raw
Quoted 79 lines · Yasuaki Ishimatsu

Hi Jiang,

Quoted 71 lines · Jiang Liu
Quoted 63 lines · Yasuaki Ishimatsu

Hi Jiang,

Quoted 52 lines · Jiang Liu
Quoted 51 lines · Yasuaki Ishimatsu

Hi Christoph,

Quoted 10 lines · Christoph Lameter
Quoted 4 lines · Yasuaki Ishimatsu

Even if you apply these patches, you cannot remove the physical memory completely since these patches are still under development. I want you to cooperate to improve the physical memory hot-remove. So please review these patches and give your comment/idea.

Could you at least give a method on how you want to do physical memory removal?

We plan to release a dynamic hardware partitionable system. It will be able to hot remove/add a system board which included memory and cpu. But as you know, Linux does not support memory hot-remove on x86 box. So I try to develop it.

Current plan to hot remove system board is to use container driver. Thus I define the system board in ACPI DSDT table as a container device. It have supported hot-add a container device. And if container device has _EJ0 ACPI method, "eject" file to remove the container device is prepared as follow:

# ls -l /sys/bus/acpi/devices/ACPI0004\:01/eject
--w-------. 1 root root 4096 Jul 10 18:19 /sys/bus/acpi/devices/ACPI0004:01/eject

When I hot-remove the container device, I echo 1 to the file as follow:

#echo 1 > /sys/bus/acpi/devices/ACPI0004\:02/eject

Then acpi_bus_trim() is called. And it calls acpi_memory_device_remove() for removing memory device. But the code does not do nothing. So I developed the continuation of the function.

Quoted 4 lines

You would have to remove all objects from the range you want to physically remove. That is only possible under special circumstances and with a limited set of objects. Even if you exclusively use ZONE_MOVEABLE you still may get cases where pages are pinned for a long time.

I know it. So my memory hot-remove plan is as follows:

1. hot-added a system board
All memory which included the system board is offline.

2. online the memory as removable page
The function has not supported yet. It is being developed by Lai as follow:
http://lkml.indiana.edu/hypermail/linux/kernel/1207.0/01478.html
If it is supported, I will be able to create movable memory.

Quoted 7 lines
Quoted 1 line

3. hot-remove the memory by container device's eject file

We have implemented a prototype to do physical node (mem + CPU + IOH) hotplug for Itanium and is now porting it to x86. But with currently solution, memory hotplug functionality may cause 10-20% performance decrease because we concentrate all DMA/Normal memory to the first NUMA node, and all other NUMA nodes only hosts ZONE_MOVABLE. We are working on solution to minimize the performance drop now.

Quoted 7 lines

Thank you for your interesting response.

I have a question. How do you move all other NUMA nodes to ZONE_MOVABLE? To use ZONE_MOVABLE, we need to use boot options like kernelcore or movablecore. But it is not enough, since the requested amount is spread evenly throughout all nodes in the system. So I think we do not have way to move all other NUMA node to ZONE_MOVABLE.

Quoted 4 lines

We have modified the ZONE_MOVABLE spreading and bootmem allocation. If the kernelcore or movablecore kernel parameters are present, we follow current behavior. If those parameter are absent and the platform supports physical hotplug, we will concentrate DMA/NORMAL memory to specific nodes.

Quoted 4 lines

That's interesting. I want to know more details, if you do not mind. Current kernel doesn't do the behavior, does it? So I think you have some patches for changing the behavior. Will you merge these patches into community kernel?

Yeah, we do have patches for that. But it's still prototype, still much work needed before sending them to the community.
Currently I'm trying to send out patches for an ACPI based system device hotplug framework, which will support processor, memory, IOH and node hotplug in a unified way. After that, I will prepare the memory hotplug code.

Thanks!
Gerry

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#49 lore raw
In reply to Yasuaki Ishimatsu

At 07/09/2012 06:21 PM, Yasuaki Ishimatsu Wrote:

Quoted 43 lines

This patch series aims to support physical memory hot-remove.

[RFC PATCH v3 1/13] memory-hotplug : rename remove_memory to offline_memory
[RFC PATCH v3 2/13] memory-hotplug : add physical memory hotplug code to acpi_memory_device_remove [RFC PATCH v3 3/13] memory-hotplug : unify argument of firmware_map_add_early/hotplug [RFC PATCH v3 4/13] memory-hotplug : remove /sys/firmware/memmap/X sysfs
[RFC PATCH v3 5/13] memory-hotplug : does not release memory region in PAGES_PER_SECTION chunks [RFC PATCH v3 6/13] memory-hotplug : add memory_block_release
[RFC PATCH v3 7/13] memory-hotplug : remove_memory calls __remove_pages
[RFC PATCH v3 8/13] memory-hotplug : check page type in get_page_bootmem
[RFC PATCH v3 9/13] memory-hotplug : move register_page_bootmem_info_node and put_page_bootmem for
sparse-vmemmap
[RFC PATCH v3 10/13] memory-hotplug : implement register_page_bootmem_info_section of sparse-vmemmap [RFC PATCH v3 11/13] memory-hotplug : free memmap of sparse-vmemmap
[RFC PATCH v3 12/13] memory-hotplug : add node_device_release
[RFC PATCH v3 13/13] memory-hotplug : remove sysfs file of node

Even if you apply these patches, you cannot remove the physical memory completely since these patches are still under development. I want you to cooperate to improve the physical memory hot-remove. So please review these patches and give your comment/idea.

The patches can free/remove following things:

- acpi_memory_info : [RFC PATCH 2/13]
- /sys/firmware/memmap/X/{end, start, type} : [RFC PATCH 4/13]

  - iomem_resource                            : [RFC PATCH 5/13]
  - mem_section and related sysfs files       : [RFC PATCH 6-11/13]
  - node and related sysfs files              : [RFC PATCH 12-13/13]

The patches cannot do following things yet:

- page table of removed memory

If you find lack of function for physical memory hot-remove, please let me know.

change log of v3:
* rebase to 3.5.0-rc6

[RFC PATCH v2 2/13]
* remove extra kobject_put()

Quoted 7 lines

* The patch was commented by Wen. Wen's comment is "acpi_memory_device_remove() should ignore a return value of remove_memory() since caller does not care the return value". But I did not change it since I think caller should care the return value. And I am trying to fix it as follow:

https://lkml.org/lkml/2012/7/5/624

acpi_memory_device_remove() will be called not only when we write 1 to /sys/bus/acpi/devices/PNP0C80:XX/eject. When we unbind it from the driver or remove the module acpi_memhotplug, this function will be called too.

I will check whether your patch can work for these two cases.

Thanks
Wen Congyang

Quoted 38 lines

[RFC PATCH v2 4/13]
* remove a firmware_memmap_entry allocated by kzmalloc()

change log of v2:
[RFC PATCH v2 2/13]
* check whether memory block is offline or not before calling offline_memory()
* check whether section is valid or not in is_memblk_offline()
* call kobject_put() for each memory_block in is_memblk_offline()

[RFC PATCH v2 3/13]
* unify the end argument of firmware_map_add_early/hotplug

[RFC PATCH v2 4/13]
* add release_firmware_map_entry() for freeing firmware_map_entry

[RFC PATCH v2 6/13]
* add release_memory_block() for freeing memory_block

[RFC PATCH v2 11/13]
* fix wrong arguments of free_pages()

---
 arch/powerpc/platforms/pseries/hotplug-memory.c |   16 +-
 arch/x86/mm/init_64.c                           |  144 ++++++++++++++++++++++++
 drivers/acpi/acpi_memhotplug.c                  |   28 ++++
 drivers/base/memory.c                           |   54 ++++++++-
 drivers/base/node.c                             |    7 +
 drivers/firmware/memmap.c                       |   78 ++++++++++++-
 include/linux/firmware-map.h                    |    6 +
 include/linux/memory.h                          |    5
 include/linux/memory_hotplug.h                  |   17 --
 include/linux/mm.h                              |    5
 mm/memory_hotplug.c                             |   98 ++++++++++++----
 mm/sparse.c                                     |    5
 12 files changed, 414 insertions(+), 49 deletions(-)
#50 lore raw

At 07/11/2012 09:52 AM, Wen Congyang Wrote:

Quoted 50 lines

At 07/09/2012 06:21 PM, Yasuaki Ishimatsu Wrote:

Quoted 49 lines

This patch series aims to support physical memory hot-remove.

[RFC PATCH v3 1/13] memory-hotplug : rename remove_memory to offline_memory
[RFC PATCH v3 2/13] memory-hotplug : add physical memory hotplug code to acpi_memory_device_remove [RFC PATCH v3 3/13] memory-hotplug : unify argument of firmware_map_add_early/hotplug [RFC PATCH v3 4/13] memory-hotplug : remove /sys/firmware/memmap/X sysfs
[RFC PATCH v3 5/13] memory-hotplug : does not release memory region in PAGES_PER_SECTION chunks [RFC PATCH v3 6/13] memory-hotplug : add memory_block_release
[RFC PATCH v3 7/13] memory-hotplug : remove_memory calls __remove_pages
[RFC PATCH v3 8/13] memory-hotplug : check page type in get_page_bootmem
[RFC PATCH v3 9/13] memory-hotplug : move register_page_bootmem_info_node and put_page_bootmem for
sparse-vmemmap
[RFC PATCH v3 10/13] memory-hotplug : implement register_page_bootmem_info_section of sparse-vmemmap [RFC PATCH v3 11/13] memory-hotplug : free memmap of sparse-vmemmap
[RFC PATCH v3 12/13] memory-hotplug : add node_device_release
[RFC PATCH v3 13/13] memory-hotplug : remove sysfs file of node

Even if you apply these patches, you cannot remove the physical memory completely since these patches are still under development. I want you to cooperate to improve the physical memory hot-remove. So please review these patches and give your comment/idea.

The patches can free/remove following things:

- acpi_memory_info : [RFC PATCH 2/13]
- /sys/firmware/memmap/X/{end, start, type} : [RFC PATCH 4/13]

  - iomem_resource                            : [RFC PATCH 5/13]
  - mem_section and related sysfs files       : [RFC PATCH 6-11/13]
  - node and related sysfs files              : [RFC PATCH 12-13/13]

The patches cannot do following things yet:

- page table of removed memory

If you find lack of function for physical memory hot-remove, please let me know.

change log of v3:
* rebase to 3.5.0-rc6

[RFC PATCH v2 2/13]
* remove extra kobject_put()

* The patch was commented by Wen. Wen's comment is "acpi_memory_device_remove() should ignore a return value of remove_memory() since caller does not care the return value". But I did not change it since I think caller should care the return value. And I am trying to fix it as follow:

Quoted 8 lines
Quoted 1 line
     https://lkml.org/lkml/2012/7/5/624

acpi_memory_device_remove() will be called not only when we write 1 to /sys/bus/acpi/devices/PNP0C80:XX/eject. When we unbind it from the driver or remove the module acpi_memhotplug, this function will be called too.

I will check whether your patch can work for these two cases.

I have checked it, and I think your patch can not work for these 2 cases.

When we unbind the device from the driver(write device name to /sys/bus/acpi/drivers/acpi_memhotplug/unbind), driver_unbind() will be called. This function does not care the return value.

When we remove the module acpi_memhotplug, acpi_memory_device_exit() will be called. This function does not care the return value too.

I don't know whether there are some other cases that acpi_memory_device_remove() will be called.

Thanks
Wen Congyang

Quoted 43 lines

Thanks
Wen Congyang

Quoted 22 lines

[RFC PATCH v2 4/13]
* remove a firmware_memmap_entry allocated by kzmalloc()

change log of v2:
[RFC PATCH v2 2/13]
* check whether memory block is offline or not before calling offline_memory()
* check whether section is valid or not in is_memblk_offline()
* call kobject_put() for each memory_block in is_memblk_offline()

[RFC PATCH v2 3/13]
* unify the end argument of firmware_map_add_early/hotplug

[RFC PATCH v2 4/13]
* add release_firmware_map_entry() for freeing firmware_map_entry

[RFC PATCH v2 6/13]
* add release_memory_block() for freeing memory_block

[RFC PATCH v2 11/13]
* fix wrong arguments of free_pages()

Quoted 16 lines
---
 arch/powerpc/platforms/pseries/hotplug-memory.c |   16 +-
 arch/x86/mm/init_64.c                           |  144 ++++++++++++++++++++++++
 drivers/acpi/acpi_memhotplug.c                  |   28 ++++
 drivers/base/memory.c                           |   54 ++++++++-
 drivers/base/node.c                             |    7 +
 drivers/firmware/memmap.c                       |   78 ++++++++++++-
 include/linux/firmware-map.h                    |    6 +
 include/linux/memory.h                          |    5
 include/linux/memory_hotplug.h                  |   17 --
 include/linux/mm.h                              |    5
 mm/memory_hotplug.c                             |   98 ++++++++++++----
 mm/sparse.c                                     |    5
 12 files changed, 414 insertions(+), 49 deletions(-)
Quoted 5 lines

--
To unsubscribe from this list: send the line "unsubscribe linux-acpi" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>