[PATCH v3 0/5]

PATCH In review Alex Chiang 2 participants 7 messages Nov 10, 2009 – Nov 12, 2009

Subsystems (3)

Memory management Documentation Driver core

lore

Share & tools

lore

Alex Chiang

author

2009-11-10 22:36 UTC #0 lore raw

v3 cover [PATCH v3 0/5]

This is v3 of the series.

I based it off of Linus's latest tree.

I did not include David Rientjes's "mm: slab allocate memory section nodemask for large systems" patch in my series, since it's not necessarily related.

Please consider for inclusion for the next merge window (v2.6.33).

Thanks,
/ac

v2 -> v3:
- rebased to Linus's latest tree (799dd75b)
- Added David Rientjes's Acked-by: flags
- dropped S390 cc's, since they are unaffected by this series

v1 -> v2: http://thread.gmane.org/gmane.linux.kernel.mm/40084/
Address David Rientjes's comments
- check return value of sysfs_create_link in register_cpu_under_node
- do /not/ convert [un]register_cpu_under_node to return void, since sparse starts whinging if you ignore sysfs_create_link()'s return value and working around sparse makes the code ugly
- adjust documentation

Added S390 maintainers to cc: for patch [1/5] as per Kame-san's suggestion. S390 may map a memory section to more than one node, causing this series to break.

---

Alex Chiang (5):
mm: add numa node symlink for memory section in sysfs
mm: refactor register_cpu_under_node()
mm: refactor unregister_cpu_under_node()
mm: add numa node symlink for cpu devices in sysfs
Documentation: ABI: /sys/devices/system/cpu/cpu#/node

 Documentation/ABI/testing/sysfs-devices-memory     |   14 ++++-
 Documentation/ABI/testing/sysfs-devices-system-cpu |   14 +++++
 Documentation/memory-hotplug.txt                   |   11 ++--
 drivers/base/node.c                                |   58 ++++++++++++++------
 4 files changed, 76 insertions(+), 21 deletions(-)

Alex Chiang

author

2009-11-10 22:36 UTC #1 lore raw

PATCH v3 1/5 mm: add numa node symlink for memory section in sysfs

Commit c04fc586c (mm: show node to memory section relationship with symlinks in sysfs) created symlinks from nodes to memory sections, e.g.

/sys/devices/system/node/node1/memory135 -> ../../memory/memory135

If you're examining the memory section though and are wondering what node it might belong to, you can find it by grovelling around in sysfs, but it's a little cumbersome.

Add a reverse symlink for each memory section that points back to the node to which it belongs.

Cc Gary Hade <[email protected]>

Cc Badari Pulavarty <[email protected]>

Cc Ingo Molnar <[email protected]>

Acked-by David Rientjes <[email protected]>

Signed-off-by Alex Chiang <[email protected]>

---

 Documentation/ABI/testing/sysfs-devices-memory |   14 +++++++++++++-
 Documentation/memory-hotplug.txt               |   11 +++++++----
 drivers/base/node.c                            |   11 ++++++++++-
 3 files changed, 30 insertions(+), 6 deletions(-)

Documentation/ABI/testing/sysfs-devices-memory +13 -1

index 9fe91c0..bf1627b 100644
@@ -60,6 +60,19 @@ Description:
 Users:		hotplug memory remove tools
 		https://w3.opensource.ibm.com/projects/powerpc-utils/
 
+
+What:		/sys/devices/system/memoryX/nodeY
+Date:		October 2009
+Contact:	Linux Memory Management list <[email protected]>
+Description:
+		When CONFIG_NUMA is enabled, a symbolic link that
+		points to the corresponding NUMA node directory.
+
+		For example, the following symbolic link is created for
+		memory section 9 on node0:
+		/sys/devices/system/memory/memory9/node0 -> ../../node/node0
+
+
 What:		/sys/devices/system/node/nodeX/memoryY
 Date:		September 2008
 Contact:	Gary Hade <[email protected]>

@@ -70,4 +83,3 @@ Description:
 		memory section directory.  For example, the following symbolic
 		link is created for memory section 9 on node0.
 		/sys/devices/system/node/node0/memory9 -> ../../memory/memory9
-

Documentation/memory-hotplug.txt +7 -4

index bbc8a6a..57e7e9c 100644
@@ -160,12 +160,15 @@ Under each section, you can see 4 files.
 NOTE:
   These directories/files appear after physical memory hotplug phase.
 
-If CONFIG_NUMA is enabled the
-/sys/devices/system/memory/memoryXXX memory section
-directories can also be accessed via symbolic links located in
-the /sys/devices/system/node/node* directories.  For example:
+If CONFIG_NUMA is enabled the memoryXXX/ directories can also be accessed
+via symbolic links located in the /sys/devices/system/node/node* directories.
+
+For example:
 /sys/devices/system/node/node0/memory9 -> ../../memory/memory9
 
+A backlink will also be created:
+/sys/devices/system/memory/memory9/node0 -> ../../node/node0
+
 --------------------------------
 4. Physical memory hot-add phase
 --------------------------------

drivers/base/node.c +10 -1

index 1fe5536..3108b21 100644
@@ -268,6 +268,7 @@ static int get_nid_for_pfn(unsigned long pfn)
 /* register memory section under specified node if it spans that node */
 int register_mem_sect_under_node(struct memory_block *mem_blk, int nid)
 {
+	int ret;
 	unsigned long pfn, sect_start_pfn, sect_end_pfn;
 
 	if (!mem_blk)

@@ -284,9 +285,15 @@ int register_mem_sect_under_node(struct memory_block *mem_blk, int nid)
 			continue;
 		if (page_nid != nid)
 			continue;
-		return sysfs_create_link_nowarn(&node_devices[nid].sysdev.kobj,
+		ret = sysfs_create_link_nowarn(&node_devices[nid].sysdev.kobj,
 					&mem_blk->sysdev.kobj,
 					kobject_name(&mem_blk->sysdev.kobj));
+		if (ret)
+			return ret;
+
+		return sysfs_create_link_nowarn(&mem_blk->sysdev.kobj,
+				&node_devices[nid].sysdev.kobj,
+				kobject_name(&node_devices[nid].sysdev.kobj));
 	}
 	/* mem section does not span the specified node */
 	return 0;

@@ -315,6 +322,8 @@ int unregister_mem_sect_under_nodes(struct memory_block *mem_blk)
 			continue;
 		sysfs_remove_link(&node_devices[nid].sysdev.kobj,
 			 kobject_name(&mem_blk->sysdev.kobj));
+		sysfs_remove_link(&mem_blk->sysdev.kobj,
+			 kobject_name(&node_devices[nid].sysdev.kobj));
 	}
 	return 0;
 }

Gary Hade

Tested-by Acked-by on v3 1/5

2009-11-12 17:26 UTC #2 lore raw

Re: [PATCH v3 1/5] mm: add numa node symlink for memory section in sysfs

Quoted 11 lines · Alex Chiang

Commit c04fc586c (mm: show node to memory section relationship with symlinks in sysfs) created symlinks from nodes to memory sections, e.g.
/sys/devices/system/node/node1/memory135 -> ../../memory/memory135
If you're examining the memory section though and are wondering what node it might belong to, you can find it by grovelling around in sysfs, but it's a little cumbersome.
Add a reverse symlink for each memory section that points back to the node to which it belongs.

Hi Alex,
I'm kinda late to the party but I finally had a chance to review and try it out on one of our systems today. Looks good to me.

Tested-by Gary Hade <[email protected]>

Acked-by Gary Hade <[email protected]>

Gary

Signature · 6 lines

Gary Hade
System x Enablement
IBM Linux Technology Center
503-578-4503  IBM T/L: 775-4503
[email protected]
http://www.ibm.com/linux/ltc

Alex Chiang

author

2009-11-10 22:36 UTC #3 lore raw

PATCH v3 2/5 mm: refactor register_cpu_under_node()

In reply to Alex Chiang

By returning early if the node is not online, we can unindent the interesting code by one level.

No functional change.

Signed-off-by Alex Chiang <[email protected]>

---

 drivers/base/node.c |   20 +++++++++++---------
 1 files changed, 11 insertions(+), 9 deletions(-)

drivers/base/node.c +11 -9

index 3108b21..ef7dd22 100644
@@ -227,16 +227,18 @@ struct node node_devices[MAX_NUMNODES];
  */
 int register_cpu_under_node(unsigned int cpu, unsigned int nid)
 {
-	if (node_online(nid)) {
-		struct sys_device *obj = get_cpu_sysdev(cpu);
-		if (!obj)
-			return 0;
-		return sysfs_create_link(&node_devices[nid].sysdev.kobj,
-					 &obj->kobj,
-					 kobject_name(&obj->kobj));
-	 }
+	struct sys_device *obj;
 
-	return 0;
+	if (!node_online(nid))
+		return 0;
+
+	obj = get_cpu_sysdev(cpu);
+	if (!obj)
+		return 0;
+
+	return sysfs_create_link(&node_devices[nid].sysdev.kobj,
+				&obj->kobj,
+				kobject_name(&obj->kobj));
 }
 
 int unregister_cpu_under_node(unsigned int cpu, unsigned int nid)

Alex Chiang

author

2009-11-10 22:36 UTC #4 lore raw

PATCH v3 3/5 mm: refactor unregister_cpu_under_node()

In reply to Alex Chiang

By returning early if the node is not online, we can unindent the interesting code by two levels.

No functional change.

Signed-off-by Alex Chiang <[email protected]>

---

 drivers/base/node.c |   18 ++++++++++++------
 1 files changed, 12 insertions(+), 6 deletions(-)

drivers/base/node.c +12 -6

index ef7dd22..ffda067 100644
@@ -243,12 +243,18 @@ int register_cpu_under_node(unsigned int cpu, unsigned int nid)
 
 int unregister_cpu_under_node(unsigned int cpu, unsigned int nid)
 {
-	if (node_online(nid)) {
-		struct sys_device *obj = get_cpu_sysdev(cpu);
-		if (obj)
-			sysfs_remove_link(&node_devices[nid].sysdev.kobj,
-					 kobject_name(&obj->kobj));
-	}
+	struct sys_device *obj;
+
+	if (!node_online(nid))
+		return 0;
+
+	obj = get_cpu_sysdev(cpu);
+	if (!obj)
+		return 0;
+
+	sysfs_remove_link(&node_devices[nid].sysdev.kobj,
+			  kobject_name(&obj->kobj));
+
 	return 0;
 }

Alex Chiang

author

2009-11-10 22:36 UTC #5 lore raw

PATCH v3 4/5 mm: add numa node symlink for cpu devices in sysfs

In reply to Alex Chiang

You can discover which CPUs belong to a NUMA node by examining /sys/devices/system/node/node#/

However, it's not convenient to go in the other direction, when looking at /sys/devices/system/cpu/cpu#/

Yes, you can muck about in sysfs, but adding these symlinks makes life a lot more convenient.

Acked-by David Rientjes <[email protected]>

Signed-off-by Alex Chiang <[email protected]>

---

 drivers/base/node.c |   11 ++++++++++-
 1 files changed, 10 insertions(+), 1 deletions(-)

drivers/base/node.c +10 -1

index ffda067..24fa962 100644
@@ -227,6 +227,7 @@ struct node node_devices[MAX_NUMNODES];
  */
 int register_cpu_under_node(unsigned int cpu, unsigned int nid)
 {
+	int ret;
 	struct sys_device *obj;
 
 	if (!node_online(nid))

@@ -236,9 +237,15 @@ int register_cpu_under_node(unsigned int cpu, unsigned int nid)
 	if (!obj)
 		return 0;
 
-	return sysfs_create_link(&node_devices[nid].sysdev.kobj,
+	ret = sysfs_create_link(&node_devices[nid].sysdev.kobj,
 				&obj->kobj,
 				kobject_name(&obj->kobj));
+	if (ret)
+		return ret;
+
+	return sysfs_create_link(&obj->kobj,
+				 &node_devices[nid].sysdev.kobj,
+				 kobject_name(&node_devices[nid].sysdev.kobj));
 }
 
 int unregister_cpu_under_node(unsigned int cpu, unsigned int nid)

@@ -254,6 +261,8 @@ int unregister_cpu_under_node(unsigned int cpu, unsigned int nid)
 
 	sysfs_remove_link(&node_devices[nid].sysdev.kobj,
 			  kobject_name(&obj->kobj));
+	sysfs_remove_link(&obj->kobj,
+			  kobject_name(&node_devices[nid].sysdev.kobj));
 
 	return 0;
 }

Alex Chiang

author

2009-11-10 22:37 UTC #6 lore raw

PATCH v3 5/5 Documentation: ABI: /sys/devices/system/cpu/cpu#/node

In reply to Alex Chiang

Describe NUMA node symlink created for CPUs when CONFIG_NUMA is set.

Cc Greg KH <[email protected]>

Cc Randy Dunlap <[email protected]>

Signed-off-by Alex Chiang <[email protected]>

---

 Documentation/ABI/testing/sysfs-devices-system-cpu |   14 ++++++++++++++
 1 files changed, 14 insertions(+), 0 deletions(-)

Documentation/ABI/testing/sysfs-devices-system-cpu +14 -0

index 5aace16..1671634 100644
@@ -94,6 +94,20 @@ Description:	Discover and change the online state of a CPU.
 		For more information, please read Documentation/cpu-hotplug.txt
 
 
+What:		/sys/devices/system/cpu/cpu#/node
+Date:		October 2009
+Contact:	Linux memory management mailing list <[email protected]>
+Description:	Discover NUMA node a CPU belongs to
+
+		When CONFIG_NUMA is enabled, a symbolic link that points
+		to the corresponding NUMA node directory.
+
+		For example, the following symlink is created for cpu42
+		in NUMA node 2:
+
+		/sys/devices/system/cpu/cpu42/node2 -> ../../node/node2
+
+
 What:		/sys/devices/system/cpu/cpu#/topology/core_id
 		/sys/devices/system/cpu/cpu#/topology/core_siblings
 		/sys/devices/system/cpu/cpu#/topology/core_siblings_list