Skip to contentSkip to conversation

find the node that a cpu belongs to

PATCH Kornilios Kourtis 3 participants 6 messages Mar 17, 2009 – Mar 19, 2009

lore
Share & tools
lore
#0 lore raw

Hi,

I was wondering if you would be interested in adding a libnuma function for determing the node that a cpu belongs to.

Something like this maybe ?

I could write a more complete patch if needed
(libnuma.h,version.ldscript,numa.3 ?)

libnuma.c renamed from libnuma.c.2 +30 -0
@@ -1784,3 +1784,34 @@ err:
 	numa_bitmask_free(mask);
 	return NULL;
 }
+
+int numa_node_from_cpu_v2(int cpu)
+{
+	struct bitmask *bmp;
+	int ncpus, nnodes, node, ret;
+
+	ncpus = numa_num_possible_cpus();
+	if (cpu > ncpus){
+		errno = ERANGE;
+		return -1;
+	}
+
+	bmp = numa_bitmask_alloc(ncpus);
+	nnodes = numa_num_configured_nodes();
+	for (node = 0; node < nnodes; node++){
+		numa_node_to_cpus_v2(node, bmp);
+		if (numa_bitmask_isbitset(bmp, cpu)){
+			ret = node;
+			goto end;
+		}
+	}
+
+	/* I don't think this is supposed to happen */
+	ret = -1;
+	errno = EINVAL;
+end:
+	numa_bitmask_free(bmp);
+	return ret;
+}
+__asm__(".symver numa_node_from_cpu_v2,numa_node_from_cpu@@libnuma_1.2");
Signature · 1 line
Kornilios Kourtis
#1 lore raw
Quoted 4 lines · Kornilios Kourtis

Hi,

I was wondering if you would be interested in adding a libnuma function for determing the node that a cpu belongs to.

Concrete use case please. Why would you want to do that in a program?

-Andi

Signature · 1 line
[email protected] -- Speaking for myself only.
#2 lore raw
Quoted 7 lines · Andi Kleen
Quoted 4 lines · Kornilios Kourtis

Hi,

I was wondering if you would be interested in adding a libnuma function for determing the node that a cpu belongs to.

Concrete use case please. Why would you want to do that in a program?

My specific case is quite simple: I have a program that creates a number of threads that are bound to run in specific cpus. I want to pre-allocate memory in the node of the cpu that the thread will run before the threads are created using numa_alloc_onnode().

Obviously, there isn't a hard requirement to include such a function in libnuma, since it could be implemented using existing functions, or make the allocations after the threads are spawned using numa_alloc_local().

On the other hand I'm guessing that it could be useful to other cases as well.

cheers,

Signature · 1 line
Kornilios Kourtis
#3 lore raw
Quoted 7 lines · Kornilios Kourtis
Quoted 6 lines · Andi Kleen
Quoted 4 lines · Kornilios Kourtis

Hi,

I was wondering if you would be interested in adding a libnuma function for determing the node that a cpu belongs to.

Quoted 6 lines
Quoted 1 line

Concrete use case please. Why would you want to do that in a program?

My specific case is quite simple: I have a program that creates a number of threads that are bound to run in specific cpus. I want to pre-allocate memory in the node of the cpu that the thread will run before the threads are created using numa_alloc_onnode().

Normally the number of threads is dependent on the data layout, isn't it? It seems like a backwards way of doing things, but ok.

BTW another known topology reporting hole is there is currently no way to report number of CPUs for a given node. Normally it's just max/cpus, but that can change with CPU hotplug. On the other hand CPU/node hotplug is currently not really supported anyways.

-Andi

Signature · 1 line
[email protected] -- Speaking for myself only.
#4 lore raw
Quoted 2 lines · Andi Kleen

Normally the number of threads is dependent on the data layout, isn't it? It seems like a backwards way of doing things, but ok.

I'm including a better patch for this.

Quoted 4 lines

BTW another known topology reporting hole is there is currently no way to report number of CPUs for a given node. Normally it's just max/cpus, but that can change with CPU hotplug. On the other hand CPU/node hotplug is currently not really supported anyways.

I took a quick look at this and I guess the most straightforward way would be to intersect the set of numa_node_to_cpus() with the set of online cpus (/sys/devices/system/cpu/online). I'm not exactly sure about the format of this file and a quick grep at kernel sources didn't reveal much. I might take another look if I find some idle time.

cheers,

Add a numa_node_of_cpu() function for retrieving the local node of a cpu.

libnuma.c +27 -0
diff -uprN numactl-2.0.3-rc2.orig/libnuma.c numactl-2.0.3-rc2/libnuma.c
@@ -1310,6 +1310,33 @@ __asm__(".symver numa_node_to_cpus_v2,nu
 make_internal_alias(numa_node_to_cpus_v1);
 make_internal_alias(numa_node_to_cpus_v2);
 
+/* report the node of the specified cpu */
+int numa_node_of_cpu(int cpu)
+{
+	struct bitmask *bmp;
+	int ncpus, nnodes, node, ret;
+
+	ncpus = numa_num_possible_cpus();
+	if (cpu > ncpus){
+		errno = EINVAL;
+		return -1;
+	}
+	bmp = numa_bitmask_alloc(ncpus);
+	nnodes = numa_num_configured_nodes();
+	for (node = 0; node < nnodes; node++){
+		numa_node_to_cpus_v2_int(node, bmp);
+		if (numa_bitmask_isbitset(bmp, cpu)){
+			ret = node;
+			goto end;
+		}
+	}
+	ret = -1;
+	errno = EINVAL;
+end:
+	numa_bitmask_free(bmp);
+	return ret;
+}
+
 
 int
 numa_run_on_node_mask_v1(const nodemask_t *mask)
numa.3 +10 -0
diff -uprN numactl-2.0.3-rc2.orig/numa.3 numactl-2.0.3-rc2/numa.3
@@ -115,6 +115,8 @@ numa \- NUMA policy library
 .BI "int numa_sched_setaffinity(pid_t " pid ", struct bitmask *" mask );
 .br
 .BI "int numa_node_to_cpus(int " node ", unsigned long *" buffer ", int " bufferlen );
+.br
+.BI "int numa_node_of_cpu(int " cpu ");
 .sp
 .BI "struct bitmask *numa_allocate_cpumask();"
 .sp
@@ -220,6 +222,7 @@ Most functions in this library are only 
 their memory.
 The exceptions to this are:
 .IR numa_node_to_cpus (),
+.IR numa_node_of_cpu (),
 .IR numa_bind (),
 .IR numa_run_on_node (),
 .IR numa_run_on_node_mask ()
@@ -730,6 +733,13 @@ will be set to
 .I ERANGE
 and \-1 returned. On success 0 is returned.
 
+.BR numa_node_of_cpu ()
+returns the node that a cpu belongs to. If the user supplies an invalid cpu
+.I errno
+will be set to
+.I EINVAL
+and \-1 will be returned.
+
 .BR numa_allocate_cpumask
 () returns a bitmask of a size equal to the kernel's cpu
 mask (kernel type cpumask_t).  In other words, large enough to represent
numa.h +3 -0
diff -uprN numactl-2.0.3-rc2.orig/numa.h numactl-2.0.3-rc2/numa.h
@@ -273,6 +273,9 @@ static inline void numa_free_cpumask(str
 /* Convert node to CPU mask. -1/errno on failure, otherwise 0. */
 int numa_node_to_cpus(int, struct bitmask *);
 
+/* report the node of the specified cpu. -1/errno on invalid cpu. */
+int numa_node_of_cpu(int cpu);
+
 /* Report distance of node1 from node2. 0 on error.*/
 int numa_distance(int node1, int node2);
 
versions.ldscript +1 -0
diff -uprN numactl-2.0.3-rc2.orig/versions.ldscript numactl-2.0.3-rc2/versions.ldscript
@@ -118,6 +118,7 @@ libnuma_1.2 {
     numa_node_size64;
     numa_node_size;
     numa_node_to_cpus;
+    numa_node_of_cpu;
     numa_num_configured_cpus;
     numa_num_configured_nodes;
     numa_num_possible_nodes;
Signature · 1 line
Kornilios Kourtis
#5 lore raw
In reply to Kornilios Kourtis

Hi Kornilios,

I applied and tested your patch (adding numa_node_of_cpu()).

It looks good. Thanks.

It is in the numactl-2.0.3-rc2.tar.gz tarball. (ftp://oss.sgi.com/www/projects/libnuma/download/)

-Cliff

Signature · 4 lines
Cliff Wickman
Silicon Graphics, Inc.
[email protected]
(651) 683-3824