find the node that a cpu belongs to
Hi,
I was wondering if you would be interested in adding a libnuma function for determing the node that a cpu belongs to.
Something like this maybe ?
I could write a more complete patch if needed
(libnuma.h,version.ldscript,numa.3 ?)
libnuma.c renamed from libnuma.c.2 +30 -0
@@ -1784,3 +1784,34 @@ err:numa_bitmask_free (mask);return NULL ; } + +int numa_node_from_cpu_v2 (int cpu) +{ +struct bitmask *bmp; +int ncpus, nnodes, node, ret; + + ncpus =numa_num_possible_cpus (); +if (cpu > ncpus){ + errno =ERANGE ; +return -1 ; + } + + bmp =numa_bitmask_alloc (ncpus); + nnodes =numa_num_configured_nodes (); +for (node =0 ; node < nnodes; node++){ +numa_node_to_cpus_v2 (node, bmp); +if (numa_bitmask_isbitset (bmp, cpu)){ + ret = node; + goto end; + } + } + +/* I don't think this is supposed to happen */ + ret =-1 ; + errno =EINVAL ; +end: +numa_bitmask_free (bmp); +return ret; +} +__asm__(".symver numa_node_from_cpu_v2,numa_node_from_cpu@@libnuma_1.2" );
Signature · 1 line
Kornilios Kourtis
Quoted 4 lines · Kornilios Kourtis
Hi,
I was wondering if you would be interested in adding a libnuma function for determing the node that a cpu belongs to.
Concrete use case please. Why would you want to do that in a program?
-Andi
Signature · 1 line
[email protected] -- Speaking for myself only.
Quoted 7 lines · Andi Kleen
Quoted 4 lines · Kornilios Kourtis
Hi,
I was wondering if you would be interested in adding a libnuma function for determing the node that a cpu belongs to.
Concrete use case please. Why would you want to do that in a program?
My specific case is quite simple: I have a program that creates a number of threads that are bound to run in specific cpus. I want to pre-allocate memory in the node of the cpu that the thread will run before the threads are created using numa_alloc_onnode().
Obviously, there isn't a hard requirement to include such a function in libnuma, since it could be implemented using existing functions, or make the allocations after the threads are spawned using numa_alloc_local().
On the other hand I'm guessing that it could be useful to other cases as well.
cheers,
Signature · 1 line
Kornilios Kourtis
Quoted 7 lines · Kornilios Kourtis
Quoted 6 lines · Andi Kleen
Quoted 4 lines · Kornilios Kourtis
Hi,
I was wondering if you would be interested in adding a libnuma function for determing the node that a cpu belongs to.
Quoted 6 lines
Quoted 1 line
Concrete use case please. Why would you want to do that in a program?
My specific case is quite simple: I have a program that creates a number of threads that are bound to run in specific cpus. I want to pre-allocate memory in the node of the cpu that the thread will run before the threads are created using
numa_alloc_onnode().
Normally the number of threads is dependent on the data layout, isn't it? It seems like a backwards way of doing things, but ok.
BTW another known topology reporting hole is there is currently no way to report number of CPUs for a given node. Normally it's just max/cpus, but that can change with CPU hotplug. On the other hand CPU/node hotplug is currently not really supported anyways.
-Andi
Signature · 1 line
[email protected] -- Speaking for myself only.
Quoted 2 lines · Andi Kleen
Normally the number of threads is dependent on the data layout, isn't it? It seems like a backwards way of doing things, but ok.
I'm including a better patch for this.
Quoted 4 lines
BTW another known topology reporting hole is there is currently no way to report number of CPUs for a given node. Normally it's just max/cpus, but that can change with CPU hotplug. On the other hand CPU/node hotplug is currently not really supported anyways.
I took a quick look at this and I guess the most straightforward way would be to intersect the set of numa_node_to_cpus() with the set of online cpus (/sys/devices/system/cpu/online). I'm not exactly sure about the format of this file and a quick grep at kernel sources didn't reveal much. I might take another look if I find some idle time.
cheers,
Add a numa_node_of_cpu() function for retrieving the local node of a cpu.
libnuma.c +27 -0
@@ -1310,6 +1310,33 @@ __asm__(".symver numa_node_to_cpus_v2,numake_internal_alias (numa_node_to_cpus_v1);make_internal_alias (numa_node_to_cpus_v2); +/* report the node of the specified cpu */ +int numa_node_of_cpu (int cpu) +{ +struct bitmask *bmp; +int ncpus, nnodes, node, ret; + + ncpus =numa_num_possible_cpus (); +if (cpu > ncpus){ + errno =EINVAL ; +return -1 ; + } + bmp =numa_bitmask_alloc (ncpus); + nnodes =numa_num_configured_nodes (); +for (node =0 ; node < nnodes; node++){ +numa_node_to_cpus_v2_int (node, bmp); +if (numa_bitmask_isbitset (bmp, cpu)){ + ret = node; + goto end; + } + } + ret =-1 ; + errno =EINVAL ; +end: +numa_bitmask_free (bmp); +return ret; +} +int numa_run_on_node_mask_v1 (const nodemask_t *mask)
numa.3 +10 -0
@@ -115,6 +115,8 @@ numa \- NUMA policy library .BI "int numa_sched_setaffinity(pid_t " pid ", struct bitmask *" mask ); .br .BI "int numa_node_to_cpus(int " node ", unsigned long *" buffer ", int " bufferlen ); +.br +.BI "int numa_node_of_cpu(int " cpu "); .sp .BI "struct bitmask *numa_allocate_cpumask();" .sp
@@ -220,6 +222,7 @@ Most functions in this library are only their memory. The exceptions to this are: .IR numa_node_to_cpus (), +.IR numa_node_of_cpu (), .IR numa_bind (), .IR numa_run_on_node (), .IR numa_run_on_node_mask ()
@@ -730,6 +733,13 @@ will be set to .I ERANGE and \-1 returned. On success 0 is returned. +.BR numa_node_of_cpu () +returns the node that a cpu belongs to. If the user supplies an invalid cpu +.I errno +will be set to +.I EINVAL +and \-1 will be returned. + .BR numa_allocate_cpumask () returns a bitmask of a size equal to the kernel's cpu mask (kernel type cpumask_t). In other words, large enough to represent
numa.h +3 -0
@@ -273,6 +273,9 @@ static inline void numa_free_cpumask(str/* Convert node to CPU mask. -1/errno on failure, otherwise 0. */ int numa_node_to_cpus (int ,struct bitmask *); +/* report the node of the specified cpu. -1/errno on invalid cpu. */ +int numa_node_of_cpu (int cpu); +/* Report distance of node1 from node2. 0 on error.*/ int numa_distance (int node1,int node2);
versions.ldscript +1 -0
@@ -118,6 +118,7 @@ libnuma_1.2 { numa_node_size64; numa_node_size; numa_node_to_cpus; + numa_node_of_cpu; numa_num_configured_cpus; numa_num_configured_nodes; numa_num_possible_nodes;
Signature · 1 line
Kornilios Kourtis
Hi Kornilios,
I applied and tested your patch (adding numa_node_of_cpu()).
It looks good. Thanks.
It is in the numactl-2.0.3-rc2.tar.gz tarball. (ftp://oss.sgi.com/www/projects/libnuma/download/)
-Cliff
Signature · 4 lines
Cliff Wickman Silicon Graphics, Inc. [email protected] (651) 683-3824