wrong parsing to /proc/self/status causes wrong out-of-range errors
Hello,
After upgrading a quad-socket quad-core machine from 2.6.27 to 2.6.29, numactl now reports that --physcpubind=X is an out-of-range CPU when 12<=X<16
(while obviously we still have 16 cores in this machine). Same problem on other machines with 2.6.29: the last 4 cores are "out-of-range".
First observed with Debian's 2.0.3-rc1 but seems to occur with 2.0.3-rc2 as well.
It appears that this is caused by set_thread_constraints() passing a wrong pointer to read_mask() when trying to gather maxproccpu and maxprocnode from /proc/self/status. It points to the second character of the mask instead of the first one, thus loosing one "f", which means 4 cores are lost.
The kernel code generating the "Cpus_allowed:" and "Mems_allowed:" masks in /proc/self/status has changed recently, so maybe the formatting changed a bit (whitespaces?).
The patch below fixes this problem by just passing a pointer to the first character after ":". read_mask/strtoul are able to skip whitespaces anyway, so no need to bother trying to guess how many whitespaces follow ":" in the caller.
Brice
libnuma.c +2 -2
@@ -479,11 +479,11 @@while (getline (&buffer, &buflen, f) >0 ) {if (strncmp (buffer,"Cpus_allowed:" ,13 ) ==0 ) - maxproccpu =read_mask (buffer +15 , numa_all_cpus_ptr); + maxproccpu =read_mask (buffer +13 , numa_all_cpus_ptr);if (strncmp (buffer,"Mems_allowed:" ,13 ) ==0 ) { maxprocnode = -read_mask (buffer +15 , numa_all_nodes_ptr); +read_mask (buffer +13 , numa_all_nodes_ptr); } }fclose (f);
Hi Brice,
Thanks for the fix.
I tested it on ia64.
It is now part of numactl-2.0.3-rc2.tar.gz at
ftp://oss.sgi.com/www/projects/libnuma/download/
BTW,
Andi, Lee and all,
We have no established release schedule that I'm aware of. 2.0.2 was Aug08
There are about 14 fixes accumulated in 2.0.3 rc1 and rc2. But only a trickle of small changes lately.
I'm inclined to announce it as 2.0.3, if you have used it and are satisfied that it's stable.
-Cliff
Quoted 54 lines · Brice Goglin
Hello,
After upgrading a quad-socket quad-core machine from 2.6.27 to 2.6.29, numactl now reports that --physcpubind=X is an out-of-range CPU when 12<=X<16
(while obviously we still have 16 cores in this machine). Same problem on other machines with 2.6.29: the last 4 cores are "out-of-range".
First observed with Debian's 2.0.3-rc1 but seems to occur with 2.0.3-rc2 as well.It appears that this is caused by
set_thread_constraints()passing a wrong pointer toread_mask()when trying to gather maxproccpu and maxprocnode from/proc/self/status. It points to the second character of the mask instead of the first one, thus loosing one "f", which means 4 cores are lost.The kernel code generating the "
Cpus_allowed:" and "Mems_allowed:" masks in/proc/self/statushas changed recently, so maybe the formatting changed a bit (whitespaces?).The patch below fixes this problem by just passing a pointer to the first character after ":". read_mask/strtoul are able to skip whitespaces anyway, so no need to bother trying to guess how many whitespaces follow ":" in the caller.
Brice
Signed-off-by Brice Goglin <[email protected]>diff -ur
numactl-2.0.3~rc1/libnuma.cnumactl-2.0.3~rc1.save/libnuma.clibnuma.c +2
-3@@ -479,11 +479,11 @@while (getline (&buffer, &buflen, f) >0 ) {if (strncmp (buffer,"Cpus_allowed:" ,13 ) ==0 ) - maxproccpu =read_mask (buffer +15 , numa_all_cpus_ptr); + maxproccpu =read_mask (buffer +13 , numa_all_cpus_ptr);if (strncmp (buffer,"Mems_allowed:" ,13 ) ==0 ) { maxprocnode = -read_mask (buffer +15 , numa_all_nodes_ptr); +read_mask (buffer +13 , numa_all_nodes_ptr); } }fclose (f); --To unsubscribe from this list: send the line "unsubscribe linux-numa" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Signature · 4 lines
Cliff Wickman Silicon Graphics, Inc. [email protected] (651) 683-3824
Quoted 8 lines · Cliff Wickman
Hi Brice,
Thanks for the fix.
I tested it on ia64.
It is now part of numactl-2.0.3-rc2.tar.gz at
ftp://oss.sgi.com/www/projects/libnuma/download/
Quoted 8 lines
BTW,
Andi, Lee and all,
We have no established release schedule that I'm aware of. 2.0.2 was Aug08
There are about 14 fixes accumulated in 2.0.3 rc1 and rc2. But only a trickle of small changes lately.
I'm inclined to announce it as 2.0.3, if you have used it and are satisfied that it's stable.
Hi, Cliff:
I have been using/testing 2.0.3-rc2 for a while. I have accumulated a couple of patches which I've been remiss for not posting. They sort of got pushed way down on my stack. One involves cleaning up what I perceived as bitmap leaks, but included some unrelated cleanup. I've meant to separate them out. One actually addresses the Cpus_allowed parsing that Brice fixed.
A while ago I sent a copy to Kornilios Kourtis [off list] to see if he was interested in moving them. He did make some mods, which I've been too distracted to check out :(, but KK was also too busy to dedicate much time. I can try to spend some time tomorrow reviewing Kornilios' version and my other patches [nothing serious, as I recall], and send them on--before the weekend, I would hope.
Or I could send you what I have as a heads up and work with you to finish testing/cleanup/... Let me know your preference.
Apologies for letting this slide,
Lee
<snip>
Hi Lee.
Quoted 25 lines · Lee Schermerhorn
Quoted 7 lines · Cliff Wickman
Andi, Lee and all,
We have no established release schedule that I'm aware of. 2.0.2 was Aug08
There are about 14 fixes accumulated in 2.0.3 rc1 and rc2. But only a trickle of small changes lately.
I'm inclined to announce it as 2.0.3, if you have used it and are satisfied that it's stable.Hi, Cliff:
I have been using/testing 2.0.3-rc2 for a while. I have accumulated a couple of patches which I've been remiss for not posting. They sort of got pushed way down on my stack. One involves cleaning up what I perceived as bitmap leaks, but included some unrelated cleanup. I've meant to separate them out. One actually addresses the
Cpus_allowedparsing that Brice fixed.A while ago I sent a copy to Kornilios Kourtis [off list] to see if he was interested in moving them. He did make some mods, which I've been too distracted to check out :(, but KK was also too busy to dedicate much time. I can try to spend some time tomorrow reviewing Kornilios' version and my other patches [nothing serious, as I recall], and send them on--before the weekend, I would hope.
Quoted 5 lines
Or I could send you what I have as a heads up and work with you to finish testing/cleanup/... Let me know your preference.
Apologies for letting this slide,
Lee
I'm not in any hurry release 2.0.3. It just looked like it was stable.
But if you and Kornilios and others are working on some issues that's
fine. Finish them when you can.
-Cliff
Signature · 4 lines
Cliff Wickman Silicon Graphics, Inc. [email protected] (651) 683-3824