Skip to contentSkip to conversation

wrong parsing to /proc/self/status causes wrong out-of-range errors

PATCH Brice Goglin 3 participants 4 messages Apr 21, 2009 – Apr 23, 2009

lore
Share & tools
lore
#0 lore raw

Hello,

After upgrading a quad-socket quad-core machine from 2.6.27 to 2.6.29, numactl now reports that --physcpubind=X is an out-of-range CPU when 12<=X<16
(while obviously we still have 16 cores in this machine). Same problem on other machines with 2.6.29: the last 4 cores are "out-of-range".
First observed with Debian's 2.0.3-rc1 but seems to occur with 2.0.3-rc2 as well.

It appears that this is caused by set_thread_constraints() passing a wrong pointer to read_mask() when trying to gather maxproccpu and maxprocnode from /proc/self/status. It points to the second character of the mask instead of the first one, thus loosing one "f", which means 4 cores are lost.

The kernel code generating the "Cpus_allowed:" and "Mems_allowed:" masks in /proc/self/status has changed recently, so maybe the formatting changed a bit (whitespaces?).

The patch below fixes this problem by just passing a pointer to the first character after ":". read_mask/strtoul are able to skip whitespaces anyway, so no need to bother trying to guess how many whitespaces follow ":" in the caller.

Brice

libnuma.c +2 -2
diff -ur numactl-2.0.3~rc1/libnuma.c numactl-2.0.3~rc1.save/libnuma.c
@@ -479,11 +479,11 @@
 
 	while (getline(&buffer, &buflen, f) > 0) {
 		if (strncmp(buffer,"Cpus_allowed:",13) == 0)
-			maxproccpu = read_mask(buffer + 15, numa_all_cpus_ptr);
+			maxproccpu = read_mask(buffer + 13, numa_all_cpus_ptr);
 
 		if (strncmp(buffer,"Mems_allowed:",13) == 0) {
 			maxprocnode =
-				read_mask(buffer + 15, numa_all_nodes_ptr);
+				read_mask(buffer + 13, numa_all_nodes_ptr);
 		}
 	}
 	fclose(f);
#1 lore raw

Hi Brice,

Thanks for the fix.

I tested it on ia64.
It is now part of numactl-2.0.3-rc2.tar.gz at
ftp://oss.sgi.com/www/projects/libnuma/download/

BTW,
Andi, Lee and all,
We have no established release schedule that I'm aware of. 2.0.2 was Aug08
There are about 14 fixes accumulated in 2.0.3 rc1 and rc2. But only a trickle of small changes lately.
I'm inclined to announce it as 2.0.3, if you have used it and are satisfied that it's stable.

-Cliff

Quoted 54 lines · Brice Goglin

Hello,

After upgrading a quad-socket quad-core machine from 2.6.27 to 2.6.29, numactl now reports that --physcpubind=X is an out-of-range CPU when 12<=X<16
(while obviously we still have 16 cores in this machine). Same problem on other machines with 2.6.29: the last 4 cores are "out-of-range".
First observed with Debian's 2.0.3-rc1 but seems to occur with 2.0.3-rc2 as well.

It appears that this is caused by set_thread_constraints() passing a wrong pointer to read_mask() when trying to gather maxproccpu and maxprocnode from /proc/self/status. It points to the second character of the mask instead of the first one, thus loosing one "f", which means 4 cores are lost.

The kernel code generating the "Cpus_allowed:" and "Mems_allowed:" masks in /proc/self/status has changed recently, so maybe the formatting changed a bit (whitespaces?).

The patch below fixes this problem by just passing a pointer to the first character after ":". read_mask/strtoul are able to skip whitespaces anyway, so no need to bother trying to guess how many whitespaces follow ":" in the caller.

Brice

diff -ur numactl-2.0.3~rc1/libnuma.c numactl-2.0.3~rc1.save/libnuma.c

libnuma.c +2 -3
@@ -479,11 +479,11 @@
 
 	while (getline(&buffer, &buflen, f) > 0) {
 		if (strncmp(buffer,"Cpus_allowed:",13) == 0)
-			maxproccpu = read_mask(buffer + 15, numa_all_cpus_ptr);
+			maxproccpu = read_mask(buffer + 13, numa_all_cpus_ptr);
 
 		if (strncmp(buffer,"Mems_allowed:",13) == 0) {
 			maxprocnode =
-				read_mask(buffer + 15, numa_all_nodes_ptr);
+				read_mask(buffer + 13, numa_all_nodes_ptr);
 		}
 	}
 	fclose(f);
 
 
--

To unsubscribe from this list: send the line "unsubscribe linux-numa" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Signature · 4 lines
Cliff Wickman
Silicon Graphics, Inc.
[email protected]
(651) 683-3824
#2 lore raw
Quoted 8 lines · Cliff Wickman

Hi Brice,

Thanks for the fix.

I tested it on ia64.
It is now part of numactl-2.0.3-rc2.tar.gz at
ftp://oss.sgi.com/www/projects/libnuma/download/

Quoted 8 lines

BTW,
Andi, Lee and all,
We have no established release schedule that I'm aware of. 2.0.2 was Aug08
There are about 14 fixes accumulated in 2.0.3 rc1 and rc2. But only a trickle of small changes lately.
I'm inclined to announce it as 2.0.3, if you have used it and are satisfied that it's stable.

Hi, Cliff:

I have been using/testing 2.0.3-rc2 for a while. I have accumulated a couple of patches which I've been remiss for not posting. They sort of got pushed way down on my stack. One involves cleaning up what I perceived as bitmap leaks, but included some unrelated cleanup. I've meant to separate them out. One actually addresses the Cpus_allowed parsing that Brice fixed.

A while ago I sent a copy to Kornilios Kourtis [off list] to see if he was interested in moving them. He did make some mods, which I've been too distracted to check out :(, but KK was also too busy to dedicate much time. I can try to spend some time tomorrow reviewing Kornilios' version and my other patches [nothing serious, as I recall], and send them on--before the weekend, I would hope.

Or I could send you what I have as a heads up and work with you to finish testing/cleanup/... Let me know your preference.

Apologies for letting this slide,
Lee

<snip>

#3 lore raw

Hi Lee.

Quoted 25 lines · Lee Schermerhorn
Quoted 7 lines · Cliff Wickman

Andi, Lee and all,
We have no established release schedule that I'm aware of. 2.0.2 was Aug08
There are about 14 fixes accumulated in 2.0.3 rc1 and rc2. But only a trickle of small changes lately.
I'm inclined to announce it as 2.0.3, if you have used it and are satisfied that it's stable.

Hi, Cliff:

I have been using/testing 2.0.3-rc2 for a while. I have accumulated a couple of patches which I've been remiss for not posting. They sort of got pushed way down on my stack. One involves cleaning up what I perceived as bitmap leaks, but included some unrelated cleanup. I've meant to separate them out. One actually addresses the Cpus_allowed parsing that Brice fixed.

A while ago I sent a copy to Kornilios Kourtis [off list] to see if he was interested in moving them. He did make some mods, which I've been too distracted to check out :(, but KK was also too busy to dedicate much time. I can try to spend some time tomorrow reviewing Kornilios' version and my other patches [nothing serious, as I recall], and send them on--before the weekend, I would hope.

Quoted 5 lines

Or I could send you what I have as a heads up and work with you to finish testing/cleanup/... Let me know your preference.

Apologies for letting this slide,
Lee

I'm not in any hurry release 2.0.3. It just looked like it was stable.

But if you and Kornilios and others are working on some issues that's
fine. Finish them when you can.

-Cliff

Signature · 4 lines
Cliff Wickman
Silicon Graphics, Inc.
[email protected]
(651) 683-3824