Skip to contentSkip to conversation

cpuset - question

DISCUSSION JaniD++ 3 participants 8 messages Nov 1, 2005

lore
Share & tools
lore

JaniD++

author
#0 lore raw

Hello, list

I have tried this:

[root@dy-xeon-1 dev]# mount -t cpuset none /dev/cpuset [root@dy-xeon-1 dev]# cd /dev/cpuset [root@dy-xeon-1 cpuset]# mkdir cpus_0 [root@dy-xeon-1 cpuset]# cd cpus_0 [root@dy-xeon-1 cpus_0]# /bin/echo 0 > cpus [root@dy-xeon-1 cpus_0]# /bin/echo 1 > mems /bin/echo: write error: Numerical result out of range [root@dy-xeon-1 cpus_0]# echo 1 >mems [root@dy-xeon-1 cpus_0]# cat mems

[root@dy-xeon-1 cpus_0]# /bin/echo $$ > tasks
/bin/echo: write error: No space left on device
[root@dy-xeon-1 cpus_0]# echo $$ >tasks
[root@dy-xeon-1 cpus_0]# cat tasks
[root@dy-xeon-1 cpus_0]#

The google, and man pages cant help.
What can i do?

Thanks

Janos

#1 lore raw
Quoted 7 lines · JaniD++

[root@dy-xeon-1 cpus_0]# /bin/echo 1 > mems /bin/echo: write error: Numerical result out of range [root@dy-xeon-1 cpus_0]# echo 1 >mems [root@dy-xeon-1 cpus_0]# cat mems

[root@dy-xeon-1 cpus_0]# /bin/echo $$ > tasks
/bin/echo: write error: No space left on device

I'm guessing you are on a multi-processor, with a single memory node, not a NUMA system with multiple memory nodes.

Or, at least, your kernel was compiled for that (with the CONFIG_NUMA option disabled).

The first echo above failed because you tried to set bit 1 in mems, but only bit 0 is valid (only one memory node).

The second echo failed too, but your shells (like most shells) builtin echo didn't display the error.

The 'cat mems' command showed that mems was not yet set, which is indeed the case.

The third and final echo above, into 'tasks' failed because you can't attach a task to a cpuset that has no memory specified.

If you had done '/bin/echo 0 > mems', it would have worked much better.

Signature · 3 lines
                  I won't rest till it's the best ...
                  Programmer, Linux Scalability
                  Paul Jackson <[email protected]> 1.925.600.0401
author
#2 lore raw

Hello,

----- Original Message -----
From: "Paul Jackson" <[email protected]>
To: "JaniD++" <[email protected]>
Cc: <[email protected]>
Sent: Tuesday, November 01, 2005 9:36 PM
Subject: Re: cpuset - question

Quoted 6 lines
Quoted 5 lines · JaniD++

[root@dy-xeon-1 cpus_0]# /bin/echo 1 > mems /bin/echo: write error: Numerical result out of range [root@dy-xeon-1 cpus_0]# echo 1 >mems [root@dy-xeon-1 cpus_0]# cat mems

Quoted 8 lines
Quoted 2 lines

[root@dy-xeon-1 cpus_0]# /bin/echo $$ > tasks
/bin/echo: write error: No space left on device

I'm guessing you are on a multi-processor, with a single memory node, not a NUMA system with multiple memory nodes.

Or, at least, your kernel was compiled for that (with the CONFIG_NUMA option disabled).

Yes, this option is disabled.
This is a dual-xeon server with HT.
With HT looks 4 CPU.

In this config i need NUMA option enabled to use cpusets?

I only need to move my 4 gnbd-client process to 4 cpuset, but i don't want to touch the memory.
This is possible, or need the CONFIG_NUMA=y option?

Thanks

Janos

Quoted 25 lines

The first echo above failed because you tried to set bit 1 in mems, but only bit 0 is valid (only one memory node).

The second echo failed too, but your shells (like most shells) builtin echo didn't display the error.

The 'cat mems' command showed that mems was not yet set, which is indeed the case.

The third and final echo above, into 'tasks' failed because you can't attach a task to a cpuset that has no memory specified.

If you had done '/bin/echo 0 > mems', it would have worked much better.

Signature · 8 lines
                  I won't rest till it's the best ...
                  Programmer, Linux Scalability
                  Paul Jackson <[email protected]> 1.925.600.0401
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/
#6 lore raw
In reply to JaniD++
Quoted 14 lines · JaniD++

Hello, list

I have tried this:

[root@dy-xeon-1 dev]# mount -t cpuset none /dev/cpuset [root@dy-xeon-1 dev]# cd /dev/cpuset [root@dy-xeon-1 cpuset]# mkdir cpus_0 [root@dy-xeon-1 cpuset]# cd cpus_0 [root@dy-xeon-1 cpus_0]# /bin/echo 0 > cpus [root@dy-xeon-1 cpus_0]# /bin/echo 1 > mems /bin/echo: write error: Numerical result out of range [root@dy-xeon-1 cpus_0]# echo 1 >mems [root@dy-xeon-1 cpus_0]# cat mems

Quoted 8 lines

[root@dy-xeon-1 cpus_0]# /bin/echo $$ > tasks
/bin/echo: write error: No space left on device
[root@dy-xeon-1 cpus_0]# echo $$ >tasks
[root@dy-xeon-1 cpus_0]# cat tasks
[root@dy-xeon-1 cpus_0]#

The google, and man pages cant help.
What can i do?

Start by telling us what kernel and patches you run, what config option you used, etc. Oh, and what you're trying to do...

Signature · 3 lines
   -bill davidsen ([email protected])
"The secret to procrastination is to put things off until the
 last possible moment - but no longer"  -me
#3 lore raw
In reply to JaniD++
Quoted 1 line

In this config i need NUMA option enabled to use cpusets?

No - if you got this far, cpusets are working for you.

You would need NUMA if you had multiple memory nodes.

I'm guessing you have one collection of RAM modules, all equally distant from the processors, which is not a NUMA (Non-Uniform-Memory-Architecture) system.

I only mentioned NUMA because if you did have NUMA hardware, then you would need to CONFIG it into to your kernel to make full use of your multiple memory nodes. I doubt that applies to you.

It looks like you have multiple (4 logical with HT) CPUs, numbered 0, 1, 2, and 3, and one Memory Node, numbered 0.

Cpusets should work for you - just "echo 0", not "echo 1" into the "mems" files. Your one and only Memory Node is numbered "0", not "1".

Actually, make that "/bin/echo", not "echo", so you can see the error messages.

Signature · 3 lines
                  I won't rest till it's the best ...
                  Programmer, Linux Scalability
                  Paul Jackson <[email protected]> 1.925.600.0401
#7 lore raw
In reply to Bill Davidsen
Quoted 2 lines · Bill

Start by telling us what kernel and patches you run, what config option you used, etc. Oh, and what you're trying to do...

That's ok, Bill. I've spent the last two years reading those sort of echo error messages from munging /dev/cpuset files on an almost daily basis.

I'm pretty sure I can see what's going on with JaniD++'s situation without needing more background information.

Signature · 3 lines
                  I won't rest till it's the best ...
                  Programmer, Linux Scalability
                  Paul Jackson <[email protected]> 1.925.600.0401
author
#4 lore raw
In reply to Paul Jackson

Thanks Paul Jackson, Bill Davidsen!

Yes, with echo 0 > mems, this is works fine! :-)

But the documentation is a little small for cpusets...

Thanks

Janos
----- Original Message -----
From: "Paul Jackson" <[email protected]>
To: "JaniD++" <[email protected]>
Cc: <[email protected]>
Sent: Tuesday, November 01, 2005 11:24 PM
Subject: Re: cpuset - question

Quoted 35 lines
Quoted 1 line

In this config i need NUMA option enabled to use cpusets?

No - if you got this far, cpusets are working for you.

You would need NUMA if you had multiple memory nodes.

I'm guessing you have one collection of RAM modules, all equally distant from the processors, which is not a NUMA (Non-Uniform-Memory-Architecture) system.

I only mentioned NUMA because if you did have NUMA hardware, then you would need to CONFIG it into to your kernel to make full use of your multiple memory nodes. I doubt that applies to you.

It looks like you have multiple (4 logical with HT) CPUs, numbered 0, 1, 2, and 3, and one Memory Node, numbered 0.

Cpusets should work for you - just "echo 0", not "echo 1" into the "mems" files. Your one and only Memory Node is numbered "0", not "1".

Actually, make that "/bin/echo", not "echo", so you can see the error messages.

Signature · 8 lines
                  I won't rest till it's the best ...
                  Programmer, Linux Scalability
                  Paul Jackson <[email protected]> 1.925.600.0401
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/
#5 lore raw
Quoted 1 line

But the documentation is a little small for cpusets...

Yes, it is.

I actually have a fairly complete document, but it contains both the user library support and the kernel support information. The user library has not yet been open sourced by SGI, but I anticipate that it will be, at which time the full documentation will become available.

Signature · 3 lines
                  I won't rest till it's the best ...
                  Programmer, Linux Scalability
                  Paul Jackson <[email protected]> 1.925.600.0401