Skip to contentSkip to conversation

cpus_allowed

DISCUSSION Rolf Fokkens 4 participants 5 messages Oct 13, 2001

lore
Share & tools
lore
#0 lore raw

Hi!

I was curious about "CPU affinity" in linux. I found some patches which add affinity in task_struct but later I found out that "cpus_allowed" in task_struct almost does the same thing.

It resulted in some new curiosity: where's cpus_allowed initialized? I can only find an assignment to cpus_allowed for softirq's but no initialization for other processes. I assume the correct init value would be "0xffffffff" or -1. Can't find it though.

I'm sure I'm overlooking something, but that doesn't help me finding the answer. So would someone be so kind to enlighten me?

Thanks,

Rolf

#1 lore raw
Quoted 11 lines · Rolf Fokkens

I was curious about "CPU affinity" in linux. I found some patches which add affinity in task_struct but later I found out that "cpus_allowed" in task_struct almost does the same thing.

It resulted in some new curiosity: where's cpus_allowed initialized? I can only find an assignment to cpus_allowed for softirq's but no initialization for other processes. I assume the correct init value would be "0xffffffff" or -1. Can't find it though.

I'm sure I'm overlooking something, but that doesn't help me finding the answer. So would someone be so kind to enlighten me?

It is initialized to -1 (0xffffffff) by struct definition at linux/kernel/sched.h. Since it is a mask, this means all CPUs (obviously).

It isn't used much. The softirq code uses it to try to keep some tasks on one CPU, for performance reasons. The Tux in-kernel httpd uses it for performance reasons. Some patches use it, too.

Most of the CPU affinity patches you see were written before cpus_allowed. They go through all sorts of trouble to do what the OS now does on its own. If you want to change CPU affinity then you just need a patch that adds a syscall or proc interface for setting the cpus_allowed mask.

I wrote a proc interface but for the life of me I can't find it. If you can I would be happy to update it to your kernel. It isn't very old, anyhow.

Andrew Morton wrote has a similar patch that is quite good. Get it at
http://www.uow.edu.au/~andrewm/linux/#cpus_allowed

Robert Love

#2 lore raw
Quoted 5 lines

Most of the CPU affinity patches you see were written before cpus_allowed. They go through all sorts of trouble to do what the OS now does on its own. If you want to change CPU affinity then you just need a patch that adds a syscall or proc interface for setting the cpus_allowed mask.

I'm still porting pset to 2.4. It is more robust than cpus_allowed and does more. My plan is to do away with cpus_allowed altogether, and just provide a compat with pset, but if enough stuff uses cpus_allowed, perhaps I'll have to leave it..

http://www.hockin.org/~thockin/pset

2.4.x patch not up yet

#3 lore raw
In reply to Rolf Fokkens
Quoted 4 lines · Rolf Fokkens

It resulted in some new curiosity: where's cpus_allowed initialized? I can only find an assignment to cpus_allowed for softirq's but no initialization for other processes. I assume the correct init value would be "0xffffffff" or -1. Can't find it though.

include/linux/sched.h: cpus_allowed: -1,

regards,

Dave.

Signature · 2 lines
| Dave Jones.        http://www.codemonkey.org.uk
| SuSE Labs
#4 lore raw
In reply to Rolf Fokkens
Quoted 7 lines · Robert Love
Quoted 2 lines · Rolf Fokkens

I'm sure I'm overlooking something, but that doesn't help me finding the answer. So would someone be so kind to enlighten me?

It is initialized to -1 (0xffffffff) by struct definition at linux/kernel/sched.h. Since it is a mask, this means all CPUs (obviously).

Silly me, I didn't check the header files.

Quoted 5 lines

Most of the CPU affinity patches you see were written before cpus_allowed. They go through all sorts of trouble to do what the OS now does on its own. If you want to change CPU affinity then you just need a patch that adds a syscall or proc interface for setting the cpus_allowed mask.

Just read the "Linux Scalability Effort" (LSE) on Sourceforge. The concept of limitiming applications to certain resources is appealing to me. With the use of cpus_allowed it's not to hard to restrict CPU power for some applications. I'm thinking of the following: we're running about 12 (!) Oracle databases on 1 Linux Server with 4 CPU's. One of the databases is a datawarehouse database which handles all kinds of heavy queries. Assuming that the machine has enough memory (which is the case) restricting this database to certain CPU's could be very useful.

I have no idea however what the right API would be. LSE suggests a ulimit like setting. In this Oracle example one listener handles connections to all databases on the machine, it does so by forking and executing the database binary with some specific environment settings per database. So ulimit won't handle it. The solution might be to run 2 listeners, one with restricted cpus_allowed and the other one with unrestricted cpus_allowed.