Skip to contentSkip to conversation

Two questions about scheduling and threading.

DISCUSSION Brian McGrew 3 participants 3 messages May 8, 2008 1 message not in the archive

lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): C4485A81.1E1EF%[email protected]

#1 lore raw

We're using the 2.6.16.16 kernel in an almost real time, multi processor and multi threaded environment.

When I start my system for the first time, I start one thread for each processor/core in the machine (is this the correct thing to do)? These threads set a busy flag, go to work and then go to sleep. I put everything to sleep as opposed to killing the threads because it saves me on average of about 400ms each time around.

My problem is, and, it is very reproducable, that if CPU0 is at 100%, none of my threads see the wakeup! It doesn't matter what the other CPU's are doing, if they're all at 0 or 100%, but if CPU0 is 100, I'm toast. Is there anyway around this?

Also, I know that we're supposed to sit back and let the scheduler do all the work for us; but, in the 2.6.16.16 kernel, is there a way to assign a specific thread and/or process to a designated processor??? I really need to be able to do this because even with the preemptive scheduling, I'm still real-time and it's not quite real-time enough!

Thanks!

-brian

#2 lore raw
Quoted 5 lines · Brian McGrew

Also, I know that we're supposed to sit back and let the scheduler do all the work for us; but, in the 2.6.16.16 kernel, is there a way to assign a specific thread and/or process to a designated processor??? I really need to be able to do this because even with the preemptive scheduling, I'm still real-time and it's not quite real-time enough!

There are some functions to set CPU affinity from user space:

sched_setaffinity()
sched_getaffinity()

And a user space program:

taskset

All have decent man pages. They'll allow you to pin a process to a particular CPU, or set the normal behavior, which is to try and keep a process on the same CPU as long as possible. There are some other utilities mentioned in the related section of the manpages - they allow you to tweak scheduler behavior.

Hope that helps.

Cheers,
Dan

Signature · 3 lines
                    /--------------- - -  -  -   -   -
                    |  Dan Noe
                    |  http://isomerica.net/~dpn/
#3 lore raw
In reply to Brian McGrew
Quoted 8 lines

When I start my system for the first time, I start one thread for each processor/core in the machine (is this the correct thing to do)? These threads set a busy flag, go to work and then go to sleep. I put everything
to sleep as opposed to killing the threads because it saves me on average of
about 400ms each time around.

That's not unreasonable. However, you may wish to create a few extra threads. Otherwise, if one thread is blocked on I/O, you can't use all the CPUs.

Quoted 5 lines

My problem is, and, it is very reproducable, that if CPU0 is at 100%, none of my threads see the wakeup! It doesn't matter what the other CPU's are doing, if they're all at 0 or 100%, but if CPU0 is 100, I'm toast. Is there
anyway around this?

Sounds like a bug. What does your wakeup code look like? Do you put your threads to sleep blocked on a condition variable? Is the c.v. code correctly using a predicate, that's something easy to screw up. It's like 'select', there's a dozen classic mistakes and someone often makes one or two of them and their code still soemtimes works.

Quoted 6 lines

Also, I know that we're supposed to sit back and let the scheduler do all the work for us; but, in the 2.6.16.16 kernel, is there a way to assign a specific thread and/or process to a designated processor??? I really need to be able to do this because even with the preemptive scheduling, I'm still
real-time and it's not quite real-time enough!

Big mistake. If you bind threads to CPUs and the thread that gets a wakeup is assigned to a CPU that's busy, the job that thread was going to do will have to wait, while other CPUs sit idle.

DS