Skip to contentSkip to conversation

SMP CPU affinity questions

DISCUSSION [email protected] 5 participants 5 messages Nov 2, 2005 – Nov 6, 2005

lore
Share & tools
lore
#0 lore raw

Hi-

I am trying to use a quad Opteron motherboard with SMP Kernel 2.6.5 for a quasi-real-time task. I need to assign all processes to specific CPUs, including interrupt handlers.
I have had success using sched_setaffinity() to set the CPU for processes I create, but I am unable, as root, to force system processes to move to another CPU. Any ideas?

I can find no documentation about how to force an interrupt handler to a specific CPU - is this possible without modifying the kernel?

--Pete

#1 lore raw
Quoted 12 lines · [email protected] wrote:

Hi-

I am trying to use a quad Opteron motherboard with SMP Kernel 2.6.5 for a quasi-real-time task. I need to assign all processes to specific CPUs, including interrupt handlers.
I have had success using sched_setaffinity() to set the CPU for processes I create, but I am unable, as root, to force system processes to move to another CPU. Any ideas?

I can find no documentation about how to force an interrupt handler to a specific CPU - is this possible without modifying the kernel?

IOApic's support binding of interrupt delivery in intel based platforms, but I am unaware of tools which force this setting by default on Linux, but someone else may be able to point you in that direction. Most folks code APIC ICC delivery to
AV_LOPRI (meaning lowest priority processor gets next interrupt). This is advantageous for cache coherency since the IRQ code
is probaby still in that processors cache. You may have to modify the kernel. Linux doesn't allow processors to be shutdown and reactiviated real time, it just starts them and lets them run, so you don;t have to worry about the case of migrating interrupts off pinned APICs. The APIC supports what you are asking for, but I am not certain anyone implemented anything other than AV_LOPRI settings by default in the IO APIC code. I would suggest you look over the IO APIC Code -- this is a lot of work, BTW.

Jeff

Quoted 9 lines

--Pete
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

#2 lore raw
Quoted 7 lines · Jeffrey V. Merkey

IOApic's support binding of interrupt delivery in intel based platforms, but I am unaware of tools which force this
setting by default on Linux, but someone else may be able to point you in that direction. Most folks code APIC ICC delivery to
AV_LOPRI (meaning lowest priority processor gets next interrupt). This is
advantageous for cache coherency since the IRQ code
is probaby still in that processors cache. You may have to modify the kernel.

Quoted 8 lines

Linux doesn't allow processors to be shutdown and
reactiviated real time, it just starts them and lets them run, so you don;t have to worry about the case of migrating interrupts
off pinned APICs. The APIC supports what you are asking for, but I am not certain anyone implemented anything other
than AV_LOPRI settings by default in the IO APIC code. I would suggest you look over the IO APIC Code -- this is a lot
of work, BTW.

Or you could just put something which writes to
/proc/irq/$IRQ/smp_affinity in your initscripts.

Cheers,
Zwane

#3 lore raw
In reply to Jeffrey V. Merkey
Quoted 16 lines · Jeffrey V. Merkey
Quoted 12 lines · [email protected] wrote:

Hi-

I am trying to use a quad Opteron motherboard with SMP Kernel 2.6.5 for a quasi-real-time task. I need to assign all processes to specific CPUs, including interrupt handlers.
I have had success using sched_setaffinity() to set the CPU for processes I create, but I am unable, as root, to force system processes to move to another CPU. Any ideas?

I can find no documentation about how to force an interrupt handler to a specific CPU - is this possible without modifying the kernel?

Quoted 4 lines

IOApic's support binding of interrupt delivery in intel based platforms, but I am unaware of tools which force this
setting by default on Linux, but someone else may be able to point you in that direction.

http://people.redhat.com/arjanv/irqbalance

#4 lore raw
In reply to [email protected]
Quoted 3 lines · On Wed, 2005-11-02 at 17:50 +0000, [email protected] wrote:

Hi-

I am trying to use a quad Opteron motherboard with SMP Kernel 2.6.5 for a quasi-real-time task.

2.6.5 is a terrible choice for soft realtime applications, the serious work on making 2.6 a viable soft RT platform didn't even start until around 2.6.8. Try 2.6.12 or newer (ideally 2.6.14).

Lee