Skip to contentSkip to conversation

ULI 2.6.12

PATCH In review Michael Raymond 3 participants 5 messages Apr 1, 2005 – Apr 4, 2005

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 ULI 2.6.12

Here's a patch of the ULI code to Tony's pre-2.6.12 tree.

I was able to show that ULIs cannot be scheduled onto other CPUs. For
every IRQ that a user requests it registers a standard handler. All IRQ handlers are called out of do_IRQ(). It's not until do_IRQ() is called that ia64_eoi() is called to complete the ACK and allow lower priority interrupts into the system.
I've added a simple stats program that makes use of ULIs to the support
package on the web site (1). We've also found a case where their low perturbation will be useful for user level profiling. I'll be writing that up next.
Thanks,
Michael
(1) http://oss.sgi.com/projects/uli/

Signature · 2 lines
Michael A. Raymond              Office: (651) 683-3434
Core OS Group                   Real-Time System Software
#1 lore raw
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Michael" = Michael Raymond <[email protected]> writes:

Michael> Here's a patch of the ULI code to Tony's pre-2.6.12 tree.
Michael> Signed-off-by: Michael A Raymond <[email protected]>

Michael> All IRQ handlers are called out of
Michael> do_IRQ(). It's not until do_IRQ() is called that ia64_eoi() Michael> is called to complete the ACK and allow lower priority Michael> interrupts into the system.

This is one thing that bothers me most about this approach. User code is invoked before acknowledging the interrupt to the interrupt controller. As a general rule, kernel code should not trust user code for its correct operation. If the ULI does not clear the interrupt, (because it is malicious, say) then there is a distinct possibility of DOS here.

However, I think that the thing that bothers me *most* about SGI's ULI approach is that a full context switch is not done. The ULI runs as if it were in the interrupted process's context. `current' isn't changed, so it runs with the privileges of the interrupted process. The *way* it runs (CPU bound, presumably) will affect the scheduler's decisions about how to run the interrupted process in the next timeslice. For most interrupt handlers this won't matter, but it'd be relatively easy to construct a malicious one to slow particular processes.

Running at interrupted process provilege doesn't matter very much, because all system calls are mapped onto uli_syscall if you're running as a user-level interrupt. However, this severely limits what you can do in a handler. For example, you can't wake threads waiting on a futex (something we find very common in a user-level interrupt handler).

--
Dr Peter Chubb http://www.gelato.unsw.edu.au peterc AT gelato.unsw.edu.au The technical we do immediately, the political takes *forever*

#2 lore raw
In reply to Michael Raymond
Quoted 9 lines · Peter Chubb

However, I think that the thing that bothers me *most* about SGI's ULI approach is that a full context switch is not done. The ULI runs as if it were in the interrupted process's context. `current' isn't changed, so it runs with the privileges of the interrupted process. The *way* it runs (CPU bound, presumably) will affect the scheduler's decisions about how to run the interrupted process in the next timeslice. For most interrupt handlers this won't matter, but it'd be relatively easy to construct a malicious one to slow particular processes.

I agree. Raymonds version looks like a hack for a very narrow special
case to me. I really prefer your more portable and useful version, and
I don't think the performance differences matter given today hardware.

#3 lore raw
In reply to Michael Raymond

These are all valid concerns.

Quoted 6 lines · Peter Chubb

This is one thing that bothers me most about this approach. User code is invoked before acknowledging the interrupt to the interrupt controller. As a general rule, kernel code should not trust user code for its correct operation. If the ULI does not clear the interrupt, (because it is malicious, say) then there is a distinct possibility of DOS here.

The idea is that only trusted code by priviledged users will be run.
Let's assume for a moment that this isn't true. My understanding of the IA64 code is the following.

- Interrupt handling ends up in __do_IRQ()
- __do_IRQ() ACKs the interrupt with an infrastructure provided function
- control passes to handle_IRQ_event()
- Each registered handler is run. These include traditional kernel drivers, ULIs, etc
- control returns through __do_IRQ() to ia64_handle_irq()
- ia64_eoi() is called to lower the blocked interrupt class and allow the interrupt to be handled again

Quoted 9 lines

However, I think that the thing that bothers me *most* about SGI's ULI approach is that a full context switch is not done. The ULI runs as if it were in the interrupted process's context. `current' isn't changed, so it runs with the privileges of the interrupted process. The *way* it runs (CPU bound, presumably) will affect the scheduler's decisions about how to run the interrupted process in the next timeslice. For most interrupt handlers this won't matter, but it'd be relatively easy to construct a malicious one to slow particular processes.

Again the idea is that the handler function was registered by root or
some priviledged developer making a conscious decision. This is currently protected by permissions on the /dev/uli file. If you'd like me to add permission checks to the kernel functions I'd be happy to do it.
Another help here is that handlers can only run for a certain amount of
time. I think the arbitrarily chosen limit right now is ~10us. A handler that has to do anything more than simple work should wake up a thread / process to do so. I can make that time limit configurable if you'd like.

Quoted 5 lines

Running at interrupted process provilege doesn't matter very much, because all system calls are mapped onto uli_syscall if you're running as a user-level interrupt. However, this severely limits what you can do in a handler. For example, you can't wake threads waiting on a futex (something we find very common in a user-level interrupt handler).

Currently its own semaphore approach is supported. If you'd like me to
switch it to use futexes I can do that.
Thanks,
Michael

Signature · 2 lines
Michael A. Raymond              Office: (651) 683-3434
Core OS Group                   Real-Time System Software
#4 lore raw
In reply to Michael Raymond
Quoted 4 lines · Michael Raymond

Again the idea is that the handler function was registered by root or
some priviledged developer making a conscious decision. This is currently protected by permissions on the /dev/uli file. If you'd like me to add permission checks to the kernel functions I'd be happy to do it.

No, that's not the point. User level interrupts are totally pointless if they can cause the same problems on misbehaviour.