Routable IRQs
Hi Thomas & Jason,
I'm dealing with an interesting situation which I'm wondering if Linux already support for.
Basically, in some TI SoCs we have what's referred to as Programmable Real-Time Unit SubSystem (PRUSS). That's essentially a really simple, low latency, single cycle architecture which is pretty darn good for bit banging peripherals (ETH, SPI, I2C, UART, etc). It's very predicatable because every instruction takes 5ns and interrupts don't cause exceptions, they just get registered.
Anyway, the interesting part is that PRUSS has 64 events (on current incarnations at least) and PRUSS has 10 physical IRQ lines to the ARM land. Each of these 64 events can be routed to any of these 10 IRQ lines. This might not be very useful on UP (AM335x & AM437x) other than the fact that soft-IP drivers running on Linux would need to guarantee they are the ones who should handle the IRQ. However, on SMP (AM57xx) we could have real tangible benefits by means of IRQ affinity, etc.
So, the question is, what is there in IRQ subsystem today for routable IRQ support ?
If a Diagram helps here's a simple one. Note that I'm not showing details on the PRUSS side, but that side can also map events pretty much any way it wants.
.--------------------. .--------------------. | HOST CPU | | PRUSS | |--------------------| |--------------------| | | | | | irq0 |<-.----------|evt0 | | | | | | | irq1 | | .-------|evt1 | | | | | | | | irq2 | '----------|evt2 | | | | | | | irq3 | | | | | | | | | | irq4 | | | . | | | | | | | irq5 | | | . | | | | | | | irq6 | | | . | | | | | | | irq7 |<----' | | | | | | | irq8 | | | | | | | | irq9 |<------------|evtN | '--------------------' '--------------------'
Given this setup, what I want to do, is let soft-IP drivers running on linux rely on standard *request_*irq() calls and DTS descrition. But I'm still considering how/if we should describe the routing itself or just go round-robin (i.o.w. irq0 -> evt0, irq1 -> evt1, ..., irq9 -> evt9, irq0 -> evt10, ...).
Thoughts ?
Signature · 1 line
balbi
Felipe,
Quoted 40 lines · Felipe Balbi
Anyway, the interesting part is that PRUSS has 64 events (on current incarnations at least) and PRUSS has 10 physical IRQ lines to the ARM land. Each of these 64 events can be routed to any of these 10 IRQ lines. This might not be very useful on UP (AM335x & AM437x) other than the fact that soft-IP drivers running on Linux would need to guarantee they are the ones who should handle the IRQ. However, on SMP (AM57xx) we could have real tangible benefits by means of IRQ affinity, etc.
So, the question is, what is there in IRQ subsystem today for routable IRQ support ?
If a Diagram helps here's a simple one. Note that I'm not showing details on the PRUSS side, but that side can also map events pretty much any way it wants.
.--------------------. .--------------------. | HOST CPU | | PRUSS | |--------------------| |--------------------| | | | | | irq0 |<-.----------|evt0 | | | | | | | irq1 | | .-------|evt1 | | | | | | | | irq2 | '----------|evt2 | | | | | | | irq3 | | | | | | | | | | irq4 | | | . | | | | | | | irq5 | | | . | | | | | | | irq6 | | | . | | | | | | | irq7 |<----' | | | | | | | irq8 | | | | | | | | irq9 |<------------|evtN | '--------------------' '--------------------'
Quoted 7 lines
Given this setup, what I want to do, is let soft-IP drivers running on linux rely on standard *request_*irq() calls and DTS descrition. But I'm still considering how/if we should describe the routing itself or just go round-robin (i.o.w. irq0 -> evt0, irq1 -> evt1, ..., irq9 -> evt9, irq0 -> evt10, ...).
Thoughts ?
I have a few questions:
- Is there a "mapping" block between PRUSS and the host interrupt controller or is this "mapping" block part of PRUSS?
- We all know how well shared interrupts work. Is there a point of supporting 64 interrupts when you only have 10 irq lines available?
- I assume that the PRUSS interrupt mapping is more or less a question of the firmware implementation. So you either have a fixed association in the firmware which is reflected in the DT description of the IP block or you need an interface to tell the PRUSS firmware which event it should map to which irq line. Is there actually a value in doing the latter?
Thanks,
tglx
Hi Thomas,
Quoted 47 lines · Thomas Gleixner
Quoted 46 lines · Felipe Balbi
Anyway, the interesting part is that PRUSS has 64 events (on current incarnations at least) and PRUSS has 10 physical IRQ lines to the ARM land. Each of these 64 events can be routed to any of these 10 IRQ lines. This might not be very useful on UP (AM335x & AM437x) other than the fact that soft-IP drivers running on Linux would need to guarantee they are the ones who should handle the IRQ. However, on SMP (AM57xx) we could have real tangible benefits by means of IRQ affinity, etc.
So, the question is, what is there in IRQ subsystem today for routable IRQ support ?
If a Diagram helps here's a simple one. Note that I'm not showing details on the PRUSS side, but that side can also map events pretty much any way it wants.
.--------------------. .--------------------. | HOST CPU | | PRUSS | |--------------------| |--------------------| | | | | | irq0 |<-.----------|evt0 | | | | | | | irq1 | | .-------|evt1 | | | | | | | | irq2 | '----------|evt2 | | | | | | | irq3 | | | | | | | | | | irq4 | | | . | | | | | | | irq5 | | | . | | | | | | | irq6 | | | . | | | | | | | irq7 |<----' | | | | | | | irq8 | | | | | | | | irq9 |<------------|evtN | '--------------------' '--------------------'Given this setup, what I want to do, is let soft-IP drivers running on linux rely on standard *request_*irq() calls and DTS descrition. But I'm still considering how/if we should describe the routing itself or just go round-robin (i.o.w. irq0 -> evt0, irq1 -> evt1, ..., irq9 -> evt9, irq0 -> evt10, ...).
Quoted 6 lines
Quoted 1 line
Thoughts ?
I have a few questions:
- Is there a "mapping" block between PRUSS and the host interrupt controller or is this "mapping" block part of PRUSS?
The description in TRM is a bit "poor", but from what I can gather, the mapping is done on an interrupt controller inside the PRUSS. However, Linux is the one who's got the driver for that INTC (well, Linux will be the one with the soft ethernet/uart/whatever IP to talk to). All of its (INTC's) registers are memory mapped to the ARM side.
Quoted 2 lines
- We all know how well shared interrupts work. Is there a point of supporting 64 interrupts when you only have 10 irq lines available?
I'm looking at these 64 events more like MSI kind of events. It's just that the events themselves can be routed to any of the 10 available HW IRQ lines.
Quoted 5 lines
- I assume that the PRUSS interrupt mapping is more or less a question of the firmware implementation. So you either have a fixed association in the firmware which is reflected in the DT description of the IP block or you need an interface to tell the PRUSS firmware which event it should map to which irq line. Is there actually a value in doing the latter?
right, I'd say the mapping is pretty static. Unless Suman has some extra information which I don't. I guess the question was really to see if there was an easy way for doing this so we don't have to mess with DTS for every other FW and their neighbor.
Chances are (or at least I'm speculating) in most cases we won't use more than 10 events anyway (Suman ?) so we might not run into any troubles.
Signature · 1 line
balbi
Felipe,
Quoted 9 lines · Felipe Balbi
Quoted 2 lines · Thomas Gleixner
- Is there a "mapping" block between PRUSS and the host interrupt controller or is this "mapping" block part of PRUSS?
The description in TRM is a bit "poor", but from what I can gather, the mapping is done on an interrupt controller inside the PRUSS. However, Linux is the one who's got the driver for that INTC (well, Linux will be the one with the soft ethernet/uart/whatever IP to talk to). All of its (INTC's) registers are memory mapped to the ARM side.
Ok. And the INTC registers include the "mapping" configuration, right?
Quoted 4 lines
Quoted 2 lines
- We all know how well shared interrupts work. Is there a point of supporting 64 interrupts when you only have 10 irq lines available?
I'm looking at these 64 events more like MSI kind of events. It's just
Well, that's fine to look at them this way, but they will end up shared no matter what.
Quoted 9 lines
that the events themselves can be routed to any of the 10 available HW IRQ lines.
Quoted 5 lines
- I assume that the PRUSS interrupt mapping is more or less a question of the firmware implementation. So you either have a fixed association in the firmware which is reflected in the DT description of the IP block or you need an interface to tell the PRUSS firmware which event it should map to which irq line. Is there actually a value in doing the latter?
Quoted 4 lines
right, I'd say the mapping is pretty static. Unless Suman has some extra information which I don't. I guess the question was really to see if there was an easy way for doing this so we don't have to mess with DTS for every other FW and their neighbor.
Well, you will need information about every other firmware simply because you need to know which events the firmware is actually using and what the purpose of the particular event is.
Assume you have a simple uart with 3 events (RX, TX, status). So how will the firmware tell you which event is which? You have a few options:
1) DT + fixed mapping scheme:
Describe the PRUSS event number in DT and have a fixed mapping scheme like the one you mentioned evt0 -> irq0 .....
2) DT + DT mapping scheme
Describe the PRUSS event number in DT and describe the mapping scheme in DT as well
3) DT + dynamic mapping scheme
Describe the PRUSS event number in DT and let your interrupt controller associate the irq number dynamically. That's kind of similar to MSI with the exception that it needs to support shared interrupts.
4) Fully dynamic association
Have a query interface to the firmware which tells you which event it uses for which particular purpose (RX, TX ...) and then establish a dynamic mapping to one of the interrupts.
Not sure which level of complexity you want :)
Thanks,
tglx
Hi,
Quoted 7 lines · Thomas Gleixner
Felipe,
Quoted 4 lines · Felipe Balbi
Quoted 2 lines · Thomas Gleixner
- Is there a "mapping" block between PRUSS and the host interrupt controller or is this "mapping" block part of PRUSS?
Quoted 7 lines
Quoted 5 lines
The description in TRM is a bit "poor", but from what I can gather, the mapping is done on an interrupt controller inside the PRUSS. However, Linux is the one who's got the driver for that INTC (well, Linux will be the one with the soft ethernet/uart/whatever IP to talk to). All of its (INTC's) registers are memory mapped to the ARM side.
Ok. And the INTC registers include the "mapping" configuration, right?
right. A bunch of 32 bit registers each with several 4 bit fields (one for each of the 64 events) where we write the physical IRQ number.
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines
- We all know how well shared interrupts work. Is there a point of supporting 64 interrupts when you only have 10 irq lines available?
I'm looking at these 64 events more like MSI kind of events. It's just
Well, that's fine to look at them this way, but they will end up shared no matter what.
sure :-)
Quoted 37 lines
Quoted 9 lines
that the events themselves can be routed to any of the 10 available HW IRQ lines.
Quoted 5 lines
- I assume that the PRUSS interrupt mapping is more or less a question of the firmware implementation. So you either have a fixed association in the firmware which is reflected in the DT description of the IP block or you need an interface to tell the PRUSS firmware which event it should map to which irq line. Is there actually a value in doing the latter?
Quoted 4 lines
right, I'd say the mapping is pretty static. Unless Suman has some extra information which I don't. I guess the question was really to see if there was an easy way for doing this so we don't have to mess with DTS for every other FW and their neighbor.
Well, you will need information about every other firmware simply because you need to know which events the firmware is actually using and what the purpose of the particular event is.
Assume you have a simple uart with 3 events (RX, TX, status). So how will the firmware tell you which event is which? You have a few options:
1) DT + fixed mapping scheme:
Describe the PRUSS event number in DT and have a fixed mapping scheme like the one you mentioned evt0 -> irq0 .....
2) DT + DT mapping scheme
Describe the PRUSS event number in DT and describe the mapping scheme in DT as well
3) DT + dynamic mapping scheme
Describe the PRUSS event number in DT and let your interrupt controller associate the irq number dynamically. That's kind of similar to MSI with the exception that it needs to support shared interrupts.
Quoted 7 lines
4) Fully dynamic association
Have a query interface to the firmware which tells you which event it uses for which particular purpose (RX, TX ...) and then establish a dynamic mapping to one of the interrupts.
Not sure which level of complexity you want :)
I guess only 1, 2 are anything worth considering, most likely. 4 would just be too much headache :-p
3 might be doable too, though a bit more complex. Suman (who has been working on this for much longer than I have) might have some extra info to add, but he's on vacations for now. Hopefully, he'll add to this thread once he's back.
cheers
Signature · 1 line
balbi