Skip to contentSkip to conversation

Preempt Real-time for ARM

DISCUSSION Daniel Walker 5 participants 9 messages Feb 5, 2005 – Feb 10, 2005

lore
Share & tools
lore
#0 lore raw

This is a release of Preempt Real-time for ARM . It includes everything up to CONFIG_PREEMPT_RT , and all of the latency tracing except interrupts off timing. The timing also excludes syscalls. This patch includes only a port to OMAP boards. However, it should be straight forward to get it working on other boards.

The biggest point of discussion relates to the interrupts in threads implementation. It is largely identical to what is implemented in the generic irq handling. However, ARM doesn't not implement generic irq handling, and will not support it in the near future. I am not in support of two different threaded interrupt implementations.

I recently made a proposal to separate the threaded interrupt handling from the generic irq handling, but I'm open to other ideas.

The patch can be found at the following url

ftp://source.mvista.com/pub/realtime/arm/common_arm_realtime.patch

The patch order is as follows

http://www.kernel.org/pub/linux/kernel/v2.6/testing/patch-2.6.11-rc3.bz2
http://people.redhat.com/~mingo/realtime-preempt/realtime-preempt-2.6.11-rc3-V0.7.38-01
common_arm_realtime.patch

Enjoy!
Daniel Walker

#1 lore raw
Quoted 5 lines · Daniel Walker

The biggest point of discussion relates to the interrupts in threads implementation. It is largely identical to what is implemented in the generic irq handling. However, ARM doesn't not implement generic irq handling, and will not support it in the near future. I am not in support of two different threaded interrupt implementations.

We have done the conversion to the generic irq handling and it works fine on a couple of machines.

I'm just waiting until the new SMP bits are there before I have another go and clean up the missing SMP bits.

tglx

#2 lore raw
Quoted 10 lines · Thomas Gleixner
Quoted 5 lines · Daniel Walker

The biggest point of discussion relates to the interrupts in threads implementation. It is largely identical to what is implemented in the generic irq handling. However, ARM doesn't not implement generic irq handling, and will not support it in the near future. I am not in support of two different threaded interrupt implementations.

We have done the conversion to the generic irq handling and it works fine on a couple of machines.

great - this would be a much preferred approach indeed.

Quoted 2 lines

I'm just waiting until the new SMP bits are there before I have another go and clean up the missing SMP bits.

any chances for (most of) these bits going upstream as well? In any case, the -RT tree can be a testbed for this.

Ingo

#3 lore raw
Quoted 5 lines · Ingo Molnar
Quoted 2 lines

I'm just waiting until the new SMP bits are there before I have another go and clean up the missing SMP bits.

any chances for (most of) these bits going upstream as well? In any case, the -RT tree can be a testbed for this.

I guess this has to be discussed with Russell in detail. The UP bits are quite simple and we can preserve the ARM oddities by tweaking the generic layer a bit. The SMP stuff was/is work in progress, but what I have seen until now is not too scary, but needs some careful thoughts.

tglx

#4 lore raw
In reply to Ingo Molnar
Quoted 11 lines · Ingo Molnar
Quoted 8 lines · Thomas Gleixner
Quoted 5 lines · Daniel Walker

The biggest point of discussion relates to the interrupts in threads implementation. It is largely identical to what is implemented in the generic irq handling. However, ARM doesn't not implement generic irq handling, and will not support it in the near future. I am not in support of two different threaded interrupt implementations.

Quoted 4 lines
Quoted 2 lines

We have done the conversion to the generic irq handling and it works fine on a couple of machines.

great - this would be a much preferred approach indeed.

Well, I remain unconvinced about the generic irq handling.

Back in 2.4 times, ARM used to use the x86 way to handle IRQs, and it caused lots of dropped IRQs for CF cards and the like, particularly in mixed level/edge triggered interrupt environments (where a mixture of level and edge based outputs are connected to edge triggered inputs.)

The ARM IRQ code got completely rewritten during 2.5 with a clean design, generated from the requirements of the machines.

This caused major changes throughout all the machine support files, and I'm _NOT_, repeat _NOT_ going to consider going back to some half baked approach which doesn't really fit the needs of the ARM architecture, just because "oh, it's generic." If it doesn't work reliably, I'm not interested.

This is especially so when it impacts so many machines in ways specific to each machine, and there's no way to get them tested in one go.

If this is to be done, doing it in the middle of a stable kernel series is NOT the time or place to do it. I have recently had people complaining about the "stability" of 2.6, particularly in relation to changes made by other people affecting drivers.

Consider these questions in relation to the generic IRQ code:

1. Does it know the difference between handling level, edge-based and "simple" IRQs? ("simple" IRQs are those which are cascaded, but don't have their own individual interrupt mask controls.)

2. Does the generic autoprobe code know which IRQs can be autoprobed and which can't? (cascade interrupts are just one example of interrupts which must not be autoprobed. There may be other reasons you wish to avoid probing other interrupts on a particular machine, which the machine support code knows about.)

3. Does the generic IRQ code know which IRQs can be claimed and which can't? (IRQs 0 to NR_IRQS aren't always claimable, even when they appear to be available - iow, desc->handler != &no_irq_type.)

4. Does it allow per "hw type" retriggering of interrupts, even if the hardware itself is not capable of such an action? (and running these interrupts at the next hardware interrupt?)

5. Does it allow control of interrupt wakeup sources?

6. Does it allow architectures to define their own irq_desc_t so that all the data for a particular IRQ is localised and contained within one data structure?

What you'll find is that the ARM interrupt structure is designed to efficiently meet the requirements of our wide range of hardware interrupt controllers, with chained interrupt controllers, with as low latency as possible.

In essence, I'm opposed to completely rewriting the ARM interrupt handling at this stage.

Signature · 4 lines
Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:  2.6 PCMCIA      - http://pcmcia.arm.linux.org.uk/
                 2.6 Serial core
#5 lore raw

Russell,

Quoted 5 lines · Russell King
Quoted 4 lines
Quoted 2 lines

We have done the conversion to the generic irq handling and it works fine on a couple of machines.

great - this would be a much preferred approach indeed.

Quoted 7 lines

Well, I remain unconvinced about the generic irq handling.

This caused major changes throughout all the machine support files, and I'm _NOT_, repeat _NOT_ going to consider going back to some half baked approach which doesn't really fit the needs of the ARM architecture, just because "oh, it's generic." If it doesn't work reliably, I'm not interested.

Agreed. Doing it just for the sake of a generic solution is not the point.

Quoted 3 lines

This is especially so when it impacts so many machines in ways specific to each machine, and there's no way to get them tested in one go.

Ack.

Quoted 10 lines

If this is to be done, doing it in the middle of a stable kernel series is NOT the time or place to do it. I have recently had people complaining about the "stability" of 2.6, particularly in relation to changes made by other people affecting drivers.

Consider these questions in relation to the generic IRQ code:

1. Does it know the difference between handling level, edge-based and "simple" IRQs? ("simple" IRQs are those which are cascaded, but don't have their own individual interrupt mask controls.)

The changes we made to the generic layer in order to make it work on ARM still call the level/edge/simple handlers.

Quoted 5 lines

2. Does the generic autoprobe code know which IRQs can be autoprobed and which can't? (cascade interrupts are just one example of interrupts which must not be autoprobed. There may be other reasons you wish to avoid probing other interrupts on a particular machine, which the machine support code knows about.)

Makes sense to add this to the generic code anyway.

Quoted 3 lines

3. Does the generic IRQ code know which IRQs can be claimed and which can't? (IRQs 0 to NR_IRQS aren't always claimable, even when they appear to be available - iow, desc->handler != &no_irq_type.)

There's a check whether a particular irq has been exclusively allocated or is available for driver use. Not sure, if it does exactly the same what you are talking about. Needs to be checked.

Quoted 3 lines

4. Does it allow per "hw type" retriggering of interrupts, even if the hardware itself is not capable of such an action? (and running these interrupts at the next hardware interrupt?)

I implemented this basically, but it's not fully tested/functional yet.

Quoted 1 line

5. Does it allow control of interrupt wakeup sources?

Are you talking about PM wakeup ? If yes, it is not there, but isn't this functionality useful for other platforms than ARM too ?

Quoted 3 lines

6. Does it allow architectures to define their own irq_desc_t so that all the data for a particular IRQ is localised and contained within one data structure?

Having a generic irq_desc_t with an architecture specific member "struct arch_irq_desc_t bla;" should do the job.

Quoted 2 lines

In essence, I'm opposed to completely rewriting the ARM interrupt handling at this stage.

As expected :)
I accept your decision, but I hope that you are not completely opposed to discuss a conversion at all, especially as other architectures will benefit from the necessary changes to the generic code too.

Thanks for the pointers so far.

tglx

#6 lore raw
In reply to Russell King
Quoted 7 lines · Russell King

What you'll find is that the ARM interrupt structure is designed to efficiently meet the requirements of our wide range of hardware interrupt controllers, with chained interrupt controllers, with as low latency as possible.

In essence, I'm opposed to completely rewriting the ARM interrupt handling at this stage.

Everyone wants this as the final solution, but all I want right now is
to have a clean patch for ARM RT .. It would be nice if ARM had the generics, but It's not _my_ goal.

All I want to do is integrate the common IRQ threading code. To do that
I need things , from Russell, like per descriptor locks .. And I need things , from Ingo, like pulling out the IRQ threading code..

Daniel

#7 lore raw
Quoted 3 lines · Daniel Walker

All I want to do is integrate the common IRQ threading code. To do that
I need things , from Russell, like per descriptor locks .. And I need things , from Ingo, like pulling out the IRQ threading code..

I've said why per-IRQ locks are incorrect for the non-RT cases on ARM, but unfortunately just repeating the reasons why it's wrong isn't getting me anywhere either. So shrug, all I can to is explain why it's wrong, and if people choose not to listen there's nothing more I can do.

Signature · 4 lines
Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:  2.6 PCMCIA      - http://pcmcia.arm.linux.org.uk/
                 2.6 Serial core
#8 lore raw
Quoted 7 lines · Russell King
Quoted 3 lines · Daniel Walker

All I want to do is integrate the common IRQ threading code. To do that
I need things , from Russell, like per descriptor locks .. And I need things , from Ingo, like pulling out the IRQ threading code..

Quoted 5 lines

I've said why per-IRQ locks are incorrect for the non-RT cases on ARM, but unfortunately just repeating the reasons why it's wrong isn't getting me anywhere either. So shrug, all I can to is explain why it's wrong, and if people choose not to listen there's nothing more I can do.

Lets summarize your main arguments from two threads - "irq_controller_lock" and this one:
(sorry, I summarized since I somehow accidently lost traack of "irq_controller_lock" thread and want to be sure I haven't missed anything)

1) if we drop ide_controller_lock we need to add per-chip lock due to read-modify-write issue

true

2) per-descriptor lock will not bring gains since
a) SMP - almost nonexistent at the moment

As Daniel said - why not look to the future - did anybody expect 3 month
ago RT enchancement for Linux?! progress is too quick - and again from
the perspective of SMP - irq_controller_lock is defective.

b) lots of contention on request_irq/free_irq - rare
seems true
c) multiple devices on the same interrupt line - rare
seems true

But in a whole it's not so unambiguously what outweighs - b)&c) against contra a)

3) per chip lock in combination with per descriptor lock
a) decreases peformance

why not to lock per-chip lock only for chips indeed require this (i.e. with read/modify/write/) and drop the locking otherwise?!

b) (quoatition):
"Yes, and then audit that no one uses different irqchip structures
covering the same register (consider a read-modify-write mask register where some IRQs are edge and others are level riggered.)"

such a register and a chip have ono-to-one relationship, do they? chip lock is something connected to _the_ chip. The above situation is definitly up to a developer and his own _fault_.

4) ARM IRQs are already "threaded"

As you said: can we _please_ get the terminology right? As Nicolas pointed you were talking about completely different "threaded".

Eugeny