Skip to contentSkip to conversation

network driver disabled interrupts in PREEMPT_RT

DISCUSSION Kristian Benoit 5 participants 11 messages Jun 13, 2005 – Jul 5, 2005 Network drivers

lore
Share & tools
lore
#0 lore raw

Hi,
I got lots of these messages when accessing the net running 2.6.12-rc6-RT-V0.7.48-25 :

"network driver disabled interrupts: tg3_start_xmit+0x0/0x629 [tg3]"

it seem to come from net/sched/sch_generic.c.

As of the response time I get, it does not seem normal.

Kristian

#1 lore raw
Quoted 7 lines · Kristian Benoit

Hi,
I got lots of these messages when accessing the net running 2.6.12-rc6-RT-V0.7.48-25 :

"network driver disabled interrupts: tg3_start_xmit+0x0/0x629 [tg3]"

it seem to come from net/sched/sch_generic.c.

does the patch below fix it?

Ingo

drivers/net/tg3.c renamed from drivers/net/tg3.c.orig +2 -2
@@ -3242,9 +3242,9 @@ static int tg3_start_xmit(struct sk_buff
 	 * So we really do need to disable interrupts when taking
 	 * tx_lock here.
 	 */
-	local_irq_save(flags);
+	local_irq_save_nort(flags);
 	if (!spin_trylock(&tp->tx_lock)) { 
-		local_irq_restore(flags);
+		local_irq_restore_nort(flags);
 		return NETDEV_TX_LOCKED; 
 	} 
 
#2 lore raw
Quoted 7 lines · Ingo Molnar
Quoted 4 lines · Kristian Benoit

Hi,
I got lots of these messages when accessing the net running 2.6.12-rc6-RT-V0.7.48-25 :

Quoted 5 lines
Quoted 3 lines

"network driver disabled interrupts: tg3_start_xmit+0x0/0x629 [tg3]"

it seem to come from net/sched/sch_generic.c.

does the patch below fix it?

Wouldn't it be much more useful to add spin_trylock_irq?

#3 lore raw
Quoted 8 lines · Christoph Hellwig
Quoted 7 lines · Ingo Molnar
Quoted 4 lines · Kristian Benoit

Hi,
I got lots of these messages when accessing the net running 2.6.12-rc6-RT-V0.7.48-25 :

Quoted 7 lines
Quoted 5 lines
Quoted 3 lines

"network driver disabled interrupts: tg3_start_xmit+0x0/0x629 [tg3]"

it seem to come from net/sched/sch_generic.c.

does the patch below fix it?

Wouldn't it be much more useful to add spin_trylock_irq?

you are right, a spin_trylock_irqsave() call would be much cleaner.

i think i should explain why you are seeing such hacks in the PREEMPT_RT
patch: generally we are trying to achieve near-100% driver API compatibility (that of course is also one of the requirements for any of this to hit mainline), so i'm marking any deviations with _nort or _rt postfixes (depending on where the change in semantics is needed). I consider them temporary hacks, so i'm usually trying to keep the original form of the code, so that i can go back later and fix it.

I had these hacks in tg3.c for some time, and i thought i could drop them now that local_irq_*() uses the soft IRQ-flag - but i was wrong. One example of the _rt/_nort marking process is e.g. the earlier RCU API related grossness, which went away with Paul's PREEMPT_RCU aproach and now PREEMPT_RT is fully compatible with the RCU API. Fortunately, assymetric local_irq_* + spin_lock_* uses (which are perfectly legal!) are relatively rare. Extending the spinlock APIs and converting all upstream code would be a good approach to solve this problem, as it's also a cleanup. (It would probably also make static lock analysis easier.)

Ingo

#4 lore raw
In reply to Christoph Hellwig
Quoted 8 lines · Christoph Hellwig
Quoted 7 lines · Ingo Molnar
Quoted 4 lines · Kristian Benoit

Hi,
I got lots of these messages when accessing the net running 2.6.12-rc6-RT-V0.7.48-25 :

Quoted 7 lines
Quoted 5 lines
Quoted 3 lines

"network driver disabled interrupts: tg3_start_xmit+0x0/0x629 [tg3]"

it seem to come from net/sched/sch_generic.c.

does the patch below fix it?

Wouldn't it be much more useful to add spin_trylock_irq?

Dave has a patch pending that should remove these IRQ disabling calls.

Cheers,

Signature · 4 lines
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
#6 lore raw
In reply to Ingo Molnar
Quoted 6 lines · Ingo Molnar
Quoted 4 lines · Kristian Benoit

Hi,
I got lots of these messages when accessing the net running 2.6.12-rc6-RT-V0.7.48-25 :

Quoted 7 lines
Quoted 3 lines

"network driver disabled interrupts: tg3_start_xmit+0x0/0x629 [tg3]"

it seem to come from net/sched/sch_generic.c.

does the patch below fix it?

Ingo

I have the same problem with an e1000 card for 2.6.12-rc6-RT-V0.7.48-32 :
#dmesg
...
network driver disabled interrupts: e1000_xmit_frame+0x0/0xbc0 [e1000]
...

#7 lore raw
Quoted 11 lines · Serge Noiraud
Quoted 10 lines · Ingo Molnar
Quoted 7 lines · Kristian Benoit

Hi,
I got lots of these messages when accessing the net running 2.6.12-rc6-RT-V0.7.48-25 :

"network driver disabled interrupts: tg3_start_xmit+0x0/0x629 [tg3]"

it seem to come from net/sched/sch_generic.c.

Quoted 8 lines
Quoted 3 lines

does the patch below fix it?

Ingo

I have the same problem with an e1000 card for 2.6.12-rc6-RT-V0.7.48-32 :
#dmesg
...
network driver disabled interrupts: e1000_xmit_frame+0x0/0xbc0 [e1000]
...

does -48-33 fix it for you?

Ingo

#8 lore raw
Quoted 22 lines · Ingo Molnar
Quoted 11 lines · Serge Noiraud
Quoted 10 lines · Ingo Molnar
Quoted 7 lines · Kristian Benoit

Hi,
I got lots of these messages when accessing the net running 2.6.12-rc6-RT-V0.7.48-25 :

"network driver disabled interrupts: tg3_start_xmit+0x0/0x629 [tg3]"

it seem to come from net/sched/sch_generic.c.

Quoted 8 lines
Quoted 3 lines

does the patch below fix it?

Ingo

I have the same problem with an e1000 card for 2.6.12-rc6-RT-V0.7.48-32 :
#dmesg
...
network driver disabled interrupts: e1000_xmit_frame+0x0/0xbc0 [e1000]
...

Quoted 3 lines

does -48-33 fix it for you?

Ingo

No and I have the tg3_start_xmit too. So the first problem is not solved. I have the following :
network driver disabled interrupts: tg3_start_xmit+0x0/0x620 [tg3]

#9 lore raw
Quoted 6 lines · Serge Noiraud
Quoted 6 lines
Quoted 5 lines

I have the same problem with an e1000 card for 2.6.12-rc6-RT-V0.7.48-32 :
#dmesg
...
network driver disabled interrupts: e1000_xmit_frame+0x0/0xbc0 [e1000]
...

Quoted 7 lines
Quoted 3 lines

does -48-33 fix it for you?

Ingo

No and I have the tg3_start_xmit too. So the first problem is not solved. I have the following : network driver disabled interrupts:
tg3_start_xmit+0x0/0x620 [tg3]

do -50-06 (or later) kernels solve this for you?

Ingo

#10 lore raw
Quoted 12 lines · Ingo Molnar
Quoted 9 lines · Serge Noiraud
Quoted 9 lines
Quoted 5 lines

I have the same problem with an e1000 card for 2.6.12-rc6-RT-V0.7.48-32 :
#dmesg
...
network driver disabled interrupts: e1000_xmit_frame+0x0/0xbc0 [e1000]
...

does -48-33 fix it for you?

Ingo

Quoted 7 lines
Quoted 3 lines

No and I have the tg3_start_xmit too. So the first problem is not solved. I have the following : network driver disabled interrupts:
tg3_start_xmit+0x0/0x620 [tg3]

do -50-06 (or later) kernels solve this for you?

Ingo

sorry for the delay. I was out of my office for 2 weeks. The last RT version I tried was 48-36. it worked.

I'm testing 50-51 wihout succes.
One DEFINE_SPINLOCK is not translated in DEFINE_RAW_SPINLOCK in the following file :
CC arch/i386/kernel/vm86.o
arch/i386/kernel/vm86.c:701: error: syntax error before ',' token make[4]: *** [arch/i386/kernel/vm86.o] Erreur 1
make[3]: *** [arch/i386/kernel] Erreur 2

is it correct ?
I think I had not this error with 50-47.