Skip to contentSkip to conversation

mac80211: NOHZ: local_softirq_pending 08

PATCH In review Michael Buesch 14 participants 27 messages Sep 11, 2009 – Oct 1, 2009 13 messages not in the archive

Subsystems (5)
lore
Share & tools
lore
#0 lore raw

Hi,

mac80211 (or some other part of the networking stack) triggers this warning in the NOHZ code:
NOHZ: local_softirq_pending 08

08 seems to be NET_RX_SOFTIRQ.

It happens, because my test driver b43 handles all RX and TX-status callbacks in process context. I guess some part of the networking stack expects RX to be in tasklet and/or softirq context.

We also have a report of this warning in wl1251, so it's probably not a b43 problem.

Signature · 1 line
Greetings, Michael.
#1 lore raw
Quoted 1 line · Michael Buesch

Hi,

Hallo,

Quoted 11 lines

mac80211 (or some other part of the networking stack) triggers this warning in the NOHZ code: NOHZ: local_softirq_pending 08

08 seems to be NET_RX_SOFTIRQ.

It happens, because my test driver b43 handles all RX and TX-status callbacks in process context. I guess some part of the networking stack expects RX to be in tasklet and/or softirq context.

We also have a report of this warning in wl1251, so it's probably not a b43 problem.

Yes, I see this with wl1251. It uses workqueues everywhere.

Signature · 1 line
Kalle Valo
#2 lore raw
Quoted 11 lines · On Friday 11 September 2009 16:57:54 Kalle Valo wrote:
Quoted 1 line · Michael Buesch

Hi,

Hallo,

Quoted 5 lines

mac80211 (or some other part of the networking stack) triggers this warning in the NOHZ code: NOHZ: local_softirq_pending 08

08 seems to be NET_RX_SOFTIRQ.

Quoted 9 lines
Quoted 6 lines

It happens, because my test driver b43 handles all RX and TX-status callbacks in process context. I guess some part of the networking stack expects RX to be in tasklet and/or softirq context.

We also have a report of this warning in wl1251, so it's probably not a b43 problem.

Yes, I see this with wl1251. It uses workqueues everywhere.

This patch seems to fix it.

net/mac80211/cfg.c +1 -1
Index: wireless-testing/net/mac80211/cfg.c
===================================================================
@@ -606,7 +606,7 @@ static void ieee80211_send_layer2_update
 	skb->dev = sta->sdata->dev;
 	skb->protocol = eth_type_trans(skb, sta->sdata->dev);
 	memset(skb->cb, 0, sizeof(skb->cb));
-	netif_rx(skb);
+	ieee80211_netif_rx(skb);
 }
 
 static void sta_apply_parameters(struct ieee80211_local *local,
net/mac80211/ieee80211_i.h +8 -0
Index: wireless-testing/net/mac80211/ieee80211_i.h
===================================================================
@@ -1053,6 +1053,14 @@ void ieee80211_tx_pending(unsigned long 
 int ieee80211_monitor_start_xmit(struct sk_buff *skb, struct net_device *dev);
 int ieee80211_subif_start_xmit(struct sk_buff *skb, struct net_device *dev);
 
+/* rx handling */
+static inline int ieee80211_netif_rx(struct sk_buff *skb)
+{
+	if (in_interrupt())
+		return netif_rx(skb);
+	return netif_rx_ni(skb);
+}
+
 /* HT */
 void ieee80211_ht_cap_ie_to_sta_ht_cap(struct ieee80211_supported_band *sband,
 				       struct ieee80211_ht_cap *ht_cap_ie,
net/mac80211/main.c +2 -2
Index: wireless-testing/net/mac80211/main.c
===================================================================
@@ -591,7 +591,7 @@ void ieee80211_tx_status(struct ieee8021
 				skb2 = skb_clone(skb, GFP_ATOMIC);
 				if (skb2) {
 					skb2->dev = prev_dev;
-					netif_rx(skb2);
+					ieee80211_netif_rx(skb2);
 				}
 			}
 
@@ -600,7 +600,7 @@ void ieee80211_tx_status(struct ieee8021
 	}
 	if (prev_dev) {
 		skb->dev = prev_dev;
-		netif_rx(skb);
+		ieee80211_netif_rx(skb);
 		skb = NULL;
 	}
 	rcu_read_unlock();
net/mac80211/rx.c +5 -5
Index: wireless-testing/net/mac80211/rx.c
===================================================================
@@ -309,7 +309,7 @@ ieee80211_rx_monitor(struct ieee80211_lo
 			skb2 = skb_clone(skb, GFP_ATOMIC);
 			if (skb2) {
 				skb2->dev = prev_dev;
-				netif_rx(skb2);
+				ieee80211_netif_rx(skb2);
 			}
 		}
 
@@ -320,7 +320,7 @@ ieee80211_rx_monitor(struct ieee80211_lo
 
 	if (prev_dev) {
 		skb->dev = prev_dev;
-		netif_rx(skb);
+		ieee80211_netif_rx(skb);
 	} else
 		dev_kfree_skb(skb);
 
@@ -1349,7 +1349,7 @@ ieee80211_deliver_skb(struct ieee80211_r
 			/* deliver to local stack */
 			skb->protocol = eth_type_trans(skb, dev);
 			memset(skb->cb, 0, sizeof(skb->cb));
-			netif_rx(skb);
+			ieee80211_netif_rx(skb);
 		}
 	}
 
@@ -1943,7 +1943,7 @@ static void ieee80211_rx_cooked_monitor(
 			skb2 = skb_clone(skb, GFP_ATOMIC);
 			if (skb2) {
 				skb2->dev = prev_dev;
-				netif_rx(skb2);
+				ieee80211_netif_rx(skb2);
 			}
 		}
 
@@ -1954,7 +1954,7 @@ static void ieee80211_rx_cooked_monitor(
 
 	if (prev_dev) {
 		skb->dev = prev_dev;
-		netif_rx(skb);
+		ieee80211_netif_rx(skb);
 		skb = NULL;
 	} else
 		goto out_free_skb;
Signature · 1 line
Greetings, Michael.
Tested-by
#4 lore raw
In reply to a message not in the archive
Quoted 9 lines · Michael Buesch
Quoted 9 lines
Quoted 9 lines

mac80211 (or some other part of the networking stack) triggers this warning in the NOHZ code: NOHZ: local_softirq_pending 08

08 seems to be NET_RX_SOFTIRQ.

It happens, because my test driver b43 handles all RX and TX-status callbacks in process context. I guess some part of the networking stack expects RX to be in tasklet and/or softirq context.

Quoted 7 lines
Quoted 5 lines
Quoted 2 lines

We also have a report of this warning in wl1251, so it's probably not a b43 problem.

Yes, I see this with wl1251. It uses workqueues everywhere.

This patch seems to fix it.

Yes, it does. Thanks.

Quoted 1 line
Signature · 1 line
Kalle Valo
#5 lore raw
In reply to a message not in the archive
Quoted 43 lines · Michael Buesch
Quoted 10 lines · On Friday 11 September 2009 16:57:54 Kalle Valo wrote:
Quoted 1 line · Michael Buesch

Hi,

Hallo,

Quoted 5 lines

mac80211 (or some other part of the networking stack) triggers this warning in the NOHZ code: NOHZ: local_softirq_pending 08

08 seems to be NET_RX_SOFTIRQ.

Quoted 8 lines
Quoted 6 lines

It happens, because my test driver b43 handles all RX and TX-status callbacks in process context. I guess some part of the networking stack expects RX to be in tasklet and/or softirq context.

We also have a report of this warning in wl1251, so it's probably not a b43 problem.

Yes, I see this with wl1251. It uses workqueues everywhere.

This patch seems to fix it.

Index: wireless-testing/net/mac80211/cfg.c
===================================================================

net/mac80211/cfg.c +1 -1
@@ -606,7 +606,7 @@ static void ieee80211_send_layer2_update
 	skb->dev = sta->sdata->dev;
 	skb->protocol = eth_type_trans(skb, sta->sdata->dev);
 	memset(skb->cb, 0, sizeof(skb->cb));
-	netif_rx(skb);
+	ieee80211_netif_rx(skb);
 }
 
 static void sta_apply_parameters(struct ieee80211_local *local,

Index: wireless-testing/net/mac80211/ieee80211_i.h
===================================================================

net/mac80211/ieee80211_i.h +0 -0
@@ -1053,6 +1053,14 @@ void ieee80211_tx_pending(unsigned long 
 int ieee80211_monitor_start_xmit(struct sk_buff *skb, struct net_device *dev);

net/mac80211/ieee80211_i.h void ieee80211_tx_pending(unsigned long · line 1063

Quoted 10 lines
net/mac80211/ieee80211_i.h +8 -0
 int ieee80211_subif_start_xmit(struct sk_buff *skb, struct net_device *dev);
 
+/* rx handling */
+static inline int ieee80211_netif_rx(struct sk_buff *skb)
+{
+	if (in_interrupt())
+		return netif_rx(skb);
+	return netif_rx_ni(skb);
+}
+

Hello Michael,

i know this NOHZ warning from the CAN stack also - but now, i know what caused this warning. I fixed it in my local tree and it works. Thanks!

As there are several users in the kernel do exact this test and call the appropriate netif_rx() function, i would suggest to create a static inline function:

static inline int netif_rx_ti(struct sk_buff *skb)

{
	if (in_interrupt())
		return netif_rx(skb);
	return netif_rx_ni(skb);
}

('ti' for test in_interrupt())

in include/linux/netdevice.h

What do you think about that?

Regards,
Oliver
--
To unsubscribe from this list: send the line "unsubscribe linux-wireless" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

#7 lore raw
In reply to a message not in the archive
Quoted 71 lines · On Friday 11 September 2009 18:07:39 Oliver Hartkopp wrote:
Quoted 43 lines · Michael Buesch
Quoted 10 lines · On Friday 11 September 2009 16:57:54 Kalle Valo wrote:
Quoted 1 line · Michael Buesch

Hi,

Hallo,

Quoted 5 lines

mac80211 (or some other part of the networking stack) triggers this warning in the NOHZ code: NOHZ: local_softirq_pending 08

08 seems to be NET_RX_SOFTIRQ.

Quoted 8 lines
Quoted 6 lines

It happens, because my test driver b43 handles all RX and TX-status callbacks in process context. I guess some part of the networking stack expects RX to be in tasklet and/or softirq context.

We also have a report of this warning in wl1251, so it's probably not a b43 problem.

Yes, I see this with wl1251. It uses workqueues everywhere.

This patch seems to fix it.

Index: wireless-testing/net/mac80211/cfg.c
===================================================================

net/mac80211/cfg.c +1 -1
@@ -606,7 +606,7 @@ static void ieee80211_send_layer2_update
 	skb->dev = sta->sdata->dev;
 	skb->protocol = eth_type_trans(skb, sta->sdata->dev);
 	memset(skb->cb, 0, sizeof(skb->cb));
-	netif_rx(skb);
+	ieee80211_netif_rx(skb);
 }
 
 static void sta_apply_parameters(struct ieee80211_local *local,

Index: wireless-testing/net/mac80211/ieee80211_i.h
===================================================================

net/mac80211/ieee80211_i.h +0 -0
@@ -1053,6 +1053,14 @@ void ieee80211_tx_pending(unsigned long 
 int ieee80211_monitor_start_xmit(struct sk_buff *skb, struct net_device *dev);

net/mac80211/ieee80211_i.h void ieee80211_tx_pending(unsigned long · line 1063

Quoted 10 lines
net/mac80211/ieee80211_i.h +8 -0
 int ieee80211_subif_start_xmit(struct sk_buff *skb, struct net_device *dev);
 
+/* rx handling */
+static inline int ieee80211_netif_rx(struct sk_buff *skb)
+{
+	if (in_interrupt())
+		return netif_rx(skb);
+	return netif_rx_ni(skb);
+}
+

Hello Michael,

i know this NOHZ warning from the CAN stack also - but now, i know what caused this warning. I fixed it in my local tree and it works. Thanks!

As there are several users in the kernel do exact this test and call the appropriate netif_rx() function, i would suggest to create a static inline function:

static inline int netif_rx_ti(struct sk_buff *skb)

{
	if (in_interrupt())
		return netif_rx(skb);
	return netif_rx_ni(skb);
}
Quoted 5 lines

('ti' for test in_interrupt())

in include/linux/netdevice.h

What do you think about that?

Yeah, I'm fine with that.

Signature · 5 lines
Greetings, Michael.
--
To unsubscribe from this list: send the line "unsubscribe linux-wireless" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
maintainer
#8 lore raw
Quoted 11 lines · Michael Buesch
Quoted 11 lines

As there are several users in the kernel do exact this test and call the appropriate netif_rx() function, i would suggest to create a static inline function:

static inline int netif_rx_ti(struct sk_buff *skb)

{
	if (in_interrupt())
		return netif_rx(skb);
	return netif_rx_ni(skb);
}
Quoted 8 lines
Quoted 5 lines

('ti' for test in_interrupt())

in include/linux/netdevice.h

What do you think about that?

Yeah, I'm fine with that.

Hi Michael,

i cooked a patch that introduces netif_rx_ti() and fixes up the problems in mac80211 and the CAN subsystem.

Currently i'm pondering whether netif_rx_ti() is needed in all cases or if there are code sections that'll never be executed from irq-context.

In theses cases netif_rx_ni() should be prefered to netif_rx_ti() to prevent the obsolete check ...

Is there any of your changes that should better use netif_rx_ni() ?

Regards,
Oliver

drivers/net/can/vcan.c +1 -1
index 6971f6c..899f3d3 100644
@@ -80,7 +80,7 @@ static void vcan_rx(struct sk_buff *skb, struct net_device *dev)
 	skb->dev       = dev;
 	skb->ip_summed = CHECKSUM_UNNECESSARY;
 
-	netif_rx(skb);
+	netif_rx_ti(skb);
 }
 
 static netdev_tx_t vcan_tx(struct sk_buff *skb, struct net_device *dev)
include/linux/netdevice.h +12 -0
index a44118b..b34c05d 100644
@@ -1503,6 +1503,18 @@ extern int		netdev_budget;
 extern void netdev_run_todo(void);
 
 /**
+ *	netif_rx_ti - test for irq context and post buffer to the network code
+ *	@skb: buffer to post
+ *
+ */
+static inline int netif_rx_ti(struct sk_buff *skb)
+{
+	if (in_interrupt())
+		return netif_rx(skb);
+	return netif_rx_ni(skb);
+}
+
+/**
  *	dev_put - release reference to device
  *	@dev: network device
  *
net/can/af_can.c +1 -1
index ef1c43a..c21e7f4 100644
@@ -278,7 +278,7 @@ int can_send(struct sk_buff *skb, int loop)
 	}
 
 	if (newskb)
-		netif_rx(newskb);
+		netif_rx_ti(newskb);
 
 	/* update statistics */
 	can_stats.tx_frames++;
net/mac80211/cfg.c +1 -1
index 5608f6c..bbcb4cb 100644
@@ -606,7 +606,7 @@ static void ieee80211_send_layer2_update(struct sta_info *sta)
 	skb->dev = sta->sdata->dev;
 	skb->protocol = eth_type_trans(skb, sta->sdata->dev);
 	memset(skb->cb, 0, sizeof(skb->cb));
-	netif_rx(skb);
+	netif_rx_ti(skb);
 }
 
 static void sta_apply_parameters(struct ieee80211_local *local,
net/mac80211/main.c +2 -2
index 797f539..1109f99 100644
@@ -591,7 +591,7 @@ void ieee80211_tx_status(struct ieee80211_hw *hw, struct sk_buff *skb)
 				skb2 = skb_clone(skb, GFP_ATOMIC);
 				if (skb2) {
 					skb2->dev = prev_dev;
-					netif_rx(skb2);
+					netif_rx_ti(skb2);
 				}
 			}
 
@@ -600,7 +600,7 @@ void ieee80211_tx_status(struct ieee80211_hw *hw, struct sk_buff *skb)
 	}
 	if (prev_dev) {
 		skb->dev = prev_dev;
-		netif_rx(skb);
+		netif_rx_ti(skb);
 		skb = NULL;
 	}
 	rcu_read_unlock();
net/mac80211/rx.c +5 -5
index c01588f..5bb7c04 100644
@@ -309,7 +309,7 @@ ieee80211_rx_monitor(struct ieee80211_local *local, struct sk_buff *origskb,
 			skb2 = skb_clone(skb, GFP_ATOMIC);
 			if (skb2) {
 				skb2->dev = prev_dev;
-				netif_rx(skb2);
+				netif_rx_ti(skb2);
 			}
 		}
 
@@ -320,7 +320,7 @@ ieee80211_rx_monitor(struct ieee80211_local *local, struct sk_buff *origskb,
 
 	if (prev_dev) {
 		skb->dev = prev_dev;
-		netif_rx(skb);
+		netif_rx_ti(skb);
 	} else
 		dev_kfree_skb(skb);
 
@@ -1349,7 +1349,7 @@ ieee80211_deliver_skb(struct ieee80211_rx_data *rx)
 			/* deliver to local stack */
 			skb->protocol = eth_type_trans(skb, dev);
 			memset(skb->cb, 0, sizeof(skb->cb));
-			netif_rx(skb);
+			netif_rx_ti(skb);
 		}
 	}
 
@@ -1943,7 +1943,7 @@ static void ieee80211_rx_cooked_monitor(struct ieee80211_rx_data *rx)
 			skb2 = skb_clone(skb, GFP_ATOMIC);
 			if (skb2) {
 				skb2->dev = prev_dev;
-				netif_rx(skb2);
+				netif_rx_ti(skb2);
 			}
 		}
 
@@ -1954,7 +1954,7 @@ static void ieee80211_rx_cooked_monitor(struct ieee80211_rx_data *rx)
 
 	if (prev_dev) {
 		skb->dev = prev_dev;
-		netif_rx(skb);
+		netif_rx_ti(skb);
 		skb = NULL;
 	} else
 		goto out_free_skb;
#10 lore raw
In reply to a message not in the archive
Quoted 30 lines · On Saturday 12 September 2009 18:41:12 Oliver Hartkopp wrote:
Quoted 11 lines · Michael Buesch
Quoted 11 lines

As there are several users in the kernel do exact this test and call the appropriate netif_rx() function, i would suggest to create a static inline function:

static inline int netif_rx_ti(struct sk_buff *skb)

{
	if (in_interrupt())
		return netif_rx(skb);
	return netif_rx_ni(skb);
}
Quoted 8 lines
Quoted 5 lines

('ti' for test in_interrupt())

in include/linux/netdevice.h

What do you think about that?

Yeah, I'm fine with that.

Hi Michael,

i cooked a patch that introduces netif_rx_ti() and fixes up the problems in mac80211 and the CAN subsystem.

Currently i'm pondering whether netif_rx_ti() is needed in all cases or if there are code sections that'll never be executed from irq-context.

Quoted 8 lines

In theses cases netif_rx_ni() should be prefered to netif_rx_ti() to prevent the obsolete check ...

Is there any of your changes that should better use netif_rx_ni() ?

Regards,
Oliver

Well, I'd say this check does not cost much at all.
If I were the net maintainer, I'd get rid of netif_rx_ni() _and_ netif_rx_ti() and do the check internally in netif_rx().
But as I don't have to decide that, I just want the mac80211 issue fixed.

Signature · 5 lines
Greetings, Michael.
--
To unsubscribe from this list: send the line "unsubscribe linux-wireless" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
#12 lore raw
In reply to a message not in the archive
Quoted 34 lines · Michael Buesch
Quoted 33 lines · On Saturday 12 September 2009 18:41:12 Oliver Hartkopp wrote:
Quoted 11 lines · Michael Buesch
Quoted 11 lines

As there are several users in the kernel do exact this test and call the appropriate netif_rx() function, i would suggest to create a static inline function:

static inline int netif_rx_ti(struct sk_buff *skb)

{
	if (in_interrupt())
		return netif_rx(skb);
	return netif_rx_ni(skb);
}
Quoted 7 lines
Quoted 5 lines

('ti' for test in_interrupt())

in include/linux/netdevice.h

What do you think about that?

Yeah, I'm fine with that.

Hi Michael,

i cooked a patch that introduces netif_rx_ti() and fixes up the problems in mac80211 and the CAN subsystem.

Currently i'm pondering whether netif_rx_ti() is needed in all cases or if there are code sections that'll never be executed from irq-context.

In theses cases netif_rx_ni() should be prefered to netif_rx_ti() to prevent the obsolete check ...

Is there any of your changes that should better use netif_rx_ni() ?

Quoted 9 lines
Quoted 3 lines

Regards,
Oliver

Well, I'd say this check does not cost much at all.
If I were the net maintainer, I'd get rid of netif_rx_ni() _and_ netif_rx_ti() and do the check internally in netif_rx().
But as I don't have to decide that, I just want the mac80211 issue fixed.

Like this?

int netif_rx(struct sk_buff *skb)

{
	int err;

	if (likely(in_interrupt()))
		err = __netif_rx(skb);
	else {
		preempt_disable();
		err = __netif_rx(skb);
		if (local_softirq_pending())
			do_softirq();
		preempt_enable();
	}

	return err;
}

I don't know how expensive in_interrupt() is ... checking the code does not give any answers to *me* ;-)

But i found

356 netif_rx()

but only

18 netif_rx_ni()

in the kernel tree.

And three of them check for in_interrupt() before using netif_rx() or netif_rx_ni() ...

Finally i would tend to introduce netif_rx_ti() in include/linux/netdevice.h as described above, for the rare code that can be used inside and outside the irq context.

I assume the affected code in the CAN stuff has to use netif_rx_ni() - but i will doublecheck that (and prepare a separate CAN patch).

For the mac80211 i would suggest to check whether you really need netif_rx()/netif_rx_ni()/netif_rx_ti() in all the regarded cases.

I assume always using netif_rx_ti() (as you proposed in the original patch) is not the most efficient approach.

Best regards,
Oliver

--
To unsubscribe from this list: send the line "unsubscribe linux-wireless" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

#13 lore raw
In reply to Oliver Hartkopp
Quoted 2 lines · Oliver Hartkopp

i cooked a patch that introduces netif_rx_ti() and fixes up the problems in mac80211 and the CAN subsystem.

Oliver,

Are you going to send this patch to Dave? If you want me to carry it instead, please resend it with a proper changelog including a Signed-off-by line. For that matter, Dave will most certainly want that as well...

Thanks!

John

Signature · 2 lines
John W. Linville		Someday the world will need a hero, and you
[email protected]			might be all we have.  Be ready.
#15 lore raw
In reply to a message not in the archive
Quoted 11 lines · John W. Linville
Quoted 2 lines · Oliver Hartkopp

i cooked a patch that introduces netif_rx_ti() and fixes up the problems in mac80211 and the CAN subsystem.

Oliver,

Are you going to send this patch to Dave? If you want me to carry it instead, please resend it with a proper changelog including a Signed-off-by line. For that matter, Dave will most certainly want that as well...

Hello John,

as i wrote here

http://marc.info/?l=linux-netdev&m=125277885910179&w=2

there are currently only three occurrences of checks that use netif_rx() and netif_rx_ni() depending on in_interrupt().

And regarding the suggested fix from Michael, that checked every(!) netif_rx() whether it is in interrupt or not, i was unsure if a netif_tx_ti() would make sense for only three cases?!?

If you think it makes sense, i can post a patch for that ... but:

Indeed it costs some additional investigation to prove whether netif_rx() or netif_rx_ni() should be used in each case. But IMHO this has to be done before providing a pump-gun function that solves the problem without thinking if we are in irq-context or not. I want to avoid that people are using netif_rx_ti() as some kind of default ...

I don't know how expensive in_interrupt() is, but it IMO should be avoided when the context for a code section can be determined in another way.

Regards,
Oliver

--
To unsubscribe from this list: send the line "unsubscribe linux-wireless" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

#16 lore raw
Quoted 28 lines · On Wednesday 30 September 2009 13:56:12 Oliver Hartkopp wrote:
Quoted 11 lines · John W. Linville
Quoted 2 lines · Oliver Hartkopp

i cooked a patch that introduces netif_rx_ti() and fixes up the problems in mac80211 and the CAN subsystem.

Oliver,

Are you going to send this patch to Dave? If you want me to carry it instead, please resend it with a proper changelog including a Signed-off-by line. For that matter, Dave will most certainly want that as well...

Hello John,

as i wrote here

http://marc.info/?l=linux-netdev&m=125277885910179&w=2

there are currently only three occurrences of checks that use netif_rx() and netif_rx_ni() depending on in_interrupt().

And regarding the suggested fix from Michael, that checked every(!) netif_rx() whether it is in interrupt or not, i was unsure if a netif_tx_ti() would make sense for only three cases?!?

If you think it makes sense, i can post a patch for that ... but:

Quoted 8 lines

Indeed it costs some additional investigation to prove whether netif_rx() or netif_rx_ni() should be used in each case. But IMHO this has to be done before providing a pump-gun function that solves the problem without thinking if we are in irq-context or not. I want to avoid that people are using netif_rx_ti() as some kind of default ...

I don't know how expensive in_interrupt() is, but it IMO should be avoided when the context for a code section can be determined in another way.

What if we just get the fix merged and discuss later whether it's worth to optimize a picosecond or not?? My patch fixes the _bug_. You can merge a more "efficient" fix later that saves one or two CPU cycles.

Signature · 1 line
Greetings, Michael.
#18 lore raw
In reply to a message not in the archive
Quoted 8 lines · Michael Buesch
Quoted 3 lines

I don't know how expensive in_interrupt() is, but it IMO should be avoided when the context for a code section can be determined in another way.

What if we just get the fix merged and discuss later whether it's worth to optimize a picosecond or not?? My patch fixes the _bug_. You can merge a more "efficient" fix later that saves one or two CPU cycles.

I agree with Michael. The bug is real and I have verified that Michael's patch fixes the issue. Better to apply the patch now, it's trivial to change the implementation if/when the network stack has support for this.

Signature · 5 lines
Kalle Valo
--
To unsubscribe from this list: send the line "unsubscribe linux-wireless" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
maintainer
#19 lore raw
Quoted 4 lines · Kalle Valo

I agree with Michael. The bug is real and I have verified that Michael's patch fixes the issue. Better to apply the patch now, it's trivial to change the implementation if/when the network stack has support for this.

FWIW, I think in mac80211 the in_interrupt() check can never return true since we postpone all RX to the tasklet. But the tasklet seems to be ok -- so should it really be in_interrupt()?

johannes

#21 lore raw
In reply to a message not in the archive
Quoted 10 lines · On Wednesday 30 September 2009 16:54:26 Johannes Berg wrote:
Quoted 4 lines · Kalle Valo

I agree with Michael. The bug is real and I have verified that Michael's patch fixes the issue. Better to apply the patch now, it's trivial to change the implementation if/when the network stack has support for this.

FWIW, I think in mac80211 the in_interrupt() check can never return true since we postpone all RX to the tasklet. But the tasklet seems to be ok -- so should it really be in_interrupt()?

I think a tasklet is also in_interrupt(), because it's a softirq. in_interrupt() returns false in process context. The problem appeared when the b43 driver started passing RX frames while being in process context (threaded IRQ). It previously was in tasklet (= softirq) context.

Signature · 5 lines
Greetings, Michael.
--
To unsubscribe from this list: send the line "unsubscribe linux-wireless" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
maintainer
#22 lore raw
Quoted 8 lines · Michael Buesch
Quoted 7 lines · On Wednesday 30 September 2009 16:54:26 Johannes Berg wrote:
Quoted 4 lines · Kalle Valo

I agree with Michael. The bug is real and I have verified that Michael's patch fixes the issue. Better to apply the patch now, it's trivial to change the implementation if/when the network stack has support for this.

Quoted 5 lines
Quoted 3 lines

FWIW, I think in mac80211 the in_interrupt() check can never return true since we postpone all RX to the tasklet. But the tasklet seems to be ok -- so should it really be in_interrupt()?

I think a tasklet is also in_interrupt(), because it's a softirq.

Ah, yes, indeed, in_interrupt() vs. in_irq().

johannes

#24 lore raw
In reply to a message not in the archive
Quoted 8 lines · Johannes Berg
Quoted 7 lines · Michael Buesch
Quoted 6 lines · On Wednesday 30 September 2009 16:54:26 Johannes Berg wrote:
Quoted 4 lines · Kalle Valo

I agree with Michael. The bug is real and I have verified that Michael's patch fixes the issue. Better to apply the patch now, it's trivial to change the implementation if/when the network stack has support for this.

Quoted 7 lines
Quoted 4 lines
Quoted 3 lines

FWIW, I think in mac80211 the in_interrupt() check can never return true since we postpone all RX to the tasklet. But the tasklet seems to be ok -- so should it really be in_interrupt()?

I think a tasklet is also in_interrupt(), because it's a softirq.

Ah, yes, indeed, in_interrupt() vs. in_irq().

Oops!

I missed that for my previous patch i added for two occurrences in the CAN sources.

I'm currently compiling the patch for netif_rx_ti() and will post it in some minutes (for CAN and mac80211) when it runs without probs.

Regards,
Oliver

#26 lore raw

PATCH 1/1 net: fix NOHZ: local_softirq_pending 08

In reply to a message not in the archive

Socket buffers that are generated and received inside softirqs or from process context must not use netif_rx() that's intended to be used from irq context only.

This patch introduces a new helper function netif_rx_ti(skb) that tests for in_interrupt() before invoking netif_rx() or netif_rx_ni().

It fixes the ratelimited kernel warning

NOHZ: local_softirq_pending 08

in the mac80211 and can subsystems.

---

drivers/net/can/vcan.c +1 -1
index 80ac563..899f3d3 100644
@@ -80,7 +80,7 @@ static void vcan_rx(struct sk_buff *skb, struct net_device *dev)
 	skb->dev       = dev;
 	skb->ip_summed = CHECKSUM_UNNECESSARY;
 
-	netif_rx_ni(skb);
+	netif_rx_ti(skb);
 }
 
 static netdev_tx_t vcan_tx(struct sk_buff *skb, struct net_device *dev)
include/linux/netdevice.h +13 -0
index 94958c1..dc8dfb2 100644
@@ -1509,6 +1509,19 @@ extern int		netdev_budget;
 extern void netdev_run_todo(void);
 
 /**
+ *	netif_rx_ti - test for irq context and post buffer to the network code
+ *	@skb: buffer to post
+ *
+ */
+static inline int netif_rx_ti(struct sk_buff *skb)
+{
+	if (in_interrupt())
+		return netif_rx(skb);
+	else
+		return netif_rx_ni(skb);
+}
+
+/**
  *	dev_put - release reference to device
  *	@dev: network device
  *
net/can/af_can.c +1 -3
index 6068321..c21e7f4 100644
@@ -199,8 +199,6 @@ static int can_create(struct net *net, struct socket *sock, int protocol)
  * @skb: pointer to socket buffer with CAN frame in data section
  * @loop: loopback for listeners on local CAN sockets (recommended default!)
  *
- * Due to the loopback this routine must not be called from hardirq context.
- *
  * Return:
  *  0 on success
  *  -ENETDOWN when the selected interface is down
@@ -280,7 +278,7 @@ int can_send(struct sk_buff *skb, int loop)
 	}
 
 	if (newskb)
-		netif_rx_ni(newskb);
+		netif_rx_ti(newskb);
 
 	/* update statistics */
 	can_stats.tx_frames++;
net/mac80211/cfg.c +1 -1
index 5608f6c..bbcb4cb 100644
@@ -606,7 +606,7 @@ static void ieee80211_send_layer2_update(struct sta_info *sta)
 	skb->dev = sta->sdata->dev;
 	skb->protocol = eth_type_trans(skb, sta->sdata->dev);
 	memset(skb->cb, 0, sizeof(skb->cb));
-	netif_rx(skb);
+	netif_rx_ti(skb);
 }
 
 static void sta_apply_parameters(struct ieee80211_local *local,
net/mac80211/main.c +2 -2
index 797f539..1109f99 100644
@@ -591,7 +591,7 @@ void ieee80211_tx_status(struct ieee80211_hw *hw, struct sk_buff *skb)
 				skb2 = skb_clone(skb, GFP_ATOMIC);
 				if (skb2) {
 					skb2->dev = prev_dev;
-					netif_rx(skb2);
+					netif_rx_ti(skb2);
 				}
 			}
 
@@ -600,7 +600,7 @@ void ieee80211_tx_status(struct ieee80211_hw *hw, struct sk_buff *skb)
 	}
 	if (prev_dev) {
 		skb->dev = prev_dev;
-		netif_rx(skb);
+		netif_rx_ti(skb);
 		skb = NULL;
 	}
 	rcu_read_unlock();
net/mac80211/rx.c +5 -5
index c01588f..5bb7c04 100644
@@ -309,7 +309,7 @@ ieee80211_rx_monitor(struct ieee80211_local *local, struct sk_buff *origskb,
 			skb2 = skb_clone(skb, GFP_ATOMIC);
 			if (skb2) {
 				skb2->dev = prev_dev;
-				netif_rx(skb2);
+				netif_rx_ti(skb2);
 			}
 		}
 
@@ -320,7 +320,7 @@ ieee80211_rx_monitor(struct ieee80211_local *local, struct sk_buff *origskb,
 
 	if (prev_dev) {
 		skb->dev = prev_dev;
-		netif_rx(skb);
+		netif_rx_ti(skb);
 	} else
 		dev_kfree_skb(skb);
 
@@ -1349,7 +1349,7 @@ ieee80211_deliver_skb(struct ieee80211_rx_data *rx)
 			/* deliver to local stack */
 			skb->protocol = eth_type_trans(skb, dev);
 			memset(skb->cb, 0, sizeof(skb->cb));
-			netif_rx(skb);
+			netif_rx_ti(skb);
 		}
 	}
 
@@ -1943,7 +1943,7 @@ static void ieee80211_rx_cooked_monitor(struct ieee80211_rx_data *rx)
 			skb2 = skb_clone(skb, GFP_ATOMIC);
 			if (skb2) {
 				skb2->dev = prev_dev;
-				netif_rx(skb2);
+				netif_rx_ti(skb2);
 			}
 		}
 
@@ -1954,7 +1954,7 @@ static void ieee80211_rx_cooked_monitor(struct ieee80211_rx_data *rx)
 
 	if (prev_dev) {
 		skb->dev = prev_dev;
-		netif_rx(skb);
+		netif_rx_ti(skb);
 		skb = NULL;
 	} else
 		goto out_free_skb;
#28 lore raw

Re: [PATCH] net: fix NOHZ: local_softirq_pending 08

In reply to a message not in the archive
Quoted 6 lines · Oliver Hartkopp

Socket buffers that are generated and received inside softirqs or from process context must not use netif_rx() that's intended to be used from irq context only.

This patch introduces a new helper function netif_rx_ti(skb) that tests for in_interrupt() before invoking netif_rx() or netif_rx_ni().

Quoted 7 lines

It fixes the ratelimited kernel warning

NOHZ: local_softirq_pending 08

in the mac80211 and can subsystems.

http://bugzilla.kernel.org/show_bug.cgi?id=14278

Signature · 6 lines
John W. Linville		Someday the world will need a hero, and you
[email protected]			might be all we have.  Be ready.
--
To unsubscribe from this list: send the line "unsubscribe linux-wireless" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
#29 lore raw

Re: [PATCH] net: fix NOHZ: local_softirq_pending 08

In reply to a message not in the archive

From: Oliver Hartkopp <[email protected]>
Date: Wed, 30 Sep 2009 20:18:25 +0200

Quoted 6 lines

Socket buffers that are generated and received inside softirqs or from process context must not use netif_rx() that's intended to be used from irq context only.

This patch introduces a new helper function netif_rx_ti(skb) that tests for in_interrupt() before invoking netif_rx() or netif_rx_ni().

Quoted 7 lines

It fixes the ratelimited kernel warning

NOHZ: local_softirq_pending 08

in the mac80211 and can subsystems.

I bet all of these code paths can use netif_receive_skb() or don't need this conditional blob at all.

Looking at some specific cases in this patch:

1) VCAN: This RX routine is only invoked from the drivers ->ndo_start_xmit() handler, and therefore like the loopback driver we know that BH's are already disabled and therefore it can always use netif_rx() safely.

Why did you convert this case?

And if this needs to be converted, why doesn't loopback need to be?

2) af_can.c: In what situation will netif_rx_ni() not be appropriate here? It should always execute in softirq or user context, now hardirq context.

And the list goes on and on, I don't really like this new conditional testing of interrupt state. As always, that's usually a red flag and as far as I can see these spots where you're changing things are only trying to receive packets in one of the two possible cases not both.

I'm not applying this until all of these details are sorted out and you add some verbosity to the commit message explaining each and every case you are changing, what contexts those cases can be called from, and from where.

Thanks.

#30 lore raw

Re: [PATCH] net: fix NOHZ: local_softirq_pending 08

Quoted 32 lines · David Miller

From: Oliver Hartkopp <[email protected]>
Date: Wed, 30 Sep 2009 20:18:25 +0200

Quoted 6 lines

Socket buffers that are generated and received inside softirqs or from process context must not use netif_rx() that's intended to be used from irq context only.

This patch introduces a new helper function netif_rx_ti(skb) that tests for in_interrupt() before invoking netif_rx() or netif_rx_ni().

Quoted 7 lines

It fixes the ratelimited kernel warning

NOHZ: local_softirq_pending 08

in the mac80211 and can subsystems.

I bet all of these code paths can use netif_receive_skb() or don't need this conditional blob at all.

Looking at some specific cases in this patch:

1) VCAN: This RX routine is only invoked from the drivers ->ndo_start_xmit() handler, and therefore like the loopback driver we know that BH's are already disabled and therefore it can always use netif_rx() safely.

Why did you convert this case?

And if this needs to be converted, why doesn't loopback need to be?

Quoted 6 lines

2) af_can.c: In what situation will netif_rx_ni() not be appropriate here? It should always execute in softirq or user context, now hardirq context.

And the list goes on and on, I don't really like this new conditional testing of interrupt state.

Hello Dave,

i'm confused about in_interrupt(), in_softirq() and in_irq() as pointed out by Johannes here:

http://marc.info/?l=linux-wireless&m=125432410405562&w=2

Indeed in the two cases for the CAN stuff (in vcan.c and af_can.c) the skb's are received in process-context and softirq-context only.

Therefore i used netif_rx_ni() in my last change of this code.

Now i was reading from Johannes that in_interrupt() is used for hardirq-context /and/ softirq-context, so i was just *unsure* and used the newly introduced netif_rx_ti() for that, which tests for in_interrupt().

Indeed i'm not really happy with that, as it is always better to check each receive case in which context it can be used and used exactly the right function for that.

So when netif_rx_ni() or netif_receive_skb() is the best i can use when in process-context or in softirq-context, i'll do it with pleasure.

And if it is like this the problematic netif_rx() calls in mac80211 need to be sorted out in detail also ...

Regards,
Oliver

#31 lore raw

Re: [PATCH] net: fix NOHZ: local_softirq_pending 08

In reply to David Miller
Quoted 1 line · On Thursday 01 October 2009 01:33:33 David Miller wrote:

I'm not applying this until all of these details are sorted out

John, please apply my fix to wireless-testing to get rid of the regression. You can revert it later, if there's a better fix available.

Signature · 1 line
Greetings, Michael.
#33 lore raw

Re: [PATCH] net: fix NOHZ: local_softirq_pending 08

In reply to a message not in the archive
Quoted 7 lines · Michael Buesch
Quoted 1 line · On Thursday 01 October 2009 01:33:33 David Miller wrote:

I'm not applying this until all of these details are sorted out

John, please apply my fix to wireless-testing to get rid of the regression. You can revert it later, if there's a better fix available.

I agree, please take Michael's patch. It's trivial to change mac80211 part whenever there's better support available.

But I don't think this is a regression because I see the bug also with 2.6.28, most probably it has been in mac80211 forever. But it's still a bug which needs to be fixed.

Signature · 1 line
Kalle Valo
#34 lore raw

Re: [PATCH] net: fix NOHZ: local_softirq_pending 08

In reply to a message not in the archive
Quoted 6 lines · Michael Buesch
Quoted 1 line · On Thursday 01 October 2009 01:33:33 David Miller wrote:

I'm not applying this until all of these details are sorted out

John, please apply my fix to wireless-testing to get rid of the regression. You can revert it later, if there's a better fix available.

I agree with davem, don't. Just fix the driver to local_bh_disable() around the rx function if necessary.

johannes

#36 lore raw

Re: [PATCH] net: fix NOHZ: local_softirq_pending 08

In reply to a message not in the archive
Quoted 10 lines · On Thursday 01 October 2009 20:42:28 Johannes Berg wrote:
Quoted 6 lines · Michael Buesch
Quoted 1 line · On Thursday 01 October 2009 01:33:33 David Miller wrote:

I'm not applying this until all of these details are sorted out

John, please apply my fix to wireless-testing to get rid of the regression. You can revert it later, if there's a better fix available.

I agree with davem, don't. Just fix the driver to local_bh_disable() around the rx function if necessary.

For the benefit of a much bigger critical section? I don't get it why this would be any better.

I _am_ going to do one thing now, however. That is ignoring any regression bugreport. (Yes, it _is_ a regression for b43)

Signature · 5 lines
Greetings, Michael.
--
To unsubscribe from this list: send the line "unsubscribe linux-wireless" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
#38 lore raw

Re: [PATCH] net: fix NOHZ: local_softirq_pending 08

In reply to a message not in the archive
Quoted 4 lines · Michael Buesch
Quoted 2 lines

I agree with davem, don't. Just fix the driver to local_bh_disable() around the rx function if necessary.

For the benefit of a much bigger critical section? I don't get it why this would be any better.

And how do you know mac80211 is actually safe with this change? It uses tasklets too. At the very least you'd have to require drivers to never mix & match the regular/irqsafe functions at all.

johannes

maintainer
#39 lore raw

Re: [PATCH] net: fix NOHZ: local_softirq_pending 08

In reply to Michael Buesch

From: Michael Buesch <[email protected]>
Date: Thu, 1 Oct 2009 21:10:32 +0200

Quoted 2 lines

For the benefit of a much bigger critical section? I don't get it why this would be any better.

Think about what you are saying when you introduce things like this into your code:

	if (in_interrupt())
		foo();
	else
		bar();

That thing there means you don't know anything about how you'll need to do locking properly, because you have no idea about even the context in which your code is executed.

Sure, you can lock for the most stringent case, but that's silly and wasteful.