Skip to contentSkip to conversation

mac80211: fix "NOHZ: local_softirq_pending 08"

RFC PATCH Superseded Johannes Stezenbach 3 participants 12 messages Nov 29, 2010 – Nov 30, 2010

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

RFC PATCH 1/1 mac80211: fix "NOHZ: local_softirq_pending 08"

ieee80211_tx_status() documentation says "This function may not be called in IRQ context", and it is called by rt2800usb from a workqueue context. Thus it needs to call netif_rx_ni() instead of netif_rx().
This change fixes the "NOHZ: local_softirq_pending 08" messages I've been getting with rt2800usb.

--

 net/mac80211/status.c |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)
net/mac80211/status.c +2 -2
index bed7e32..ba8dd8e 100644
@@ -403,7 +403,7 @@ void ieee80211_tx_status(struct ieee80211_hw *hw, struct sk_buff *skb)
 				skb2 = skb_clone(skb, GFP_ATOMIC);
 				if (skb2) {
 					skb2->dev = prev_dev;
-					netif_rx(skb2);
+					netif_rx_ni(skb2);
 				}
 			}
 
@@ -412,7 +412,7 @@ void ieee80211_tx_status(struct ieee80211_hw *hw, struct sk_buff *skb)
 	}
 	if (prev_dev) {
 		skb->dev = prev_dev;
-		netif_rx(skb);
+		netif_rx_ni(skb);
 		skb = NULL;
 	}
 	rcu_read_unlock();
maintainer
#1 lore raw

net/mac80211/status.c void ieee80211_tx_status(struct ieee80211_hw *hw, struct sk_buff *skb) · line 406

Quoted 9 lines · Johannes Stezenbach

ieee80211_tx_status() documentation says "This function may not be called in IRQ context", and it is called by rt2800usb from a workqueue context. Thus it needs to call netif_rx_ni() instead of netif_rx().
This change fixes the "NOHZ: local_softirq_pending 08" messages I've been getting with rt2800usb.

net/mac80211/status.c +1 -1
-					netif_rx(skb2);
+					netif_rx_ni(skb2);

That's kinda pointless though for drivers that already call it from a tasklet or similar -- how about instead adding an ieee80211_tx_status_ni() inline along the lines of ieee80211_rx_ni()?

johannes

#2 lore raw
Quoted 8 lines · Johannes Berg
Quoted 6 lines · Johannes Stezenbach

ieee80211_tx_status() documentation says "This function may not be called in IRQ context", and it is called by rt2800usb from a workqueue context. Thus it needs to call netif_rx_ni() instead of netif_rx().
This change fixes the "NOHZ: local_softirq_pending 08" messages I've been getting with rt2800usb.

Quoted 6 lines

net/mac80211/status.c void ieee80211_tx_status(struct ieee80211_hw *hw, struct sk_buff *skb) · line 406

Quoted 2 lines
net/mac80211/status.c +1 -1
-					netif_rx(skb2);
+					netif_rx_ni(skb2);

That's kinda pointless though for drivers that already call it from a tasklet or similar -- how about instead adding an ieee80211_tx_status_ni() inline along the lines of ieee80211_rx_ni()?

It gets confusing...
There already is ieee80211_tx_status_irqsafe(), but you want a third option, right?

Johannes

maintainer
#3 lore raw
Quoted 7 lines · Johannes Stezenbach
Quoted 3 lines

That's kinda pointless though for drivers that already call it from a tasklet or similar -- how about instead adding an ieee80211_tx_status_ni() inline along the lines of ieee80211_rx_ni()?

It gets confusing...
There already is ieee80211_tx_status_irqsafe(), but you want a third option, right?

Yes: _irqsafe has to go through the tasklet, _ni just has to disable local BHs.

johannes

#4 lore raw

RFC PATCH v2 1/1 mac80211: fix "NOHZ: local_softirq_pending 08"

ieee80211_tx_status() documentation says "This function may not be called in IRQ context", and it is called by rt2800usb from a workqueue context. However, ieee80211_tx_status() is meant to be called from tasklets and thus uses netif_rx(). Add a new ieee80211_tx_status_ni() which does the same thing but uses netif_rx_ni() instead of netif_rx().

This change fixes the "NOHZ: local_softirq_pending 08" messages I've been getting with rt2800usb.

---
v2: add ieee80211_tx_status_ni()

Note: the patch is now incomplete, rt2x00lib_txdone() is used by several drivers and I'm currently checking if ieee80211_tx_status_ni() can be used by all of them of if I need to split rt2x00lib_txdone() in two versions. An updated patch with the rt2x00
changes will follow, but I'm seeking feedback if the base change here is OK.

 include/net/mac80211.h |   23 +++++++++++++++++++----
 net/mac80211/status.c  |   19 ++++++++++++++++---
 2 files changed, 35 insertions(+), 7 deletions(-)
include/net/mac80211.h +19 -4
index eaa4aff..df42312 100644
@@ -2055,8 +2055,8 @@ static inline void ieee80211_rx_ni(struct ieee80211_hw *hw,
  *
  * This function may not be called in IRQ context. Calls to this function
  * for a single hardware must be synchronized against each other. Calls
- * to this function and ieee80211_tx_status_irqsafe() may not be mixed
- * for a single hardware.
+ * to this function, ieee80211_tx_status_ni() and ieee80211_tx_status_irqsafe()
+ * may not be mixed for a single hardware.
  *
  * @hw: the hardware the frame was transmitted by
  * @skb: the frame that was transmitted, owned by mac80211 after this call
@@ -2065,13 +2065,28 @@ void ieee80211_tx_status(struct ieee80211_hw *hw,
 			 struct sk_buff *skb);
 
 /**
+ * ieee80211_tx_status_ni - transmit status callback (in process context)
+ *
+ * Like ieee80211_tx_status() but can be called in process context.
+ *
+ * Calls to this function, ieee80211_tx_status() and
+ * ieee80211_tx_status_irqsafe() may not be mixed
+ * for a single hardware.
+ *
+ * @hw: the hardware the frame was transmitted by
+ * @skb: the frame that was transmitted, owned by mac80211 after this call
+ */
+void ieee80211_tx_status_ni(struct ieee80211_hw *hw,
+			    struct sk_buff *skb);
+
+/**
  * ieee80211_tx_status_irqsafe - IRQ-safe transmit status callback
  *
  * Like ieee80211_tx_status() but can be called in IRQ context
  * (internally defers to a tasklet.)
  *
- * Calls to this function and ieee80211_tx_status() may not be mixed for a
- * single hardware.
+ * Calls to this function, ieee80211_tx_status() and
+ * ieee80211_tx_status_ni() may not be mixed for a single hardware.
  *
  * @hw: the hardware the frame was transmitted by
  * @skb: the frame that was transmitted, owned by mac80211 after this call
net/mac80211/status.c +16 -3
index bed7e32..ce4d239 100644
@@ -170,7 +170,8 @@ static void ieee80211_frame_acked(struct sta_info *sta, struct sk_buff *skb)
  */
 #define STA_LOST_PKT_THRESHOLD	50
 
-void ieee80211_tx_status(struct ieee80211_hw *hw, struct sk_buff *skb)
+void __ieee80211_tx_status(struct ieee80211_hw *hw, struct sk_buff *skb,
+			   int (*netif_rx_func)(struct sk_buff *skb))
 {
 	struct sk_buff *skb2;
 	struct ieee80211_hdr *hdr = (struct ieee80211_hdr *) skb->data;
@@ -403,7 +404,7 @@ void ieee80211_tx_status(struct ieee80211_hw *hw, struct sk_buff *skb)
 				skb2 = skb_clone(skb, GFP_ATOMIC);
 				if (skb2) {
 					skb2->dev = prev_dev;
-					netif_rx(skb2);
+					netif_rx_func(skb2);
 				}
 			}
 
@@ -412,10 +413,22 @@ void ieee80211_tx_status(struct ieee80211_hw *hw, struct sk_buff *skb)
 	}
 	if (prev_dev) {
 		skb->dev = prev_dev;
-		netif_rx(skb);
+		netif_rx_func(skb);
 		skb = NULL;
 	}
 	rcu_read_unlock();
 	dev_kfree_skb(skb);
 }
+
+void ieee80211_tx_status(struct ieee80211_hw *hw, struct sk_buff *skb)
+{
+	__ieee80211_tx_status(hw, skb, netif_rx);
+}
 EXPORT_SYMBOL(ieee80211_tx_status);
+
+void ieee80211_tx_status_ni(struct ieee80211_hw *hw,
+			    struct sk_buff *skb)
+{
+	__ieee80211_tx_status(hw, skb, netif_rx_ni);
+}
+EXPORT_SYMBOL(ieee80211_tx_status_ni);
maintainer
#5 lore raw

net/mac80211/status.c static void ieee80211_frame_acked(struct sta_info *sta, struct sk_buff *skb) · line 174

Quoted 3 lines · Johannes Stezenbach
net/mac80211/status.c +2 -1
-void ieee80211_tx_status(struct ieee80211_hw *hw, struct sk_buff *skb)
+void __ieee80211_tx_status(struct ieee80211_hw *hw, struct sk_buff *skb,
+			   int (*netif_rx_func)(struct sk_buff *skb))

This seems weird to me -- why not just do

static inline void ieee80211_tx_status_ni(...)

{
	local_bh_disable();
	ieee80211_tx_status(...);
	local_bh_enable();
}

johannes

#6 lore raw

RFC PATCH v3 1/1 mac80211: add ieee80211_tx_status_ni()

ieee80211_tx_status() documentation says "This function may not be called in IRQ context", and it is called by rt2800usb from a workqueue context. However, ieee80211_tx_status() is meant to be called from tasklets and thus uses netif_rx(). Add a new ieee80211_tx_status_ni() which does the same thing but is safe to be called from process context.

Using ieee80211_tx_status_ni() for rt2800usb fixes the "NOHZ: local_softirq_pending 08" messages I've been getting.

---
v2: add ieee80211_tx_status_ni()
v3: simplify accorindg to review

Note: the patch is now incomplete, rt2x00lib_txdone() is used by several drivers, rt2800usb calls it from a workqueue while e.g. while rt2800pci calls it from a tasklet (rt2800pci_txstatus_tasklet). I need to sort this out with the rt2x00 developers.

 include/net/mac80211.h |   28 ++++++++++++++++++++++++----
 1 files changed, 24 insertions(+), 4 deletions(-)
include/net/mac80211.h +24 -4
index eaa4aff..e411cf8 100644
@@ -2055,8 +2055,8 @@ static inline void ieee80211_rx_ni(struct ieee80211_hw *hw,
  *
  * This function may not be called in IRQ context. Calls to this function
  * for a single hardware must be synchronized against each other. Calls
- * to this function and ieee80211_tx_status_irqsafe() may not be mixed
- * for a single hardware.
+ * to this function, ieee80211_tx_status_ni() and ieee80211_tx_status_irqsafe()
+ * may not be mixed for a single hardware.
  *
  * @hw: the hardware the frame was transmitted by
  * @skb: the frame that was transmitted, owned by mac80211 after this call
@@ -2065,13 +2065,33 @@ void ieee80211_tx_status(struct ieee80211_hw *hw,
 			 struct sk_buff *skb);
 
 /**
+ * ieee80211_tx_status_ni - transmit status callback (in process context)
+ *
+ * Like ieee80211_tx_status() but can be called in process context.
+ *
+ * Calls to this function, ieee80211_tx_status() and
+ * ieee80211_tx_status_irqsafe() may not be mixed
+ * for a single hardware.
+ *
+ * @hw: the hardware the frame was transmitted by
+ * @skb: the frame that was transmitted, owned by mac80211 after this call
+ */
+static inline void ieee80211_tx_status_ni(struct ieee80211_hw *hw,
+					  struct sk_buff *skb)
+{
+	local_bh_disable();
+	ieee80211_tx_status(hw, skb);
+	local_bh_enable();
+}
+
+/**
  * ieee80211_tx_status_irqsafe - IRQ-safe transmit status callback
  *
  * Like ieee80211_tx_status() but can be called in IRQ context
  * (internally defers to a tasklet.)
  *
- * Calls to this function and ieee80211_tx_status() may not be mixed for a
- * single hardware.
+ * Calls to this function, ieee80211_tx_status() and
+ * ieee80211_tx_status_ni() may not be mixed for a single hardware.
  *
  * @hw: the hardware the frame was transmitted by
  * @skb: the frame that was transmitted, owned by mac80211 after this call
#7 lore raw

PATCH v4 1/1 mac80211/rt2x00: add ieee80211_tx_status_ni()

All rt2x00 drivers except rt2800pci call ieee80211_tx_status() from a workqueue, which causes "NOHZ: local_softirq_pending 08" messages.

To fix it, add ieee80211_tx_status_ni() similar to ieee80211_rx_ni() which can be called from process context, and call it from rt2x00lib_txdone(). For the rt2800pci special case a driver flag is introduced.

---
v2: add ieee80211_tx_status_ni()
v3: simplify accorindg to review
v4: add rt2x00 as first user of ieee80211_tx_status_ni()

The DRIVER_REQUIRE_TASKLET_CONTEXT flag may seem ugly, but there is a pending rewrite of the rt2800pci irq handling code by Helmut, so we decided to use this easy solution.

Note: Only tested with rt2800usb.

 drivers/net/wireless/rt2x00/rt2800pci.c |    1 +
 drivers/net/wireless/rt2x00/rt2x00.h    |    1 +
 drivers/net/wireless/rt2x00/rt2x00dev.c |    9 ++++++---
 include/net/mac80211.h                  |   28 ++++++++++++++++++++++++----
 4 files changed, 32 insertions(+), 7 deletions(-)
drivers/net/wireless/rt2x00/rt2800pci.c +1 -0
index b48d2d3..3ef0228 100644
@@ -932,6 +932,7 @@ static int rt2800pci_probe_hw(struct rt2x00_dev *rt2x00dev)
 	__set_bit(DRIVER_REQUIRE_DMA, &rt2x00dev->flags);
 	__set_bit(DRIVER_REQUIRE_L2PAD, &rt2x00dev->flags);
 	__set_bit(DRIVER_REQUIRE_TXSTATUS_FIFO, &rt2x00dev->flags);
+	__set_bit(DRIVER_REQUIRE_TASKLET_CONTEXT, &rt2x00dev->flags);
 	if (!modparam_nohwcrypt)
 		__set_bit(CONFIG_SUPPORT_HW_CRYPTO, &rt2x00dev->flags);
 	__set_bit(DRIVER_SUPPORT_LINK_TUNING, &rt2x00dev->flags);
drivers/net/wireless/rt2x00/rt2x00.h +1 -0
index 556b744..28ea59a 100644
@@ -670,6 +670,7 @@ enum rt2x00_flags {
 	DRIVER_REQUIRE_COPY_IV,
 	DRIVER_REQUIRE_L2PAD,
 	DRIVER_REQUIRE_TXSTATUS_FIFO,
+	DRIVER_REQUIRE_TASKLET_CONTEXT,
 
 	/*
 	 * Driver features
drivers/net/wireless/rt2x00/rt2x00dev.c +6 -3
index 87a421b..fa74acd 100644
@@ -376,9 +376,12 @@ void rt2x00lib_txdone(struct queue_entry *entry,
 	 * through a mac80211 library call (RTS/CTS) then we should not
 	 * send the status report back.
 	 */
-	if (!(skbdesc_flags & SKBDESC_NOT_MAC80211))
-		ieee80211_tx_status(rt2x00dev->hw, entry->skb);
-	else
+	if (!(skbdesc_flags & SKBDESC_NOT_MAC80211)) {
+		if (test_bit(DRIVER_REQUIRE_TASKLET_CONTEXT, &rt2x00dev->flags))
+			ieee80211_tx_status(rt2x00dev->hw, entry->skb);
+		else
+			ieee80211_tx_status_ni(rt2x00dev->hw, entry->skb);
+	} else
 		dev_kfree_skb_any(entry->skb);
 
 	/*
include/net/mac80211.h +24 -4
index eaa4aff..e411cf8 100644
@@ -2055,8 +2055,8 @@ static inline void ieee80211_rx_ni(struct ieee80211_hw *hw,
  *
  * This function may not be called in IRQ context. Calls to this function
  * for a single hardware must be synchronized against each other. Calls
- * to this function and ieee80211_tx_status_irqsafe() may not be mixed
- * for a single hardware.
+ * to this function, ieee80211_tx_status_ni() and ieee80211_tx_status_irqsafe()
+ * may not be mixed for a single hardware.
  *
  * @hw: the hardware the frame was transmitted by
  * @skb: the frame that was transmitted, owned by mac80211 after this call
@@ -2065,13 +2065,33 @@ void ieee80211_tx_status(struct ieee80211_hw *hw,
 			 struct sk_buff *skb);
 
 /**
+ * ieee80211_tx_status_ni - transmit status callback (in process context)
+ *
+ * Like ieee80211_tx_status() but can be called in process context.
+ *
+ * Calls to this function, ieee80211_tx_status() and
+ * ieee80211_tx_status_irqsafe() may not be mixed
+ * for a single hardware.
+ *
+ * @hw: the hardware the frame was transmitted by
+ * @skb: the frame that was transmitted, owned by mac80211 after this call
+ */
+static inline void ieee80211_tx_status_ni(struct ieee80211_hw *hw,
+					  struct sk_buff *skb)
+{
+	local_bh_disable();
+	ieee80211_tx_status(hw, skb);
+	local_bh_enable();
+}
+
+/**
  * ieee80211_tx_status_irqsafe - IRQ-safe transmit status callback
  *
  * Like ieee80211_tx_status() but can be called in IRQ context
  * (internally defers to a tasklet.)
  *
- * Calls to this function and ieee80211_tx_status() may not be mixed for a
- * single hardware.
+ * Calls to this function, ieee80211_tx_status() and
+ * ieee80211_tx_status_ni() may not be mixed for a single hardware.
  *
  * @hw: the hardware the frame was transmitted by
  * @skb: the frame that was transmitted, owned by mac80211 after this call
Signature · 1 line
1.7.2.3
maintainer
#8 lore raw

Re: [PATCH v4] mac80211/rt2x00: add ieee80211_tx_status_ni()

Quoted 6 lines · Johannes Stezenbach

* This function may not be called in IRQ context. Calls to this function
* for a single hardware must be synchronized against each other. Calls
- * to this function and ieee80211_tx_status_irqsafe() may not be mixed
- * for a single hardware.
+ * to this function, ieee80211_tx_status_ni() and ieee80211_tx_status_irqsafe()
+ * may not be mixed for a single hardware.

I'm ok with this, although technically you can mix ieee80211_tx_status_ni() and ieee80211_tx_status(), just not either or both of them with _irqsafe().

johannes

#9 lore raw

Re: [PATCH v4] mac80211/rt2x00: add ieee80211_tx_status_ni()

Quoted 9 lines · Johannes Berg
Quoted 6 lines · Johannes Stezenbach

* This function may not be called in IRQ context. Calls to this function
* for a single hardware must be synchronized against each other. Calls
- * to this function and ieee80211_tx_status_irqsafe() may not be mixed
- * for a single hardware.
+ * to this function, ieee80211_tx_status_ni() and ieee80211_tx_status_irqsafe()
+ * may not be mixed for a single hardware.

Quoted 3 lines

I'm ok with this, although technically you can mix ieee80211_tx_status_ni() and ieee80211_tx_status(), just not either or both of them with _irqsafe().

I copied these from ieee80211_rx() etc. since I wasn't sure if there's not some subtlety I didn't get. One can mix ieee80211_rx() and ieee80211_rx_ni(), too, right?

Thanks
Johannes

maintainer
#10 lore raw

Re: [PATCH v4] mac80211/rt2x00: add ieee80211_tx_status_ni()

Quoted 10 lines · Johannes Stezenbach
Quoted 9 lines · Johannes Berg
Quoted 6 lines · Johannes Stezenbach

* This function may not be called in IRQ context. Calls to this function
* for a single hardware must be synchronized against each other. Calls
- * to this function and ieee80211_tx_status_irqsafe() may not be mixed
- * for a single hardware.
+ * to this function, ieee80211_tx_status_ni() and ieee80211_tx_status_irqsafe()
+ * may not be mixed for a single hardware.

Quoted 7 lines
Quoted 3 lines

I'm ok with this, although technically you can mix ieee80211_tx_status_ni() and ieee80211_tx_status(), just not either or both of them with _irqsafe().

I copied these from ieee80211_rx() etc. since I wasn't sure if there's not some subtlety I didn't get. One can mix ieee80211_rx() and ieee80211_rx_ni(), too, right?

Yes. Note that all this doesn't make any sense though, since you must never call them concurrently. And it's hard to imagine a situation where a single driver calls _ni and non-_ni versions, while making sure they can't both happen at the same time ...

johannes

#11 lore raw

Re: [PATCH v4] mac80211/rt2x00: add ieee80211_tx_status_ni()

Quoted 15 lines · Johannes Berg
Quoted 14 lines · Johannes Stezenbach
Quoted 12 lines · Johannes Berg
Quoted 6 lines · Johannes Stezenbach

* This function may not be called in IRQ context. Calls to this function
* for a single hardware must be synchronized against each other. Calls
- * to this function and ieee80211_tx_status_irqsafe() may not be mixed
- * for a single hardware.
+ * to this function, ieee80211_tx_status_ni() and ieee80211_tx_status_irqsafe()
+ * may not be mixed for a single hardware.

I'm ok with this, although technically you can mix ieee80211_tx_status_ni() and ieee80211_tx_status(), just not either or both of them with _irqsafe().

Quoted 8 lines
Quoted 3 lines

I copied these from ieee80211_rx() etc. since I wasn't sure if there's not some subtlety I didn't get. One can mix ieee80211_rx() and ieee80211_rx_ni(), too, right?

Yes. Note that all this doesn't make any sense though, since you must never call them concurrently. And it's hard to imagine a situation where a single driver calls _ni and non-_ni versions, while making sure they can't both happen at the same time ...

Maybe someone can clarify the wording of the comments and send a follow-on patch?

Thanks,

John

Signature · 2 lines
John W. Linville		Someday the world will need a hero, and you
[email protected]			might be all we have.  Be ready.