SCHED: Fix unnecesary driver entries when queue is stopped

Dave,

This patch addresses an issue with multiqueue devices and non-multiqueue qdiscs which is causing performance issues. This patch should be considered for both 2.6.23-stable and 2.6.24 upstream. Basically, if a driver is using the netif_*_subqueue() calls, then qdisc_restart() will happily call hard_start_xmit() even if subqueue 0 is stopped, which is bad. This re-adds the check for the subqueue state.

Note that this check was removed when qdisc_restart() was rewritten. At that time though, we didn't understand the full effect of multiqueue with respect to the qdiscs and queue management from a driver to kernel perspective. Since the driver doesn't know what qdisc capabilities live above it, it needs to decide to use the queue or subqueue functions ahead of time. This patch is just cleaning up a miss from that rewrite.

Patch 1 is for 2.6.24, patch 2 is for 2.6.23 stable.

Thanks,

PJ Waskiewicz <[email protected]>

The only qdiscs that check subqueue state before dequeue'ing are PRIO and RR. The other qdiscs, including the default pfifo_fast qdisc, will allow traffic bound for subqueue 0 through to hard_start_xmit. The check for netif_queue_stopped() is done above in pkt_sched.h, so it is unnecessary for qdisc_restart(). However, if the underlying driver is multiqueue capable, and only sets queue states on subqueues, this will allow packets to enter the driver when it's currently unable to process packets, resulting in expensive requeues and driver entries. This patch re-adds the check for the subqueue status before calling hard_start_xmit, so we can try and avoid the driver entry when the queues are stopped.

---

 net/sched/sch_generic.c |    5 +++--
 1 files changed, 3 insertions(+), 2 deletions(-)

index fa1a6f4..e595e65 100644
@@ -134,7 +134,7 @@ static inline int qdisc_restart(struct net_device *dev)
 {
 	struct Qdisc *q = dev->qdisc;
 	struct sk_buff *skb;
-	int ret;
+	int ret = NETDEV_TX_BUSY;
 
 	/* Dequeue packet */
 	if (unlikely((skb = dev_dequeue_skb(dev, q)) == NULL))

@@ -145,7 +145,8 @@ static inline int qdisc_restart(struct net_device *dev)
 	spin_unlock(&dev->queue_lock);
 
 	HARD_TX_LOCK(dev, smp_processor_id());
-	ret = dev_hard_start_xmit(skb, dev);
+	if (!netif_subqueue_stopped(dev, skb))
+		ret = dev_hard_start_xmit(skb, dev);
 	HARD_TX_UNLOCK(dev);
 
 	spin_lock(&dev->queue_lock);

From: PJ Waskiewicz <[email protected]>
Date: Tue, 13 Nov 2007 09:44:50 -0800

Applied, and I'll queue up the other one for -stable, thanks.