sched: refactor NOLOCK qdiscs

After the previous patch, for NOLOCK qdiscs, q->seqlock is always held when the dequeue() is invoked, we can drop any additional locking to protect such operation.

Signed-off-by Paolo Abeni <[email protected]>

---
 include/linux/skb_array.h | 5 +++++
 net/sched/sch_generic.c   | 4 ++--
 2 files changed, 7 insertions(+), 2 deletions(-)

include/linux/skb_array.h +5 -0

index a6b6e8bb3d7b..62d9b0a6329f 100644
@@ -97,6 +97,11 @@ static inline bool skb_array_empty_any(struct skb_array *a)
 	return ptr_ring_empty_any(&a->ring);
 }
 
+static inline struct sk_buff *__skb_array_consume(struct skb_array *a)
+{
+	return __ptr_ring_consume(&a->ring);
+}
+
 static inline struct sk_buff *skb_array_consume(struct skb_array *a)
 {
 	return ptr_ring_consume(&a->ring);

net/sched/sch_generic.c +2 -2

index a126f16bc30b..760ab1b09f8b 100644
@@ -656,7 +656,7 @@ static struct sk_buff *pfifo_fast_dequeue(struct Qdisc *qdisc)
 		if (__skb_array_empty(q))
 			continue;
 
-		skb = skb_array_consume_bh(q);
+		skb = __skb_array_consume(q);
 	}
 	if (likely(skb)) {
 		qdisc_qstats_cpu_backlog_dec(qdisc, skb);

@@ -697,7 +697,7 @@ static void pfifo_fast_reset(struct Qdisc *qdisc)
 		if (!q->ring.queue)
 			continue;
 
-		while ((skb = skb_array_consume_bh(q)) != NULL)
+		while ((skb = __skb_array_consume(q)) != NULL)
 			kfree_skb(skb);
 	}
 

Signature · 1 line

2.14.3

Quoted 4 lines · Paolo Abeni

After the previous patch, for NOLOCK qdiscs, q->seqlock is always held when the dequeue() is invoked, we can drop any additional locking to protect such operation.

Quoted 5 lines

Signed-off-by Paolo Abeni <[email protected]>

---
 include/linux/skb_array.h | 5 +++++
 net/sched/sch_generic.c   | 4 ++--
 2 files changed, 7 insertions(+), 2 deletions(-)

Is the seqlock taken during qdisc_change_tx_queue_len? We need to prevent that racing with dequeue.

Quoted 11 lines · Michael S. Tsirkin

Quoted 9 lines · Paolo Abeni

After the previous patch, for NOLOCK qdiscs, q->seqlock is always held when the dequeue() is invoked, we can drop any additional locking to protect such operation.

Signed-off-by Paolo Abeni <[email protected]>

---
 include/linux/skb_array.h | 5 +++++
 net/sched/sch_generic.c   | 4 ++--
 2 files changed, 7 insertions(+), 2 deletions(-)

Quoted 2 lines

Is the seqlock taken during qdisc_change_tx_queue_len? We need to prevent that racing with dequeue.

Thanks for the head-up! I missed that code-path.

I'll add the lock in qdisc_change_tx_queue_len() in v2.

Quoted 5 lines · Michael S. Tsirkin

Quoted 4 lines · Paolo Abeni

After the previous patch, for NOLOCK qdiscs, q->seqlock is always held when the dequeue() is invoked, we can drop any additional locking to protect such operation.

Quoted 8 lines

Quoted 5 lines

Signed-off-by Paolo Abeni <[email protected]>

---
 include/linux/skb_array.h | 5 +++++
 net/sched/sch_generic.c   | 4 ++--
 2 files changed, 7 insertions(+), 2 deletions(-)

Is the seqlock taken during qdisc_change_tx_queue_len? We need to prevent that racing with dequeue.

Thanks for the head-up! I missed that code-path.

I'll add the lock in qdisc_change_tx_queue_len() in v2.

Thanks you,

Paolo

With the introduction of NOLOCK qdiscs, pfifo_fast performances in the uncontended scenario degraded measurably, especially after the commit eb82a9944792 ("net: sched, fix OOO packets with pfifo_fast").

This series restore the pfifo_fast performances in such scenario back the previous level, mainly reducing the number of atomic operations required to perform the qdisc_run() call. Even performances in the contended scenario increase measurably.

Note: This series is on top of:

sched: manipulate __QDISC_STATE_RUNNING in qdisc_run_* helpers