netpoll: fix race between skb_queue_len and skb_dequeue
Subsystems (2)
PATCH 1/1 netpoll: fix race between skb_queue_len and skb_dequeue
This race will break the messages order.
Sequence of events in the problem case:
Assume there is one skb "A" in skb_queue and the next action of netpoll_send_skb() is: sending out "B" skb.
The right order of messages should be: send A first, then send B.
But as following orders, it will send B first, then send A.
CPU0 CPU1
queue_process() netpoll_send_skb(B)
A = skb_dequeue()
/****
A was removed from skb_queue,
so skb_queue_len = 0
****/
Check skb_queue_len == 0 Yes
sendout (B)sendout (A)
The solution is to introduce the txq_skbuff_len in netpoll_info struct. The txq_skbuff_len will keep the skb queue lenght utill the skb was sent out, so that this race condition will gone.
--- include/linux/netpoll.h | 1 + net/core/netpoll.c | 8 ++++++-- 2 files changed, 7 insertions(+), 2 deletions(-)
include/linux/netpoll.h +1 -0
@@ -27,6 +27,7 @@ struct netpoll_info {atomic_t refcnt;int rx_flags;spinlock_t rx_lock; +__u32 txq_skbuff_len;struct netpoll *rx_np;/* netpoll that registered an rx_hook */ struct sk_buff_head arp_tx;/* list of arp requests to reply to */ struct sk_buff_head txq;
net/core/netpoll.c +6 -2
@@ -78,11 +78,14 @@ static void queue_process(struct work_struct *work)__netif_tx_unlock (txq);local_irq_restore (flags); + npinfo->txq_skbuff_len =skb_queue_len (&npinfo->txq);schedule_delayed_work (&npinfo->tx_work,HZ /10 );return ; }__netif_tx_unlock (txq);local_irq_restore (flags); + + npinfo->txq_skbuff_len =skb_queue_len (&npinfo->txq); } }
@@ -291,7 +294,7 @@ static void netpoll_send_skb(struct netpoll *np, struct sk_buff *skb) }/* don't get messages out of order, and no recursion */ -if (skb_queue_len (&npinfo->txq) ==0 && !netpoll_owner_active (dev)) { +if (npinfo->txq_skbuff_len ==0 && !netpoll_owner_active (dev)) {struct netdev_queue *txq;unsigned long flags;
@@ -329,7 +332,8 @@ static void netpoll_send_skb(struct netpoll *np, struct sk_buff *skb)if (status !=NETDEV_TX_OK ) {skb_queue_tail (&npinfo->txq, skb); -schedule_delayed_work (&npinfo->tx_work,0 ); + npinfo->txq_skbuff_len =skb_queue_len (&npinfo->txq); +schedule_delayed_work (&npinfo->tx_work,0 ); } }
Signature · 1 line
1.6.0.4
Quoted 9 lines · DDD
This race will break the messages order.
Sequence of events in the problem case:
Assume there is one skb "A" in
skb_queueand the next action ofnetpoll_send_skb()is: sending out "B" skb.
The right order of messages should be: send A first, then send B.But as following orders, it will send B first, then send A.
I would say no, the order of messages A and B queued on different CPUs is undefined. The only issue is that we can queue a message A on CPU0, then causally trigger a message on CPU1 B that arrives first. But bear in mind that the message A >>may never arrive<< because the message is about a lockup that kills processing of delayed work.
Generally speaking, queueing should be a last ditch effort. We should instead aim to deliver all messages immediately, even if they might be out of order. Because out of order is better than not arriving at all.
Signature · 1 line
http://selenic.com : development and support for Mercurial and Linux
Quoted 17 lines · Matt Mackall
Quoted 9 lines · DDD
This race will break the messages order.
Sequence of events in the problem case:
Assume there is one skb "A" in
skb_queueand the next action ofnetpoll_send_skb()is: sending out "B" skb.
The right order of messages should be: send A first, then send B.But as following orders, it will send B first, then send A.
I would say no, the order of messages A and B queued on different CPUs is undefined. The only issue is that we can queue a message A on CPU0, then causally trigger a message on CPU1 B that arrives first. But bear in mind that the message A >>may never arrive<< because the message is about a lockup that kills processing of delayed work.
Quoted 3 lines
Generally speaking, queueing should be a last ditch effort. We should instead aim to deliver all messages immediately, even if they might be out of order. Because out of order is better than not arriving at all.
Ah, yes, out of order is better than not arriving at all. :-)
Especially it is based on UDP, so I don't think the order of messages is important. :-)
I take back this patch.
Thank you very much,
Dongdong