Skip to contentSkip to conversation

virtio_net: invoke softirqs after __napi_schedule

PATCH In review Michael S. Tsirkin 3 participants 3 messages May 16, 2012 – May 17, 2012

Subsystems (4)
Networking Network drivers virtio core virtio net driver
lore
Share & tools
lore

Michael S. Tsirkin

author maintainer
#0 lore raw

PATCH 1/1 virtio_net: invoke softirqs after __napi_schedule

__napi_schedule might raise softirq but nothing
causes do_softirq to trigger, so it does not in fact
run. As a result,
the error message "NOHZ: local_softirq_pending 08"
sometimes occurs during boot of a KVM guest when the network service is started and we are oom:

...
Bringing up loopback interface: [ OK ]
Bringing up interface eth0:
Determining IP information for eth0...NOHZ: local_softirq_pending 08
done.
[ OK ]
...

Further, receive queue processing might get delayed indefinitely until some interrupt triggers:
virtio_net expected napi to be run immediately.

One way to cause do_softirq to be executed is by invoking local_bh_enable(). As __napi_schedule is normally called from bh or irq context, this seems to make sense: disable bh before __napi_schedule and enable afterwards.

Reported-by Ulrich Obergfell <[email protected]>
Tested-by Ulrich Obergfell <[email protected]>
Signed-off-by Michael S. Tsirkin <[email protected]>

---

To test, one can hack try_fill_recv to always report oom. I'm not sure it's not too late for 3.4, but we can try. Rusty, could you review ASAP pls?

 drivers/net/virtio_net.c |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)
drivers/net/virtio_net.c +2 -0
index af8acc8..cbefe67 100644
@@ -492,7 +492,9 @@ static void virtnet_napi_enable(struct virtnet_info *vi)
 	 * We synchronize against interrupts via NAPI_STATE_SCHED */
 	if (napi_schedule_prep(&vi->napi)) {
 		virtqueue_disable_cb(vi->rvq);
+		local_bh_disable();
 		__napi_schedule(&vi->napi);
+		local_bh_enable();
 	}
 }
Signature · 1 line
MST
Acked-by on v1 1/1
#1 lore raw
Quoted 25 lines · Michael S. Tsirkin

__napi_schedule might raise softirq but nothing
causes do_softirq to trigger, so it does not in fact
run. As a result,
the error message "NOHZ: local_softirq_pending 08"
sometimes occurs during boot of a KVM guest when the network service is started and we are oom:

...
Bringing up loopback interface: [ OK ]
Bringing up interface eth0:
Determining IP information for eth0...NOHZ: local_softirq_pending 08
done.
[ OK ]
...

Further, receive queue processing might get delayed indefinitely until some interrupt triggers:
virtio_net expected napi to be run immediately.

One way to cause do_softirq to be executed is by invoking local_bh_enable(). As __napi_schedule is normally called from bh or irq context, this seems to make sense: disable bh before __napi_schedule and enable afterwards.

Quoted 8 lines
Reported-by Ulrich Obergfell <[email protected]>
Tested-by Ulrich Obergfell <[email protected]>
Signed-off-by Michael S. Tsirkin <[email protected]>

---

To test, one can hack try_fill_recv to always report oom. I'm not sure it's not too late for 3.4, but we can try. Rusty, could you review ASAP pls?

It's missing a big comment: it's a very complicated way of calling do_softirq().

Indeed, this function is only used when we are not in interrupt context. It's not hot at all, in any ideal scenario.

Acked-by Rusty Russell <[email protected]>
maintainer
#2 lore raw

From: Rusty Russell <[email protected]>
Date: Thu, 17 May 2012 13:02:53 +0930

Quoted 29 lines · Michael S. Tsirkin
Quoted 28 lines

__napi_schedule might raise softirq but nothing
causes do_softirq to trigger, so it does not in fact
run. As a result,
the error message "NOHZ: local_softirq_pending 08"
sometimes occurs during boot of a KVM guest when the network service is started and we are oom:

...
Bringing up loopback interface: [ OK ]
Bringing up interface eth0:
Determining IP information for eth0...NOHZ: local_softirq_pending 08
done.
[ OK ]
...

Further, receive queue processing might get delayed indefinitely until some interrupt triggers:
virtio_net expected napi to be run immediately.

One way to cause do_softirq to be executed is by invoking local_bh_enable(). As __napi_schedule is normally called from bh or irq context, this seems to make sense: disable bh before __napi_schedule and enable afterwards.

Reported-by Ulrich Obergfell <[email protected]>
Tested-by Ulrich Obergfell <[email protected]>
Signed-off-by Michael S. Tsirkin <[email protected]>

...

Quoted 1 line
Acked-by Rusty Russell <[email protected]>

Michael, you're best to submit this directly to Linus as I just made what I hope is my last push to him for 3.4 today.