veth: schedule the peer's NAPI after XDP_TX when only GRO is enabled
Subsystems (4)
cover veth: schedule the peer's NAPI after XDP_TX when only GRO is enabled
When an XDP program on a veth device returns XDP_TX and the peer has GRO enabled but no XDP program, the frames are queued on the peer's ring, but the peer's NAPI is never scheduled to process them. Once the ring is full, every XDP_TX frame is dropped.
Commit 0e672f306a28 ("veth: check for NAPI instead of xdp_prog before xmit of XDP frame") made such a peer work as an XDP_REDIRECT target, where veth_xdp_xmit() both queues the frames and schedules the peer's NAPI. XDP_TX queues its frames through the same function but schedules the NAPI in veth_xdp_flush(), which still checks xdp_prog.
The usual workaround is to load a dummy XDP program on the peer [1]. That turns off TSO on the device, and every skb it sends then goes through the peer's XDP path, where most of them are copied. In our tests, TCP sent from a namespace dropped from 22.1 to 14.8 Gbit/s with the dummy program. With this fix the program is no longer needed.
Patch 1 makes veth_xdp_flush() check the peer's napi pointer, as veth_xdp_xmit() does. Patch 2 adds a test for this case to veth.sh.
[1] https://lore.kernel.org/all/[email protected]/
---
Bochao Cao's migration of the XDP feature test [2] skips XDP_TX on veth when the peer has no XDP program, which is the case patch 1 fixes.
[2] https://lore.kernel.org/netdev/[email protected]/
Tianyi Gao (2):
veth: schedule the peer's NAPI after XDP_TX when only GRO is enabled
selftests: net: veth: test XDP_TX to a peer with GRO enabled
drivers/net/veth.c | 2 +- .../testing/selftests/net/lib/xdp_dummy.bpf.c | 6 +++++ tools/testing/selftests/net/veth.sh | 25 +++++++++++++++++++ 3 files changed, 32 insertions(+), 1 deletion(-)
base-commit: 513444cc62e231a11c9b15ea88de0890da9b2b4f
Signature · 1 line
2.55.0
PATCH 1/2 veth: schedule the peer's NAPI after XDP_TX when only GRO is enabled
veth_xdp_xmit() queues XDP_TX frames on the peer's ring if the peer has NAPI. A peer with GRO enabled and no XDP program also has NAPI.
veth_xdp_flush() then schedules the peer's NAPI, but only if the peer's xdp_prog is set, so for such a peer the frames are never processed. Nothing else schedules that NAPI by default.
Once the ring is full, every XDP_TX frame is dropped, and so are the skbs that veth_xmit() queues on the same ring.
Commit 0e672f306a28 ("veth: check for NAPI instead of xdp_prog before xmit of XDP frame") changed the check in veth_xdp_xmit() but not the one in veth_xdp_flush(). Make veth_xdp_flush() check the napi pointer instead of xdp_prog, as veth_xdp_xmit() does.
--- drivers/net/veth.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/veth.c +1 -1
@@ -612,7 +612,7 @@ static void veth_xdp_flush(struct veth_rq *rq, struct veth_xdp_tx_bq *bq) rcv_priv =netdev_priv (rcv); rcv_rq = &rcv_priv->rq[veth_select_rxq (rcv)];/* xdp_ring is initialized on receive side? */ -if (unlikely (!rcu_access_pointer (rcv_rq->xdp_prog))) +if (unlikely (!rcu_access_pointer (rcv_rq->napi))) goto out;__veth_xdp_flush (rcv_rq);
Signature · 1 line
2.55.0
Quoted 26 lines · Tianyi Gao
veth_xdp_xmit()queuesXDP_TXframes on the peer's ring if the peer has NAPI. A peer with GRO enabled and no XDP program also has NAPI.
veth_xdp_flush()then schedules the peer's NAPI, but only if the peer'sxdp_progis set, so for such a peer the frames are never processed. Nothing else schedules that NAPI by default.Once the ring is full, every
XDP_TXframe is dropped, and so are the skbs thatveth_xmit()queues on the same ring.Commit
0e672f306a28("veth: check for NAPI instead ofxdp_progbefore xmit of XDP frame") changed the check inveth_xdp_xmit()but not the one inveth_xdp_flush(). Makeveth_xdp_flush()check the napi pointer instead ofxdp_prog, asveth_xdp_xmit()does.Fixes 0e672f306a28 ("veth: check for NAPI instead of xdp_prog before xmit of XDP frame")Signed-off-by Tianyi Gao <[email protected]>--- drivers/net/veth.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)drivers/net/veth.c +0
-0
drivers/net/veth.c static void veth_xdp_flush(struct veth_rq *rq, struct veth_xdp_tx_bq *bq) · line 618
Quoted 8 lines
drivers/net/veth.c +1
-1@@ -612,7 +612,7 @@ static void veth_xdp_flush(struct veth_rq *rq, struct veth_xdp_tx_bq *bq) rcv_priv =netdev_priv (rcv); rcv_rq = &rcv_priv->rq[veth_select_rxq (rcv)];/* xdp_ring is initialized on receive side? */ -if (unlikely (!rcu_access_pointer (rcv_rq->xdp_prog))) +if (unlikely (!rcu_access_pointer (rcv_rq->napi))) goto out;__veth_xdp_flush (rcv_rq);
Quoted 10 lines
veth_xdp_xmit()queuesXDP_TXframes on the peer's ring if the peer has NAPI. A peer with GRO enabled and no XDP program also has NAPI.
veth_xdp_flush()then schedules the peer's NAPI, but only if the peer'sxdp_progis set, so for such a peer the frames are never processed. Nothing else schedules that NAPI by default.Once the ring is full, every
XDP_TXframe is dropped, and so are the skbs thatveth_xmit()queues on the same ring.
Quoted 7 lines
Commit
0e672f306a28("veth: check for NAPI instead ofxdp_progbefore xmit of XDP frame") changed the check inveth_xdp_xmit()but not the one inveth_xdp_flush(). Makeveth_xdp_flush()check the napi pointer instead ofxdp_prog, asveth_xdp_xmit()does.Fixes 0e672f306a28 ("veth: check for NAPI instead of xdp_prog before xmit of XDP frame")Signed-off-by Tianyi Gao <[email protected]>
Quoted 20 lines
--- drivers/net/veth.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)drivers/net/veth.c +1
-1@@ -612,7 +612,7 @@ static void veth_xdp_flush(struct veth_rq *rq, struct veth_xdp_tx_bq *bq) rcv_priv =netdev_priv (rcv); rcv_rq = &rcv_priv->rq[veth_select_rxq (rcv)];/* xdp_ring is initialized on receive side? */ -if (unlikely (!rcu_access_pointer (rcv_rq->xdp_prog))) +if (unlikely (!rcu_access_pointer (rcv_rq->napi))) goto out;__veth_xdp_flush (rcv_rq);Signature · 1 line
2.55.0
Jesper Dangaard Brouer
Acked-byQuoted 15 lines · Lorenzo Bianconi
Quoted 15 lines
veth_xdp_xmit()queuesXDP_TXframes on the peer's ring if the peer has NAPI. A peer with GRO enabled and no XDP program also has NAPI.
veth_xdp_flush()then schedules the peer's NAPI, but only if the peer'sxdp_progis set, so for such a peer the frames are never processed. Nothing else schedules that NAPI by default.Once the ring is full, every
XDP_TXframe is dropped, and so are the skbs thatveth_xmit()queues on the same ring.Commit
0e672f306a28("veth: check for NAPI instead ofxdp_progbefore xmit of XDP frame") changed the check inveth_xdp_xmit()but not the one inveth_xdp_flush(). Makeveth_xdp_flush()check the napi pointer instead ofxdp_prog, asveth_xdp_xmit()does.
Quoted 4 lines
Quoted 2 lines
Fixes 0e672f306a28 ("veth: check for NAPI instead of xdp_prog before xmit of XDP frame")Signed-off-by Tianyi Gao <[email protected]>Acked-by Lorenzo Bianconi <[email protected]>
LGTM
Quoted 20 lines
Quoted 20 lines
--- drivers/net/veth.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)drivers/net/veth.c +1
-1@@ -612,7 +612,7 @@ static void veth_xdp_flush(struct veth_rq *rq, struct veth_xdp_tx_bq *bq) rcv_priv =netdev_priv (rcv); rcv_rq = &rcv_priv->rq[veth_select_rxq (rcv)];/* xdp_ring is initialized on receive side? */ -if (unlikely (!rcu_access_pointer (rcv_rq->xdp_prog))) +if (unlikely (!rcu_access_pointer (rcv_rq->napi))) goto out;__veth_xdp_flush (rcv_rq);Signature · 1 line
2.55.0
PATCH 2/2 selftests: net: veth: test XDP_TX to a peer with GRO enabled
In reply to Tianyi GaoTest that XDP_TX frames reach a peer that has GRO enabled and no XDP program. GRO enables NAPI on the peer, so the frames are queued on the peer's ring, and the peer's NAPI must be scheduled to process them.
Attach a program that returns XDP_TX for every frame, send 5 echo requests and check that all of them were sent back and that the peer's NAPI received them, as counted in its xdp_packets statistic. IPv6 is disabled on both devices, so that only the echo requests are counted.
Add the program to xdp_dummy.bpf.c in a section of its own, so that the tests that load "section xdp" still get XDP_PASS.
---
The XDP_TX program goes into xdp_dummy.bpf.c as a new section instead of a new file, since it is a single line; 75cc19c8ff89 ("selftests:
drv-net: add xdp cases for ping.py") added the xdp.frags variant there the same way. libbpf does not know the section name "xdp_tx", but iproute2 loads it, as tc loads the custom sections in nat6to4.bpf.c.
.../testing/selftests/net/lib/xdp_dummy.bpf.c | 6 +++++ tools/testing/selftests/net/veth.sh | 25 +++++++++++++++++++ 2 files changed, 31 insertions(+)
tools/testing/selftests/net/lib/xdp_dummy.bpf.c +6 -0
@@ -16,4 +16,10 @@ int xdp_dummy_prog_frags(struct xdp_md *ctx)return XDP_PASS ; } +SEC("xdp_tx") +int xdp_dummy_tx (struct xdp_md *ctx) +{ +return XDP_TX ; +} + char _license[] SEC("license") = "GPL";
tools/testing/selftests/net/veth.sh +25 -0
@@ -334,6 +334,31 @@ ip netns exec $NS_DST ethtool -K veth$DST rx-udp-gro-forwarding on chk_gro " - aggregation with TSO off" 1 cleanup +create_ns +ip netns exec $NS_SRC sysctl -qw net.ipv6.conf.veth$SRC.disable_ipv6=1 +ip netns exec $NS_DST sysctl -qw net.ipv6.conf.veth$DST.disable_ipv6=1 +ip netns exec $NS_SRC ethtool -K veth$SRC gro on +ip -n $NS_DST link set dev veth$DST xdp object ${BPF_FILE} section xdp_tx +# veth$DST sends every frame back, so neither ARP nor ping gets an answer +mac=`ip netns exec $NS_DST cat /sys/class/net/veth$DST/address` +ip -n $NS_SRC neigh replace $BM_NET_V4$DST lladdr $mac dev veth$SRC +ip netns exec $NS_SRC ping -qc 5 -i 0.01 -W 1 $BM_NET_V4$DST >/dev/null +tx=`ip netns exec $NS_DST ethtool -S veth$DST | \ + awk '/rx_queue_[0-9]+_xdp_tx:/ {s += $2} END {print s + 0}'` +rx=`ip netns exec $NS_SRC ethtool -S veth$SRC | \ + awk '/rx_queue_[0-9]+_xdp_packets:/ {s += $2} END {print s + 0}'` +printf "%-60s" "xdp_tx to peer with gro on" +if [ $tx -ne 5 ]; then + echo " fail - $tx frames sent back, expected 5" + ret=1 +elif [ $rx -ne 5 ]; then + echo " fail - $rx frames received by the peer NAPI, expected 5" + ret=1 +else + echo " ok " +fi +cleanup + create_ns CUR_TX=1
Signature · 1 line
2.55.0
This series was applied to netdev/net.git (main) by Jakub Kicinski <[email protected]>:
Hello:
This series was applied to netdev/net.git (main)
by Jakub Kicinski <[email protected]>:
Quoted 5 lines · On Fri, 16 Oct 2026 22:42:15 +0100 you wrote:
When an XDP program on a veth device returns
XDP_TXand the peer has GRO enabled but no XDP program, the frames are queued on the peer's ring, but the peer's NAPI is never scheduled to process them. Once the ring is full, everyXDP_TXframe is dropped.
Quoted 7 lines
Commit
0e672f306a28("veth: check for NAPI instead ofxdp_progbefore xmit of XDP frame") made such a peer work as anXDP_REDIRECTtarget, whereveth_xdp_xmit()both queues the frames and schedules the peer's NAPI.XDP_TXqueues its frames through the same function but schedules the NAPI inveth_xdp_flush(), which still checksxdp_prog.[...]
Here is the summary with links:
- [net,1/2] veth: schedule the peer's NAPI after XDP_TX when only GRO is enabled
https://git.kernel.org/netdev/net/c/f16b8c6a656e - [net,2/2] selftests: net: veth: test XDP_TX to a peer with GRO enabled https://git.kernel.org/netdev/net/c/a70575ae9e81
You are awesome, thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html