Skip to contentSkip to conversation

veth: schedule the peer's NAPI after XDP_TX when only GRO is enabled

PATCH Applied Tianyi Gao 5 participants 7 messages Oct 16 – Oct 22

Subsystems (4)
lore
Share & tools
lore

Tianyi Gao

author
#0 lore raw

cover veth: schedule the peer's NAPI after XDP_TX when only GRO is enabled

When an XDP program on a veth device returns XDP_TX and the peer has GRO enabled but no XDP program, the frames are queued on the peer's ring, but the peer's NAPI is never scheduled to process them. Once the ring is full, every XDP_TX frame is dropped.

Commit 0e672f306a28 ("veth: check for NAPI instead of xdp_prog before xmit of XDP frame") made such a peer work as an XDP_REDIRECT target, where veth_xdp_xmit() both queues the frames and schedules the peer's NAPI. XDP_TX queues its frames through the same function but schedules the NAPI in veth_xdp_flush(), which still checks xdp_prog.

The usual workaround is to load a dummy XDP program on the peer [1]. That turns off TSO on the device, and every skb it sends then goes through the peer's XDP path, where most of them are copied. In our tests, TCP sent from a namespace dropped from 22.1 to 14.8 Gbit/s with the dummy program. With this fix the program is no longer needed.

Patch 1 makes veth_xdp_flush() check the peer's napi pointer, as veth_xdp_xmit() does. Patch 2 adds a test for this case to veth.sh.

[1] https://lore.kernel.org/all/[email protected]/

---
Bochao Cao's migration of the XDP feature test [2] skips XDP_TX on veth when the peer has no XDP program, which is the case patch 1 fixes.

[2] https://lore.kernel.org/netdev/[email protected]/

Tianyi Gao (2):
veth: schedule the peer's NAPI after XDP_TX when only GRO is enabled
selftests: net: veth: test XDP_TX to a peer with GRO enabled

 drivers/net/veth.c                            |  2 +-
 .../testing/selftests/net/lib/xdp_dummy.bpf.c |  6 +++++
 tools/testing/selftests/net/veth.sh           | 25 +++++++++++++++++++
 3 files changed, 32 insertions(+), 1 deletion(-)

base-commit: 513444cc62e231a11c9b15ea88de0890da9b2b4f

Signature · 1 line
2.55.0
author
#1 lore raw

PATCH 1/2 veth: schedule the peer's NAPI after XDP_TX when only GRO is enabled

veth_xdp_xmit() queues XDP_TX frames on the peer's ring if the peer has NAPI. A peer with GRO enabled and no XDP program also has NAPI.

veth_xdp_flush() then schedules the peer's NAPI, but only if the peer's xdp_prog is set, so for such a peer the frames are never processed. Nothing else schedules that NAPI by default.

Once the ring is full, every XDP_TX frame is dropped, and so are the skbs that veth_xmit() queues on the same ring.

Commit 0e672f306a28 ("veth: check for NAPI instead of xdp_prog before xmit of XDP frame") changed the check in veth_xdp_xmit() but not the one in veth_xdp_flush(). Make veth_xdp_flush() check the napi pointer instead of xdp_prog, as veth_xdp_xmit() does.

Fixes 0e672f306a28 ("veth: check for NAPI instead of xdp_prog before xmit of XDP frame")
---
 drivers/net/veth.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/veth.c +1 -1
index 643b97dc5245..9405ffac598a 100644
@@ -612,7 +612,7 @@ static void veth_xdp_flush(struct veth_rq *rq, struct veth_xdp_tx_bq *bq)
 	rcv_priv = netdev_priv(rcv);
 	rcv_rq = &rcv_priv->rq[veth_select_rxq(rcv)];
 	/* xdp_ring is initialized on receive side? */
-	if (unlikely(!rcu_access_pointer(rcv_rq->xdp_prog)))
+	if (unlikely(!rcu_access_pointer(rcv_rq->napi)))
 		goto out;
 
 	__veth_xdp_flush(rcv_rq);
Signature · 1 line
2.55.0
Reviewed-by on v1 1/2
#2 lore raw
Quoted 26 lines · Tianyi Gao

veth_xdp_xmit() queues XDP_TX frames on the peer's ring if the peer has NAPI. A peer with GRO enabled and no XDP program also has NAPI.

veth_xdp_flush() then schedules the peer's NAPI, but only if the peer's xdp_prog is set, so for such a peer the frames are never processed. Nothing else schedules that NAPI by default.

Once the ring is full, every XDP_TX frame is dropped, and so are the skbs that veth_xmit() queues on the same ring.

Commit 0e672f306a28 ("veth: check for NAPI instead of xdp_prog before xmit of XDP frame") changed the check in veth_xdp_xmit() but not the one in veth_xdp_flush(). Make veth_xdp_flush() check the napi pointer instead of xdp_prog, as veth_xdp_xmit() does.

Fixes 0e672f306a28 ("veth: check for NAPI instead of xdp_prog before xmit of XDP frame")
---
  drivers/net/veth.c | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/veth.c +0 -0
index 643b97dc5245..9405ffac598a 100644

drivers/net/veth.c static void veth_xdp_flush(struct veth_rq *rq, struct veth_xdp_tx_bq *bq) · line 618

Quoted 8 lines
drivers/net/veth.c +1 -1
@@ -612,7 +612,7 @@ static void veth_xdp_flush(struct veth_rq *rq, struct veth_xdp_tx_bq *bq)
  	rcv_priv = netdev_priv(rcv);
  	rcv_rq = &rcv_priv->rq[veth_select_rxq(rcv)];
  	/* xdp_ring is initialized on receive side? */
-	if (unlikely(!rcu_access_pointer(rcv_rq->xdp_prog)))
+	if (unlikely(!rcu_access_pointer(rcv_rq->napi)))
  		goto out;
 
  	__veth_xdp_flush(rcv_rq);
Acked-by on v1 1/2
#3 lore raw
In reply to Tianyi Gao
Quoted 10 lines

veth_xdp_xmit() queues XDP_TX frames on the peer's ring if the peer has NAPI. A peer with GRO enabled and no XDP program also has NAPI.

veth_xdp_flush() then schedules the peer's NAPI, but only if the peer's xdp_prog is set, so for such a peer the frames are never processed. Nothing else schedules that NAPI by default.

Once the ring is full, every XDP_TX frame is dropped, and so are the skbs that veth_xmit() queues on the same ring.

Quoted 7 lines

Commit 0e672f306a28 ("veth: check for NAPI instead of xdp_prog before xmit of XDP frame") changed the check in veth_xdp_xmit() but not the one in veth_xdp_flush(). Make veth_xdp_flush() check the napi pointer instead of xdp_prog, as veth_xdp_xmit() does.

Fixes 0e672f306a28 ("veth: check for NAPI instead of xdp_prog before xmit of XDP frame")
Quoted 20 lines
---
 drivers/net/veth.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/veth.c +1 -1
index 643b97dc5245..9405ffac598a 100644
@@ -612,7 +612,7 @@ static void veth_xdp_flush(struct veth_rq *rq, struct veth_xdp_tx_bq *bq)
 	rcv_priv = netdev_priv(rcv);
 	rcv_rq = &rcv_priv->rq[veth_select_rxq(rcv)];
 	/* xdp_ring is initialized on receive side? */
-	if (unlikely(!rcu_access_pointer(rcv_rq->xdp_prog)))
+	if (unlikely(!rcu_access_pointer(rcv_rq->napi)))
 		goto out;
 
 	__veth_xdp_flush(rcv_rq);
Signature · 1 line
2.55.0
#4 lore raw
Quoted 15 lines · Lorenzo Bianconi
Quoted 15 lines

veth_xdp_xmit() queues XDP_TX frames on the peer's ring if the peer has NAPI. A peer with GRO enabled and no XDP program also has NAPI.

veth_xdp_flush() then schedules the peer's NAPI, but only if the peer's xdp_prog is set, so for such a peer the frames are never processed. Nothing else schedules that NAPI by default.

Once the ring is full, every XDP_TX frame is dropped, and so are the skbs that veth_xmit() queues on the same ring.

Commit 0e672f306a28 ("veth: check for NAPI instead of xdp_prog before xmit of XDP frame") changed the check in veth_xdp_xmit() but not the one in veth_xdp_flush(). Make veth_xdp_flush() check the napi pointer instead of xdp_prog, as veth_xdp_xmit() does.

Quoted 4 lines
Quoted 2 lines
Fixes 0e672f306a28 ("veth: check for NAPI instead of xdp_prog before xmit of XDP frame")

LGTM

Quoted 20 lines
Quoted 20 lines
---
  drivers/net/veth.c | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/veth.c +1 -1
index 643b97dc5245..9405ffac598a 100644
@@ -612,7 +612,7 @@ static void veth_xdp_flush(struct veth_rq *rq, struct veth_xdp_tx_bq *bq)
  	rcv_priv = netdev_priv(rcv);
  	rcv_rq = &rcv_priv->rq[veth_select_rxq(rcv)];
  	/* xdp_ring is initialized on receive side? */
-	if (unlikely(!rcu_access_pointer(rcv_rq->xdp_prog)))
+	if (unlikely(!rcu_access_pointer(rcv_rq->napi)))
  		goto out;
 
  	__veth_xdp_flush(rcv_rq);
Signature · 1 line
2.55.0
author
#5 lore raw

PATCH 2/2 selftests: net: veth: test XDP_TX to a peer with GRO enabled

In reply to Tianyi Gao

Test that XDP_TX frames reach a peer that has GRO enabled and no XDP program. GRO enables NAPI on the peer, so the frames are queued on the peer's ring, and the peer's NAPI must be scheduled to process them.

Attach a program that returns XDP_TX for every frame, send 5 echo requests and check that all of them were sent back and that the peer's NAPI received them, as counted in its xdp_packets statistic. IPv6 is disabled on both devices, so that only the echo requests are counted.

Add the program to xdp_dummy.bpf.c in a section of its own, so that the tests that load "section xdp" still get XDP_PASS.

---
The XDP_TX program goes into xdp_dummy.bpf.c as a new section instead of a new file, since it is a single line; 75cc19c8ff89 ("selftests:
drv-net: add xdp cases for ping.py") added the xdp.frags variant there the same way. libbpf does not know the section name "xdp_tx", but iproute2 loads it, as tc loads the custom sections in nat6to4.bpf.c.

 .../testing/selftests/net/lib/xdp_dummy.bpf.c |  6 +++++
 tools/testing/selftests/net/veth.sh           | 25 +++++++++++++++++++
 2 files changed, 31 insertions(+)
tools/testing/selftests/net/lib/xdp_dummy.bpf.c +6 -0
index e73fab3edd9f..95abd67cc184 100644
@@ -16,4 +16,10 @@ int xdp_dummy_prog_frags(struct xdp_md *ctx)
 	return XDP_PASS;
 }
 
+SEC("xdp_tx")
+int xdp_dummy_tx(struct xdp_md *ctx)
+{
+	return XDP_TX;
+}
+
 char _license[] SEC("license") = "GPL";
tools/testing/selftests/net/veth.sh +25 -0
index 8b867e0675a7..0167f78604ef 100755
@@ -334,6 +334,31 @@ ip netns exec $NS_DST ethtool -K veth$DST rx-udp-gro-forwarding on
 chk_gro "        - aggregation with TSO off" 1
 cleanup
 
+create_ns
+ip netns exec $NS_SRC sysctl -qw net.ipv6.conf.veth$SRC.disable_ipv6=1
+ip netns exec $NS_DST sysctl -qw net.ipv6.conf.veth$DST.disable_ipv6=1
+ip netns exec $NS_SRC ethtool -K veth$SRC gro on
+ip -n $NS_DST link set dev veth$DST xdp object ${BPF_FILE} section xdp_tx
+# veth$DST sends every frame back, so neither ARP nor ping gets an answer
+mac=`ip netns exec $NS_DST cat /sys/class/net/veth$DST/address`
+ip -n $NS_SRC neigh replace $BM_NET_V4$DST lladdr $mac dev veth$SRC
+ip netns exec $NS_SRC ping -qc 5 -i 0.01 -W 1 $BM_NET_V4$DST >/dev/null
+tx=`ip netns exec $NS_DST ethtool -S veth$DST | \
+    awk '/rx_queue_[0-9]+_xdp_tx:/ {s += $2} END {print s + 0}'`
+rx=`ip netns exec $NS_SRC ethtool -S veth$SRC | \
+    awk '/rx_queue_[0-9]+_xdp_packets:/ {s += $2} END {print s + 0}'`
+printf "%-60s" "xdp_tx to peer with gro on"
+if [ $tx -ne 5 ]; then
+	echo " fail - $tx frames sent back, expected 5"
+	ret=1
+elif [ $rx -ne 5 ]; then
+	echo " fail - $rx frames received by the peer NAPI, expected 5"
+	ret=1
+else
+	echo " ok "
+fi
+cleanup
+
 create_ns
 
 CUR_TX=1
Signature · 1 line
2.55.0
#6 lore raw
In reply to Tianyi Gao
This series was applied to netdev/net.git (main) by Jakub Kicinski <[email protected]>:

Hello:

This series was applied to netdev/net.git (main)
by Jakub Kicinski <[email protected]>:

Quoted 5 lines · On Fri, 16 Oct 2026 22:42:15 +0100 you wrote:

When an XDP program on a veth device returns XDP_TX and the peer has GRO enabled but no XDP program, the frames are queued on the peer's ring, but the peer's NAPI is never scheduled to process them. Once the ring is full, every XDP_TX frame is dropped.

Quoted 7 lines

Commit 0e672f306a28 ("veth: check for NAPI instead of xdp_prog before xmit of XDP frame") made such a peer work as an XDP_REDIRECT target, where veth_xdp_xmit() both queues the frames and schedules the peer's NAPI. XDP_TX queues its frames through the same function but schedules the NAPI in veth_xdp_flush(), which still checks xdp_prog.

[...]

Here is the summary with links:
- [net,1/2] veth: schedule the peer's NAPI after XDP_TX when only GRO is enabled

    https://git.kernel.org/netdev/net/c/f16b8c6a656e
  - [net,2/2] selftests: net: veth: test XDP_TX to a peer with GRO enabled
    https://git.kernel.org/netdev/net/c/a70575ae9e81

You are awesome, thank you!

Signature · 2 lines
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html