Skip to contentSkip to conversation

xsk: set network header on skb at generic transmit

PATCH In review Jason Xing 3 participants 3 messages Nov 20

Subsystems (5)
lore
Share & tools
lore

Jason Xing

author
#0 lore raw

PATCH v4 1/1 xsk: set network header on skb at generic transmit

When constructing an skb in the XSK generic transmit path, neither xsk_build_skb() nor xsk_build_skb_zerocopy() calls skb_reset_network_header(). After __finalize_skb_around(), the network_header offset defaults to 0 (pointing to skb->head). The subsequent skb_reserve(skb, hr) moves skb->data forward by hr bytes, but leaves network_header still at 0. This makes skb_network_offset() return a negative value (-hr).

When such an skb reaches a tunnel driver like ipgre_xmit(), the call chain pskb_inet_may_pull() -> pskb_network_may_pull_reason() computes:

pskb_may_pull_reason(skb, skb_network_offset(skb) + nhlen)

The negative int from skb_network_offset() is implicitly promoted to unsigned int, wrapping around to a huge value. This triggers the DEBUG_NET_WARN_ON_ONCE(len > INT_MAX) in pskb_may_pull_reason().

Call trace from syzkaller:

  ipgre_xmit+0x958/0xcd0 net/ipv4/ip_gre.c:658
  __netdev_start_xmit include/linux/netdevice.h:5400
  netdev_start_xmit include/linux/netdevice.h:5409
  __dev_direct_xmit+0x4b6/0x730 net/core/dev.c:4942
  __xsk_generic_xmit net/xdp/xsk.c:1079
  xsk_generic_xmit+0x277a/0x40a0 net/xdp/xsk.c:1120

Reset the network header so skb_network_offset() is valid (0). For the non-CHECKSUM_PARTIAL case, unlike AF_PACKET, AF_XDP does not need to locate the real L2/L3 boundary: an AF_XDP frame is always a complete, opaque blob from userspace. So resetting the network header is enough to restore the invariant and avoid the crash.

Note this is not sufficient for CHECKSUM_PARTIAL skbs, where drivers derive the L2/L3 split from the header offsets for checksum offload; that will be handled separately after this patch.

Fixes 3914d88f7608 ("xsk: Respect device's headroom and tailroom on generic xmit path")
Reported-by [email protected]

---
V4

1. use the one-liner change to fix non csum case at a minimum. As for
csum, since the blamed commit is different, I plan to cook a patch to
fix that[1].
2. revise the commit message accordingly.
3. add James reported-by tag since he spotted this and sent this in
private.
[1]: https://lore.kernel.org/all/CAL+tcoB8f7H1ptcEk1iVH065AcUNPEL2V1gKoJacwr3zndfPWA@mail.gmail.com/

V3

1. remove the misleading description about af_packet

V2

1. add more clarification about why we don't probe the header like packet at the end of the commit message (Jakub)

---
 net/xdp/xsk.c | 2 ++
 1 file changed, 2 insertions(+)
net/xdp/xsk.c +2 -0
index f22f9a2cdbbe..52d538160e0a 100644
@@ -948,6 +948,8 @@ static int xsk_skb_init_misc(struct sk_buff *skb, struct xdp_sock *xs,
 {
 	int err;
 
+	skb_reset_network_header(skb);
+
 	err = xsk_skb_destructor_set_addr(skb, addr);
 	if (unlikely(err))
 		return err;
Signature · 1 line
2.43.7
Tested-by on v4 1/1
#1 lore raw
Quoted 1 line · Jason Xing

Reset the network header so skb_network_offset() is valid (0).

Hi Jason,

I tested v4 on net main (d7f2d9b6c), comparing the same commit with and without the patch on arm64 and x86_64.

Without the patch, an AF_XDP copy-mode skb sent through a macvlan reaches the lower device's netfilter egress hook with skb_network_offset() equal to -hr, and @nh reads return stale headroom, including another user's data. With the patch, @nh starts at the frame on every packet (300,000 per run). The same test returned only the frame's own bytes, and the issue I reported no longer reproduces.

Thanks for the quick turnaround.

James

maintainer Reviewed-by on v4 1/1
#2 lore raw
In reply to Jason Xing
Quoted 35 lines · Jason Xing

When constructing an skb in the XSK generic transmit path, neither xsk_build_skb() nor xsk_build_skb_zerocopy() calls skb_reset_network_header(). After __finalize_skb_around(), the network_header offset defaults to 0 (pointing to skb->head). The subsequent skb_reserve(skb, hr) moves skb->data forward by hr bytes, but leaves network_header still at 0. This makes skb_network_offset() return a negative value (-hr).

When such an skb reaches a tunnel driver like ipgre_xmit(), the call chain pskb_inet_may_pull() -> pskb_network_may_pull_reason() computes:

pskb_may_pull_reason(skb, skb_network_offset(skb) + nhlen)

The negative int from skb_network_offset() is implicitly promoted to unsigned int, wrapping around to a huge value. This triggers the DEBUG_NET_WARN_ON_ONCE(len > INT_MAX) in pskb_may_pull_reason().

Call trace from syzkaller:

  ipgre_xmit+0x958/0xcd0 net/ipv4/ip_gre.c:658
  __netdev_start_xmit include/linux/netdevice.h:5400
  netdev_start_xmit include/linux/netdevice.h:5409
  __dev_direct_xmit+0x4b6/0x730 net/core/dev.c:4942
  __xsk_generic_xmit net/xdp/xsk.c:1079
  xsk_generic_xmit+0x277a/0x40a0 net/xdp/xsk.c:1120

Reset the network header so skb_network_offset() is valid (0). For the non-CHECKSUM_PARTIAL case, unlike AF_PACKET, AF_XDP does not need to locate the real L2/L3 boundary: an AF_XDP frame is always a complete, opaque blob from userspace. So resetting the network header is enough to restore the invariant and avoid the crash.

Note this is not sufficient for CHECKSUM_PARTIAL skbs, where drivers derive the L2/L3 split from the header offsets for checksum offload; that will be handled separately after this patch.

Quoted 6 lines
Fixes 3914d88f7608 ("xsk: Respect device's headroom and tailroom on generic xmit path")
Reported-by [email protected]
Quoted 39 lines

---
V4

1. use the one-liner change to fix non csum case at a minimum. As for
csum, since the blamed commit is different, I plan to cook a patch to
fix that[1].
2. revise the commit message accordingly.
3. add James reported-by tag since he spotted this and sent this in
private.
[1]: https://lore.kernel.org/all/CAL+tcoB8f7H1ptcEk1iVH065AcUNPEL2V1gKoJacwr3zndfPWA@mail.gmail.com/

V3

1. remove the misleading description about af_packet

V2

1. add more clarification about why we don't probe the header like packet at the end of the commit message (Jakub)

---
 net/xdp/xsk.c | 2 ++
 1 file changed, 2 insertions(+)
net/xdp/xsk.c +2 -0
index f22f9a2cdbbe..52d538160e0a 100644
@@ -948,6 +948,8 @@ static int xsk_skb_init_misc(struct sk_buff *skb, struct xdp_sock *xs,
 {
 	int err;
 
+	skb_reset_network_header(skb);
+
 	err = xsk_skb_destructor_set_addr(skb, addr);
 	if (unlikely(err))
 		return err;
Signature · 1 line
2.43.7