bpf: Update transport_header when encapsulating UDP tunnel in lwt
Subsystems (5)
v4 cover bpf: Update transport_header when encapsulating UDP tunnel in lwt
Currently, bpf_lwt_push_ip_encap() does not update skb->transport_header. When a driver, e.g. ice, reuses the stale skb->transport_header to offload checksum computation to NIC hardware, VxLAN packets encapsulated by bpf_lwt_push_encap() helper may be dropped due to incorrect checksum.
Update skb->transport_header in bpf_lwt_push_ip_encap() whenever the encapsulated packet uses UDP, so checksum offload works correctly.
Changes:
v3 -> v4:
* Address comments from Emil:
* Make the logic of skb_set_transport_header() clearer in patch #1.
* Fold the code of fexit_lwt_push_ip_encap() into test_lwt_ip_encap.c in
patch #2.
* Resolve assorted issues of test in patch #2.
* v3: https://lore.kernel.org/bpf/[email protected]/
v2 -> v3:
* Drop patch #1 and #2 of v2 that aim to resolve potential issues reported by sashiko (per Alexei).
* Check target IP version and UDP tunnel in test (per sashiko).
* v2: https://lore.kernel.org/bpf/[email protected]/
v1 -> v2:
* Address sashiko's reviews:
* Fix TOCTOU issue in lwt to avoid changing hdr after checks.
* Add check iph->ihl < 5 in lwt to avoid infinite-loop in MIPS driver.
* Update comment style in selftests with BPF comment style.
* v1: https://lore.kernel.org/bpf/[email protected]/
Leon Hwang (2):
bpf: Update transport_header when encapsulating UDP tunnel in lwt selftests/bpf: Add tests to verify the fix of encapsulating VxLAN in
lwt
net/core/lwt_bpf.c | 12 ++ .../selftests/bpf/prog_tests/lwt_ip_encap.c | 145 ++++++++++++++++ .../selftests/bpf/progs/test_lwt_ip_encap.c | 155 ++++++++++++++++-- 3 files changed, 302 insertions(+), 10 deletions(-)
Signature · 1 line
2.54.0
PATCH v4 1/2 bpf: Update transport_header when encapsulating UDP tunnel in lwt
Currently, bpf_lwt_push_ip_encap() does not update skb->transport_header. When a driver, e.g. ice, reuses the stale skb->transport_header to offload checksum computation to NIC hardware, VxLAN packets encapsulated by bpf_lwt_push_encap() helper may be dropped due to incorrect checksum.
Update skb->transport_header in bpf_lwt_push_ip_encap() whenever the encapsulated packet uses UDP, so checksum offload works correctly.
--- net/core/lwt_bpf.c | 12 ++++++++++++ 1 file changed, 12 insertions(+)
net/core/lwt_bpf.c +12 -0
@@ -599,6 +599,7 @@ static int handle_gso_encap(struct sk_buff *skb, bool ipv4, int encap_len)int bpf_lwt_push_ip_encap (struct sk_buff *skb,void *hdr,u32 len,bool ingress) { +bool is_udp_tunnel;struct iphdr *iph;bool ipv4;int err;
@@ -612,10 +613,16 @@ int bpf_lwt_push_ip_encap(struct sk_buff *skb, void *hdr, u32 len, bool ingress) ipv4 = true;if (unlikely (len < iph->ihl *4 ))return -EINVAL ; + is_udp_tunnel = iph->protocol ==IPPROTO_UDP ; +if (unlikely (is_udp_tunnel && len < iph->ihl *4 +sizeof (struct udphdr ))) +return -EINVAL ; }else if (iph->version ==6 ) { ipv4 = false;if (unlikely (len <sizeof (struct ipv6hdr )))return -EINVAL ; + is_udp_tunnel = ((struct ipv6hdr *)iph)->nexthdr ==NEXTHDR_UDP ; +if (unlikely (is_udp_tunnel && len <sizeof (struct ipv6hdr ) +sizeof (struct udphdr ))) +return -EINVAL ; }else {return -EINVAL ; }
@@ -637,6 +644,11 @@ int bpf_lwt_push_ip_encap(struct sk_buff *skb, void *hdr, u32 len, bool ingress)if (ingress)skb_postpush_rcsum (skb, iph, len);skb_reset_network_header (skb); +if (is_udp_tunnel) { +size_t iph_sz = ipv4 ? iph->ihl *4 :sizeof (struct ipv6hdr ); + +skb_set_transport_header (skb,skb_network_offset (skb) + iph_sz); + }memcpy (skb_network_header (skb), hdr, len);bpf_compute_data_pointers (skb);skb_clear_hash (skb);
Signature · 1 line
2.54.0
[email protected]
AI reviewThank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
Pre-existing issues:
- [High] Missing IPv4 header length (ihl) minimum bounds check in bpf_lwt_push_ip_encap allows denial of service via ip_fast_csum.
- [High] Time-of-Check to Time-of-Use (TOCTOU) vulnerability in bpf_lwt_push_ip_encap due to re-reading from eBPF map memory. --
commit 8004c2f9aa8d7cbb7b762ab79a4e3272c20af453
Author: Leon Hwang <[email protected]>
bpf: Update transport_header when encapsulating UDP tunnel in lwt
This commit updates skb->transport_header in bpf_lwt_push_ip_encap() when encapsulating a packet that uses UDP. This ensures that checksum offloading works correctly when a driver reuses the transport header.
net/core/lwt_bpf.c int bpf_lwt_push_ip_encap(struct sk_buff *skb, void *hdr, u32 len, bool ingress) · line 615
Quoted 8 lines
net/core/lwt_bpf.c +0
-0@@ -612,10 +613,16 @@ int bpf_lwt_push_ip_encap(struct sk_buff *skb, void *hdr, u32 len, bool ingress) ipv4 = true;if (unlikely (len < iph->ihl *4 ))return -EINVAL ;
[Severity: High]
This is a pre-existing issue, but does bpf_lwt_push_ip_encap() need to verify that iph->ihl is at least 5?
An eBPF program can pass an IPv4 header with ihl = 0 and len = 20. This bypasses the length validation because 20 < 0 is false. Later in the function, this unvalidated ihl is passed to ip_fast_csum(iph, iph->ihl) with a length of 0. On architectures like MIPS and PowerPC, could this cause the ip_fast_csum implementation to blindly decrement or loop until reaching the end pointer, triggering an infinite loop or kernel panic?
[Severity: High]
This isn't a bug introduced by this patch, but could this validation suffer from a Time-of-Check to Time-of-Use (TOCTOU) race?
The hdr argument can point to mutable eBPF map memory. A concurrent modification by userspace or another CPU could alter iph->ihl between this validation and its later uses. For example, if iph->ihl is mutated to 0 after this check, it bypasses validation and the subsequent memcpy() copies the mutated ihl to the SKB, leading to the same ip_fast_csum() kernel panic described above.
Should the function copy the header into a local stack buffer or the SKB first, and then perform all validations on that immutable copy?
Quoted 4 lines
net/core/lwt_bpf.c +3
-0+ is_udp_tunnel = iph->protocol ==IPPROTO_UDP ; +if (unlikely (is_udp_tunnel && len < iph->ihl *4 +sizeof (struct udphdr ))) +return -EINVAL ; }else if (iph->version ==6 ) {
[ ... ]
net/core/lwt_bpf.c int bpf_lwt_push_ip_encap(struct sk_buff *skb, void *hdr, u32 len, bool ingress) · line 651
Quoted 9 lines
net/core/lwt_bpf.c +5
-0@@ -637,6 +644,11 @@ int bpf_lwt_push_ip_encap(struct sk_buff *skb, void *hdr, u32 len, bool ingress)if (ingress)skb_postpush_rcsum (skb, iph, len);skb_reset_network_header (skb); +if (is_udp_tunnel) { +size_t iph_sz = ipv4 ? iph->ihl *4 :sizeof (struct ipv6hdr ); + +skb_set_transport_header (skb,skb_network_offset (skb) + iph_sz); + }
[Severity: High]
This is a pre-existing issue, but does this newly added code introduce an additional TOCTOU race by re-reading iph->ihl?
Because iph->ihl is read again here from mutable eBPF map memory to calculate iph_sz, a concurrent modification could result in an incorrect size calculation. Could this allow skb_set_transport_header() to be set to a malicious out-of-bounds offset?
Quoted 3 lines
net/core/lwt_bpf.c +0
-0memcpy (skb_network_header (skb), hdr, len);bpf_compute_data_pointers (skb);skb_clear_hash (skb);
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Quoted 7 lines · On 2/6/26 23:23, [email protected] wrote:
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
Pre-existing issues:
- [High] Missing IPv4 header length (ihl) minimum bounds check inbpf_lwt_push_ip_encapallows denial of service viaip_fast_csum.
- [High] Time-of-Check to Time-of-Use (TOCTOU) vulnerability inbpf_lwt_push_ip_encapdue to re-reading from eBPF map memory. --
As Alexei suggested in v2: right now the priority is to fix the issues that normal users can hit and not bots.
So, this series aims to resolve the VxLAN packet-drop issue caused by stale skb->transport_header on ice driver, excluding the issues reported by bots.
Thanks,
Leon
[email protected]
AI reviewCan iph->ihl == 0 slip past this check? Since len is a u32, iph->ihl * 4 is 0 when iph->ihl is 0, and len < 0 is never true, so the buffer is accepted without returning -EINVAL.
net/core/lwt_bpf.c int bpf_lwt_push_ip_encap(struct sk_buff *skb, void *hdr, u32 len, bool ingress) · line 615
Quoted 8 lines
net/core/lwt_bpf.c +0
-0@@ -612,10 +613,16 @@ int bpf_lwt_push_ip_encap(struct sk_buff *skb, void *hdr, u32 len, bool ingress) ipv4 = true;if (unlikely (len < iph->ihl *4 ))return -EINVAL ;
Can iph->ihl == 0 slip past this check? Since len is a u32, iph->ihl * 4 is 0 when iph->ihl is 0, and len < 0 is never true, so the buffer is accepted without returning -EINVAL.
ip_fast_csum() is later called with this iph->ihl, and on some architectures (PowerPC, MIPS) that implementation assumes ihl >= 5 and decrements the loop counter, which can underflow when ihl is 0.
Would an explicit iph->ihl >= 5 guard be needed here, for example:
if (unlikely(iph->ihl < 5 || len < iph->ihl * 4)) return -EINVAL;
This was raised in review of v1 and an earlier revision added the iph->ihl < 5 guard, but the current revision appears to have dropped it again.
Quoted 14 lines
net/core/lwt_bpf.c +0
-0is_udp_tunnel = iph->protocol ==IPPROTO_UDP ;if (unlikely (is_udp_tunnel && len < iph->ihl *4 +sizeof (struct udphdr )))return -EINVAL ; }else if (iph->version ==6 ) { ipv4 = false;if (unlikely (len <sizeof (struct ipv6hdr )))return -EINVAL ; is_udp_tunnel = ((struct ipv6hdr *)iph)->nexthdr ==NEXTHDR_UDP ;if (unlikely (is_udp_tunnel && len <sizeof (struct ipv6hdr ) +sizeof (struct udphdr )))return -EINVAL ; }else {return -EINVAL ; }
net/core/lwt_bpf.c int bpf_lwt_push_ip_encap(struct sk_buff *skb, void *hdr, u32 len, bool ingress) · line 651
Quoted 8 lines
net/core/lwt_bpf.c +5
-0@@ -637,6 +644,11 @@ int bpf_lwt_push_ip_encap(struct sk_buff *skb, void *hdr, u32 len, bool ingress)if (ingress)skb_postpush_rcsum (skb, iph, len);skb_reset_network_header (skb); +if (is_udp_tunnel) { +size_t iph_sz = ipv4 ? iph->ihl *4 :sizeof (struct ipv6hdr ); + +skb_set_transport_header (skb,skb_network_offset (skb) + iph_sz); + }
Can iph->ihl change between the initial length validation and this read?
The hdr pointer, and therefore iph, refers to a buffer provided by the BPF program, which can be concurrently modified by userspace through a shared BPF map. iph->ihl is re-read here to compute iph_sz, and this read happens before the memcpy() of the header into the skb:
if (is_udp_tunnel) {
size_t iph_sz = ipv4 ? iph->ihl * 4 : sizeof(struct ipv6hdr);
skb_set_transport_header(skb, skb_network_offset(skb) + iph_sz);
}
memcpy(skb_network_header(skb), hdr, len);If iph->ihl is concurrently changed to a larger value such as 15 after the initial validation, can the transport header offset be set past skb->tail, leading to out-of-bounds access when drivers later dereference the transport header for checksum updates?
The suggestion in the v1 discussion was to move skb_set_transport_header() after the memcpy() so it reads from the copied-in header, but the current revision still reads iph->ihl before the memcpy().
net/core/lwt_bpf.c int bpf_lwt_push_ip_encap(struct sk_buff *skb, void *hdr, u32 len, bool ingress) · line 654
Quoted 3 lines
net/core/lwt_bpf.c +0
-0memcpy (skb_network_header (skb), hdr, len);bpf_compute_data_pointers (skb);skb_clear_hash (skb);
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/26830439032
PATCH v4 2/2 selftests/bpf: Add tests to verify the fix of encapsulating VxLAN in lwt
In reply to Leon HwangAdd two tests to verify the transport header of skb has been set when encapsulate VxLAN using bpf_lwt_push_encap() helper.
1. VxLAN over IPv4.
2. VxLAN over IPv6.
Without the fix, the tests would fail:
lwt_ip_encap_vxlan:FAIL:transport_hdr offset unexpected transport_hdr offset: actual 70 != expected 20
#208 lwt_ip_encap_vxlan_ipv4:FAIL
lwt_ip_encap_vxlan:FAIL:transport_hdr offset unexpected transport_hdr offset: actual 110 != expected 40
#209 lwt_ip_encap_vxlan_ipv6:FAIL
The unexpected offsets are: outer encap headers
(IPv4: iphdr+udp+vxlan+eth = 50 bytes, IPv6: ipv6hdr+udp+vxlan+eth = 70 bytes) plus the inner IP header (20 or 40 bytes), because without the fix transport_header still points at the inner transport layer instead of the outer UDP header.
--- .../selftests/bpf/prog_tests/lwt_ip_encap.c | 145 ++++++++++++++++ .../selftests/bpf/progs/test_lwt_ip_encap.c | 155 ++++++++++++++++-- 2 files changed, 290 insertions(+), 10 deletions(-)
tools/testing/selftests/bpf/prog_tests/lwt_ip_encap.c +145 -0
@@ -3,6 +3,7 @@#include "network_helpers.h" #include "test_progs.h" +#include "test_lwt_ip_encap.skel.h" #define BPF_FILE "test_lwt_ip_encap.bpf.o"
@@ -32,6 +33,9 @@#define IP6_ADDR_8 "fb08::1"#define IP6_ADDR_GRE "fb10::1" +#define IP4_ADDR_VXLAN "172.16.17.100" +#define IP6_ADDR_VXLAN "fb11::1" +#define IP6_ADDR_SRC IP6_ADDR_1#define IP6_ADDR_DST IP6_ADDR_4
@@ -538,3 +542,144 @@ void test_lwt_ip_encap_ipv4(void)if (test__start_subtest ("ingress" ))lwt_ip_encap (IPV4_ENCAP ,INGRESS ,"" ); } + +/* +* VxLAN Setup/topology: +* +* NS1 (IP*_ADDR_1) NS2 NS3 (IP*_ADDR_4) +* [ping src] +* | top route +* veth1 (LWT encap) <<-- veth2 veth3 <<-- veth4 (ping dst) +* | ^ +* (bottom route) | (inner pkt) +* v bottom route | +* veth5 -->> veth6 veth7 -->> veth8 (vxlan decap) +* (IP*_ADDR_VXLAN) +* +* Add the VxLAN endpoint addresses to NS3's veth8, create standard +* VxLAN decap devices bound to those addresses, and install routes so +* NS1/NS2 can reach the endpoints via the bottom route. NS2 here is to +* make sure the LWT-encap VxLAN packets are routed to NS3 correctly. +*/ +static int setup_vxlan_routes (const char *ns3,const char *ns1,const char *ns2) +{ +struct nstoken *nstoken; + + nstoken =open_netns (ns3); +if (!ASSERT_OK_PTR (nstoken,"open ns3 for vxlan" )) +return -1 ; + +SYS (fail_close,"ip a add %s/32 dev veth8" ,IP4_ADDR_VXLAN ); +SYS (fail_close,"ip -6 a add %s/128 dev veth8" ,IP6_ADDR_VXLAN ); +/* +* Standard VxLAN devices to decap the encapsulated packets. The inner +* Ethernet frame uses a broadcast dst MAC so the IP stack accepts it +* without ARP or FDB configuration. +*/ +SYS (fail_close,"ip link add vxlan4 type vxlan id 1 dstport 4789 local %s dev veth8 nolearning noudpcsum" , +IP4_ADDR_VXLAN ); +SYS (fail_close,"ip link set vxlan4 up" ); +SYS (fail_close,"ip link add vxlan6 type vxlan id 1 dstport 4789 local %s dev veth8 nolearning udp6zerocsumrx" , +IP6_ADDR_VXLAN ); +SYS (fail_close,"ip link set vxlan6 up" ); +close_netns (nstoken); + +SYS (fail,"ip -n %s route add %s/32 dev veth5 via %s" , + ns1,IP4_ADDR_VXLAN ,IP4_ADDR_6 ); +SYS (fail,"ip -n %s route add %s/32 dev veth7 via %s" , + ns2,IP4_ADDR_VXLAN ,IP4_ADDR_8 ); +SYS (fail,"ip -n %s -6 route add %s/128 dev veth5 via %s" , + ns1,IP6_ADDR_VXLAN ,IP6_ADDR_6 ); +SYS (fail,"ip -n %s -6 route add %s/128 dev veth7 via %s" , + ns2,IP6_ADDR_VXLAN ,IP6_ADDR_8 ); +return 0 ; + +fail_close: +close_netns (nstoken); +fail: +return -1 ; +} + +static void lwt_ip_encap_vxlan (bool ipv4_encap) +{ +char ns1[NETNS_NAME_SIZE ] =NETNS_BASE "-1-" ; +char ns2[NETNS_NAME_SIZE ] =NETNS_BASE "-2-" ; +char ns3[NETNS_NAME_SIZE ] =NETNS_BASE "-3-" ; +const char *sec = ipv4_encap ?"encap_vxlan" :"encap_vxlan6" ; +int expected_offset = ipv4_encap ? (int )sizeof (struct iphdr ) + : (int )sizeof (struct ipv6hdr ); +struct test_lwt_ip_encap *skel =NULL ; +int thdr_offset, err; + +if (!ASSERT_OK (create_ns (ns1,NETNS_NAME_SIZE ),"create ns1" )) + goto out; +if (!ASSERT_OK (create_ns (ns2,NETNS_NAME_SIZE ),"create ns2" )) + goto out; +if (!ASSERT_OK (create_ns (ns3,NETNS_NAME_SIZE ),"create ns3" )) + goto out; + +if (!ASSERT_OK (setup_network (ns1, ns2, ns3,"" ),"setup network" )) + goto out; + +if (!ASSERT_OK (setup_vxlan_routes (ns3, ns1, ns2),"setup vxlan routes" )) + goto out; + + skel =test_lwt_ip_encap__open (); +if (!ASSERT_OK_PTR (skel,"test_lwt_ip_encap__open" )) + goto out; + +bpf_program__set_autoload (skel->progs.bpf_lwt_encap_gre, false); +bpf_program__set_autoload (skel->progs.bpf_lwt_encap_gre6, false); +bpf_program__set_autoload (skel->progs.bpf_lwt_encap_vxlan, false); +bpf_program__set_autoload (skel->progs.bpf_lwt_encap_vxlan6, false); +bpf_program__set_autoload (skel->progs.fexit_lwt_push_ip_encap, true); + skel->rodata->tgt_ip_version = ipv4_encap ?4 :6 ; + + err =test_lwt_ip_encap__load (skel); +if (!ASSERT_OK (err,"test_lwt_ip_encap__load" )) + goto out; + + err =test_lwt_ip_encap__attach (skel); +if (!ASSERT_OK (err,"test_lwt_ip_encap__attach" )) + goto out; + +/* Remove the direct NS2->DST route so packets must go via LWT encap. */ +SYS (out,"ip -n %s route del %s/32 dev veth3" , ns2,IP4_ADDR_DST ); +SYS (out,"ip -n %s -6 route del %s/128 dev veth3" , ns2,IP6_ADDR_DST ); + +if (ipv4_encap) +SYS (out,"ip -n %s route add %s encap bpf xmit obj %s sec %s dev veth1" , + ns1,IP4_ADDR_DST ,BPF_FILE , sec); +else +SYS (out,"ip -n %s -6 route add %s encap bpf xmit obj %s sec %s dev veth1" , + ns1,IP6_ADDR_DST ,BPF_FILE , sec); + + skel->bss->fexit_triggered = false; + +if (ipv4_encap) +SYS (out,"ip netns exec %s ping -c 1 -W1 %s" , ns1,IP4_ADDR_DST ); +else +SYS (out,"ip netns exec %s ping6 -c 1 -W1 %s" , ns1,IP6_ADDR_DST ); + +if (!ASSERT_TRUE (skel->bss->fexit_triggered,"fexit_triggered" )) + goto out; + + thdr_offset = (int )skel->bss->transport_hdr - (int )skel->bss->network_hdr; +ASSERT_EQ (thdr_offset, expected_offset,"transport_hdr offset" ); + +out: +test_lwt_ip_encap__destroy (skel); +SYS_NOFAIL ("ip netns del %s" , ns1); +SYS_NOFAIL ("ip netns del %s" , ns2); +SYS_NOFAIL ("ip netns del %s" , ns3); +} + +void test_lwt_ip_encap_vxlan_ipv4 (void ) +{ +lwt_ip_encap_vxlan (IPV4_ENCAP ); +} + +void test_lwt_ip_encap_vxlan_ipv6 (void ) +{ +lwt_ip_encap_vxlan (IPV6_ENCAP ); +}
tools/testing/selftests/bpf/progs/test_lwt_ip_encap.c +145 -10
173 lines not on this page · raw message
Signature · 1 line
2.54.0
This series was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:
Hello:
This series was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:
Quoted 9 lines · On Tue, 2 Jun 2026 23:09:29 +0800 you wrote:
Currently,
bpf_lwt_push_ip_encap()does not updateskb->transport_header. When a driver, e.g. ice, reuses the staleskb->transport_headerto offload checksum computation to NIC hardware, VxLAN packets encapsulated bybpf_lwt_push_encap()helper may be dropped due to incorrect checksum.Update
skb->transport_headerinbpf_lwt_push_ip_encap()whenever the encapsulated packet uses UDP, so checksum offload works correctly.[...]
Here is the summary with links:
- [bpf,v4,1/2] bpf: Update transport_header when encapsulating UDP tunnel in lwt
https://git.kernel.org/bpf/bpf-next/c/82d7d0adbc67 - [bpf,v4,2/2] selftests/bpf: Add tests to verify the fix of encapsulating VxLAN in lwt https://git.kernel.org/bpf/bpf-next/c/5477d55f351f
You are awesome, thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html