Skip to contentSkip to conversation

bpf: Update transport_header when encapsulating UDP tunnel in lwt

PATCH Superseded Leon Hwang 6 participants 13 messages Jun 1 – Jun 2

Subsystems (5)
Networking Network drivers Kselftest BPF +1 more subsystems

v3 of 4 Superseded Latest is v4 →

lore
Share & tools
lore

Leon Hwang

author
#0 lore raw

v3 cover bpf: Update transport_header when encapsulating UDP tunnel in lwt

Currently, bpf_lwt_push_ip_encap() does not update skb->transport_header. When a driver, e.g. ice, reuses the stale skb->transport_header to offload checksum computation to NIC hardware, VxLAN packets encapsulated by bpf_lwt_push_encap() helper may be dropped due to incorrect checksum.

Update skb->transport_header in bpf_lwt_push_ip_encap() whenever the encapsulated packet uses UDP, so checksum offload works correctly.

Changes:
v2 -> v3:
* Drop patch #1 and #2 of v2 that aim to resolve potential issues reported by sashiko (per Alexei).
* Check target IP version and UDP tunnel in test (per sashiko).
* v2: https://lore.kernel.org/bpf/[email protected]/

v1 -> v2:
* Address sashiko's reviews:
* Fix TOCTOU issue in lwt to avoid changing hdr after checks.
* Add check iph->ihl < 5 in lwt to avoid infinite-loop in MIPS driver.
* Update comment style in selftests with BPF comment style.
* v1: https://lore.kernel.org/bpf/[email protected]/

Leon Hwang (2):
bpf: Update transport_header when encapsulating UDP tunnel in lwt selftests/bpf: Add tests to verify the fix of encapsulating VxLAN in
lwt

 net/core/lwt_bpf.c                            |  11 ++
 .../selftests/bpf/prog_tests/lwt_ip_encap.c   | 164 ++++++++++++++++++
 .../selftests/bpf/progs/test_lwt_ip_encap.c   | 112 ++++++++++++
 .../bpf/progs/test_lwt_ip_encap_fix.c         |  44 +++++
 4 files changed, 331 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/progs/test_lwt_ip_encap_fix.c
Signature · 1 line
2.54.0
author
#1 lore raw

PATCH v3 1/2 bpf: Update transport_header when encapsulating UDP tunnel in lwt

Currently, bpf_lwt_push_ip_encap() does not update skb->transport_header. When a driver, e.g. ice, reuses the stale skb->transport_header to offload checksum computation to NIC hardware, VxLAN packets encapsulated by bpf_lwt_push_encap() helper may be dropped due to incorrect checksum.

Update skb->transport_header in bpf_lwt_push_ip_encap() whenever the encapsulated packet uses UDP, so checksum offload works correctly.

Fixes 52f278774e79 ("bpf: implement BPF_LWT_ENCAP_IP mode in bpf_lwt_push_encap")
Cc Leon Hwang <[email protected]>
Signed-off-by Leon Hwang <[email protected]>
---
 net/core/lwt_bpf.c | 11 +++++++++++
 1 file changed, 11 insertions(+)
net/core/lwt_bpf.c +11 -0
index f71ef82a5f3d..65d1dfbf3312 100644
@@ -599,6 +599,7 @@ static int handle_gso_encap(struct sk_buff *skb, bool ipv4, int encap_len)
 
 int bpf_lwt_push_ip_encap(struct sk_buff *skb, void *hdr, u32 len, bool ingress)
 {
+	bool is_udp_tunnel;
 	struct iphdr *iph;
 	bool ipv4;
 	int err;

@@ -612,10 +613,16 @@ int bpf_lwt_push_ip_encap(struct sk_buff *skb, void *hdr, u32 len, bool ingress)
 		ipv4 = true;
 		if (unlikely(len < iph->ihl * 4))
 			return -EINVAL;
+		is_udp_tunnel = iph->protocol == IPPROTO_UDP;
+		if (unlikely(is_udp_tunnel && len < iph->ihl * 4 + sizeof(struct udphdr)))
+			return -EINVAL;
 	} else if (iph->version == 6) {
 		ipv4 = false;
 		if (unlikely(len < sizeof(struct ipv6hdr)))
 			return -EINVAL;
+		is_udp_tunnel = ((struct ipv6hdr *)iph)->nexthdr == NEXTHDR_UDP;
+		if (unlikely(is_udp_tunnel && len < sizeof(struct ipv6hdr) + sizeof(struct udphdr)))
+			return -EINVAL;
 	} else {
 		return -EINVAL;
 	}

@@ -637,6 +644,10 @@ int bpf_lwt_push_ip_encap(struct sk_buff *skb, void *hdr, u32 len, bool ingress)
 	if (ingress)
 		skb_postpush_rcsum(skb, iph, len);
 	skb_reset_network_header(skb);
+	if (ipv4 && is_udp_tunnel)
+		skb_set_transport_header(skb, skb_network_offset(skb) + iph->ihl * 4);
+	else if (!ipv4 && is_udp_tunnel)
+		skb_set_transport_header(skb, skb_network_offset(skb) + sizeof(struct ipv6hdr));
 	memcpy(skb_network_header(skb), hdr, len);
 	bpf_compute_data_pointers(skb);
 	skb_clear_hash(skb);
Signature · 1 line
2.54.0
AI review
#2 lore raw
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:

New issues:
- [High] Time-of-Check to Time-of-Use (TOCTOU) vulnerability in transport header offset calculation leads to out-of-bounds write.

Pre-existing issues:
- [Critical] Missing validation of iph->ihl >= 5 allows unprivileged BPF programs to trigger a kernel panic in ip_fast_csum. --

commit 0be2d8b388e17b2000afd2e38ff89f5f956084ae
Author: Leon Hwang <[email protected]>

bpf: Update transport_header when encapsulating UDP tunnel in lwt

This commit updates bpf_lwt_push_ip_encap() to set the skb transport header whenever the encapsulated packet uses UDP. This ensures that drivers relying on the transport header for hardware checksum offload do not drop packets due to incorrect checksums.

Cc Leon Hwang <[email protected]>

net/core/lwt_bpf.c int bpf_lwt_push_ip_encap(struct sk_buff *skb, void *hdr, u32 len, bool ingress) · line 615

Quoted 8 lines
net/core/lwt_bpf.c +0 -0
index f71ef82a5f3d3..65d1dfbf3312b 100644
@@ -612,10 +613,16 @@ int bpf_lwt_push_ip_encap(struct sk_buff *skb, void *hdr, u32 len, bool ingress)
 		ipv4 = true;
 		if (unlikely(len < iph->ihl * 4))
 			return -EINVAL;

[Severity: Critical]
This is a pre-existing issue, but does this code properly validate the minimum IPv4 header length?

Because len is an unsigned u32, if a BPF program provides a buffer where iph->ihl is 0, iph->ihl * 4 will be 0. Since len < 0 is never true, the check trivially passes without returning -EINVAL.

Later in the function, ip_fast_csum() is called with iph->ihl. On some architectures (like PowerPC and MIPS), the ip_fast_csum() implementation assumes ihl >= 5 and subtracts from it, causing an integer underflow in the loop counter that leads to a deterministic kernel panic.

Should there be an explicit check that enforces iph->ihl >= 5 before the length calculation?

Quoted 4 lines
net/core/lwt_bpf.c +3 -0
+		is_udp_tunnel = iph->protocol == IPPROTO_UDP;
+		if (unlikely(is_udp_tunnel && len < iph->ihl * 4 + sizeof(struct udphdr)))
+			return -EINVAL;
 	} else if (iph->version == 6) {

[ ... ]

net/core/lwt_bpf.c int bpf_lwt_push_ip_encap(struct sk_buff *skb, void *hdr, u32 len, bool ingress) · line 648

Quoted 6 lines
net/core/lwt_bpf.c +2 -0
@@ -637,6 +644,10 @@ int bpf_lwt_push_ip_encap(struct sk_buff *skb, void *hdr, u32 len, bool ingress)
 	if (ingress)
 		skb_postpush_rcsum(skb, iph, len);
 	skb_reset_network_header(skb);
+	if (ipv4 && is_udp_tunnel)
+		skb_set_transport_header(skb, skb_network_offset(skb) + iph->ihl * 4);

[Severity: High]
Can iph->ihl change between the initial length validation and this call?

The hdr pointer (and thus iph) points to a buffer provided by the BPF program, which might be concurrently modified by userspace via a shared BPF map. Because skb_cow_head() acts as a compiler barrier, the compiler will reload iph->ihl from this untrusted memory here.

If it is concurrently changed to a larger value (like 15) after the initial validation, the transport header offset could be set beyond the end of the SKB data (skb->tail).

Could this lead to out-of-bounds memory writes when network drivers later dereference this transport header to update checksums?

Quoted 3 lines
net/core/lwt_bpf.c +2 -0
+	else if (!ipv4 && is_udp_tunnel)
+		skb_set_transport_header(skb, skb_network_offset(skb) + sizeof(struct ipv6hdr));
 	memcpy(skb_network_header(skb), hdr, len);
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
#3 lore raw
In reply to Leon Hwang
Quoted 46 lines · Leon Hwang

Currently, bpf_lwt_push_ip_encap() does not update skb->transport_header. When a driver, e.g. ice, reuses the stale skb->transport_header to offload checksum computation to NIC hardware, VxLAN packets encapsulated by bpf_lwt_push_encap() helper may be dropped due to incorrect checksum.

Update skb->transport_header in bpf_lwt_push_ip_encap() whenever the encapsulated packet uses UDP, so checksum offload works correctly.

Fixes 52f278774e79 ("bpf: implement BPF_LWT_ENCAP_IP mode in bpf_lwt_push_encap")
Cc Leon Hwang <[email protected]>
Signed-off-by Leon Hwang <[email protected]>
---
 net/core/lwt_bpf.c | 11 +++++++++++
 1 file changed, 11 insertions(+)
net/core/lwt_bpf.c +7 -0
index f71ef82a5f3d..65d1dfbf3312 100644
@@ -599,6 +599,7 @@ static int handle_gso_encap(struct sk_buff *skb, bool ipv4, int encap_len)
 
 int bpf_lwt_push_ip_encap(struct sk_buff *skb, void *hdr, u32 len, bool ingress)
 {
+       bool is_udp_tunnel;
        struct iphdr *iph;
        bool ipv4;
        int err;

@@ -612,10 +613,16 @@ int bpf_lwt_push_ip_encap(struct sk_buff *skb, void *hdr, u32 len, bool ingress)
                ipv4 = true;
                if (unlikely(len < iph->ihl * 4))
                        return -EINVAL;
+               is_udp_tunnel = iph->protocol == IPPROTO_UDP;
+               if (unlikely(is_udp_tunnel && len < iph->ihl * 4 + sizeof(struct udphdr)))
+                       return -EINVAL;
        } else if (iph->version == 6) {
                ipv4 = false;
                if (unlikely(len < sizeof(struct ipv6hdr)))
                        return -EINVAL;
+               is_udp_tunnel = ((struct ipv6hdr *)iph)->nexthdr == NEXTHDR_UDP;
+               if (unlikely(is_udp_tunnel && len < sizeof(struct ipv6hdr) + sizeof(struct udphdr)))
+                       return -EINVAL;
        } else {
                return -EINVAL;
        }

net/core/lwt_bpf.c int bpf_lwt_push_ip_encap(struct sk_buff *skb, void *hdr, u32 len, bool ingress) · line 646

Quoted 3 lines
net/core/lwt_bpf.c +0 -0
@@ -637,6 +644,10 @@ int bpf_lwt_push_ip_encap(struct sk_buff *skb, void *hdr, u32 len, bool ingress)
        if (ingress)
                skb_postpush_rcsum(skb, iph, len);
        skb_reset_network_header(skb);

iph might point to freed memory after skb_cow_head(), leading to UAF in iph->ihl

This issue is already there in the skb_postpush_rcsum(skb, iph, len) call above.

Quoted 10 lines
net/core/lwt_bpf.c +4 -1
+       if (ipv4 && is_udp_tunnel)
+               skb_set_transport_header(skb, skb_network_offset(skb) + iph->ihl * 4);
+       else if (!ipv4 && is_udp_tunnel)
+               skb_set_transport_header(skb, skb_network_offset(skb) + sizeof(struct ipv6hdr));
        memcpy(skb_network_header(skb), hdr, len);
        bpf_compute_data_pointers(skb);
        skb_clear_hash(skb);
--

2.54.0

#4 lore raw
Quoted 54 lines · Eric Dumazet
Quoted 46 lines · Leon Hwang

Currently, bpf_lwt_push_ip_encap() does not update skb->transport_header. When a driver, e.g. ice, reuses the stale skb->transport_header to offload checksum computation to NIC hardware, VxLAN packets encapsulated by bpf_lwt_push_encap() helper may be dropped due to incorrect checksum.

Update skb->transport_header in bpf_lwt_push_ip_encap() whenever the encapsulated packet uses UDP, so checksum offload works correctly.

Fixes 52f278774e79 ("bpf: implement BPF_LWT_ENCAP_IP mode in bpf_lwt_push_encap")
Cc Leon Hwang <[email protected]>
Signed-off-by Leon Hwang <[email protected]>
---
 net/core/lwt_bpf.c | 11 +++++++++++
 1 file changed, 11 insertions(+)
net/core/lwt_bpf.c +7 -0
index f71ef82a5f3d..65d1dfbf3312 100644
@@ -599,6 +599,7 @@ static int handle_gso_encap(struct sk_buff *skb, bool ipv4, int encap_len)
 
 int bpf_lwt_push_ip_encap(struct sk_buff *skb, void *hdr, u32 len, bool ingress)
 {
+       bool is_udp_tunnel;
        struct iphdr *iph;
        bool ipv4;
        int err;

@@ -612,10 +613,16 @@ int bpf_lwt_push_ip_encap(struct sk_buff *skb, void *hdr, u32 len, bool ingress)
                ipv4 = true;
                if (unlikely(len < iph->ihl * 4))
                        return -EINVAL;
+               is_udp_tunnel = iph->protocol == IPPROTO_UDP;
+               if (unlikely(is_udp_tunnel && len < iph->ihl * 4 + sizeof(struct udphdr)))
+                       return -EINVAL;
        } else if (iph->version == 6) {
                ipv4 = false;
                if (unlikely(len < sizeof(struct ipv6hdr)))
                        return -EINVAL;
+               is_udp_tunnel = ((struct ipv6hdr *)iph)->nexthdr == NEXTHDR_UDP;
+               if (unlikely(is_udp_tunnel && len < sizeof(struct ipv6hdr) + sizeof(struct udphdr)))
+                       return -EINVAL;
        } else {
                return -EINVAL;
        }

net/core/lwt_bpf.c int bpf_lwt_push_ip_encap(struct sk_buff *skb, void *hdr, u32 len, bool ingress) · line 646

Quoted 3 lines
net/core/lwt_bpf.c +0 -0
@@ -637,6 +644,10 @@ int bpf_lwt_push_ip_encap(struct sk_buff *skb, void *hdr, u32 len, bool ingress)
        if (ingress)
                skb_postpush_rcsum(skb, iph, len);
        skb_reset_network_header(skb);

iph might point to freed memory after skb_cow_head(), leading to UAF in iph->ihl

Quoted 1 line

This issue is already there in the skb_postpush_rcsum(skb, iph, len) call above.

Ignore this, I missed the context about iph, which does not point to skb->head

AI review
#5 lore raw
In reply to Leon Hwang
Since len is a u32, if iph->ihl is 0 then iph->ihl * 4 is 0 and the "len < iph->ihl * 4" check is never true, so it passes without returning -EINVAL.

net/core/lwt_bpf.c int bpf_lwt_push_ip_encap(struct sk_buff *skb, void *hdr, u32 len, bool ingress) · line 615

Quoted 8 lines
net/core/lwt_bpf.c +0 -0
index f71ef82a5f3d..65d1dfbf3312 100644
@@ -612,10 +613,16 @@ int bpf_lwt_push_ip_encap(struct sk_buff *skb, void *hdr, u32 len, bool ingress)
 		ipv4 = true;
 		if (unlikely(len < iph->ihl * 4))
 			return -EINVAL;

Since len is a u32, if iph->ihl is 0 then iph->ihl * 4 is 0 and the "len < iph->ihl * 4" check is never true, so it passes without returning -EINVAL.

Later ip_fast_csum() is called with iph->ihl, and on some architectures (PowerPC, MIPS) that implementation assumes ihl >= 5 and decrements the loop counter, which underflows when ihl is 0.

Should there be an explicit iph->ihl >= 5 check here, for example:

	if (unlikely(iph->ihl < 5 || len < iph->ihl * 4))
		return -EINVAL;

This was raised in review of v1 and v2 added the guard, but v3 appears to have dropped it again. The earlier discussion is here:

https://lore.kernel.org/bpf/[email protected]

Quoted 13 lines
net/core/lwt_bpf.c +6 -0
+		is_udp_tunnel = iph->protocol == IPPROTO_UDP;
+		if (unlikely(is_udp_tunnel && len < iph->ihl * 4 + sizeof(struct udphdr)))
+			return -EINVAL;
 	} else if (iph->version == 6) {
 		ipv4 = false;
 		if (unlikely(len < sizeof(struct ipv6hdr)))
 			return -EINVAL;
+		is_udp_tunnel = ((struct ipv6hdr *)iph)->nexthdr == NEXTHDR_UDP;
+		if (unlikely(is_udp_tunnel && len < sizeof(struct ipv6hdr) + sizeof(struct udphdr)))
+			return -EINVAL;
 	} else {
 		return -EINVAL;
 	}

[ ... ]

net/core/lwt_bpf.c int bpf_lwt_push_ip_encap(struct sk_buff *skb, void *hdr, u32 len, bool ingress) · line 651

Quoted 9 lines
net/core/lwt_bpf.c +4 -0
@@ -637,6 +644,10 @@ int bpf_lwt_push_ip_encap(struct sk_buff *skb, void *hdr, u32 len, bool ingress)
 	if (ingress)
 		skb_postpush_rcsum(skb, iph, len);
 	skb_reset_network_header(skb);
+	if (ipv4 && is_udp_tunnel)
+		skb_set_transport_header(skb, skb_network_offset(skb) + iph->ihl * 4);
+	else if (!ipv4 && is_udp_tunnel)
+		skb_set_transport_header(skb, skb_network_offset(skb) + sizeof(struct ipv6hdr));
 	memcpy(skb_network_header(skb), hdr, len);

Can iph->ihl change between the initial length validation and this call?

The hdr pointer (and therefore iph) refers to a buffer provided by the BPF program, which can be concurrently modified by userspace through a shared BPF map. Because skb_cow_head() acts as a compiler barrier, the compiler reloads iph->ihl from that untrusted memory here.

This read also happens before the memcpy() into the skb:

	skb_set_transport_header(skb, skb_network_offset(skb) + iph->ihl * 4);
	memcpy(skb_network_header(skb), hdr, len);

If iph->ihl is concurrently changed to a larger value (such as 15) after the initial validation, can the transport header offset be set beyond skb->tail, leading to out-of-bounds writes when drivers later dereference the transport header for checksum updates?

In the v1 discussion the suggestion was to move skb_set_transport_header() after the memcpy() so it reads from the copied-in header, but the current code still reads iph->ihl before the memcpy(). See:

https://lore.kernel.org/bpf/[email protected]

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/26763883406

reviewer
#6 lore raw
In reply to Leon Hwang
Quoted 45 lines · Leon Hwang

Currently, bpf_lwt_push_ip_encap() does not update skb->transport_header. When a driver, e.g. ice, reuses the stale skb->transport_header to offload checksum computation to NIC hardware, VxLAN packets encapsulated by bpf_lwt_push_encap() helper may be dropped due to incorrect checksum.

Update skb->transport_header in bpf_lwt_push_ip_encap() whenever the encapsulated packet uses UDP, so checksum offload works correctly.

Fixes 52f278774e79 ("bpf: implement BPF_LWT_ENCAP_IP mode in bpf_lwt_push_encap")
Cc Leon Hwang <[email protected]>
Signed-off-by Leon Hwang <[email protected]>
---
 net/core/lwt_bpf.c | 11 +++++++++++
 1 file changed, 11 insertions(+)
net/core/lwt_bpf.c +7 -0
index f71ef82a5f3d..65d1dfbf3312 100644
@@ -599,6 +599,7 @@ static int handle_gso_encap(struct sk_buff *skb, bool ipv4, int encap_len)
 
 int bpf_lwt_push_ip_encap(struct sk_buff *skb, void *hdr, u32 len, bool ingress)
 {
+	bool is_udp_tunnel;
 	struct iphdr *iph;
 	bool ipv4;
 	int err;

@@ -612,10 +613,16 @@ int bpf_lwt_push_ip_encap(struct sk_buff *skb, void *hdr, u32 len, bool ingress)
 		ipv4 = true;
 		if (unlikely(len < iph->ihl * 4))
 			return -EINVAL;
+		is_udp_tunnel = iph->protocol == IPPROTO_UDP;
+		if (unlikely(is_udp_tunnel && len < iph->ihl * 4 + sizeof(struct udphdr)))
+			return -EINVAL;
 	} else if (iph->version == 6) {
 		ipv4 = false;
 		if (unlikely(len < sizeof(struct ipv6hdr)))
 			return -EINVAL;
+		is_udp_tunnel = ((struct ipv6hdr *)iph)->nexthdr == NEXTHDR_UDP;
+		if (unlikely(is_udp_tunnel && len < sizeof(struct ipv6hdr) + sizeof(struct udphdr)))
+			return -EINVAL;
 	} else {
 		return -EINVAL;
 	}

net/core/lwt_bpf.c int bpf_lwt_push_ip_encap(struct sk_buff *skb, void *hdr, u32 len, bool ingress) · line 650

Quoted 7 lines
net/core/lwt_bpf.c +4 -0
@@ -637,6 +644,10 @@ int bpf_lwt_push_ip_encap(struct sk_buff *skb, void *hdr, u32 len, bool ingress)
 	if (ingress)
 		skb_postpush_rcsum(skb, iph, len);
 	skb_reset_network_header(skb);
+	if (ipv4 && is_udp_tunnel)
+		skb_set_transport_header(skb, skb_network_offset(skb) + iph->ihl * 4);
+	else if (!ipv4 && is_udp_tunnel)
+		skb_set_transport_header(skb, skb_network_offset(skb) + sizeof(struct ipv6hdr));

Minor nit: Could we do something like

if (is_udp_tunnel) {
	size_t hrdsz = ipv4 ? iph->ihl * 4 : sizeof(struct ipv6hdr);
	skb_set_transport_header(skb9, skb_network_offset(skb)) + hdrsz;
}

to make the logic clearer?

Quoted 3 lines
net/core/lwt_bpf.c +0 -0
 	memcpy(skb_network_header(skb), hdr, len);
 	bpf_compute_data_pointers(skb);
 	skb_clear_hash(skb);
author
#7 lore raw
Quoted 1 line · Emil Tsalapatis

On Mon Jun 1, 2026 at 11:02 AM EDT, Leon Hwang wrote:

[...]

Quoted 11 lines

net/core/lwt_bpf.c int bpf_lwt_push_ip_encap(struct sk_buff *skb, void *hdr, u32 len, bool ingress) · line 650

Quoted 8 lines
net/core/lwt_bpf.c +4 -0
@@ -637,6 +644,10 @@ int bpf_lwt_push_ip_encap(struct sk_buff *skb, void *hdr, u32 len, bool ingress)
 	if (ingress)
 		skb_postpush_rcsum(skb, iph, len);
 	skb_reset_network_header(skb);
+	if (ipv4 && is_udp_tunnel)
+		skb_set_transport_header(skb, skb_network_offset(skb) + iph->ihl * 4);
+	else if (!ipv4 && is_udp_tunnel)
+		skb_set_transport_header(skb, skb_network_offset(skb) + sizeof(struct ipv6hdr));

Minor nit: Could we do something like

Quoted 7 lines
if (is_udp_tunnel) {
	size_t hrdsz = ipv4 ? iph->ihl * 4 : sizeof(struct ipv6hdr);
	skb_set_transport_header(skb9, skb_network_offset(skb)) + hdrsz;
}

to make the logic clearer?

Make sense.

Thanks,
Leon

Quoted 4 lines
Quoted 3 lines
net/core/lwt_bpf.c +0 -0
 	memcpy(skb_network_header(skb), hdr, len);
 	bpf_compute_data_pointers(skb);
 	skb_clear_hash(skb);
#8 lore raw
In reply to Leon Hwang
Quoted 45 lines · Leon Hwang

Currently, bpf_lwt_push_ip_encap() does not update skb->transport_header. When a driver, e.g. ice, reuses the stale skb->transport_header to offload checksum computation to NIC hardware, VxLAN packets encapsulated by bpf_lwt_push_encap() helper may be dropped due to incorrect checksum.

Update skb->transport_header in bpf_lwt_push_ip_encap() whenever the encapsulated packet uses UDP, so checksum offload works correctly.

Fixes 52f278774e79 ("bpf: implement BPF_LWT_ENCAP_IP mode in bpf_lwt_push_encap")
Cc Leon Hwang <[email protected]>
Signed-off-by Leon Hwang <[email protected]>
---
  net/core/lwt_bpf.c | 11 +++++++++++
  1 file changed, 11 insertions(+)
net/core/lwt_bpf.c +7 -0
index f71ef82a5f3d..65d1dfbf3312 100644
@@ -599,6 +599,7 @@ static int handle_gso_encap(struct sk_buff *skb, bool ipv4, int encap_len)
 
  int bpf_lwt_push_ip_encap(struct sk_buff *skb, void *hdr, u32 len, bool ingress)
  {
+	bool is_udp_tunnel;
  	struct iphdr *iph;
  	bool ipv4;
  	int err;

@@ -612,10 +613,16 @@ int bpf_lwt_push_ip_encap(struct sk_buff *skb, void *hdr, u32 len, bool ingress)
  		ipv4 = true;
  		if (unlikely(len < iph->ihl * 4))
  			return -EINVAL;
+		is_udp_tunnel = iph->protocol == IPPROTO_UDP;
+		if (unlikely(is_udp_tunnel && len < iph->ihl * 4 + sizeof(struct udphdr)))
+			return -EINVAL;
  	} else if (iph->version == 6) {
  		ipv4 = false;
  		if (unlikely(len < sizeof(struct ipv6hdr)))
  			return -EINVAL;
+		is_udp_tunnel = ((struct ipv6hdr *)iph)->nexthdr == NEXTHDR_UDP;
+		if (unlikely(is_udp_tunnel && len < sizeof(struct ipv6hdr) + sizeof(struct udphdr)))
+			return -EINVAL;
  	} else {
  		return -EINVAL;
  	}

net/core/lwt_bpf.c int bpf_lwt_push_ip_encap(struct sk_buff *skb, void *hdr, u32 len, bool ingress) · line 650

Quoted 7 lines
net/core/lwt_bpf.c +4 -0
@@ -637,6 +644,10 @@ int bpf_lwt_push_ip_encap(struct sk_buff *skb, void *hdr, u32 len, bool ingress)
  	if (ingress)
  		skb_postpush_rcsum(skb, iph, len);
  	skb_reset_network_header(skb);
+	if (ipv4 && is_udp_tunnel)
+		skb_set_transport_header(skb, skb_network_offset(skb) + iph->ihl * 4);
+	else if (!ipv4 && is_udp_tunnel)
+		skb_set_transport_header(skb, skb_network_offset(skb) + sizeof(struct ipv6hdr));

I think GRE is also affected, why not unconditionally set transport header to network_offset + outer_ip_ihl regardless of outer protocol??

Quoted 3 lines
net/core/lwt_bpf.c +0 -0
  	memcpy(skb_network_header(skb), hdr, len);
  	bpf_compute_data_pointers(skb);
  	skb_clear_hash(skb);
author
#9 lore raw
Quoted 2 lines · Jiayuan Chen

On 6/1/26 11:02 PM, Leon Hwang wrote:

[...]

Quoted 7 lines
Quoted 7 lines
net/core/lwt_bpf.c +2 -0
@@ -637,6 +644,10 @@ int bpf_lwt_push_ip_encap(struct sk_buff *skb,
 void *hdr, u32 len, bool ingress)
       if (ingress)
           skb_postpush_rcsum(skb, iph, len);
       skb_reset_network_header(skb);
+    if (ipv4 && is_udp_tunnel)
+        skb_set_transport_header(skb, skb_network_offset(skb) + iph-
Quoted 9 lines

Review int bpf_lwt_push_ip_encap(struct sk_buff *skb, · line 653

Quoted 4 lines
(quoted diff) +2 -0
 >ihl * 4);
+    else if (!ipv4 && is_udp_tunnel)
+        skb_set_transport_header(skb, skb_network_offset(skb) +
 sizeof(struct ipv6hdr));

I think GRE is also affected, why not unconditionally set transport header to network_offset + outer_ip_ihl regardless of outer protocol??

Probably, yes. But, I'm not sure about this.

At Shopee, we encountered the issue that the VxLAN packets encapsulated with lwt-bpf were dropped on ice driver. And, it is to be fixed by this patch.

IOW, if someone encounter the same issue for GRE packets, he can fix the issue by the same way.

Thanks,
Leon

Quoted 3 lines
Quoted 3 lines

      memcpy(skb_network_header(skb), hdr, len);
      bpf_compute_data_pointers(skb);
      skb_clear_hash(skb);

author
#10 lore raw

PATCH v3 2/2 selftests/bpf: Add tests to verify the fix of encapsulating VxLAN in lwt

In reply to Leon Hwang

Add two tests to verify the transport header of skb has been set when encapsulate VxLAN using bpf_lwt_push_encap() helper.

1. VxLAN over IPv4.
2. VxLAN over IPv6.

Without the fix, the tests would fail:

lwt_ip_encap_vxlan:FAIL:transport_hdr offset unexpected transport_hdr offset: actual 70 != expected 20
#208 lwt_ip_encap_vxlan_ipv4:FAIL
lwt_ip_encap_vxlan:FAIL:transport_hdr offset unexpected transport_hdr offset: actual 110 != expected 40
#209 lwt_ip_encap_vxlan_ipv6:FAIL

Assisted-by Claude:claude-sonnet-4-6
Cc Leon Hwang <[email protected]>
Signed-off-by Leon Hwang <[email protected]>
---
 .../selftests/bpf/prog_tests/lwt_ip_encap.c   | 164 ++++++++++++++++++
 .../selftests/bpf/progs/test_lwt_ip_encap.c   | 112 ++++++++++++
 .../bpf/progs/test_lwt_ip_encap_fix.c         |  44 +++++
 3 files changed, 320 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/progs/test_lwt_ip_encap_fix.c
tools/testing/selftests/bpf/prog_tests/lwt_ip_encap.c +164 -0
index b6391af5f6f9..50104d847fde 100644
@@ -1,8 +1,11 @@
 // SPDX-License-Identifier: GPL-2.0-only
+#include <linux/ip.h>
+#include <linux/ipv6.h>
 #include <netinet/in.h>
 
 #include "network_helpers.h"
 #include "test_progs.h"
+#include "test_lwt_ip_encap_fix.skel.h"
 
 #define BPF_FILE "test_lwt_ip_encap.bpf.o"
 

@@ -35,6 +38,10 @@
 #define IP6_ADDR_SRC IP6_ADDR_1
 #define IP6_ADDR_DST IP6_ADDR_4
 
+/* VxLAN tunnel endpoints, reachable via the bottom route (veth5/6/7/8). */
+#define IP4_ADDR_VXLAN  "172.16.17.100"
+#define IP6_ADDR_VXLAN  "fb20::1"
+
 /* Setup/topology:
  *
  *    NS1             NS2             NS3

@@ -538,3 +545,160 @@ void test_lwt_ip_encap_ipv4(void)
 	if (test__start_subtest("ingress"))
 		lwt_ip_encap(IPV4_ENCAP, INGRESS, "");
 }
+
+/*
+ * VxLAN Setup/topology:
+ *
+ * NS1 (IP*_ADDR_1)                NS2                  NS3 (IP*_ADDR_4)
+ *       [ping src]
+ *           |                          top route
+ *         veth1 (LWT encap)  <<-- veth2        veth3  <<-- veth4 (ping dst)
+ *           |                                                ^
+ *       (bottom route)                                       | (inner pkt)
+ *           v                        bottom route            |
+ *         veth5              -->> veth6        veth7  -->> veth8 (vxlan decap)
+ *                                                          (IP*_ADDR_VXLAN)
+ *
+ * Add the VxLAN endpoint addresses to NS3's veth8, create standard
+ * VxLAN decap devices bound to those addresses, and install routes so
+ * NS1/NS2 can reach the endpoints via the bottom route.
+ */
+static int setup_vxlan_routes(const char *ns3, const char *ns1, const char *ns2)
+{
+	struct nstoken *nstoken;
+
+	nstoken = open_netns(ns3);
+	if (!ASSERT_OK_PTR(nstoken, "open ns3 for vxlan"))
+		return -1;
+
+	SYS(fail_close, "ip    a add %s/32  dev veth8", IP4_ADDR_VXLAN);
+	SYS(fail_close, "ip -6 a add %s/128 dev veth8", IP6_ADDR_VXLAN);
+	/*
+	 * Standard VxLAN devices to decap the encapsulated packets.  The inner
+	 * Ethernet frame uses a broadcast dst MAC so the IP stack accepts it
+	 * without ARP or FDB configuration.
+	 */
+	SYS(fail_close, "ip link add vxlan4 type vxlan id 1 dstport 4789 local %s dev veth8 nolearning noudpcsum",
+	    IP4_ADDR_VXLAN);
+	SYS(fail_close, "ip link set vxlan4 up");
+	SYS(fail_close, "ip link add vxlan6 type vxlan id 1 dstport 4789 local %s dev veth8 nolearning udp6zerocsumrx",
+	    IP6_ADDR_VXLAN);
+	SYS(fail_close, "ip link set vxlan6 up");
+	close_netns(nstoken);
+
+	SYS(fail, "ip -n %s    route add %s/32  dev veth5 via %s",
+	    ns1, IP4_ADDR_VXLAN, IP4_ADDR_6);
+	SYS(fail, "ip -n %s    route add %s/32  dev veth7 via %s",
+	    ns2, IP4_ADDR_VXLAN, IP4_ADDR_8);
+	SYS(fail, "ip -n %s -6 route add %s/128 dev veth5 via %s",
+	    ns1, IP6_ADDR_VXLAN, IP6_ADDR_6);
+	SYS(fail, "ip -n %s -6 route add %s/128 dev veth7 via %s",
+	    ns2, IP6_ADDR_VXLAN, IP6_ADDR_8);
+	return 0;
+
+fail_close:
+	close_netns(nstoken);
+fail:
+	return -1;
+}
+
+/*
+ * VxLAN encap tests (IPv4-outer and IPv6-outer variants).
+ *
+ * Test 1 - functional: the BPF LWT xmit program encapsulates the packet
+ *   (protocol=UDP, port=4789) and re-routes it without dropping it.
+ *   Verified by ping success.
+ *
+ * Test 2 - fix verification: after bpf_lwt_push_ip_encap() the
+ *   skb->transport_header must point at the outer UDP header, i.e.
+ *   transport_header - network_header == sizeof(outer IP header).
+ *   Without the fix the transport_header still points at the inner
+ *   transport layer, giving a wrong (larger) offset.
+ */
+static void lwt_ip_encap_vxlan(bool ipv4_encap)
+{
+	char ns1[NETNS_NAME_SIZE] = NETNS_BASE "-1-";
+	char ns2[NETNS_NAME_SIZE] = NETNS_BASE "-2-";
+	char ns3[NETNS_NAME_SIZE] = NETNS_BASE "-3-";
+	const char *sec = ipv4_encap ? "encap_vxlan" : "encap_vxlan6";
+	int expected_offset = ipv4_encap ? (int)sizeof(struct iphdr)
+					 : (int)sizeof(struct ipv6hdr);
+	struct test_lwt_ip_encap_fix *skel = NULL;
+	int thdr_offset, err;
+
+	if (!ASSERT_OK(create_ns(ns1, NETNS_NAME_SIZE), "create ns1"))
+		goto out;
+	if (!ASSERT_OK(create_ns(ns2, NETNS_NAME_SIZE), "create ns2"))
+		goto out;
+	if (!ASSERT_OK(create_ns(ns3, NETNS_NAME_SIZE), "create ns3"))
+		goto out;
+
+	if (!ASSERT_OK(setup_network(ns1, ns2, ns3, ""), "setup network"))
+		goto out;
+
+	if (!ASSERT_OK(setup_vxlan_routes(ns3, ns1, ns2), "setup vxlan routes"))
+		goto out;
+
+	/*
+	 * Attach fexit to bpf_lwt_push_ip_encap() before installing the
+	 * LWT route so we don't miss the first encap call.
+	 */
+	skel = test_lwt_ip_encap_fix__open();
+	if (!ASSERT_OK_PTR(skel, "test_lwt_ip_encap_fix__open"))
+		goto out;
+
+	skel->rodata->tgt_ip_version = ipv4_encap ? 4 : 6;
+
+	err = test_lwt_ip_encap_fix__load(skel);
+	if (!ASSERT_OK(err, "test_lwt_ip_encap_fix__load"))
+		goto out;
+
+	err = test_lwt_ip_encap_fix__attach(skel);
+	if (!ASSERT_OK(err, "test_lwt_ip_encap_fix__attach"))
+		goto out;
+
+	/* Remove the direct NS2->DST route so packets must go via LWT encap. */
+	SYS(out, "ip -n %s    route del %s/32  dev veth3", ns2, IP4_ADDR_DST);
+	SYS(out, "ip -n %s -6 route del %s/128 dev veth3", ns2, IP6_ADDR_DST);
+
+	/* Install the VxLAN BPF LWT xmit route. */
+	if (ipv4_encap)
+		SYS(out, "ip -n %s route add %s encap bpf xmit obj %s sec %s dev veth1",
+		    ns1, IP4_ADDR_DST, BPF_FILE, sec);
+	else
+		SYS(out, "ip -n %s -6 route add %s encap bpf xmit obj %s sec %s dev veth1",
+		    ns1, IP6_ADDR_DST, BPF_FILE, sec);
+
+	skel->bss->fexit_triggered = false;
+	if (ipv4_encap)
+		SYS(out, "ip netns exec %s ping  -c 1 -W1 %s", ns1, IP4_ADDR_DST);
+	else
+		SYS(out, "ip netns exec %s ping6 -c 1 -W1 %s", ns1, IP6_ADDR_DST);
+
+	/* Test 1: fexit triggered means bpf_lwt_push_ip_encap() succeeded. */
+	if (!ASSERT_TRUE(skel->bss->fexit_triggered, "fexit_triggered"))
+		goto out;
+
+	/*
+	 * Test 2: transport_header must sit immediately after the outer IP
+	 * header, pointing at the UDP header of the VxLAN encap.
+	 */
+	thdr_offset = (int)skel->bss->transport_hdr - (int)skel->bss->network_hdr;
+	ASSERT_EQ(thdr_offset, expected_offset, "transport_hdr offset");
+
+out:
+	test_lwt_ip_encap_fix__destroy(skel);
+	SYS_NOFAIL("ip netns del %s", ns1);
+	SYS_NOFAIL("ip netns del %s", ns2);
+	SYS_NOFAIL("ip netns del %s", ns3);
+}
+
+void test_lwt_ip_encap_vxlan_ipv4(void)
+{
+	lwt_ip_encap_vxlan(IPV4_ENCAP);
+}
+
+void test_lwt_ip_encap_vxlan_ipv6(void)
+{
+	lwt_ip_encap_vxlan(IPV6_ENCAP);
+}
tools/testing/selftests/bpf/progs/test_lwt_ip_encap.c +112 -0

124 lines not on this page · raw message

tools/testing/selftests/bpf/progs/test_lwt_ip_encap_fix.c new file +44 -0

44 lines not on this page · raw message

Signature · 1 line
2.54.0
reviewer
#11 lore raw

Re: [PATCH bpf v3 2/2] selftests/bpf: Add tests to verify the fix of encapsulating VxLAN in lwt

Quoted 40 lines · Leon Hwang

Add two tests to verify the transport header of skb has been set when encapsulate VxLAN using bpf_lwt_push_encap() helper.

1. VxLAN over IPv4.
2. VxLAN over IPv6.

Without the fix, the tests would fail:

lwt_ip_encap_vxlan:FAIL:transport_hdr offset unexpected transport_hdr offset: actual 70 != expected 20
#208 lwt_ip_encap_vxlan_ipv4:FAIL
lwt_ip_encap_vxlan:FAIL:transport_hdr offset unexpected transport_hdr offset: actual 110 != expected 40
#209 lwt_ip_encap_vxlan_ipv6:FAIL

Assisted-by Claude:claude-sonnet-4-6
Cc Leon Hwang <[email protected]>
Signed-off-by Leon Hwang <[email protected]>
---
 .../selftests/bpf/prog_tests/lwt_ip_encap.c   | 164 ++++++++++++++++++
 .../selftests/bpf/progs/test_lwt_ip_encap.c   | 112 ++++++++++++
 .../bpf/progs/test_lwt_ip_encap_fix.c         |  44 +++++
 3 files changed, 320 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/progs/test_lwt_ip_encap_fix.c
tools/testing/selftests/bpf/prog_tests/lwt_ip_encap.c +3 -0
index b6391af5f6f9..50104d847fde 100644
@@ -1,8 +1,11 @@
 // SPDX-License-Identifier: GPL-2.0-only
+#include <linux/ip.h>
+#include <linux/ipv6.h>
 #include <netinet/in.h>
 
 #include "network_helpers.h"
 #include "test_progs.h"
+#include "test_lwt_ip_encap_fix.skel.h"
 
 #define BPF_FILE "test_lwt_ip_encap.bpf.o"

tools/testing/selftests/bpf/prog_tests/lwt_ip_encap.c line 43

Quoted 6 lines
tools/testing/selftests/bpf/prog_tests/lwt_ip_encap.c +3 -0
@@ -35,6 +38,10 @@
 #define IP6_ADDR_SRC IP6_ADDR_1
 #define IP6_ADDR_DST IP6_ADDR_4
 
+/* VxLAN tunnel endpoints, reachable via the bottom route (veth5/6/7/8). */
+#define IP4_ADDR_VXLAN  "172.16.17.100"
+#define IP6_ADDR_VXLAN  "fb20::1"

There's a whole series of existing IP definitions at the top of the file, we should put those new ones right below them. And fb20 -> fb11 to keep with the pattern imo.

Quoted 12 lines
tools/testing/selftests/bpf/prog_tests/lwt_ip_encap.c +5 -0
+
 /* Setup/topology:
  *
  *    NS1             NS2             NS3

@@ -538,3 +545,160 @@ void test_lwt_ip_encap_ipv4(void)
 	if (test__start_subtest("ingress"))
 		lwt_ip_encap(IPV4_ENCAP, INGRESS, "");
 }
+
+/*
+ * VxLAN Setup/topology:
+ *

tools/testing/selftests/bpf/prog_tests/lwt_ip_encap.c void test_lwt_ip_encap_ipv4(void) · line 561

Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/lwt_ip_encap.c +10 -0
+ * NS1 (IP*_ADDR_1)                NS2                  NS3 (IP*_ADDR_4)
+ *       [ping src]
+ *           |                          top route
+ *         veth1 (LWT encap)  <<-- veth2        veth3  <<-- veth4 (ping dst)
+ *           |                                                ^
+ *       (bottom route)                                       | (inner pkt)
+ *           v                        bottom route            |
+ *         veth5              -->> veth6        veth7  -->> veth8 (vxlan decap)
+ *                                                          (IP*_ADDR_VXLAN)
+ *

Not sure if this is rendering weird for me but NS2 could be tabbed over once more for clarity.

Quoted 46 lines
tools/testing/selftests/bpf/prog_tests/lwt_ip_encap.c +46 -0
+ * Add the VxLAN endpoint addresses to NS3's veth8, create standard
+ * VxLAN decap devices bound to those addresses, and install routes so
+ * NS1/NS2 can reach the endpoints via the bottom route.
+ */
+static int setup_vxlan_routes(const char *ns3, const char *ns1, const char *ns2)
+{
+	struct nstoken *nstoken;
+
+	nstoken = open_netns(ns3);
+	if (!ASSERT_OK_PTR(nstoken, "open ns3 for vxlan"))
+		return -1;
+
+	SYS(fail_close, "ip    a add %s/32  dev veth8", IP4_ADDR_VXLAN);
+	SYS(fail_close, "ip -6 a add %s/128 dev veth8", IP6_ADDR_VXLAN);
+	/*
+	 * Standard VxLAN devices to decap the encapsulated packets.  The inner
+	 * Ethernet frame uses a broadcast dst MAC so the IP stack accepts it
+	 * without ARP or FDB configuration.
+	 */
+	SYS(fail_close, "ip link add vxlan4 type vxlan id 1 dstport 4789 local %s dev veth8 nolearning noudpcsum",
+	    IP4_ADDR_VXLAN);
+	SYS(fail_close, "ip link set vxlan4 up");
+	SYS(fail_close, "ip link add vxlan6 type vxlan id 1 dstport 4789 local %s dev veth8 nolearning udp6zerocsumrx",
+	    IP6_ADDR_VXLAN);
+	SYS(fail_close, "ip link set vxlan6 up");
+	close_netns(nstoken);
+
+	SYS(fail, "ip -n %s    route add %s/32  dev veth5 via %s",
+	    ns1, IP4_ADDR_VXLAN, IP4_ADDR_6);
+	SYS(fail, "ip -n %s    route add %s/32  dev veth7 via %s",
+	    ns2, IP4_ADDR_VXLAN, IP4_ADDR_8);
+	SYS(fail, "ip -n %s -6 route add %s/128 dev veth5 via %s",
+	    ns1, IP6_ADDR_VXLAN, IP6_ADDR_6);
+	SYS(fail, "ip -n %s -6 route add %s/128 dev veth7 via %s",
+	    ns2, IP6_ADDR_VXLAN, IP6_ADDR_8);
+	return 0;
+
+fail_close:
+	close_netns(nstoken);
+fail:
+	return -1;
+}
+
+/*
+ * VxLAN encap tests (IPv4-outer and IPv6-outer variants).
+ *

tools/testing/selftests/bpf/prog_tests/lwt_ip_encap.c void test_lwt_ip_encap_ipv4(void) · line 617

Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/lwt_ip_encap.c +10 -0
+ * Test 1 - functional: the BPF LWT xmit program encapsulates the packet
+ *   (protocol=UDP, port=4789) and re-routes it without dropping it.
+ *   Verified by ping success.
+ *
+ * Test 2 - fix verification: after bpf_lwt_push_ip_encap() the
+ *   skb->transport_header must point at the outer UDP header, i.e.
+ *   transport_header - network_header == sizeof(outer IP header).
+ *   Without the fix the transport_header still points at the inner
+ *   transport layer, giving a wrong (larger) offset.
+ */

The Test 2 bullet regurgitates the AI's context, can you rephrase it so that it's not framed as a fix but as a test?

Quoted 109 lines
tools/testing/selftests/bpf/prog_tests/lwt_ip_encap.c +87 -0
+static void lwt_ip_encap_vxlan(bool ipv4_encap)
+{
+	char ns1[NETNS_NAME_SIZE] = NETNS_BASE "-1-";
+	char ns2[NETNS_NAME_SIZE] = NETNS_BASE "-2-";
+	char ns3[NETNS_NAME_SIZE] = NETNS_BASE "-3-";
+	const char *sec = ipv4_encap ? "encap_vxlan" : "encap_vxlan6";
+	int expected_offset = ipv4_encap ? (int)sizeof(struct iphdr)
+					 : (int)sizeof(struct ipv6hdr);
+	struct test_lwt_ip_encap_fix *skel = NULL;
+	int thdr_offset, err;
+
+	if (!ASSERT_OK(create_ns(ns1, NETNS_NAME_SIZE), "create ns1"))
+		goto out;
+	if (!ASSERT_OK(create_ns(ns2, NETNS_NAME_SIZE), "create ns2"))
+		goto out;
+	if (!ASSERT_OK(create_ns(ns3, NETNS_NAME_SIZE), "create ns3"))
+		goto out;
+
+	if (!ASSERT_OK(setup_network(ns1, ns2, ns3, ""), "setup network"))
+		goto out;
+
+	if (!ASSERT_OK(setup_vxlan_routes(ns3, ns1, ns2), "setup vxlan routes"))
+		goto out;
+
+	/*
+	 * Attach fexit to bpf_lwt_push_ip_encap() before installing the
+	 * LWT route so we don't miss the first encap call.
+	 */
+	skel = test_lwt_ip_encap_fix__open();
+	if (!ASSERT_OK_PTR(skel, "test_lwt_ip_encap_fix__open"))
+		goto out;
+
+	skel->rodata->tgt_ip_version = ipv4_encap ? 4 : 6;
+
+	err = test_lwt_ip_encap_fix__load(skel);
+	if (!ASSERT_OK(err, "test_lwt_ip_encap_fix__load"))
+		goto out;
+
+	err = test_lwt_ip_encap_fix__attach(skel);
+	if (!ASSERT_OK(err, "test_lwt_ip_encap_fix__attach"))
+		goto out;
+
+	/* Remove the direct NS2->DST route so packets must go via LWT encap. */
+	SYS(out, "ip -n %s    route del %s/32  dev veth3", ns2, IP4_ADDR_DST);
+	SYS(out, "ip -n %s -6 route del %s/128 dev veth3", ns2, IP6_ADDR_DST);
+
+	/* Install the VxLAN BPF LWT xmit route. */
+	if (ipv4_encap)
+		SYS(out, "ip -n %s route add %s encap bpf xmit obj %s sec %s dev veth1",
+		    ns1, IP4_ADDR_DST, BPF_FILE, sec);
+	else
+		SYS(out, "ip -n %s -6 route add %s encap bpf xmit obj %s sec %s dev veth1",
+		    ns1, IP6_ADDR_DST, BPF_FILE, sec);
+
+	skel->bss->fexit_triggered = false;
+	if (ipv4_encap)
+		SYS(out, "ip netns exec %s ping  -c 1 -W1 %s", ns1, IP4_ADDR_DST);
+	else
+		SYS(out, "ip netns exec %s ping6 -c 1 -W1 %s", ns1, IP6_ADDR_DST);
+
+	/* Test 1: fexit triggered means bpf_lwt_push_ip_encap() succeeded. */
+	if (!ASSERT_TRUE(skel->bss->fexit_triggered, "fexit_triggered"))
+		goto out;
+
+	/*
+	 * Test 2: transport_header must sit immediately after the outer IP
+	 * header, pointing at the UDP header of the VxLAN encap.
+	 */
+	thdr_offset = (int)skel->bss->transport_hdr - (int)skel->bss->network_hdr;
+	ASSERT_EQ(thdr_offset, expected_offset, "transport_hdr offset");
+
+out:
+	test_lwt_ip_encap_fix__destroy(skel);
+	SYS_NOFAIL("ip netns del %s", ns1);
+	SYS_NOFAIL("ip netns del %s", ns2);
+	SYS_NOFAIL("ip netns del %s", ns3);
+}
+
+void test_lwt_ip_encap_vxlan_ipv4(void)
+{
+	lwt_ip_encap_vxlan(IPV4_ENCAP);
+}
+
+void test_lwt_ip_encap_vxlan_ipv6(void)
+{
+	lwt_ip_encap_vxlan(IPV6_ENCAP);
+}
tools/testing/selftests/bpf/progs/test_lwt_ip_encap.c +5 -0
index d6cb986e7533..36f0fc682ffb 100644
@@ -2,8 +2,10 @@
 #include <stddef.h>
 #include <string.h>
 #include <linux/bpf.h>
+#include <linux/if_ether.h>
 #include <linux/ip.h>
 #include <linux/ipv6.h>
+#include <linux/udp.h>
 #include <bpf/bpf_helpers.h>
 #include <bpf/bpf_endian.h>
 

@@ -82,4 +84,114 @@ int bpf_lwt_encap_gre6(struct __sk_buff *skb)
 	return BPF_LWT_REROUTE;
 }
 
+struct vxlanhdr {
+	__be32 vx_flags;  /* I flag = 0x08000000 (valid VNI) */
+	__be32 vx_vni;    /* VNI in top 24 bits */

tools/testing/selftests/bpf/progs/test_lwt_ip_encap.c int bpf_lwt_encap_gre6(struct __sk_buff *skb) · line 99

Quoted 10 lines
tools/testing/selftests/bpf/progs/test_lwt_ip_encap.c +10 -0
+};
+
+#define VXLAN_PORT  4789
+#define VXLAN_FLAGS 0x08000000
+#define VXLAN_VNI   1
+
+static const __u8 bcast[ETH_ALEN] = {
+	0xff, 0xff, 0xff, 0xff, 0xff, 0xff,
+};
+

q +static const __u8 srcmac[ETH_ALEN] = {

tools/testing/selftests/bpf/progs/test_lwt_ip_encap.c int bpf_lwt_encap_gre6(struct __sk_buff *skb) · line 112

Quoted 12 lines
tools/testing/selftests/bpf/progs/test_lwt_ip_encap.c +12 -0
+	0x02, 0x00, 0x00, 0x00, 0x00, 0x01,
+};
+
+SEC("encap_vxlan")
+int bpf_lwt_encap_vxlan(struct __sk_buff *skb)
+{
+	struct encap_hdr {
+		struct iphdr    iph;
+		struct udphdr   udph;
+		struct vxlanhdr vxh;
+		struct ethhdr   eth;
+	} __attribute__((__packed__)) /* packed is required to avoid padding */ hdr;

Comment is unnecessary here.

tools/testing/selftests/bpf/progs/test_lwt_ip_encap.c int bpf_lwt_encap_gre6(struct __sk_buff *skb) · line 124

Quoted 12 lines
tools/testing/selftests/bpf/progs/test_lwt_ip_encap.c +12 -0
+	int err;
+
+	memset(&hdr, 0, sizeof(hdr));
+
+	hdr.iph.ihl      = 5;
+	hdr.iph.version  = 4;
+	hdr.iph.ttl      = 0x40;
+	hdr.iph.protocol = 17; /* IPPROTO_UDP */
+	hdr.iph.tot_len  = bpf_htons(skb->len + sizeof(hdr));
+#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__
+	hdr.iph.saddr = 0x640510ac;  /* 172.16.5.100  */
+	hdr.iph.daddr = 0x641110ac;  /* 172.16.17.100 */

Ideally want to keep the addresses we are hardcoding here and the the addresses we're declaring in the userspace part in sync. Here we've hardcoded the addresses three multiple ways (be, le, and string in the userspace part).

tools/testing/selftests/bpf/progs/test_lwt_ip_encap_fix.c

Quoted 74 lines
tools/testing/selftests/bpf/progs/test_lwt_ip_encap.c +72 -0
+#elif __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__
+	hdr.iph.saddr = 0xac100564;  /* 172.16.5.100 */
+	hdr.iph.daddr = 0xac101164;  /* 172.16.17.100 */
+#else
+#error "Fix your compiler's __BYTE_ORDER__?!"
+#endif
+
+	hdr.udph.source = bpf_htons(VXLAN_PORT);
+	hdr.udph.dest   = bpf_htons(VXLAN_PORT);
+	hdr.udph.len    = bpf_htons(skb->len + sizeof(hdr.udph) + sizeof(hdr.vxh) +
+				    sizeof(hdr.eth));
+
+	hdr.vxh.vx_flags = bpf_htonl(VXLAN_FLAGS);
+	hdr.vxh.vx_vni   = bpf_htonl(VXLAN_VNI << 8);
+
+	__builtin_memcpy(hdr.eth.h_dest, bcast, ETH_ALEN);
+	__builtin_memcpy(hdr.eth.h_source, srcmac, ETH_ALEN);
+	hdr.eth.h_proto = bpf_htons(ETH_P_IP);
+
+	err = bpf_lwt_push_encap(skb, BPF_LWT_ENCAP_IP, &hdr, sizeof(hdr));
+	if (err)
+		return BPF_DROP;
+
+	return BPF_LWT_REROUTE;
+}
+
+SEC("encap_vxlan6")
+int bpf_lwt_encap_vxlan6(struct __sk_buff *skb)
+{
+	struct encap_hdr {
+		struct ipv6hdr  ip6hdr;
+		struct udphdr   udph;
+		struct vxlanhdr vxh;
+		struct ethhdr   eth;
+	} __attribute__((__packed__)) /* packed is required to avoid padding */ hdr;
+	int err;
+
+	memset(&hdr, 0, sizeof(hdr));
+
+	hdr.ip6hdr.version     = 6;
+	hdr.ip6hdr.nexthdr     = 17; /* IPPROTO_UDP */
+	hdr.ip6hdr.hop_limit   = 0x40;
+	hdr.ip6hdr.payload_len = bpf_htons(skb->len + sizeof(hdr.udph) + sizeof(hdr.vxh) +
+					   sizeof(hdr.eth));
+	/* fb05::1 */
+	hdr.ip6hdr.saddr.s6_addr[0]  = 0xfb;
+	hdr.ip6hdr.saddr.s6_addr[1]  = 0x05;
+	hdr.ip6hdr.saddr.s6_addr[15] = 1;
+	/* fb20::1 */
+	hdr.ip6hdr.daddr.s6_addr[0]  = 0xfb;
+	hdr.ip6hdr.daddr.s6_addr[1]  = 0x20;
+	hdr.ip6hdr.daddr.s6_addr[15] = 1;
+
+	hdr.udph.source = bpf_htons(VXLAN_PORT);
+	hdr.udph.dest   = bpf_htons(VXLAN_PORT);
+	hdr.udph.len    = bpf_htons(skb->len + sizeof(hdr.udph) + sizeof(hdr.vxh) +
+				    sizeof(hdr.eth));
+
+	hdr.vxh.vx_flags = bpf_htonl(VXLAN_FLAGS);
+	hdr.vxh.vx_vni   = bpf_htonl(VXLAN_VNI << 8);
+
+	__builtin_memcpy(hdr.eth.h_dest, bcast, ETH_ALEN);
+	__builtin_memcpy(hdr.eth.h_source, srcmac, ETH_ALEN);
+	hdr.eth.h_proto = bpf_htons(ETH_P_IPV6);
+
+	err = bpf_lwt_push_encap(skb, BPF_LWT_ENCAP_IP, &hdr, sizeof(hdr));
+	if (err)
+		return BPF_DROP;
+
+	return BPF_LWT_REROUTE;
+}
+
 char _license[] SEC("license") = "GPL";
tools/testing/selftests/bpf/progs/test_lwt_ip_encap_fix.c +0 -0

We definitely need to change the name here this is straight from the AI.

tools/testing/selftests/bpf/progs/test_lwt_ip_encap_fix.c line 7

Quoted 12 lines

new file mode 100644
index 000000000000..6945f83b94f2

tools/testing/selftests/bpf/progs/test_lwt_ip_encap_fix.c new file +7 -0
@@ -0,0 +1,44 @@
+// SPDX-License-Identifier: GPL-2.0
+#include "vmlinux.h"
+#include <bpf/bpf_helpers.h>
+#include <bpf/bpf_tracing.h>
+#include <bpf/bpf_core_read.h>
+
+#define NEXTHDR_UDP		17	/* UDP message. */

Unnecessary, we hardcode 17 in the other test.

Quoted 37 lines
tools/testing/selftests/bpf/progs/test_lwt_ip_encap_fix.c new file +37 -0
+
+volatile const int tgt_ip_version;
+
+__u16 transport_hdr = 0;
+__u16 network_hdr = 0;
+bool fexit_triggered = false;
+
+/*
+ * bpf_lwt_push_ip_encap(struct sk_buff *skb, void *hdr, u32 len, bool ingress)
+ *
+ * After a successful push the transport_header must point at the outer
+ * transport header (UDP for VxLAN), i.e.
+ *   transport_header - network_header == sizeof(outer IP header)
+ */
+SEC("fexit/bpf_lwt_push_ip_encap")
+int BPF_PROG(fexit_lwt_push_ip_encap, struct sk_buff *skb, void *hdr, u32 len, bool ingress,
+	     int retval)
+{
+	struct iphdr *iph;
+
+	if (retval || fexit_triggered)
+		return 0;
+
+	iph = (typeof(iph)) (skb->head + skb->network_header);
+	if (iph->version != tgt_ip_version)
+		return 0;
+
+	if ((iph->version == 4 && iph->protocol == IPPROTO_UDP) ||
+	    (iph->version == 6 && ((struct ipv6hdr *)iph)->nexthdr == NEXTHDR_UDP)) {
+		fexit_triggered = true;
+		transport_hdr   = BPF_CORE_READ(skb, transport_header);
+		network_hdr     = BPF_CORE_READ(skb, network_header);
+	}
+	return 0;
+}
+
+char _license[] SEC("license") = "GPL";
author
#12 lore raw

Re: [PATCH bpf v3 2/2] selftests/bpf: Add tests to verify the fix of encapsulating VxLAN in lwt

Quoted 1 line · Emil Tsalapatis

On Mon Jun 1, 2026 at 11:02 AM EDT, Leon Hwang wrote:

[...]

Quoted 4 lines
Quoted 4 lines
tools/testing/selftests/bpf/prog_tests/lwt_ip_encap.c +0 -0
@@ -35,6 +38,10 @@
 #define IP6_ADDR_SRC IP6_ADDR_1
 #define IP6_ADDR_DST IP6_ADDR_4
Quoted 8 lines

tools/testing/selftests/bpf/prog_tests/lwt_ip_encap.c line 43

Quoted 3 lines
tools/testing/selftests/bpf/prog_tests/lwt_ip_encap.c +3 -0
+/* VxLAN tunnel endpoints, reachable via the bottom route (veth5/6/7/8). */
+#define IP4_ADDR_VXLAN  "172.16.17.100"
+#define IP6_ADDR_VXLAN  "fb20::1"

There's a whole series of existing IP definitions at the top of the file, we should put those new ones right below them. And fb20 -> fb11 to keep with the pattern imo.

Ack.

Quoted 16 lines
Quoted 16 lines
tools/testing/selftests/bpf/prog_tests/lwt_ip_encap.c +9 -0
+
 /* Setup/topology:
  *
  *    NS1             NS2             NS3

@@ -538,3 +545,160 @@ void test_lwt_ip_encap_ipv4(void)
 	if (test__start_subtest("ingress"))
 		lwt_ip_encap(IPV4_ENCAP, INGRESS, "");
 }
+
+/*
+ * VxLAN Setup/topology:
+ *
+ * NS1 (IP*_ADDR_1)                NS2                  NS3 (IP*_ADDR_4)
+ *       [ping src]
+ *           |                          top route
+ *         veth1 (LWT encap)  <<-- veth2        veth3  <<-- veth4 (ping dst)
Quoted 9 lines

tools/testing/selftests/bpf/prog_tests/lwt_ip_encap.c void test_lwt_ip_encap_ipv4(void) · line 561

Quoted 6 lines
tools/testing/selftests/bpf/prog_tests/lwt_ip_encap.c +6 -0
+ *           |                                                ^
+ *       (bottom route)                                       | (inner pkt)
+ *           v                        bottom route            |
+ *         veth5              -->> veth6        veth7  -->> veth8 (vxlan decap)
+ *                                                          (IP*_ADDR_VXLAN)
+ *

Not sure if this is rendering weird for me but NS2 could be tabbed over once more for clarity.

NS2 here is to make sure the LWT-encap VxLAN packets can be routed correctly like other lwt tests.

Quoted 4 lines
Quoted 4 lines
tools/testing/selftests/bpf/prog_tests/lwt_ip_encap.c +4 -0
+ * Add the VxLAN endpoint addresses to NS3's veth8, create standard
+ * VxLAN decap devices bound to those addresses, and install routes so
+ * NS1/NS2 can reach the endpoints via the bottom route.
+ */

[...]

Quoted 14 lines
Quoted 14 lines
tools/testing/selftests/bpf/prog_tests/lwt_ip_encap.c +14 -0
+fail_close:
+	close_netns(nstoken);
+fail:
+	return -1;
+}
+
+/*
+ * VxLAN encap tests (IPv4-outer and IPv6-outer variants).
+ *
+ * Test 1 - functional: the BPF LWT xmit program encapsulates the packet
+ *   (protocol=UDP, port=4789) and re-routes it without dropping it.
+ *   Verified by ping success.
+ *
+ * Test 2 - fix verification: after bpf_lwt_push_ip_encap() the
Quoted 9 lines
Quoted 5 lines
+ *   skb->transport_header must point at the outer UDP header, i.e.
+ *   transport_header - network_header == sizeof(outer IP header).
+ *   Without the fix the transport_header still points at the inner
+ *   transport layer, giving a wrong (larger) offset.

+ */

The Test 2 bullet regurgitates the AI's context, can you rephrase it so that it's not framed as a fix but as a test?

Hmm, better to drop this comment, I think.

Quoted 6 lines
Quoted 6 lines
tools/testing/selftests/bpf/prog_tests/lwt_ip_encap.c +6 -0
+static void lwt_ip_encap_vxlan(bool ipv4_encap)
+{
+	char ns1[NETNS_NAME_SIZE] = NETNS_BASE "-1-";
+	char ns2[NETNS_NAME_SIZE] = NETNS_BASE "-2-";
+	char ns3[NETNS_NAME_SIZE] = NETNS_BASE "-3-";
+	const char *sec = ipv4_encap ? "encap_vxlan" : "encap_vxlan6";

[...]

Quoted 4 lines
Quoted 4 lines
tools/testing/selftests/bpf/progs/test_lwt_ip_encap.c +4 -0
+
+SEC("encap_vxlan")
+int bpf_lwt_encap_vxlan(struct __sk_buff *skb)
+{
Quoted 9 lines

tools/testing/selftests/bpf/progs/test_lwt_ip_encap.c int bpf_lwt_encap_gre6(struct __sk_buff *skb) · line 112

Quoted 6 lines
tools/testing/selftests/bpf/progs/test_lwt_ip_encap.c +6 -0
+	struct encap_hdr {
+		struct iphdr    iph;
+		struct udphdr   udph;
+		struct vxlanhdr vxh;
+		struct ethhdr   eth;
+	} __attribute__((__packed__)) /* packed is required to avoid padding */ hdr;

Comment is unnecessary here.

Ack.

Quoted 9 lines
Quoted 9 lines
tools/testing/selftests/bpf/progs/test_lwt_ip_encap.c +9 -0
+	int err;
+
+	memset(&hdr, 0, sizeof(hdr));
+
+	hdr.iph.ihl      = 5;
+	hdr.iph.version  = 4;
+	hdr.iph.ttl      = 0x40;
+	hdr.iph.protocol = 17; /* IPPROTO_UDP */
+	hdr.iph.tot_len  = bpf_htons(skb->len + sizeof(hdr));
Quoted 9 lines

tools/testing/selftests/bpf/progs/test_lwt_ip_encap.c int bpf_lwt_encap_gre6(struct __sk_buff *skb) · line 124

Quoted 3 lines
tools/testing/selftests/bpf/progs/test_lwt_ip_encap.c +3 -0
+#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__
+	hdr.iph.saddr = 0x640510ac;  /* 172.16.5.100  */
+	hdr.iph.daddr = 0x641110ac;  /* 172.16.17.100 */

Ideally want to keep the addresses we are hardcoding here and the the addresses we're declaring in the userspace part in sync. Here we've hardcoded the addresses three multiple ways (be, le, and string in the userspace part).

I think it's OK to hardcode them here like the gre tests.

Quoted 27 lines
Quoted 27 lines
tools/testing/selftests/bpf/progs/test_lwt_ip_encap.c +27 -0
+#elif __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__
+	hdr.iph.saddr = 0xac100564;  /* 172.16.5.100 */
+	hdr.iph.daddr = 0xac101164;  /* 172.16.17.100 */
+#else
+#error "Fix your compiler's __BYTE_ORDER__?!"
+#endif
+
+	hdr.udph.source = bpf_htons(VXLAN_PORT);
+	hdr.udph.dest   = bpf_htons(VXLAN_PORT);
+	hdr.udph.len    = bpf_htons(skb->len + sizeof(hdr.udph) + sizeof(hdr.vxh) +
+				    sizeof(hdr.eth));
+
+	hdr.vxh.vx_flags = bpf_htonl(VXLAN_FLAGS);
+	hdr.vxh.vx_vni   = bpf_htonl(VXLAN_VNI << 8);
+
+	__builtin_memcpy(hdr.eth.h_dest, bcast, ETH_ALEN);
+	__builtin_memcpy(hdr.eth.h_source, srcmac, ETH_ALEN);
+	hdr.eth.h_proto = bpf_htons(ETH_P_IP);
+
+	err = bpf_lwt_push_encap(skb, BPF_LWT_ENCAP_IP, &hdr, sizeof(hdr));
+	if (err)
+		return BPF_DROP;
+
+	return BPF_LWT_REROUTE;
+}
+
+SEC("encap_vxlan6")
Quoted 8 lines

tools/testing/selftests/bpf/progs/test_lwt_ip_encap.c int bpf_lwt_encap_gre6(struct __sk_buff *skb) · line 159

Quoted 8 lines
tools/testing/selftests/bpf/progs/test_lwt_ip_encap.c +8 -0
+int bpf_lwt_encap_vxlan6(struct __sk_buff *skb)
+{
+	struct encap_hdr {
+		struct ipv6hdr  ip6hdr;
+		struct udphdr   udph;
+		struct vxlanhdr vxh;
+		struct ethhdr   eth;
+	} __attribute__((__packed__)) /* packed is required to avoid padding */ hdr;

Will drop the comment.

Quoted 2 lines
Quoted 2 lines
(quoted diff) +2 -0
+	int err;
+

[...]

Quoted 8 lines

tools/testing/selftests/bpf/progs/test_lwt_ip_encap_fix.c

Quoted 5 lines
tools/testing/selftests/bpf/progs/test_lwt_ip_encap.c +3 -0
+	return BPF_LWT_REROUTE;
+}
+
 char _license[] SEC("license") = "GPL";
tools/testing/selftests/bpf/progs/test_lwt_ip_encap_fix.c +0 -0

We definitely need to change the name here this is straight from the AI.

Will fold it into test_lwt_ip_encap.c to avoid creating a new file.

Quoted 6 lines
Quoted 6 lines

new file mode 100644
index 000000000000..6945f83b94f2

tools/testing/selftests/bpf/progs/test_lwt_ip_encap_fix.c new file +1 -0
@@ -0,0 +1,44 @@
+// SPDX-License-Identifier: GPL-2.0
Quoted 9 lines

tools/testing/selftests/bpf/progs/test_lwt_ip_encap_fix.c line 7

Quoted 6 lines
tools/testing/selftests/bpf/progs/test_lwt_ip_encap_fix.c new file +6 -0
+#include "vmlinux.h"
+#include <bpf/bpf_helpers.h>
+#include <bpf/bpf_tracing.h>
+#include <bpf/bpf_core_read.h>
+
+#define NEXTHDR_UDP		17	/* UDP message. */

Unnecessary, we hardcode 17 in the other test.

Ack.

Thanks,
Leon

[...]