fou: ensure GUE headers have enough headroom
Subsystems (4)
PATCH 1/1 fou: ensure GUE headers have enough headroom
ipgre_changelink() installs GUE encapsulation before it publishes the new GRE header length and updates dev->needed_headroom. The transmit path does not serialize with RTNL, so it can interleave as follows:
CPU 0 (ipgre_changelink) CPU 1 (ipgre_xmit)
install GUE encapsulation
reserve the old needed_headroom
publish larger GRE flags
update tunnel->tun_hlen
push the larger GRE header push the GUE and UDP headers
update dev->needed_headroom
With REMCSUM, the new layout can push 16 bytes of GRE and 20 bytes of GUE/UDP headers into an skb with only 32 bytes of actual headroom. The final UDP push writes four bytes before skb->head.
With the update window widened, the kernel reported:
skbuff: skb_under_panic: ... len:128 put:8 ... dev:gre0poc kernel BUG at net/core/skbuff.c:214!
Oops: invalid opcode: 0000 [#1] SMP KASAN NOPTI Call Trace: skb_push fou_build_udp gue_build_header ip_tunnel_xmit __gre_xmit ipgre_xmit
Make __gue_build_header() ensure space for both the GUE header it is about to push and the UDP header that follows. On normally sized skbs the check is a no-op. If configuration changes race with transmission, skb_cow_head() expands the head before either GUE write, or returns an error without modifying the packet.
--- net/ipv4/fou_core.c | 2 ++ 1 file changed, 2 insertions(+)
net/ipv4/fou_core.c +2 -0
@@ -980,6 +980,8 @@ int __gue_build_header(struct sk_buff *skb, struct ip_tunnel_encap *e, skb, 0, 0, false); hdrlen =sizeof (struct guehdr ) + optlen; +if (skb_cow_head (skb, hdrlen +sizeof (struct udphdr ))) +return -ENOMEM ;skb_push (skb, hdrlen);
Signature · 1 line
2.43.0
Quoted 49 lines · Chengfeng Ye
ipgre_changelink()installs GUE encapsulation before it publishes the new GRE header length and updatesdev->needed_headroom. The transmit path does not serialize with RTNL, so it can interleave as follows:CPU 0 (
ipgre_changelink) CPU 1 (ipgre_xmit)
install GUE encapsulation
reserve the oldneeded_headroom
publish larger GRE flags
updatetunnel->tun_hlen
push the larger GRE header push the GUE and UDP headers
updatedev->needed_headroomWith REMCSUM, the new layout can push 16 bytes of GRE and 20 bytes of GUE/UDP headers into an skb with only 32 bytes of actual headroom. The final UDP push writes four bytes before
skb->head.With the update window widened, the kernel reported:
skbuff:
skb_under_panic: ... len:128 put:8 ... dev:gre0poc kernel BUG atnet/core/skbuff.c:214!Oops: invalid opcode: 0000 [#1] SMP KASAN NOPTI Call Trace: skb_push fou_build_udp gue_build_header ip_tunnel_xmit __gre_xmit ipgre_xmitMake
__gue_build_header()ensure space for both the GUE header it is about to push and the UDP header that follows. On normally sized skbs the check is a no-op. If configuration changes race with transmission,skb_cow_head()expands the head before either GUE write, or returns an error without modifying the packet.Fixes dd9d598c6657 ("ip_gre: add the support for i/o_flags update via netlink")Assisted-by Codex:gpt-5Signed-off-by Chengfeng Ye <[email protected]>--- net/ipv4/fou_core.c | 2 ++ 1 file changed, 2 insertions(+)net/ipv4/fou_core.c +0
-0
net/ipv4/fou_core.c int __gue_build_header(struct sk_buff *skb, struct ip_tunnel_encap *e, · line 984
Quoted 5 lines
net/ipv4/fou_core.c +2
-0@@ -980,6 +980,8 @@ int __gue_build_header(struct sk_buff *skb, struct ip_tunnel_encap *e, skb, 0, 0, false); hdrlen =sizeof (struct guehdr ) + optlen; +if (skb_cow_head (skb, hdrlen +sizeof (struct udphdr ))) +return -ENOMEM ;
My understanding is this takes into account the UDP header pushed in fou_build_udp. Isn't fou_build_header also affected by the same issue then? (If so check the IPv6 paths too).
Also please check Sashiko's output,
https://sashiko.dev/#/patchset/20260801060115.3538849-1-nicoyip.dev%40gmail.com
PATCH v2 1/1 fou: ensure GUE headers have enough headroom
In reply to Chengfeng Yeipgre_changelink() installs GUE encapsulation before it publishes the new GRE header length and updates dev->needed_headroom. The transmit path does not serialize with RTNL, so it can interleave as follows:
CPU 0 (ipgre_changelink) CPU 1 (ipgre_xmit)
install GUE encapsulation
reserve the old needed_headroom
publish larger GRE flags
update tunnel->tun_hlen
push the larger GRE header push the GUE and UDP headers
update dev->needed_headroom
With REMCSUM, the new layout can push 16 bytes of GRE and 20 bytes of GUE/UDP headers into an skb with only 32 bytes of actual headroom. The final UDP push writes four bytes before skb->head.
With the update window widened, the kernel reported:
skbuff: skb_under_panic: ... len:128 put:8 ... dev:gre0poc kernel BUG at net/core/skbuff.c:214!
Oops: invalid opcode: 0000 [#1] SMP KASAN NOPTI Call Trace: skb_push fou_build_udp gue_build_header ip_tunnel_xmit __gre_xmit ipgre_xmit
Make __gue_build_header() ensure space for both the GUE header it is about to push and the UDP header that follows. On normally sized skbs the check is a no-op. If configuration changes race with transmission, skb_cow_head() expands the head before either GUE write, or returns an error without modifying the packet.
skb_cow_head() may move skb->head. Refresh skb-derived inner_iph pointers after encapsulation, and read the tunnel fragment and TTL fields before ip_tunnel_encap() so ip_tunnel_xmit() does not dereference tnl_params after a possible skb head reallocation.
---
Changes in v2:
- Refresh skb-derived inner_iph pointers after ip_tunnel_encap() and read frag_off and ttl from tnl_params before ip_tunnel_encap(), because the new skb_cow_head() in the GUE builder can move skb->head.
net/ipv4/fou_core.c | 2 ++ net/ipv4/ip_tunnel.c | 7 +++++-- 2 files changed, 7 insertions(+), 2 deletions(-)
net/ipv4/fou_core.c +2 -0
@@ -980,6 +980,8 @@ int __gue_build_header(struct sk_buff *skb, struct ip_tunnel_encap *e, skb, 0, 0, false); hdrlen =sizeof (struct guehdr ) + optlen; +if (skb_cow_head (skb, hdrlen +sizeof (struct udphdr ))) +return -ENOMEM ;skb_push (skb, hdrlen);
net/ipv4/ip_tunnel.c +5 -2
@@ -606,6 +606,7 @@ void ip_md_tunnel_xmit(struct sk_buff *skb, struct net_device *dev,if (ip_tunnel_encap (skb, &tun_info->encap, &proto, &fl4) <0 ) goto tx_error; + inner_iph = (const struct iphdr *)skb_inner_network_header (skb); use_cache =ip_tunnel_dst_cache_usable (skb, tun_info);if (use_cache)
@@ -765,8 +766,12 @@ void ip_tunnel_xmit(struct sk_buff *skb, struct net_device *dev, tunnel->net,READ_ONCE (tunnel->parms.link), tunnel->fwmark, skb_get_hash(skb), 0); + df = tnl_params->frag_off; + ttl = tnl_params->ttl; +if (ip_tunnel_encap (skb, &tunnel->encap, &protocol, &fl4) <0 ) goto tx_error; + inner_iph = (const struct iphdr *)skb_inner_network_header (skb);if (connected && md) { use_cache =ip_tunnel_dst_cache_usable (skb, tun_info);
@@ -799,7 +804,6 @@ void ip_tunnel_xmit(struct sk_buff *skb, struct net_device *dev, goto tx_error; } - df = tnl_params->frag_off;if (payload_protocol ==htons (ETH_P_IP ) && !tunnel->ignore_df) df |= (inner_iph->frag_off &htons (IP_DF ));
@@ -821,7 +825,6 @@ void ip_tunnel_xmit(struct sk_buff *skb, struct net_device *dev, } tos =ip_tunnel_ecn_encap (tos, inner_iph, skb); - ttl = tnl_params->ttl;if (ttl ==0 ) {if (payload_protocol ==htons (ETH_P_IP )) ttl = inner_iph->ttl;
Signature · 1 line
2.43.0
Ido Schimmel
maintainerQuoted 36 lines · Chengfeng Ye
ipgre_changelink()installs GUE encapsulation before it publishes the new GRE header length and updatesdev->needed_headroom. The transmit path does not serialize with RTNL, so it can interleave as follows:CPU 0 (
ipgre_changelink) CPU 1 (ipgre_xmit)
install GUE encapsulation
reserve the oldneeded_headroom
publish larger GRE flags
updatetunnel->tun_hlen
push the larger GRE header push the GUE and UDP headers
updatedev->needed_headroomWith REMCSUM, the new layout can push 16 bytes of GRE and 20 bytes of GUE/UDP headers into an skb with only 32 bytes of actual headroom. The final UDP push writes four bytes before
skb->head.With the update window widened, the kernel reported:
skbuff:
skb_under_panic: ... len:128 put:8 ... dev:gre0poc kernel BUG atnet/core/skbuff.c:214!Oops: invalid opcode: 0000 [#1] SMP KASAN NOPTI Call Trace: skb_push fou_build_udp gue_build_header ip_tunnel_xmit __gre_xmit ipgre_xmitMake
__gue_build_header()ensure space for both the GUE header it is about to push and the UDP header that follows. On normally sized skbs the check is a no-op. If configuration changes race with transmission,skb_cow_head()expands the head before either GUE write, or returns an error without modifying the packet.
Quoted 4 lines
skb_cow_head()may moveskb->head. Refresh skb-derivedinner_iphpointers after encapsulation, and read the tunnel fragment and TTL fields beforeip_tunnel_encap()soip_tunnel_xmit()does not dereferencetnl_paramsafter a possible skb head reallocation.
It is up to the caller of ip_tunnel_encap() to make sure that there is enough headroom in the packet. Otherwise, what is the point of the encap_hlen() callback? Also, adding skb_cow_head() in the build_header() callback forces every caller to refresh pointers to skb->head.
There's a WIP patch to fix the same problem in IPv6 [1]. I suggest doing something similar in ip_tunnel_xmit() and ip_md_tunnel_xmit(). Note that IPv6 reserves the headroom before the encap push, so the snapshot is enough there. IPv4 reserves it after, so we also need an skb_cow_head() before ip_tunnel_encap().
Quoted 6 lines · Ido Schimmel
It is up to the caller of
ip_tunnel_encap()to make sure that there is enough headroom in the packet. Otherwise, what is the point of theencap_hlen()callback? Also, addingskb_cow_head()in thebuild_header()callback forces every caller to refresh pointers toskb->head.
Quoted 7 lines
There's a WIP patch to fix the same problem in IPv6 [1]. I suggest doing something similar in
ip_tunnel_xmit()andip_md_tunnel_xmit(). Note that IPv6 reserves the headroom before the encap push, so the snapshot is enough there. IPv4 reserves it after, so we also need anskb_cow_head()beforeip_tunnel_encap().
Thanks for the guide, I just created a v3 to fix the problem using the similar way as that ipv6 patch by adjusting the callsite.
Best regards,
Chengfeng