net: fix quite a few dst_cache crashes reported by syzbot
Subsystems (4)
cover net: fix quite a few dst_cache crashes reported by syzbot
There are two kinds of crashes reported many times by syzbot with no reproducer. Call Traces are like:
BUG: KASAN: slab-out-of-bounds in rt_cache_valid+0x158/0x190
net/ipv4/route.c:1556
rt_cache_valid+0x158/0x190 net/ipv4/route.c:1556
__mkroute_output net/ipv4/route.c:2332 [inline]
ip_route_output_key_hash_rcu+0x819/0x2d50 net/ipv4/route.c:2564
ip_route_output_key_hash+0x1ef/0x360 net/ipv4/route.c:2393
__ip_route_output_key include/net/route.h:125 [inline]
ip_route_output_flow+0x28/0xc0 net/ipv4/route.c:2651
ip_route_output_key include/net/route.h:135 [inline]
...or:
kasan: GPF could be caused by NULL-ptr deref or user memory access
RIP: 0010:dst_dev_put+0x24/0x290 net/core/dst.c:168
<IRQ>
rt_fibinfo_free_cpus net/ipv4/fib_semantics.c:200 [inline]
free_fib_info_rcu+0x2e1/0x490 net/ipv4/fib_semantics.c:217
__rcu_reclaim kernel/rcu/rcu.h:240 [inline]
rcu_do_batch kernel/rcu/tree.c:2437 [inline]
invoke_rcu_callbacks kernel/rcu/tree.c:2716 [inline]
rcu_process_callbacks+0x100a/0x1ac0 kernel/rcu/tree.c:2697
...They were caused by the fib_nh_common percpu member 'nhc_pcpu_rth_output' overwritten by another percpu variable 'dev->tstats' access overflow in tipc udp media xmit path when counting packets on a non tunnel device.
The fix is to make udp tunnel work with no tunnel device by allowing not to count packets on the tstats when the tunnel dev is NULL in Patches 1/3 and 2/3, then pass a NULL tunnel dev in tipc_udp_tunnel() in Patch 3/3.
Xin Long (3):
ip_tunnel: allow not to count pkts on tstats by setting skb's dev to
NULL
ip6_tunnel: allow not to count pkts on tstats by passing dev as NULL
tipc: pass tunnel dev as NULL to udp_tunnel(6)_xmit_skb
include/net/ip6_tunnel.h | 9 ++++++--- net/ipv4/ip_tunnel_core.c | 9 ++++++--- net/tipc/udp_media.c | 8 +++----- 3 files changed, 15 insertions(+), 11 deletions(-)
Signature · 1 line
2.1.0
PATCH 1/3 ip_tunnel: allow not to count pkts on tstats by setting skb's dev to NULL
iptunnel_xmit() works as a common function, also used by a udp tunnel which doesn't have to have a tunnel device, like how TIPC works with udp media.
In these cases, we should allow not to count pkts on dev's tstats, so that udp tunnel can work with no tunnel device safely.
--- net/ipv4/ip_tunnel_core.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-)
net/ipv4/ip_tunnel_core.c +6 -3
@@ -89,9 +89,12 @@ void iptunnel_xmit(struct sock *sk, struct rtable *rt, struct sk_buff *skb,__ip_select_ident (net, iph,skb_shinfo (skb)->gso_segs ?:1 ); err =ip_local_out (net, sk, skb); -if (unlikely (net_xmit_eval (err))) - pkt_len =0 ; -iptunnel_xmit_stats (dev, pkt_len); + +if (dev) { +if (unlikely (net_xmit_eval (err))) + pkt_len =0 ; +iptunnel_xmit_stats (dev, pkt_len); + } }EXPORT_SYMBOL_GPL (iptunnel_xmit);
Signature · 1 line
2.1.0
PATCH 2/3 ip6_tunnel: allow not to count pkts on tstats by passing dev as NULL
A similar fix to Patch "ip_tunnel: allow not to count pkts on tstats by setting skb's dev to NULL" is also needed by ip6_tunnel.
--- include/net/ip6_tunnel.h | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-)
include/net/ip6_tunnel.h +6 -3
@@ -158,9 +158,12 @@ static inline void ip6tunnel_xmit(struct sock *sk, struct sk_buff *skb,memset (skb->cb,0 ,sizeof (struct inet6_skb_parm )); pkt_len = skb->len -skb_inner_network_offset (skb); err =ip6_local_out (dev_net (skb_dst (skb)->dev), sk, skb); -if (unlikely (net_xmit_eval (err))) - pkt_len =-1 ; -iptunnel_xmit_stats (dev, pkt_len); + +if (dev) { +if (unlikely (net_xmit_eval (err))) + pkt_len =-1 ; +iptunnel_xmit_stats (dev, pkt_len); + } }#endif #endif
Signature · 1 line
2.1.0
PATCH 3/3 tipc: pass tunnel dev as NULL to udp_tunnel(6)_xmit_skb
udp_tunnel(6)_xmit_skb() called by tipc_udp_xmit() expects a tunnel device to count packets on dev->tstats, a perpcu variable. However, TIPC is using udp tunnel with no tunnel device, and pass the lower dev, like veth device that only initializes dev->lstats(a perpcu variable) when creating it.
Later iptunnel_xmit_stats() called by ip(6)tunnel_xmit() thinks the dev as a tunnel device, and uses dev->tstats instead of dev->lstats. tstats' each pointer points to a bigger struct than lstats, so when tstats->tx_bytes is increased, other percpu variable's members could be overwritten.
syzbot has reported quite a few crashes due to fib_nh_common percpu member 'nhc_pcpu_rth_output' overwritten, call traces are like:
BUG: KASAN: slab-out-of-bounds in rt_cache_valid+0x158/0x190
net/ipv4/route.c:1556
rt_cache_valid+0x158/0x190 net/ipv4/route.c:1556
__mkroute_output net/ipv4/route.c:2332 [inline]
ip_route_output_key_hash_rcu+0x819/0x2d50 net/ipv4/route.c:2564
ip_route_output_key_hash+0x1ef/0x360 net/ipv4/route.c:2393
__ip_route_output_key include/net/route.h:125 [inline]
ip_route_output_flow+0x28/0xc0 net/ipv4/route.c:2651
ip_route_output_key include/net/route.h:135 [inline]
...or:
kasan: GPF could be caused by NULL-ptr deref or user memory access
RIP: 0010:dst_dev_put+0x24/0x290 net/core/dst.c:168
<IRQ>
rt_fibinfo_free_cpus net/ipv4/fib_semantics.c:200 [inline]
free_fib_info_rcu+0x2e1/0x490 net/ipv4/fib_semantics.c:217
__rcu_reclaim kernel/rcu/rcu.h:240 [inline]
rcu_do_batch kernel/rcu/tree.c:2437 [inline]
invoke_rcu_callbacks kernel/rcu/tree.c:2716 [inline]
rcu_process_callbacks+0x100a/0x1ac0 kernel/rcu/tree.c:2697
...The issue exists since tunnel stats update is moved to iptunnel_xmit by Commit 039f50629b7f ("ip_tunnel: Move stats update to iptunnel_xmit()"), and here to fix it by passing a NULL tunnel dev to udp_tunnel(6)_xmit_skb so that the packets counting won't happen on dev->tstats.
--- net/tipc/udp_media.c | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-)
net/tipc/udp_media.c +3 -5
@@ -176,7 +176,6 @@ static int tipc_udp_xmit(struct net *net, struct sk_buff *skb, goto tx_error; } - skb->dev = rt->dst.dev; ttl =ip4_dst_hoplimit (&rt->dst); udp_tunnel_xmit_skb(rt, ub->ubsock->sk, skb, src->ipv4.s_addr, dst->ipv4.s_addr, 0, ttl, 0, src->port,
@@ -195,10 +194,9 @@ static int tipc_udp_xmit(struct net *net, struct sk_buff *skb,if (err) goto tx_error; ttl =ip6_dst_hoplimit (ndst); - err =udp_tunnel6_xmit_skb (ndst, ub->ubsock->sk, skb, - ndst->dev, &src->ipv6, - &dst->ipv6,0 , ttl,0 , src->port, - dst->port, false); + err =udp_tunnel6_xmit_skb (ndst, ub->ubsock->sk, skb,NULL , + &src->ipv6, &dst->ipv6,0 , ttl,0 , + src->port, dst->port, false);#endif }return err;
Signature · 1 line
2.1.0
David Miller
maintainerFrom: Xin Long <[email protected]>
Date: Mon, 17 Jun 2019 21:34:12 +0800
Quoted 2 lines
There are two kinds of crashes reported many times by syzbot with no reproducer. Call Traces are like:
...
Quoted 7 lines
They were caused by the
fib_nh_commonpercpu member 'nhc_pcpu_rth_output' overwritten by another percpu variable 'dev->tstats' access overflow in tipc udp media xmit path when counting packets on a non tunnel device.The fix is to make udp tunnel work with no tunnel device by allowing not to count packets on the tstats when the tunnel dev is NULL in Patches 1/3 and 2/3, then pass a NULL tunnel dev in
tipc_udp_tunnel()in Patch 3/3.
Series applied and queued up for -stable.
Thanks for putting all of those syzbot reported by tags in there.