ipv4: hold a consistent view of rt->dst.dev under RCU
Subsystems (3)
PATCH v2 1/1 ipv4: hold a consistent view of rt->dst.dev under RCU
From: Xuanqiang Luo <[email protected]>
rt_flush_dev() walks the per-CPU uncached route list and rewrites rt->dst.dev in-place to blackhole_netdev under spin_lock_bh(). This lock does not exclude RCU readers, which may load rt->dst.dev multiple times within a single rcu_read_lock() region.
ip_rt_send_redirect() is a typical example: it reads rt->dst.dev three times to obtain in_dev, the L3 master ifindex, and net. A concurrent device unregistration can repoint rt->dst.dev to blackhole_netdev between those reads, making the reader combine state from two different net_devices — for instance, an in_dev from the real device but a netns and peer lookup from the blackhole device. ip_rt_get_source() has the same problem: it reads rt->dst.dev four times to obtain the output ifindex, the netns, and the source address, so a concurrent flush can cause the source selection to mix state from different devices.
Take a single dst_dev_rcu() snapshot of rt->dst.dev at the start of each affected RCU reader and use that snapshot throughout, so concurrent flushes cannot cause mid-function inconsistency. Publish the in-place write in rt_flush_dev() with rcu_assign_pointer() to match the readers.
---
v2:
- Use dst_dev_rcu() and dev_net_rcu() for the RCU readers.
- Use rcu_assign_pointer() when publishing the uncached route device replacement.
- Slightly adjust the commit message wording because this issue was found by inspection, not from an observed user-visible failure.
v1: https://lore.kernel.org/all/[email protected]/
net/ipv4/route.c | 29 +++++++++++++++++------------ 1 file changed, 17 insertions(+), 12 deletions(-)
net/ipv4/route.c +17 -12
@@ -873,6 +873,7 @@ static void ipv4_negative_advice(struct sock *sk,void ip_rt_send_redirect (struct sk_buff *skb) {struct rtable *rt =skb_rtable (skb); +struct net_device *dev;struct in_device *in_dev;struct inet_peer *peer;struct net *net;
@@ -880,15 +881,16 @@ void ip_rt_send_redirect(struct sk_buff *skb)int vif;rcu_read_lock (); - in_dev =__in_dev_get_rcu (rt->dst.dev); + dev =dst_dev_rcu (&rt->dst); + in_dev =__in_dev_get_rcu (dev);if (!in_dev || !IN_DEV_TX_REDIRECTS (in_dev)) {rcu_read_unlock ();return ; } log_martians =IN_DEV_LOG_MARTIANS (in_dev); - vif =l3mdev_master_ifindex_rcu (rt->dst.dev); + vif =l3mdev_master_ifindex_rcu (dev); - net =dev_net (rt->dst.dev); + net =dev_net_rcu (dev); peer =inet_getpeer_v4 (net->ipv4.peers,ip_hdr (skb)->saddr, vif);if (!peer) {rcu_read_unlock ();
@@ -1287,29 +1289,32 @@ void ip_rt_get_source(u8 *addr, struct sk_buff *skb, struct rtable *rt) {__be32 src; -if (rt_is_output_route (rt)) +rcu_read_lock (); +if (rt_is_output_route (rt)) { src =ip_hdr (skb)->saddr; -else { + }else {struct fib_result res;struct iphdr *iph =ip_hdr (skb); +struct net_device *dev =dst_dev_rcu (&rt->dst); +struct net *net =dev_net_rcu (dev);struct flowi4 fl4 = { .daddr = iph->daddr, .saddr = iph->saddr, .flowi4_dscp =ip4h_dscp (iph), - .flowi4_oif = rt->dst.dev->ifindex, + .flowi4_oif = dev->ifindex, .flowi4_iif = skb->dev->ifindex, .flowi4_mark = skb->mark, }; -rcu_read_lock (); -if (fib_lookup (dev_net (rt->dst.dev), &fl4, &res,0 ) ==0 ) - src =fib_result_prefsrc (dev_net (rt->dst.dev), &res); +if (fib_lookup (net, &fl4, &res,0 ) ==0 ) + src =fib_result_prefsrc (net, &res);else - src =inet_select_addr (rt->dst.dev, + src =inet_select_addr (dev,rt_nexthop (rt, iph->daddr),RT_SCOPE_UNIVERSE ); -rcu_read_unlock (); } +rcu_read_unlock (); +memcpy (addr, &src,4 ); }
@@ -1565,7 +1570,7 @@ void rt_flush_dev(struct net_device *dev)list_for_each_entry_safe (rt, safe, &ul->head, dst.rt_uncached) {if (rt->dst.dev != dev)continue ; - rt->dst.dev = blackhole_netdev; +rcu_assign_pointer (rt->dst.dev_rcu, blackhole_netdev);netdev_ref_replace (dev, blackhole_netdev, &rt->dst.dev_tracker,GFP_ATOMIC );list_del_init (&rt->dst.rt_uncached);
Signature · 1 line
2.43.0
Ido Schimmel
maintainerQuoted 7 lines · On Wed, Jul 01, 2026 at 11:24:34AM +0800, [email protected] wrote:
From: Xuanqiang Luo <[email protected]>
rt_flush_dev()walks the per-CPU uncached route list and rewritesrt->dst.devin-place toblackhole_netdevunderspin_lock_bh(). This lock does not exclude RCU readers, which may loadrt->dst.devmultiple times within a singlercu_read_lock()region.
Quoted 10 lines
ip_rt_send_redirect()is a typical example: it readsrt->dst.devthree times to obtainin_dev, the L3 master ifindex, and net. A concurrent device unregistration can repointrt->dst.devtoblackhole_netdevbetween those reads, making the reader combine state from two differentnet_devices— for instance, anin_devfrom the real device but a netns and peer lookup from the blackhole device.ip_rt_get_source()has the same problem: it readsrt->dst.devfour times to obtain the output ifindex, the netns, and the source address, so a concurrent flush can cause the source selection to mix state from different devices.
Why only change ip_rt_send_redirect() and ip_rt_get_source() when the patch is titled "ipv4: hold a consistent view of rt->dst.dev under RCU"? What is the criterion?
Quoted 6 lines
Take a single
dst_dev_rcu()snapshot ofrt->dst.devat the start of each affected RCU reader and use that snapshot throughout, so concurrent flushes cannot cause mid-function inconsistency. Publish the in-place write inrt_flush_dev()withrcu_assign_pointer()to match the readers.
The rt_flush_dev() change should be a separate change. Note that dst_dev_put() was already converted to use rcu_assign_pointer().
Quoted 2 lines
Fixes caacf05e5ad1a ("ipv4: Properly purge netdev references on uncached routes.")
Please remove the Fixes tag given you are targeting net-next.
Quoted 19 lines
Signed-off-by Xuanqiang Luo <[email protected]>---
v2:
- Usedst_dev_rcu()anddev_net_rcu()for the RCU readers.
- Usercu_assign_pointer()when publishing the uncached route device replacement.
- Slightly adjust the commit message wording because this issue was found by inspection, not from an observed user-visible failure.v1: https://lore.kernel.org/all/[email protected]/
net/ipv4/route.c | 29 +++++++++++++++++------------ 1 file changed, 17 insertions(+), 12 deletions(-)net/ipv4/route.c +0
-0
net/ipv4/route.c static void ipv4_negative_advice(struct sock *sk, · line 876
Quoted 4 lines
net/ipv4/route.c +1
-0@@ -873,6 +873,7 @@ static void ipv4_negative_advice(struct sock *sk,void ip_rt_send_redirect (struct sk_buff *skb) {struct rtable *rt =skb_rtable (skb); +struct net_device *dev;
https://docs.kernel.org/process/maintainer-netdev.html#local-variable-ordering-reverse-xmas-tree-rcs
Same in other places.
Quoted 18 lines · Ido Schimmel
Quoted 7 lines · On Wed, Jul 01, 2026 at 11:24:34AM +0800,[email protected] wrote:
From: Xuanqiang Luo<[email protected]>
rt_flush_dev()walks the per-CPU uncached route list and rewritesrt->dst.devin-place toblackhole_netdevunderspin_lock_bh(). This lock does not exclude RCU readers, which may loadrt->dst.devmultiple times within a singlercu_read_lock()region.Quoted 10 lines
ip_rt_send_redirect()is a typical example: it readsrt->dst.devthree times to obtainin_dev, the L3 master ifindex, and net. A concurrent device unregistration can repointrt->dst.devtoblackhole_netdevbetween those reads, making the reader combine state from two differentnet_devices— for instance, anin_devfrom the real device but a netns and peer lookup from the blackhole device.ip_rt_get_source()has the same problem: it readsrt->dst.devfour times to obtain the output ifindex, the netns, and the source address, so a concurrent flush can cause the source selection to mix state from different devices.
Quoted 3 lines
Why only change
ip_rt_send_redirect()andip_rt_get_source()when the patch is titled "ipv4: hold a consistent view ofrt->dst.devunder RCU"? What is the criterion?
Thanks! You are right, the subject is too broad. I will make them more accurate in the next version.
Quoted 8 lines
Quoted 5 lines
Take a single
dst_dev_rcu()snapshot ofrt->dst.devat the start of each affected RCU reader and use that snapshot throughout, so concurrent flushes cannot cause mid-function inconsistency. Publish the in-place write inrt_flush_dev()withrcu_assign_pointer()to match the readers.The
rt_flush_dev()change should be a separate change. Note thatdst_dev_put()was already converted to usercu_assign_pointer().
I will split the rt_flush_dev() change into a separate patch.
Quoted 2 lines
Quoted 1 line
Fixes caacf05e5ad1a ("ipv4: Properly purge netdev references on uncached routes.")Please remove the Fixes tag given you are targeting net-next.
Just to clarify: is the suggestion to drop the Fixes tag here solely because this patch is targeted at net-next? Or are there any other reasons?
Quoted 25 lines
Quoted 19 lines
Signed-off-by Xuanqiang Luo<[email protected]>---
v2:
- Usedst_dev_rcu()anddev_net_rcu()for the RCU readers.
- Usercu_assign_pointer()when publishing the uncached route device replacement.
- Slightly adjust the commit message wording because this issue was found by inspection, not from an observed user-visible failure.v1:https://lore.kernel.org/all/[email protected]/
net/ipv4/route.c | 29 +++++++++++++++++------------ 1 file changed, 17 insertions(+), 12 deletions(-)net/ipv4/route.c +0
-0net/ipv4/route.c static void ipv4_negative_advice(struct sock *sk, · line 876
Quoted 4 lines
net/ipv4/route.c +1
-0@@ -873,6 +873,7 @@ static void ipv4_negative_advice(struct sock *sk,void ip_rt_send_redirect (struct sk_buff *skb) {struct rtable *rt =skb_rtable (skb); +struct net_device *dev;https://docs.kernel.org/process/maintainer-netdev.html#local-variable-ordering-reverse-xmas-tree-rcs
Quoted 1 line
Same in other places.
Thanks for pointing this out!
I will fix the local variable ordering in the next version.
Paolo Abeni
maintainerQuoted 34 lines · luoxuanqiang
Quoted 18 lines · Ido Schimmel
Quoted 7 lines · On Wed, Jul 01, 2026 at 11:24:34AM +0800,[email protected] wrote:
From: Xuanqiang Luo<[email protected]>
rt_flush_dev()walks the per-CPU uncached route list and rewritesrt->dst.devin-place toblackhole_netdevunderspin_lock_bh(). This lock does not exclude RCU readers, which may loadrt->dst.devmultiple times within a singlercu_read_lock()region.Quoted 10 lines
ip_rt_send_redirect()is a typical example: it readsrt->dst.devthree times to obtainin_dev, the L3 master ifindex, and net. A concurrent device unregistration can repointrt->dst.devtoblackhole_netdevbetween those reads, making the reader combine state from two differentnet_devices— for instance, anin_devfrom the real device but a netns and peer lookup from the blackhole device.ip_rt_get_source()has the same problem: it readsrt->dst.devfour times to obtain the output ifindex, the netns, and the source address, so a concurrent flush can cause the source selection to mix state from different devices.Quoted 3 lines
Why only change
ip_rt_send_redirect()andip_rt_get_source()when the patch is titled "ipv4: hold a consistent view ofrt->dst.devunder RCU"? What is the criterion?Thanks! You are right, the subject is too broad. I will make them more accurate in the next version.
Quoted 8 lines
Quoted 5 lines
Take a single
dst_dev_rcu()snapshot ofrt->dst.devat the start of each affected RCU reader and use that snapshot throughout, so concurrent flushes cannot cause mid-function inconsistency. Publish the in-place write inrt_flush_dev()withrcu_assign_pointer()to match the readers.The
rt_flush_dev()change should be a separate change. Note thatdst_dev_put()was already converted to usercu_assign_pointer().
Quoted 8 lines
I will split the
rt_flush_dev()change into a separate patch.Quoted 2 lines
Quoted 1 line
Fixes caacf05e5ad1a ("ipv4: Properly purge netdev references on uncached routes.")Please remove the Fixes tag given you are targeting net-next.
Just to clarify: is the suggestion to drop the Fixes tag here solely because this patch is targeted at net-next? Or are there any other reasons?
Generally speaking, yes: net-next patches should not include a fixes tag unless the blamed commit is on net-next only.
More specifically, this patch is really a behavior improvement and not a vertical fix, as such we want to avoid it propagating on stable trees, as the fixes tag sometimes does.
/P
Quoted 41 lines · Paolo Abeni
Quoted 33 lines · luoxuanqiang
Quoted 18 lines · Ido Schimmel
Quoted 7 lines · On Wed, Jul 01, 2026 at 11:24:34AM +0800,[email protected] wrote:
From: Xuanqiang Luo<[email protected]>
rt_flush_dev()walks the per-CPU uncached route list and rewritesrt->dst.devin-place toblackhole_netdevunderspin_lock_bh(). This lock does not exclude RCU readers, which may loadrt->dst.devmultiple times within a singlercu_read_lock()region.Quoted 10 lines
ip_rt_send_redirect()is a typical example: it readsrt->dst.devthree times to obtainin_dev, the L3 master ifindex, and net. A concurrent device unregistration can repointrt->dst.devtoblackhole_netdevbetween those reads, making the reader combine state from two differentnet_devices— for instance, anin_devfrom the real device but a netns and peer lookup from the blackhole device.ip_rt_get_source()has the same problem: it readsrt->dst.devfour times to obtain the output ifindex, the netns, and the source address, so a concurrent flush can cause the source selection to mix state from different devices.Quoted 3 lines
Why only change
ip_rt_send_redirect()andip_rt_get_source()when the patch is titled "ipv4: hold a consistent view ofrt->dst.devunder RCU"? What is the criterion?Thanks! You are right, the subject is too broad. I will make them more accurate in the next version.
Quoted 8 lines
Quoted 5 lines
Take a single
dst_dev_rcu()snapshot ofrt->dst.devat the start of each affected RCU reader and use that snapshot throughout, so concurrent flushes cannot cause mid-function inconsistency. Publish the in-place write inrt_flush_dev()withrcu_assign_pointer()to match the readers.The
rt_flush_dev()change should be a separate change. Note thatdst_dev_put()was already converted to usercu_assign_pointer().Quoted 7 lines
I will split the
rt_flush_dev()change into a separate patch.Quoted 2 lines
Quoted 1 line
Fixes caacf05e5ad1a ("ipv4: Properly purge netdev references on uncached routes.")Please remove the Fixes tag given you are targeting net-next.
Just to clarify: is the suggestion to drop the Fixes tag here solely because this patch is targeted at net-next? Or are there any other reasons?
Quoted 9 lines
Generally speaking, yes: net-next patches should not include a fixes tag unless the blamed commit is on net-next only.
More specifically, this patch is really a behavior improvement and not a vertical fix, as such we want to avoid it propagating on stable trees, as the fixes tag sometimes does.
/P
Many thanks for the detailed explanation!