Skip to contentSkip to conversation

ipv4: hold a consistent view of rt->dst.dev under RCU

PATCH In review [email protected] 3 participants 5 messages Jul 1 – Jul 7

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 ipv4: hold a consistent view of rt->dst.dev under RCU

From: Xuanqiang Luo <[email protected]>

rt_flush_dev() walks the per-CPU uncached route list and rewrites rt->dst.dev in-place to blackhole_netdev under spin_lock_bh(). This lock does not exclude RCU readers, which may load rt->dst.dev multiple times within a single rcu_read_lock() region.

ip_rt_send_redirect() is a typical example: it reads rt->dst.dev three times to obtain in_dev, the L3 master ifindex, and net. A concurrent device unregistration can repoint rt->dst.dev to blackhole_netdev between those reads, making the reader combine state from two different net_devices — for instance, an in_dev from the real device but a netns and peer lookup from the blackhole device. ip_rt_get_source() has the same problem: it reads rt->dst.dev four times to obtain the output ifindex, the netns, and the source address, so a concurrent flush can cause the source selection to mix state from different devices.

Take a single dst_dev_rcu() snapshot of rt->dst.dev at the start of each affected RCU reader and use that snapshot throughout, so concurrent flushes cannot cause mid-function inconsistency. Publish the in-place write in rt_flush_dev() with rcu_assign_pointer() to match the readers.

Fixes caacf05e5ad1a ("ipv4: Properly purge netdev references on uncached routes.")

---
v2:
- Use dst_dev_rcu() and dev_net_rcu() for the RCU readers.
- Use rcu_assign_pointer() when publishing the uncached route device replacement.
- Slightly adjust the commit message wording because this issue was found by inspection, not from an observed user-visible failure.

v1: https://lore.kernel.org/all/[email protected]/

 net/ipv4/route.c | 29 +++++++++++++++++------------
 1 file changed, 17 insertions(+), 12 deletions(-)
net/ipv4/route.c +17 -12
index 3f3de5164d6e5..57f38467e6d0c 100644
@@ -873,6 +873,7 @@ static void ipv4_negative_advice(struct sock *sk,
 void ip_rt_send_redirect(struct sk_buff *skb)
 {
 	struct rtable *rt = skb_rtable(skb);
+	struct net_device *dev;
 	struct in_device *in_dev;
 	struct inet_peer *peer;
 	struct net *net;
@@ -880,15 +881,16 @@ void ip_rt_send_redirect(struct sk_buff *skb)
 	int vif;
 
 	rcu_read_lock();
-	in_dev = __in_dev_get_rcu(rt->dst.dev);
+	dev = dst_dev_rcu(&rt->dst);
+	in_dev = __in_dev_get_rcu(dev);
 	if (!in_dev || !IN_DEV_TX_REDIRECTS(in_dev)) {
 		rcu_read_unlock();
 		return;
 	}
 	log_martians = IN_DEV_LOG_MARTIANS(in_dev);
-	vif = l3mdev_master_ifindex_rcu(rt->dst.dev);
+	vif = l3mdev_master_ifindex_rcu(dev);
 
-	net = dev_net(rt->dst.dev);
+	net = dev_net_rcu(dev);
 	peer = inet_getpeer_v4(net->ipv4.peers, ip_hdr(skb)->saddr, vif);
 	if (!peer) {
 		rcu_read_unlock();
@@ -1287,29 +1289,32 @@ void ip_rt_get_source(u8 *addr, struct sk_buff *skb, struct rtable *rt)
 {
 	__be32 src;
 
-	if (rt_is_output_route(rt))
+	rcu_read_lock();
+	if (rt_is_output_route(rt)) {
 		src = ip_hdr(skb)->saddr;
-	else {
+	} else {
 		struct fib_result res;
 		struct iphdr *iph = ip_hdr(skb);
+		struct net_device *dev = dst_dev_rcu(&rt->dst);
+		struct net *net = dev_net_rcu(dev);
 		struct flowi4 fl4 = {
 			.daddr = iph->daddr,
 			.saddr = iph->saddr,
 			.flowi4_dscp = ip4h_dscp(iph),
-			.flowi4_oif = rt->dst.dev->ifindex,
+			.flowi4_oif = dev->ifindex,
 			.flowi4_iif = skb->dev->ifindex,
 			.flowi4_mark = skb->mark,
 		};
 
-		rcu_read_lock();
-		if (fib_lookup(dev_net(rt->dst.dev), &fl4, &res, 0) == 0)
-			src = fib_result_prefsrc(dev_net(rt->dst.dev), &res);
+		if (fib_lookup(net, &fl4, &res, 0) == 0)
+			src = fib_result_prefsrc(net, &res);
 		else
-			src = inet_select_addr(rt->dst.dev,
+			src = inet_select_addr(dev,
 					       rt_nexthop(rt, iph->daddr),
 					       RT_SCOPE_UNIVERSE);
-		rcu_read_unlock();
 	}
+	rcu_read_unlock();
+
 	memcpy(addr, &src, 4);
 }
 
@@ -1565,7 +1570,7 @@ void rt_flush_dev(struct net_device *dev)
 		list_for_each_entry_safe(rt, safe, &ul->head, dst.rt_uncached) {
 			if (rt->dst.dev != dev)
 				continue;
-			rt->dst.dev = blackhole_netdev;
+			rcu_assign_pointer(rt->dst.dev_rcu, blackhole_netdev);
 			netdev_ref_replace(dev, blackhole_netdev,
 					   &rt->dst.dev_tracker, GFP_ATOMIC);
 			list_del_init(&rt->dst.rt_uncached);
Signature · 1 line
2.43.0
maintainer
#1 lore raw
Quoted 7 lines · On Wed, Jul 01, 2026 at 11:24:34AM +0800, [email protected] wrote:

From: Xuanqiang Luo <[email protected]>

rt_flush_dev() walks the per-CPU uncached route list and rewrites rt->dst.dev in-place to blackhole_netdev under spin_lock_bh(). This lock does not exclude RCU readers, which may load rt->dst.dev multiple times within a single rcu_read_lock() region.

Quoted 10 lines

ip_rt_send_redirect() is a typical example: it reads rt->dst.dev three times to obtain in_dev, the L3 master ifindex, and net. A concurrent device unregistration can repoint rt->dst.dev to blackhole_netdev between those reads, making the reader combine state from two different net_devices — for instance, an in_dev from the real device but a netns and peer lookup from the blackhole device. ip_rt_get_source() has the same problem: it reads rt->dst.dev four times to obtain the output ifindex, the netns, and the source address, so a concurrent flush can cause the source selection to mix state from different devices.

Why only change ip_rt_send_redirect() and ip_rt_get_source() when the patch is titled "ipv4: hold a consistent view of rt->dst.dev under RCU"? What is the criterion?

Quoted 6 lines

Take a single dst_dev_rcu() snapshot of rt->dst.dev at the start of each affected RCU reader and use that snapshot throughout, so concurrent flushes cannot cause mid-function inconsistency. Publish the in-place write in rt_flush_dev() with rcu_assign_pointer() to match the readers.

The rt_flush_dev() change should be a separate change. Note that dst_dev_put() was already converted to use rcu_assign_pointer().

Quoted 2 lines
Fixes caacf05e5ad1a ("ipv4: Properly purge netdev references on uncached routes.")

Please remove the Fixes tag given you are targeting net-next.

Quoted 19 lines

---
v2:
- Use dst_dev_rcu() and dev_net_rcu() for the RCU readers.
- Use rcu_assign_pointer() when publishing the uncached route device replacement.
- Slightly adjust the commit message wording because this issue was found by inspection, not from an observed user-visible failure.

v1: https://lore.kernel.org/all/[email protected]/

 net/ipv4/route.c | 29 +++++++++++++++++------------
 1 file changed, 17 insertions(+), 12 deletions(-)
net/ipv4/route.c +0 -0
index 3f3de5164d6e5..57f38467e6d0c 100644

net/ipv4/route.c static void ipv4_negative_advice(struct sock *sk, · line 876

Quoted 4 lines
net/ipv4/route.c +1 -0
@@ -873,6 +873,7 @@ static void ipv4_negative_advice(struct sock *sk,
 void ip_rt_send_redirect(struct sk_buff *skb)
 {
 	struct rtable *rt = skb_rtable(skb);
+	struct net_device *dev;

https://docs.kernel.org/process/maintainer-netdev.html#local-variable-ordering-reverse-xmas-tree-rcs

Same in other places.

#2 lore raw
Quoted 18 lines · Ido Schimmel
Quoted 7 lines · On Wed, Jul 01, 2026 at 11:24:34AM +0800,[email protected] wrote:

From: Xuanqiang Luo<[email protected]>

rt_flush_dev() walks the per-CPU uncached route list and rewrites rt->dst.dev in-place to blackhole_netdev under spin_lock_bh(). This lock does not exclude RCU readers, which may load rt->dst.dev multiple times within a single rcu_read_lock() region.

Quoted 10 lines

ip_rt_send_redirect() is a typical example: it reads rt->dst.dev three times to obtain in_dev, the L3 master ifindex, and net. A concurrent device unregistration can repoint rt->dst.dev to blackhole_netdev between those reads, making the reader combine state from two different net_devices — for instance, an in_dev from the real device but a netns and peer lookup from the blackhole device. ip_rt_get_source() has the same problem: it reads rt->dst.dev four times to obtain the output ifindex, the netns, and the source address, so a concurrent flush can cause the source selection to mix state from different devices.

Quoted 3 lines

Why only change ip_rt_send_redirect() and ip_rt_get_source() when the patch is titled "ipv4: hold a consistent view of rt->dst.dev under RCU"? What is the criterion?

Thanks! You are right, the subject is too broad. I will make them more accurate in the next version.

Quoted 8 lines
Quoted 5 lines

Take a single dst_dev_rcu() snapshot of rt->dst.dev at the start of each affected RCU reader and use that snapshot throughout, so concurrent flushes cannot cause mid-function inconsistency. Publish the in-place write in rt_flush_dev() with rcu_assign_pointer() to match the readers.

The rt_flush_dev() change should be a separate change. Note that dst_dev_put() was already converted to use rcu_assign_pointer().

I will split the rt_flush_dev() change into a separate patch.

Quoted 2 lines
Quoted 1 line
Fixes caacf05e5ad1a ("ipv4: Properly purge netdev references on uncached routes.")

Please remove the Fixes tag given you are targeting net-next.

Just to clarify: is the suggestion to drop the Fixes tag here solely because this patch is targeted at net-next? Or are there any other reasons?

Quoted 25 lines
Quoted 19 lines

---
v2:
- Use dst_dev_rcu() and dev_net_rcu() for the RCU readers.
- Use rcu_assign_pointer() when publishing the uncached route device replacement.
- Slightly adjust the commit message wording because this issue was found by inspection, not from an observed user-visible failure.

v1:https://lore.kernel.org/all/[email protected]/

  net/ipv4/route.c | 29 +++++++++++++++++------------
  1 file changed, 17 insertions(+), 12 deletions(-)
net/ipv4/route.c +0 -0
index 3f3de5164d6e5..57f38467e6d0c 100644

net/ipv4/route.c static void ipv4_negative_advice(struct sock *sk, · line 876

Quoted 4 lines
net/ipv4/route.c +1 -0
@@ -873,6 +873,7 @@ static void ipv4_negative_advice(struct sock *sk,
  void ip_rt_send_redirect(struct sk_buff *skb)
  {
  	struct rtable *rt = skb_rtable(skb);
+	struct net_device *dev;

https://docs.kernel.org/process/maintainer-netdev.html#local-variable-ordering-reverse-xmas-tree-rcs

Quoted 1 line

Same in other places.

Thanks for pointing this out!
I will fix the local variable ordering in the next version.

maintainer
#3 lore raw
Quoted 34 lines · luoxuanqiang
Quoted 18 lines · Ido Schimmel
Quoted 7 lines · On Wed, Jul 01, 2026 at 11:24:34AM +0800,[email protected] wrote:

From: Xuanqiang Luo<[email protected]>

rt_flush_dev() walks the per-CPU uncached route list and rewrites rt->dst.dev in-place to blackhole_netdev under spin_lock_bh(). This lock does not exclude RCU readers, which may load rt->dst.dev multiple times within a single rcu_read_lock() region.

Quoted 10 lines

ip_rt_send_redirect() is a typical example: it reads rt->dst.dev three times to obtain in_dev, the L3 master ifindex, and net. A concurrent device unregistration can repoint rt->dst.dev to blackhole_netdev between those reads, making the reader combine state from two different net_devices — for instance, an in_dev from the real device but a netns and peer lookup from the blackhole device. ip_rt_get_source() has the same problem: it reads rt->dst.dev four times to obtain the output ifindex, the netns, and the source address, so a concurrent flush can cause the source selection to mix state from different devices.

Quoted 3 lines

Why only change ip_rt_send_redirect() and ip_rt_get_source() when the patch is titled "ipv4: hold a consistent view of rt->dst.dev under RCU"? What is the criterion?

Thanks! You are right, the subject is too broad. I will make them more accurate in the next version.

Quoted 8 lines
Quoted 5 lines

Take a single dst_dev_rcu() snapshot of rt->dst.dev at the start of each affected RCU reader and use that snapshot throughout, so concurrent flushes cannot cause mid-function inconsistency. Publish the in-place write in rt_flush_dev() with rcu_assign_pointer() to match the readers.

The rt_flush_dev() change should be a separate change. Note that dst_dev_put() was already converted to use rcu_assign_pointer().

Quoted 8 lines

I will split the rt_flush_dev() change into a separate patch.

Quoted 2 lines
Quoted 1 line
Fixes caacf05e5ad1a ("ipv4: Properly purge netdev references on uncached routes.")

Please remove the Fixes tag given you are targeting net-next.

Just to clarify: is the suggestion to drop the Fixes tag here solely because this patch is targeted at net-next? Or are there any other reasons?

Generally speaking, yes: net-next patches should not include a fixes tag unless the blamed commit is on net-next only.

More specifically, this patch is really a behavior improvement and not a vertical fix, as such we want to avoid it propagating on stable trees, as the fixes tag sometimes does.

/P

#4 lore raw
Quoted 41 lines · Paolo Abeni
Quoted 33 lines · luoxuanqiang
Quoted 18 lines · Ido Schimmel
Quoted 7 lines · On Wed, Jul 01, 2026 at 11:24:34AM +0800,[email protected] wrote:

From: Xuanqiang Luo<[email protected]>

rt_flush_dev() walks the per-CPU uncached route list and rewrites rt->dst.dev in-place to blackhole_netdev under spin_lock_bh(). This lock does not exclude RCU readers, which may load rt->dst.dev multiple times within a single rcu_read_lock() region.

Quoted 10 lines

ip_rt_send_redirect() is a typical example: it reads rt->dst.dev three times to obtain in_dev, the L3 master ifindex, and net. A concurrent device unregistration can repoint rt->dst.dev to blackhole_netdev between those reads, making the reader combine state from two different net_devices — for instance, an in_dev from the real device but a netns and peer lookup from the blackhole device. ip_rt_get_source() has the same problem: it reads rt->dst.dev four times to obtain the output ifindex, the netns, and the source address, so a concurrent flush can cause the source selection to mix state from different devices.

Quoted 3 lines

Why only change ip_rt_send_redirect() and ip_rt_get_source() when the patch is titled "ipv4: hold a consistent view of rt->dst.dev under RCU"? What is the criterion?

Thanks! You are right, the subject is too broad. I will make them more accurate in the next version.

Quoted 8 lines
Quoted 5 lines

Take a single dst_dev_rcu() snapshot of rt->dst.dev at the start of each affected RCU reader and use that snapshot throughout, so concurrent flushes cannot cause mid-function inconsistency. Publish the in-place write in rt_flush_dev() with rcu_assign_pointer() to match the readers.

The rt_flush_dev() change should be a separate change. Note that dst_dev_put() was already converted to use rcu_assign_pointer().

Quoted 7 lines

I will split the rt_flush_dev() change into a separate patch.

Quoted 2 lines
Quoted 1 line
Fixes caacf05e5ad1a ("ipv4: Properly purge netdev references on uncached routes.")

Please remove the Fixes tag given you are targeting net-next.

Just to clarify: is the suggestion to drop the Fixes tag here solely because this patch is targeted at net-next? Or are there any other reasons?

Quoted 9 lines

Generally speaking, yes: net-next patches should not include a fixes tag unless the blamed commit is on net-next only.

More specifically, this patch is really a behavior improvement and not a vertical fix, as such we want to avoid it propagating on stable trees, as the fixes tag sometimes does.

/P

Many thanks for the detailed explanation!