Skip to contentSkip to conversation

ipv6: Fix suspicious RCU usage in route hardware flags notification

PATCH Applied Ido Schimmel 4 participants 8 messages Oct 19 – Oct 20

Subsystems (5)
lore
Share & tools
lore

Ido Schimmel

author maintainer
#0 lore raw

cover ipv6: Fix suspicious RCU usage in route hardware flags notification

fib6_info_hw_flags_set() is called by drivers to update the hardware flags of an IPv6 route and to emit a notification when fib_notify_on_flag_change is set. Drivers call it from process context without RTNL or RCU, so when the route uses a nexthop object, building the notification triggers suspicious RCU usage splats.

Patch #1 fixes the problem by holding RCU while building the notification.

Patch #2 adds IPv4 and IPv6 test cases for routes that use nexthop objects. Without patch #1, the IPv6 one triggers the splats.

Ido Schimmel (2):
ipv6: Hold RCU when emitting route hardware flags notification
selftests: netdevsim: Add FIB notification tests for nexthop objects

 net/ipv6/route.c                              |  7 +-
 .../net/netdevsim/fib_notifications.sh        | 66 +++++++++++++++++++
 2 files changed, 72 insertions(+), 1 deletion(-)
Signature · 1 line
2.56.0
author maintainer
#1 lore raw

PATCH 1/2 ipv6: Hold RCU when emitting route hardware flags notification

fib6_info_hw_flags_set() is called by drivers from process context without RTNL or RCU. When the route uses a nexthop object, rt6_nlmsg_size() and rt6_fill_node() access the nexthop's RCU protected nexthop info and group pointers via rcu_dereference_rtnl(), resulting in suspicious RCU usage splats [1].

Fix by holding RCU while sizing and filling the notification, like fib_alias_hw_flags_set() already does. Switch to GFP_ATOMIC for the skb allocation accordingly.

Reproducer:

 # echo "10 0" > /sys/bus/netdevsim/new_device
 # sysctl -wq net.ipv6.fib_notify_on_flag_change=1
 # sysctl -wq net.ipv4.nexthop_compat_mode=1
 # ip -6 nexthop add id 1 blackhole
 # ip -6 route add 2001:db8:1::1/128 nhid 1

[1]
WARNING: suspicious RCU usage 7.3.0-rc7-virtme-g191aa41548b2 #38 Not tainted net/ipv4/nexthop.c:1539 suspicious rcu_dereference_check() usage! [...]

Workqueue: events nsim_fib_event_work
Call Trace:
 <TASK>
 dump_stack_lvl (lib/dump_stack.c:94 lib/dump_stack.c:120)
 lockdep_rcu_suspicious.cold (kernel/locking/lockdep.c:6972)
 nexthop_for_each_fib6_nh (net/ipv4/nexthop.c:1539 (discriminator 16))
 rt6_nlmsg_size (net/ipv6/route.c:5728)
 fib6_info_hw_flags_set (net/ipv6/route.c:6503)
 nsim_fib6_rt_add (drivers/net/netdevsim/fib.c:671 drivers/net/netdevsim/fib.c:694)
 nsim_fib6_rt_insert (drivers/net/netdevsim/fib.c:759)
 nsim_fib_event_work (drivers/net/netdevsim/fib.c:856 drivers/net/netdevsim/fib.c:889 drivers/net/netdevsim/fib.c:1493)
 process_one_work (kernel/workqueue.c:3396)
 worker_thread (kernel/workqueue.c:3479 kernel/workqueue.c:3560)
 kthread (kernel/kthread.c:436)
 ret_from_fork (arch/x86/kernel/process.c:158)
 ret_from_fork_asm (arch/x86/entry/entry_64.S:245)
 </TASK>
Fixes 907eea486888 ("net: ipv6: Emit notification when fib hardware flags are changed")
Assisted-by LLM
---
 net/ipv6/route.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)
net/ipv6/route.c +6 -1
index a5e955e56e79..ab355e86ea54 100644
@@ -6500,7 +6500,9 @@ void fib6_info_hw_flags_set(struct net *net, struct fib6_info *f6i,
 	if (!fib_notify_on_flag_change)
 		return;
 
-	skb = nlmsg_new(rt6_nlmsg_size(f6i), GFP_KERNEL);
+	rcu_read_lock();
+
+	skb = nlmsg_new(rt6_nlmsg_size(f6i), GFP_ATOMIC);
 	if (!skb) {
 		err = -ENOBUFS;
 		goto errout;
@@ -6513,10 +6515,13 @@ void fib6_info_hw_flags_set(struct net *net, struct fib6_info *f6i,
 		goto errout;
 	}
 
+	rcu_read_unlock();
+
 	rtnl_notify(skb, net, 0, RTNLGRP_IPV6_ROUTE, NULL, GFP_KERNEL);
 	return;
 
 errout:
+	rcu_read_unlock();
 	rtnl_set_sk_err(net, RTNLGRP_IPV6_ROUTE, err);
 }
 EXPORT_SYMBOL(fib6_info_hw_flags_set);
Signature · 1 line
2.56.0
maintainer Reviewed-by on v1 1/2
#2 lore raw

Re: [PATCH net 1/2] ipv6: Hold RCU when emitting route hardware flags notification

Quoted 10 lines · Ido Schimmel

fib6_info_hw_flags_set() is called by drivers from process context without RTNL or RCU. When the route uses a nexthop object, rt6_nlmsg_size() and rt6_fill_node() access the nexthop's RCU protected nexthop info and group pointers via rcu_dereference_rtnl(), resulting in suspicious RCU usage splats [1].

Fix by holding RCU while sizing and filling the notification, like fib_alias_hw_flags_set() already does. Switch to GFP_ATOMIC for the skb allocation accordingly.

maintainer Reviewed-by on v1 1/2
#3 lore raw

Re: [PATCH net 1/2] ipv6: Hold RCU when emitting route hardware flags notification

In reply to Ido Schimmel
Quoted 41 lines · Ido Schimmel

fib6_info_hw_flags_set() is called by drivers from process context without RTNL or RCU. When the route uses a nexthop object, rt6_nlmsg_size() and rt6_fill_node() access the nexthop's RCU protected nexthop info and group pointers via rcu_dereference_rtnl(), resulting in suspicious RCU usage splats [1].

Fix by holding RCU while sizing and filling the notification, like fib_alias_hw_flags_set() already does. Switch to GFP_ATOMIC for the skb allocation accordingly.

Reproducer:

 # echo "10 0" > /sys/bus/netdevsim/new_device
 # sysctl -wq net.ipv6.fib_notify_on_flag_change=1
 # sysctl -wq net.ipv4.nexthop_compat_mode=1
 # ip -6 nexthop add id 1 blackhole
 # ip -6 route add 2001:db8:1::1/128 nhid 1

[1]
WARNING: suspicious RCU usage 7.3.0-rc7-virtme-g191aa41548b2 #38 Not tainted net/ipv4/nexthop.c:1539 suspicious rcu_dereference_check() usage! [...]

Workqueue: events nsim_fib_event_work
Call Trace:
 <TASK>
 dump_stack_lvl (lib/dump_stack.c:94 lib/dump_stack.c:120)
 lockdep_rcu_suspicious.cold (kernel/locking/lockdep.c:6972)
 nexthop_for_each_fib6_nh (net/ipv4/nexthop.c:1539 (discriminator 16))
 rt6_nlmsg_size (net/ipv6/route.c:5728)
 fib6_info_hw_flags_set (net/ipv6/route.c:6503)
 nsim_fib6_rt_add (drivers/net/netdevsim/fib.c:671 drivers/net/netdevsim/fib.c:694)
 nsim_fib6_rt_insert (drivers/net/netdevsim/fib.c:759)
 nsim_fib_event_work (drivers/net/netdevsim/fib.c:856 drivers/net/netdevsim/fib.c:889 drivers/net/netdevsim/fib.c:1493)
 process_one_work (kernel/workqueue.c:3396)
 worker_thread (kernel/workqueue.c:3479 kernel/workqueue.c:3560)
 kthread (kernel/kthread.c:436)
 ret_from_fork (arch/x86/kernel/process.c:158)
 ret_from_fork_asm (arch/x86/entry/entry_64.S:245)
 </TASK>
Quoted 9 lines
Fixes 907eea486888 ("net: ipv6: Emit notification when fib hardware flags are changed")
Assisted-by LLM
---
 net/ipv6/route.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)
author maintainer
#4 lore raw

PATCH 2/2 selftests: netdevsim: Add FIB notification tests for nexthop objects

In reply to Ido Schimmel

Add IPv4 and IPv6 test cases that program a route that uses a nexthop object and check that two notifications are emitted for it, with the second one carrying the RTM_F_TRAP flag.

The new route_nexthop_check() helper follows the same pattern as the existing route_{addition,deletion,replacement}_check() helpers.

Without "ipv6: Hold RCU when emitting route hardware flags notification", the IPv6 test case passes, but triggers suspicious RCU usage splats. The IPv4 test case is added for completeness.

Assisted-by LLM
---
 .../net/netdevsim/fib_notifications.sh        | 66 +++++++++++++++++++
 1 file changed, 66 insertions(+)
tools/testing/selftests/drivers/net/netdevsim/fib_notifications.sh +66 -0
index 9896580c3d85..c49d6deb621a 100755
@@ -8,10 +8,12 @@ ALL_TESTS="
 	ipv4_route_deletion_test
 	ipv4_route_replacement_test
 	ipv4_route_offload_failed_test
+	ipv4_route_nexthop_test
 	ipv6_route_addition_test
 	ipv6_route_deletion_test
 	ipv6_route_replacement_test
 	ipv6_route_offload_failed_test
+	ipv6_route_nexthop_test
 "
 
 NETDEVSIM_PATH=/sys/bus/netdevsim/
@@ -269,6 +271,50 @@ ipv4_route_offload_failed_test()
 	log_test "IPv4 route offload failed"
 }
 
+route_nexthop_check()
+{
+	local ip=$1; shift
+	local route=$1; shift
+	local nhid=$1; shift
+	local expected_num_notifications=$1; shift
+	local outfile
+
+	ip netns exec testns1 sysctl -qw net.$ip.fib_notify_on_flag_change=1
+
+	outfile=$(mktemp)
+
+	$IP monitor route &> $outfile &
+	sleep 1
+	$IP route add $route nhid $nhid
+	sleep 1
+	kill_process %%
+
+	route_notify_check $outfile $expected_num_notifications
+	rm -f $outfile
+
+	$IP route del $route nhid $nhid
+}
+
+ipv4_route_nexthop_test()
+{
+	RET=0
+
+	local ip="ipv4"
+	local route=192.0.2.0/24
+	local nhid=1
+
+	$IP -4 nexthop add id $nhid dev dummy1
+
+	# Make sure two notifications will be emitted for a programmed route
+	# that uses a nexthop object.
+	local expected_num_notifications=2
+	route_nexthop_check $ip $route $nhid $expected_num_notifications
+
+	$IP nexthop del id $nhid
+
+	log_test "IPv4 route with nexthop object"
+}
+
 ipv6_route_addition_test()
 {
 	RET=0
@@ -385,6 +431,26 @@ ipv6_route_offload_failed_test()
 	log_test "IPv6 route offload failed"
 }
 
+ipv6_route_nexthop_test()
+{
+	RET=0
+
+	local ip="ipv6"
+	local route=2001:db8:1::/64
+	local nhid=1
+
+	$IP -6 nexthop add id $nhid dev dummy1
+
+	# Make sure two notifications will be emitted for a programmed route
+	# that uses a nexthop object.
+	local expected_num_notifications=2
+	route_nexthop_check $ip $route $nhid $expected_num_notifications
+
+	$IP nexthop del id $nhid
+
+	log_test "IPv6 route with nexthop object"
+}
+
 setup_prepare()
 {
 	modprobe netdevsim &> /dev/null
Signature · 1 line
2.56.0
maintainer Reviewed-by on v1 2/2
#5 lore raw

Re: [PATCH net 2/2] selftests: netdevsim: Add FIB notification tests for nexthop objects

Quoted 8 lines · Ido Schimmel

Add IPv4 and IPv6 test cases that program a route that uses a nexthop object and check that two notifications are emitted for it, with the second one carrying the RTM_F_TRAP flag.

The new route_nexthop_check() helper follows the same pattern as the existing route_{addition,deletion,replacement}_check() helpers.

Quoted 7 lines

Without "ipv6: Hold RCU when emitting route hardware flags notification", the IPv6 test case passes, but triggers suspicious RCU usage splats. The IPv4 test case is added for completeness.

Assisted-by LLM

---

maintainer Reviewed-by on v1 2/2
#6 lore raw

Re: [PATCH net 2/2] selftests: netdevsim: Add FIB notification tests for nexthop objects

In reply to Ido Schimmel
Quoted 11 lines · Ido Schimmel

Add IPv4 and IPv6 test cases that program a route that uses a nexthop object and check that two notifications are emitted for it, with the second one carrying the RTM_F_TRAP flag.

The new route_nexthop_check() helper follows the same pattern as the existing route_{addition,deletion,replacement}_check() helpers.

Without "ipv6: Hold RCU when emitting route hardware flags notification", the IPv6 test case passes, but triggers suspicious RCU usage splats. The IPv4 test case is added for completeness.

Quoted 6 lines
Assisted-by LLM
---
 .../net/netdevsim/fib_notifications.sh        | 66 +++++++++++++++++++
 1 file changed, 66 insertions(+)
#7 lore raw
In reply to Ido Schimmel
This series was applied to netdev/net.git (main) by Jakub Kicinski <[email protected]>:

Hello:

This series was applied to netdev/net.git (main)
by Jakub Kicinski <[email protected]>:

Quoted 10 lines · On Mon, 19 Oct 2026 17:03:35 +0300 you wrote:

fib6_info_hw_flags_set() is called by drivers to update the hardware flags of an IPv6 route and to emit a notification when fib_notify_on_flag_change is set. Drivers call it from process context without RTNL or RCU, so when the route uses a nexthop object, building the notification triggers suspicious RCU usage splats.

Patch #1 fixes the problem by holding RCU while building the notification.

[...]

Here is the summary with links:
- [net,1/2] ipv6: Hold RCU when emitting route hardware flags notification

    https://git.kernel.org/netdev/net/c/7c9cb1edaf8f
  - [net,2/2] selftests: netdevsim: Add FIB notification tests for nexthop objects
    https://git.kernel.org/netdev/net/c/b5dc3fde8b92

You are awesome, thank you!

Signature · 2 lines
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html