Skip to contentSkip to conversation

ipmr: Fix RCU list debugging warning

PATCH RESEND Superseded Amol Grover 4 participants 9 messages May 9, 2020 – May 13, 2020

Subsystems (3)

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

PATCH RESEND 1/2 ipmr: Fix RCU list debugging warning

ipmr_for_each_table() macro uses list_for_each_entry_rcu() for traversing outside of an RCU read side critical section but under the protection of rtnl_mutex. Hence, add the corresponding lockdep expression to silence the following false-positive warning at boot:

[    4.319347] =============================
[    4.319349] WARNING: suspicious RCU usage
[    4.319351] 5.5.4-stable #17 Tainted: G            E
[    4.319352] -----------------------------
[    4.319354] net/ipv4/ipmr.c:1757 RCU-list traversed in non-reader section!!
---
 net/ipv4/ipmr.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
net/ipv4/ipmr.c +2 -1
index 6e68def66822..99c864eb6e34 100644
@@ -110,7 +110,8 @@ static void ipmr_expire_process(struct timer_list *t);
 
 #ifdef CONFIG_IP_MROUTE_MULTIPLE_TABLES
 #define ipmr_for_each_table(mrt, net) \
-	list_for_each_entry_rcu(mrt, &net->ipv4.mr_tables, list)
+	list_for_each_entry_rcu(mrt, &net->ipv4.mr_tables, list, \
+				lockdep_rtnl_is_held())
 
 static struct mr_table *ipmr_mr_table_iter(struct net *net,
 					   struct mr_table *mrt)
Signature · 1 line
2.24.1
#1 lore raw

PATCH RESEND 2/2 ipmr: Add lockdep expression to ipmr_for_each_table macro

ipmr_for_each_table() uses list_for_each_entry_rcu() for traversing outside of an RCU read-side critical section but under the protection of pernet_ops_rwsem. Hence add the corresponding lockdep expression to silence the following false-positive warning at boot:

[    0.645292] =============================
[    0.645294] WARNING: suspicious RCU usage
[    0.645296] 5.5.4-stable #17 Not tainted
[    0.645297] -----------------------------
[    0.645299] net/ipv4/ipmr.c:136 RCU-list traversed in non-reader section!!
---
 net/ipv4/ipmr.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)
net/ipv4/ipmr.c +4 -3
index 99c864eb6e34..950ffe9943da 100644
@@ -109,9 +109,10 @@ static void mroute_clean_tables(struct mr_table *mrt, int flags);
 static void ipmr_expire_process(struct timer_list *t);
 
 #ifdef CONFIG_IP_MROUTE_MULTIPLE_TABLES
-#define ipmr_for_each_table(mrt, net) \
-	list_for_each_entry_rcu(mrt, &net->ipv4.mr_tables, list, \
-				lockdep_rtnl_is_held())
+#define ipmr_for_each_table(mrt, net)					\
+	list_for_each_entry_rcu(mrt, &net->ipv4.mr_tables, list,	\
+				lockdep_rtnl_is_held() ||		\
+				lockdep_is_held(&pernet_ops_rwsem))
 
 static struct mr_table *ipmr_mr_table_iter(struct net *net,
 					   struct mr_table *mrt)
Signature · 6 lines
2.24.1

_______________________________________________
Linux-kernel-mentees mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/linux-kernel-mentees
maintainer
#2 lore raw

Re: [PATCH net 2/2 RESEND] ipmr: Add lockdep expression to ipmr_for_each_table macro

Quoted 5 lines · On Sat, 9 May 2020 12:52:44 +0530 Amol Grover wrote:

ipmr_for_each_table() uses list_for_each_entry_rcu() for traversing outside of an RCU read-side critical section but under the protection of pernet_ops_rwsem. Hence add the corresponding lockdep expression to silence the following false-positive warning at boot:

Thanks for the fix, the warning has been annoying me as well!

Quoted 5 lines
[    0.645292] =============================
[    0.645294] WARNING: suspicious RCU usage
[    0.645296] 5.5.4-stable #17 Not tainted
[    0.645297] -----------------------------
[    0.645299] net/ipv4/ipmr.c:136 RCU-list traversed in non-reader section!!

please provide a fuller stack trace, it would have helped the review

Quoted 10 lines
---
 net/ipv4/ipmr.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)
net/ipv4/ipmr.c +0 -0
index 99c864eb6e34..950ffe9943da 100644

net/ipv4/ipmr.c static void mroute_clean_tables(struct mr_table *mrt, int flags); · line 115

Quoted 10 lines
net/ipv4/ipmr.c +4 -3
@@ -109,9 +109,10 @@ static void mroute_clean_tables(struct mr_table *mrt, int flags);
 static void ipmr_expire_process(struct timer_list *t);
 
 #ifdef CONFIG_IP_MROUTE_MULTIPLE_TABLES
-#define ipmr_for_each_table(mrt, net) \
-	list_for_each_entry_rcu(mrt, &net->ipv4.mr_tables, list, \
-				lockdep_rtnl_is_held())
+#define ipmr_for_each_table(mrt, net)					\
+	list_for_each_entry_rcu(mrt, &net->ipv4.mr_tables, list,	\
+				lockdep_rtnl_is_held() ||		\
+				lockdep_is_held(&pernet_ops_rwsem))

This is a strange condition, IMHO. How can we be fine with either lock.. This is supposed to be the writer side lock, one can't have two writer side locks..

I think what is happening is this:

ipmr_net_init() -> ipmr_rules_init() -> ipmr_new_table()

ipmr_new_table() returns an existing table if there is one, but obviously none can exist at init. So a better fix would be:

#define ipmr_for_each_table(mrt, net)					\
	list_for_each_entry_rcu(mrt, &net->ipv4.mr_tables, list,	\
				lockdep_rtnl_is_held() ||		\
				list_empty(&net->ipv4.mr_tables))

Thoughts?

#3 lore raw

Re: [PATCH net 2/2 RESEND] ipmr: Add lockdep expression to ipmr_for_each_table macro

Quoted 46 lines · Jakub Kicinski
Quoted 5 lines · On Sat, 9 May 2020 12:52:44 +0530 Amol Grover wrote:

ipmr_for_each_table() uses list_for_each_entry_rcu() for traversing outside of an RCU read-side critical section but under the protection of pernet_ops_rwsem. Hence add the corresponding lockdep expression to silence the following false-positive warning at boot:

Thanks for the fix, the warning has been annoying me as well!

Quoted 5 lines
[    0.645292] =============================
[    0.645294] WARNING: suspicious RCU usage
[    0.645296] 5.5.4-stable #17 Not tainted
[    0.645297] -----------------------------
[    0.645299] net/ipv4/ipmr.c:136 RCU-list traversed in non-reader section!!

please provide a fuller stack trace, it would have helped the review

Quoted 10 lines
---
 net/ipv4/ipmr.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)
net/ipv4/ipmr.c +0 -0
index 99c864eb6e34..950ffe9943da 100644

net/ipv4/ipmr.c static void mroute_clean_tables(struct mr_table *mrt, int flags); · line 115

Quoted 10 lines
net/ipv4/ipmr.c +4 -3
@@ -109,9 +109,10 @@ static void mroute_clean_tables(struct mr_table *mrt, int flags);
 static void ipmr_expire_process(struct timer_list *t);
 
 #ifdef CONFIG_IP_MROUTE_MULTIPLE_TABLES
-#define ipmr_for_each_table(mrt, net) \
-	list_for_each_entry_rcu(mrt, &net->ipv4.mr_tables, list, \
-				lockdep_rtnl_is_held())
+#define ipmr_for_each_table(mrt, net)					\
+	list_for_each_entry_rcu(mrt, &net->ipv4.mr_tables, list,	\
+				lockdep_rtnl_is_held() ||		\
+				lockdep_is_held(&pernet_ops_rwsem))

This is a strange condition, IMHO. How can we be fine with either lock.. This is supposed to be the writer side lock, one can't have two writer side locks..

I think what is happening is this:

ipmr_net_init() -> ipmr_rules_init() -> ipmr_new_table()

Quoted 8 lines

ipmr_new_table() returns an existing table if there is one, but obviously none can exist at init. So a better fix would be:

#define ipmr_for_each_table(mrt, net)					\
	list_for_each_entry_rcu(mrt, &net->ipv4.mr_tables, list,	\
				lockdep_rtnl_is_held() ||		\
				list_empty(&net->ipv4.mr_tables))

(adding Stephen)

Hi Jakub,

Thank you for your suggestion about this patch.
Here is a stack trace for ipmr.c:

[    1.515015] TCP: Hash tables configured (established 8192 bind 8192)
[    1.516790] UDP hash table entries: 512 (order: 3, 49152 bytes, linear)
[    1.518177] UDP-Lite hash table entries: 512 (order: 3, 49152 bytes, linear)
[    1.519805]
[    1.520178] =============================
[    1.520982] WARNING: suspicious RCU usage
[    1.521798] 5.7.0-rc2-00006-gb35af6a26b7c6f #1 Not tainted
[    1.522910] -----------------------------
[    1.523671] net/ipv4/ipmr.c:136 RCU-list traversed in non-reader section!!
[    1.525218]
[    1.525218] other info that might help us debug this:
[    1.525218]
[    1.526731]
[    1.526731] rcu_scheduler_active = 2, debug_locks = 1
[    1.528004] 1 lock held by swapper/1:
[    1.528714]  #0: c20be1d8 (pernet_ops_rwsem){+.+.}-{3:3}, at: register_pernet_subsys+0xd/0x30
[    1.530433]
[    1.530433] stack backtrace:
[    1.531262] CPU: 0 PID: 1 Comm: swapper Not tainted 5.7.0-rc2-00006-gb35af6a26b7c6f #1
[    1.532729] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014
[    1.534305] Call Trace:
[    1.534758]  ? ipmr_get_table+0x3c/0x70
[    1.535430]  ? ipmr_new_table+0x1c/0x60
[    1.536173]  ? ipmr_net_init+0x7b/0x170
[    1.536923]  ? register_pernet_subsys+0xd/0x30
[    1.537810]  ? ops_init+0x1a0/0x1e0
[    1.538518]  ? kmem_cache_create_usercopy+0x28a/0x350
[    1.539752]  ? register_pernet_operations+0xc9/0x1c0
[    1.540630]  ? ipv4_offload_init+0x65/0x65
[    1.541451]  ? register_pernet_subsys+0x19/0x30
[    1.542357]  ? ip_mr_init+0x28/0xff
[    1.543079]  ? inet_init+0x17b/0x249
[    1.543773]  ? do_one_initcall+0xc5/0x240
[    1.544532]  ? parse_args+0x192/0x350
[    1.545266]  ? rcu_read_lock_sched_held+0x2f/0x60
[    1.546180]  ? trace_initcall_level+0x61/0x93
[    1.547061]  ? kernel_init_freeable+0x112/0x18a
[    1.547978]  ? kernel_init_freeable+0x12b/0x18a
[    1.548974]  ? rest_init+0x220/0x220
[    1.549792]  ? kernel_init+0x8/0x100
[    1.550548]  ? rest_init+0x220/0x220
[    1.551288]  ? schedule_tail_wrapper+0x6/0x8
[    1.552136]  ? rest_init+0x220/0x220
[    1.552873]  ? ret_from_fork+0x2e/0x38

ALso, there is a similar warning for ip6mr.c :

=============================
WARNING: suspicious RCU usage 5.7.0-rc4-next-20200507-syzkaller #0 Not tainted
-----------------------------
net/ipv6/ip6mr.c:124 RCU-list traversed in non-reader section!!

other info that might help us debug this:

rcu_scheduler_active = 2, debug_locks = 1
1 lock held by swapper/0/1:
#0: ffffffff8a7aae30 (pernet_ops_rwsem){+.+.}-{3:3}, at: register_pernet_subsys+0x16/0x40 net/core/net_namespace.c:1257

stack backtrace:
CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.7.0-rc4-next-20200507-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Call Trace:
__dump_stack lib/dump_stack.c:77 [inline]
dump_stack+0x18f/0x20d lib/dump_stack.c:118
ip6mr_get_table+0x153/0x180 net/ipv6/ip6mr.c:124
ip6mr_new_table+0x1b/0x70 net/ipv6/ip6mr.c:382
ip6mr_rules_init net/ipv6/ip6mr.c:236 [inline]
ip6mr_net_init+0x133/0x3f0 net/ipv6/ip6mr.c:1310
ops_init+0xaf/0x420 net/core/net_namespace.c:151
__register_pernet_operations net/core/net_namespace.c:1140 [inline]
register_pernet_operations+0x346/0x840 net/core/net_namespace.c:1217
register_pernet_subsys+0x25/0x40 net/core/net_namespace.c:1258
ip6_mr_init+0x49/0x152 net/ipv6/ip6mr.c:1363
inet6_init+0x1d7/0x6dc net/ipv6/af_inet6.c:1037
do_one_initcall+0x10a/0x7d0 init/main.c:1159
do_initcall_level init/main.c:1232 [inline]
do_initcalls init/main.c:1248 [inline]
do_basic_setup init/main.c:1268 [inline]
kernel_init_freeable+0x501/0x5ae init/main.c:1454
kernel_init+0xd/0x1bb init/main.c:1359
ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:351

Segment Routing with IPv6
mip6: Mobile IPv6
sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver
ip6_gre: GRE over IPv6 tunneling driver

Quoted 1 line

Thoughts?

Do you think a similar fix (the one you suggested) is also applicable in the ip6mr case.

Thank you,
Madhuparna

maintainer
#4 lore raw

Re: [PATCH net 2/2 RESEND] ipmr: Add lockdep expression to ipmr_for_each_table macro

Quoted 25 lines · On Tue, 12 May 2020 10:47:05 +0530 Madhuparna Bhowmik wrote:
Quoted 17 lines

net/ipv4/ipmr.c static void mroute_clean_tables(struct mr_table *mrt, int flags); · line 115

Quoted 8 lines

#ifdef CONFIG_IP_MROUTE_MULTIPLE_TABLES

net/ipv4/ipmr.c +4 -3
-#define ipmr_for_each_table(mrt, net) \
-	list_for_each_entry_rcu(mrt, &net->ipv4.mr_tables, list, \
-				lockdep_rtnl_is_held())
+#define ipmr_for_each_table(mrt, net)					\
+	list_for_each_entry_rcu(mrt, &net->ipv4.mr_tables, list,	\
+				lockdep_rtnl_is_held() ||		\
+				lockdep_is_held(&pernet_ops_rwsem))  

This is a strange condition, IMHO. How can we be fine with either lock.. This is supposed to be the writer side lock, one can't have two writer side locks..

I think what is happening is this:

ipmr_net_init() -> ipmr_rules_init() -> ipmr_new_table()

Quoted 8 lines

ipmr_new_table() returns an existing table if there is one, but obviously none can exist at init. So a better fix would be:

#define ipmr_for_each_table(mrt, net)					\
	list_for_each_entry_rcu(mrt, &net->ipv4.mr_tables, list,	\
				lockdep_rtnl_is_held() ||		\
				list_empty(&net->ipv4.mr_tables))
Quoted 8 lines

(adding Stephen)

Hi Jakub,

Thank you for your suggestion about this patch.
Here is a stack trace for ipmr.c:

[...]

Thanks!

Quoted 4 lines
Quoted 1 line

Thoughts?

Do you think a similar fix (the one you suggested) is also applicable in the ip6mr case.

Yes, looking at the code it seems ip6mr has the exact same flow for netns init.

#5 lore raw

Re: [PATCH net 2/2 RESEND] ipmr: Add lockdep expression to ipmr_for_each_table macro

Quoted 37 lines · Jakub Kicinski
Quoted 25 lines · On Tue, 12 May 2020 10:47:05 +0530 Madhuparna Bhowmik wrote:
Quoted 17 lines

net/ipv4/ipmr.c static void mroute_clean_tables(struct mr_table *mrt, int flags); · line 115

Quoted 8 lines

#ifdef CONFIG_IP_MROUTE_MULTIPLE_TABLES

net/ipv4/ipmr.c +4 -3
-#define ipmr_for_each_table(mrt, net) \
-	list_for_each_entry_rcu(mrt, &net->ipv4.mr_tables, list, \
-				lockdep_rtnl_is_held())
+#define ipmr_for_each_table(mrt, net)					\
+	list_for_each_entry_rcu(mrt, &net->ipv4.mr_tables, list,	\
+				lockdep_rtnl_is_held() ||		\
+				lockdep_is_held(&pernet_ops_rwsem))  

This is a strange condition, IMHO. How can we be fine with either lock.. This is supposed to be the writer side lock, one can't have two writer side locks..

I think what is happening is this:

ipmr_net_init() -> ipmr_rules_init() -> ipmr_new_table()

Quoted 8 lines

ipmr_new_table() returns an existing table if there is one, but obviously none can exist at init. So a better fix would be:

#define ipmr_for_each_table(mrt, net)					\
	list_for_each_entry_rcu(mrt, &net->ipv4.mr_tables, list,	\
				lockdep_rtnl_is_held() ||		\
				list_empty(&net->ipv4.mr_tables))
Quoted 8 lines

(adding Stephen)

Hi Jakub,

Thank you for your suggestion about this patch.
Here is a stack trace for ipmr.c:

[...]

Thanks!

Quoted 7 lines
Quoted 4 lines
Quoted 1 line

Thoughts?

Do you think a similar fix (the one you suggested) is also applicable in the ip6mr case.

Yes, looking at the code it seems ip6mr has the exact same flow for netns init.

Alright, thanks a lot.
I will send a patch for ip6mr.c soon.

Thank you,
Madhuparna

#6 lore raw

Re: [PATCH net 2/2 RESEND] ipmr: Add lockdep expression to ipmr_for_each_table macro

In reply to Madhuparna Bhowmik
Quoted 47 lines · Madhuparna Bhowmik
Quoted 46 lines · Jakub Kicinski
Quoted 5 lines · On Sat, 9 May 2020 12:52:44 +0530 Amol Grover wrote:

ipmr_for_each_table() uses list_for_each_entry_rcu() for traversing outside of an RCU read-side critical section but under the protection of pernet_ops_rwsem. Hence add the corresponding lockdep expression to silence the following false-positive warning at boot:

Thanks for the fix, the warning has been annoying me as well!

Quoted 5 lines
[    0.645292] =============================
[    0.645294] WARNING: suspicious RCU usage
[    0.645296] 5.5.4-stable #17 Not tainted
[    0.645297] -----------------------------
[    0.645299] net/ipv4/ipmr.c:136 RCU-list traversed in non-reader section!!

please provide a fuller stack trace, it would have helped the review

Quoted 10 lines
---
 net/ipv4/ipmr.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)
net/ipv4/ipmr.c +0 -0
index 99c864eb6e34..950ffe9943da 100644

net/ipv4/ipmr.c static void mroute_clean_tables(struct mr_table *mrt, int flags); · line 115

Quoted 10 lines
net/ipv4/ipmr.c +4 -3
@@ -109,9 +109,10 @@ static void mroute_clean_tables(struct mr_table *mrt, int flags);
 static void ipmr_expire_process(struct timer_list *t);
 
 #ifdef CONFIG_IP_MROUTE_MULTIPLE_TABLES
-#define ipmr_for_each_table(mrt, net) \
-	list_for_each_entry_rcu(mrt, &net->ipv4.mr_tables, list, \
-				lockdep_rtnl_is_held())
+#define ipmr_for_each_table(mrt, net)					\
+	list_for_each_entry_rcu(mrt, &net->ipv4.mr_tables, list,	\
+				lockdep_rtnl_is_held() ||		\
+				lockdep_is_held(&pernet_ops_rwsem))

This is a strange condition, IMHO. How can we be fine with either lock.. This is supposed to be the writer side lock, one can't have two writer side locks..

I think what is happening is this:

ipmr_net_init() -> ipmr_rules_init() -> ipmr_new_table()

Quoted 8 lines
Quoted 8 lines

ipmr_new_table() returns an existing table if there is one, but obviously none can exist at init. So a better fix would be:

#define ipmr_for_each_table(mrt, net)					\
	list_for_each_entry_rcu(mrt, &net->ipv4.mr_tables, list,	\
				lockdep_rtnl_is_held() ||		\
				list_empty(&net->ipv4.mr_tables))

Jakub, I agree, this condition looks better (and correct) than the one I proposed. I'll do the changes as necessary. Also, do you want me to add the full trace to the git commit body as well? I omitted it on purpose to not make it messy.

Quoted 43 lines

(adding Stephen)

Hi Jakub,

Thank you for your suggestion about this patch.
Here is a stack trace for ipmr.c:

[    1.515015] TCP: Hash tables configured (established 8192 bind 8192)
[    1.516790] UDP hash table entries: 512 (order: 3, 49152 bytes, linear)
[    1.518177] UDP-Lite hash table entries: 512 (order: 3, 49152 bytes, linear)
[    1.519805]
[    1.520178] =============================
[    1.520982] WARNING: suspicious RCU usage
[    1.521798] 5.7.0-rc2-00006-gb35af6a26b7c6f #1 Not tainted
[    1.522910] -----------------------------
[    1.523671] net/ipv4/ipmr.c:136 RCU-list traversed in non-reader section!!
[    1.525218]
[    1.525218] other info that might help us debug this:
[    1.525218]
[    1.526731]
[    1.526731] rcu_scheduler_active = 2, debug_locks = 1
[    1.528004] 1 lock held by swapper/1:
[    1.528714]  #0: c20be1d8 (pernet_ops_rwsem){+.+.}-{3:3}, at: register_pernet_subsys+0xd/0x30
[    1.530433]
[    1.530433] stack backtrace:
[    1.531262] CPU: 0 PID: 1 Comm: swapper Not tainted 5.7.0-rc2-00006-gb35af6a26b7c6f #1
[    1.532729] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014
[    1.534305] Call Trace:
[    1.534758]  ? ipmr_get_table+0x3c/0x70
[    1.535430]  ? ipmr_new_table+0x1c/0x60
[    1.536173]  ? ipmr_net_init+0x7b/0x170
[    1.536923]  ? register_pernet_subsys+0xd/0x30
[    1.537810]  ? ops_init+0x1a0/0x1e0
[    1.538518]  ? kmem_cache_create_usercopy+0x28a/0x350
[    1.539752]  ? register_pernet_operations+0xc9/0x1c0
[    1.540630]  ? ipv4_offload_init+0x65/0x65
[    1.541451]  ? register_pernet_subsys+0x19/0x30
[    1.542357]  ? ip_mr_init+0x28/0xff
[    1.543079]  ? inet_init+0x17b/0x249
[    1.543773]  ? do_one_initcall+0xc5/0x240
[    1.544532]  ? parse_args+0x192/0x350
[    1.545266]  ? rcu_read_lock_sched_held+0x2f/0x60
[    1.546180]  ? trace_initcall_level+0x61/0x93
Quoted 9 lines
[    1.547061]  ? kernel_init_freeable+0x112/0x18a
[    1.547978]  ? kernel_init_freeable+0x12b/0x18a
[    1.548974]  ? rest_init+0x220/0x220
[    1.549792]  ? kernel_init+0x8/0x100
[    1.550548]  ? rest_init+0x220/0x220
[    1.551288]  ? schedule_tail_wrapper+0x6/0x8
[    1.552136]  ? rest_init+0x220/0x220
[    1.552873]  ? ret_from_fork+0x2e/0x38

Thank you for the stacktrace Madhuparna.

Quoted 49 lines

ALso, there is a similar warning for ip6mr.c :

=============================
WARNING: suspicious RCU usage 5.7.0-rc4-next-20200507-syzkaller #0 Not tainted
-----------------------------
net/ipv6/ip6mr.c:124 RCU-list traversed in non-reader section!!

other info that might help us debug this:

rcu_scheduler_active = 2, debug_locks = 1
1 lock held by swapper/0/1:
#0: ffffffff8a7aae30 (pernet_ops_rwsem){+.+.}-{3:3}, at: register_pernet_subsys+0x16/0x40 net/core/net_namespace.c:1257

stack backtrace:
CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.7.0-rc4-next-20200507-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Call Trace:
__dump_stack lib/dump_stack.c:77 [inline]
dump_stack+0x18f/0x20d lib/dump_stack.c:118
ip6mr_get_table+0x153/0x180 net/ipv6/ip6mr.c:124
ip6mr_new_table+0x1b/0x70 net/ipv6/ip6mr.c:382
ip6mr_rules_init net/ipv6/ip6mr.c:236 [inline]
ip6mr_net_init+0x133/0x3f0 net/ipv6/ip6mr.c:1310
ops_init+0xaf/0x420 net/core/net_namespace.c:151
__register_pernet_operations net/core/net_namespace.c:1140 [inline]
register_pernet_operations+0x346/0x840 net/core/net_namespace.c:1217
register_pernet_subsys+0x25/0x40 net/core/net_namespace.c:1258
ip6_mr_init+0x49/0x152 net/ipv6/ip6mr.c:1363
inet6_init+0x1d7/0x6dc net/ipv6/af_inet6.c:1037
do_one_initcall+0x10a/0x7d0 init/main.c:1159
do_initcall_level init/main.c:1232 [inline]
do_initcalls init/main.c:1248 [inline]
do_basic_setup init/main.c:1268 [inline]
kernel_init_freeable+0x501/0x5ae init/main.c:1454
kernel_init+0xd/0x1bb init/main.c:1359
ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:351

Segment Routing with IPv6
mip6: Mobile IPv6
sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver
ip6_gre: GRE over IPv6 tunneling driver

Quoted 1 line

Thoughts?

Do you think a similar fix (the one you suggested) is also applicable in the ip6mr case.

Thank you,
Madhuparna

maintainer
#7 lore raw

Re: [PATCH net 2/2 RESEND] ipmr: Add lockdep expression to ipmr_for_each_table macro

Quoted 17 lines · On Tue, 12 May 2020 22:47:10 +0530 Amol Grover wrote:
Quoted 8 lines
Quoted 8 lines

This is a strange condition, IMHO. How can we be fine with either lock.. This is supposed to be the writer side lock, one can't have two writer side locks..

I think what is happening is this:

ipmr_net_init() -> ipmr_rules_init() -> ipmr_new_table()

Quoted 8 lines
Quoted 8 lines

ipmr_new_table() returns an existing table if there is one, but obviously none can exist at init. So a better fix would be:

#define ipmr_for_each_table(mrt, net)					\
	list_for_each_entry_rcu(mrt, &net->ipv4.mr_tables, list,	\
				lockdep_rtnl_is_held() ||		\
				list_empty(&net->ipv4.mr_tables))
Quoted 4 lines

Jakub, I agree, this condition looks better (and correct) than the one I proposed. I'll do the changes as necessary. Also, do you want me to add the full trace to the git commit body as well? I omitted it on purpose to not make it messy.

In this case we can leave it at the depth of IPMR code + the caller, so:

[    1.534758]  ? ipmr_get_table+0x3c/0x70
[    1.535430]  ? ipmr_new_table+0x1c/0x60
[    1.536173]  ? ipmr_net_init+0x7b/0x170
[    1.536923]  ? register_pernet_subsys+0xd/0x30

This makes it clear that the problem happens at net namespace init.

Thanks!

#8 lore raw

Re: [PATCH net 2/2 RESEND] ipmr: Add lockdep expression to ipmr_for_each_table macro

In reply to Madhuparna Bhowmik

Hi all,

Quoted 8 lines · On Tue, 12 May 2020 10:47:05 +0530 Madhuparna Bhowmik…
Quoted 8 lines

I think what is happening is this:

ipmr_net_init() -> ipmr_rules_init() -> ipmr_new_table()

ipmr_new_table() returns an existing table if there is one, but obviously none can exist at init. So a better fix would be:

Quoted 6 lines
Quoted 5 lines
#define ipmr_for_each_table(mrt, net)					\
	list_for_each_entry_rcu(mrt, &net->ipv4.mr_tables, list,	\
				lockdep_rtnl_is_held() ||		\
				list_empty(&net->ipv4.mr_tables))

(adding Stephen)

I have changed the patch in my fixes tree to this.

Signature · 2 lines
Cheers,
Stephen Rothwell