Skip to contentSkip to conversation

rtnetlink: Synchronze net in rtnl_unregister()

PATCH In review Dmitry Safonov 2 participants 6 messages Feb 25, 2019 – Feb 26, 2019

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 rtnetlink: Synchronze net in rtnl_unregister()

rtnl_unregister() unsets handler from table, which is protected by rtnl_lock or RCU. At this moment only dump handlers access the table with rcu_lock(). Every other user accesses under rtnl.

Callers may expect that rtnl_unregister() prevents any further handlers calls, alike rtnl_unregister_all(). And they do expect it.

I've looked on in-tree caller uses:
br_mdb: safe, but in err-path br_netlink_init()
fib_rules: safe - err-path is very early in __init
ip6mr: safe - following unregister_pernet_subsys() calls internally rcu_barrier()
qrtr: safe - following sock_unregister() calls internally synchronize_rcu()

While it's possible to document that rtnl_unregister() requires synchronize_net() afterwards - unlike rtnl_unregister_all(), I believe the module exit is very much slow-path.

Issue seems to be very theoretical and unlikely, so I'm not Cc'ing stable tree.

Fixes 6853dd488119 ("rtnetlink: protect handler table with rcu")
Cc "David S. Miller" <[email protected]>
Cc Florian Westphal <[email protected]>
Cc "Hannes Frederic Sowa" <[email protected]>
---
 net/core/rtnetlink.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
net/core/rtnetlink.c +3 -1
index 5ea1bed08ede..3db70da4f951 100644
@@ -308,7 +308,9 @@ int rtnl_unregister(int protocol, int msgtype)
 	rcu_assign_pointer(tab[msgindex], NULL);
 	rtnl_unlock();
 
-	kfree_rcu(link, rcu);
+	synchronize_net();
+
+	kfree(link);
 
 	return 0;
 }
Signature · 1 line
2.20.1
#1 lore raw
Quoted 4 lines · Dmitry Safonov

rtnl_unregister() unsets handler from table, which is protected by rtnl_lock or RCU. At this moment only dump handlers access the table with rcu_lock(). Every other user accesses under rtnl.

Quoted 8 lines

Callers may expect that rtnl_unregister() prevents any further handlers calls, alike rtnl_unregister_all(). And they do expect it.

I've looked on in-tree caller uses:
br_mdb: safe, but in err-path br_netlink_init()
fib_rules: safe - err-path is very early in __init
ip6mr: safe - following unregister_pernet_subsys() calls internally rcu_barrier()
qrtr: safe - following sock_unregister() calls internally synchronize_rcu()

If rcu_barrier() was needed, then all callers should use it.

If synchronize_rcu() was needed, then all callers should use it.

But mixing is probably wrong.

Quoted 4 lines

While it's possible to document that rtnl_unregister() requires synchronize_net() afterwards - unlike rtnl_unregister_all(), I believe the module exit is very much slow-path.

rtnl_unregister_all() needs the sychronize_rcu() at this moment because of the kfree(tab), not because of the kfree_rcu(link, rcu);

Quoted 19 lines

Issue seems to be very theoretical and unlikely, so I'm not Cc'ing stable tree.

Fixes 6853dd488119 ("rtnetlink: protect handler table with rcu")
Cc "David S. Miller" <[email protected]>
Cc Florian Westphal <[email protected]>
Cc "Hannes Frederic Sowa" <[email protected]>
---
 net/core/rtnetlink.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
net/core/rtnetlink.c +0 -0
index 5ea1bed08ede..3db70da4f951 100644

net/core/rtnetlink.c int rtnl_unregister(int protocol, int msgtype) · line 313

Quoted 7 lines
net/core/rtnetlink.c +3 -1
@@ -308,7 +308,9 @@ int rtnl_unregister(int protocol, int msgtype)
 	rcu_assign_pointer(tab[msgindex], NULL);
 	rtnl_unlock();
 
-	kfree_rcu(link, rcu);
+	synchronize_net();
+
+	kfree(link);

I really do not see a difference here (other than this being much slower of course)

If the caller needs rcu_barrier(), then add it in the caller ?

#2 lore raw

Hi Eric,

Quoted 7 lines · Eric Dumazet
Quoted 3 lines · Dmitry Safonov

While it's possible to document that rtnl_unregister() requires synchronize_net() afterwards - unlike rtnl_unregister_all(), I believe the module exit is very much slow-path.

rtnl_unregister_all() needs the sychronize_rcu() at this moment because of the kfree(tab), not because of the kfree_rcu(link, rcu);

I may be wrong here, but shouldn't we wait for grace period to elapse by the reason that rtnl_msg_handlers are protected by RCU, not only by rtnl? Like, without synchronize_net() in rtnl_unregister() - what prevents module exit race to say, rtnetlink_rcv_msg()=>rtnl_get_link()?

Quoted 14 lines

net/core/rtnetlink.c int rtnl_unregister(int protocol, int msgtype) · line 313

Quoted 10 lines
net/core/rtnetlink.c +3 -1
@@ -308,7 +308,9 @@ int rtnl_unregister(int protocol, int msgtype)
 	rcu_assign_pointer(tab[msgindex], NULL);
 	rtnl_unlock();
 
-	kfree_rcu(link, rcu);
+	synchronize_net();
+
+	kfree(link);

I really do not see a difference here (other than this being much slower of course)

Quoted 1 line

If the caller needs rcu_barrier(), then add it in the caller ?

Well, sure - but it seems confusing that rtnl_unregister() will require synchronize_rcu(), while rtnl_unregister_all() will not.
And I thought no one would care about another synchronize_rcu() in exit path.

Thanks,
Dmitry

#3 lore raw
Quoted 8 lines · Dmitry Safonov

Hi Eric,

Quoted 5 lines · Eric Dumazet
Quoted 3 lines · Dmitry Safonov

While it's possible to document that rtnl_unregister() requires synchronize_net() afterwards - unlike rtnl_unregister_all(), I believe the module exit is very much slow-path.

Quoted 7 lines
Quoted 2 lines

rtnl_unregister_all() needs the sychronize_rcu() at this moment because of the kfree(tab), not because of the kfree_rcu(link, rcu);

I may be wrong here, but shouldn't we wait for grace period to elapse by the reason that rtnl_msg_handlers are protected by RCU, not only by rtnl? Like, without synchronize_net() in rtnl_unregister() - what prevents module exit race to say, rtnetlink_rcv_msg()=>rtnl_get_link()?

There is synchronize_rcu() in free_module() - but I believe it's a bit too far in unloading. Maybe, I'm missing another call on the way.

Thanks,
Dmitry

#4 lore raw
In reply to Dmitry Safonov
Quoted 31 lines · Dmitry Safonov

Hi Eric,

Quoted 7 lines · Eric Dumazet
Quoted 3 lines · Dmitry Safonov

While it's possible to document that rtnl_unregister() requires synchronize_net() afterwards - unlike rtnl_unregister_all(), I believe the module exit is very much slow-path.

rtnl_unregister_all() needs the sychronize_rcu() at this moment because of the kfree(tab), not because of the kfree_rcu(link, rcu);

I may be wrong here, but shouldn't we wait for grace period to elapse by the reason that rtnl_msg_handlers are protected by RCU, not only by rtnl? Like, without synchronize_net() in rtnl_unregister() - what prevents module exit race to say, rtnetlink_rcv_msg()=>rtnl_get_link()?

Quoted 14 lines

net/core/rtnetlink.c int rtnl_unregister(int protocol, int msgtype) · line 313

Quoted 10 lines
net/core/rtnetlink.c +3 -1
@@ -308,7 +308,9 @@ int rtnl_unregister(int protocol, int msgtype)
 	rcu_assign_pointer(tab[msgindex], NULL);
 	rtnl_unlock();
 
-	kfree_rcu(link, rcu);
+	synchronize_net();
+
+	kfree(link);

I really do not see a difference here (other than this being much slower of course)

Quoted 4 lines
Quoted 1 line

If the caller needs rcu_barrier(), then add it in the caller ?

Well, sure - but it seems confusing that rtnl_unregister() will require synchronize_rcu(), while rtnl_unregister_all() will not.

rtnl_unregister_all() is a different beast, since it removes the whole rtnl_msg_handlers[protocol]

rtnl_unregister() only removes a subset, with different usages.

Quoted 2 lines

And I thought no one would care about another synchronize_rcu() in exit path.

We definitely care about things be done properly.

If synchronize_rcu() is needed there, be it, but kfree_rcu() seems to be fine.

In any case, I believe you need to more carefully explain what is the problem here, because I could not really see it.

#5 lore raw
Quoted 11 lines · Eric Dumazet
Quoted 2 lines · Dmitry Safonov

Well, sure - but it seems confusing that rtnl_unregister() will require synchronize_rcu(), while rtnl_unregister_all() will not.

rtnl_unregister_all() is a different beast, since it removes the whole rtnl_msg_handlers[protocol]

rtnl_unregister() only removes a subset, with different usages.

Quoted 2 lines

And I thought no one would care about another synchronize_rcu() in exit path.

Quoted 6 lines

We definitely care about things be done properly.

If synchronize_rcu() is needed there, be it, but kfree_rcu() seems to be fine.

In any case, I believe you need to more carefully explain what is the problem here, because I could not really see it.

Ugh, sorry - it seems that I haven't had enough coffee today.

I've read again rtnetlink_rcv_msg(), who is the only user of rtnl_msg_handlers[] under read_lock, and it calls try_module_get(owner).

Sorry for the noise,
Dmitry