rds: mark bound socket with SOCK_RCU_FREE
Subsystems (3)
PATCH 1/1 rds: mark bound socket with SOCK_RCU_FREE
When a rds sock is bound, it is inserted into the bind_hash_table which is protected by RCU. But when releasing rd sock, after it is removed from this hash table, it is freed immediately without respecting RCU grace period. This could cause some use-after-free as reported by syzbot.
Mark the rds sock as SOCK_RCU_FREE before inserting it into the bind_hash_table, so that it would be always freed after a RCU grace period.
--- net/rds/bind.c | 1 + 1 file changed, 1 insertion(+)
net/rds/bind.c +1 -0
@@ -235,6 +235,7 @@ int rds_bind(struct socket *sock, struct sockaddr *uaddr, int addr_len) goto out; } +sock_set_flag (sk,SOCK_RCU_FREE ); ret =rds_add_bound (rs, binding_addr, &port, scope_id);if (ret) goto out;
Signature · 1 line
2.14.4
Quoted 7 lines · Cong Wang
When a rds sock is bound, it is inserted into the
bind_hash_tablewhich is protected by RCU. But when releasing rd sock, after it is removed from this hash table, it is freed immediately without respecting RCU grace period. This could cause some use-after-free as reported by syzbot.
I have no objection to the change itself, but the syzbot failures are caused for a very simple reason: we need synchronize_net() in rds_release before we remove the rds_sock from the bind_hash_table.
I already pointed this out in
https://www.spinics.net/lists/netdev/msg475074.html
I think the objection to synchronize_net() is that it can cause perf issues (I'm told that rds_release() has been known to be held up by other threads in rcu critical sections?) but I personally dont see any other alternative to this (other than going back to rwlock, instead of rcu)
--Sowmini
Quoted 6 lines · Cong Wang
When a rds sock is bound, it is inserted into the
bind_hash_tablewhich is protected by RCU. But when releasing rd sock, after it is removed from this hash table, it is freed immediately without respecting RCU grace period. This could cause some use-after-free as reported by syzbot.
Indeed.
Quoted 19 lines
Mark the rds sock as
SOCK_RCU_FREEbefore inserting it into thebind_hash_table, so that it would be always freed after a RCU grace period.Reported-and-tested-by [email protected]Cc Sowmini Varadhan <[email protected]>Cc Santosh Shilimkar <[email protected]>Signed-off-by Cong Wang <[email protected]>--- net/rds/bind.c | 1 + 1 file changed, 1 insertion(+)net/rds/bind.c +0
-0
net/rds/bind.c int rds_bind(struct socket *sock, struct sockaddr *uaddr, int addr_len) · line 241
Quoted 7 lines
net/rds/bind.c +1
-0@@ -235,6 +235,7 @@ int rds_bind(struct socket *sock, struct sockaddr *uaddr, int addr_len) goto out; } +sock_set_flag (sk,SOCK_RCU_FREE ); ret =rds_add_bound (rs, binding_addr, &port, scope_id);if (ret) goto out;
I wasn't aware of this "SOCK_RCU_FREE" so really thanks for this patch. Have been scratching my head over this for a while thinking about generic provision at sk level to synchronize. This is much
better than adding the sync at upper layer.
It does have the tax for slowing down RDS for other kernel components rcu sync but anyway this hole needs to be plugged so am fine to go ahead with this change. Thanks for the patch.
FWIW,
Quoted 12 lines · Santosh Shilimkar
Quoted 6 lines · Cong Wang
When a rds sock is bound, it is inserted into the
bind_hash_tablewhich is protected by RCU. But when releasing rd sock, after it is removed from this hash table, it is freed immediately without respecting RCU grace period. This could cause some use-after-free as reported by syzbot.Indeed.
Quoted 3 lines
Mark the rds sock as
SOCK_RCU_FREEbefore inserting it into thebind_hash_table, so that it would be always freed after a RCU grace period.
So I'm not sure I understand.
Yes, Cong's fix may eliminate *some* of the syzbot failures, but the basic problem is not solved.
To take one example of possible races (one that was discussed in
https://www.spinics.net/lists/netdev/msg475074.html)rds_recv_incoming->rds_find_bound is being called in rds_send_worker context and the rds_find_bound code is
63 rs = rhashtable_lookup_fast(&bind_hash_table, &key, ht_parms);
64 if (rs && !sock_flag(rds_rs_to_sk(rs), SOCK_DEAD))
65 rds_sock_addref(rs);
66 else
67 rs = NULL;
68 After we find an rs at line 63, how can we be sure that the entire logic of rds_release does not execute on another cpu, and free the rs, before we hit line 64 with the bad rs?
Normally synchronize_rcu() or synchronize_net() in rds_release() would ensure this. How do we ensure this with SOCK_RCU_FREE (or is the intention to just reduce *some* of the syzbot failures)?
--Sowmini
Quoted 32 lines · Sowmini Varadhan
Quoted 12 lines · Santosh Shilimkar
Quoted 6 lines · Cong Wang
When a rds sock is bound, it is inserted into the
bind_hash_tablewhich is protected by RCU. But when releasing rd sock, after it is removed from this hash table, it is freed immediately without respecting RCU grace period. This could cause some use-after-free as reported by syzbot.Indeed.
Quoted 3 lines
Mark the rds sock as
SOCK_RCU_FREEbefore inserting it into thebind_hash_table, so that it would be always freed after a RCU grace period.So I'm not sure I understand.
Yes, Cong's fix may eliminate *some* of the syzbot failures, but the basic problem is not solved.
To take one example of possible races (one that was discussed in
https://www.spinics.net/lists/netdev/msg475074.html)rds_recv_incoming->rds_find_boundis being called inrds_send_workercontext and therds_find_boundcode is63 rs = rhashtable_lookup_fast(&bind_hash_table, &key, ht_parms); 64 if (rs && !sock_flag(rds_rs_to_sk(rs), SOCK_DEAD)) 65 rds_sock_addref(rs); 66 else 67 rs = NULL; 68
Quoted 3 lines
After we find an rs at line 63, how can we be sure that the entire logic of
rds_releasedoes not execute on another cpu, and free the rs, before we hit line 64 with the bad rs?
This is a different problem. The RCU grace period should be just extended:
net/rds/bind.c +0 -0
@@ -76,11 +76,13 @@ struct rds_sock *rds_find_bound(const struct
in6_addr *addr, __be16 port,
struct rds_sock *rs;
__rds_create_bind_key(key, addr, port, scope_id);
- rs = rhashtable_lookup_fast(&bind_hash_table, key, ht_parms);
+ rcu_read_lock();
+ rs = rhashtable_lookup(&bind_hash_table, key, ht_parms);
if (rs && !sock_flag(rds_rs_to_sk(rs), SOCK_DEAD))
rds_sock_addref(rs);
else
rs = NULL;+ rcu_read_unlock();
rdsdebug("returning rs %p for %pI6c:%u\n", rs, addr,
ntohs(port));
Quoted 4 lines
Normally
synchronize_rcu()orsynchronize_net()inrds_release()would ensure this. How do we ensure this withSOCK_RCU_FREE(or is the intention to just reduce *some* of the syzbot failures)?
Although sock release path is not a very hot path, but blocking it isn't a good idea either, especially when you can use call_rcu(), which has the same effect.
I don't see any reason we should prefer synchronize_rcu() here.
Review
Quoted 10 lines · Cong Wang
__rds_create_bind_key(key, addr, port, scope_id);(quoted diff) +3
-1- rs = rhashtable_lookup_fast(&bind_hash_table, key, ht_parms); + rcu_read_lock(); + rs = rhashtable_lookup(&bind_hash_table, key, ht_parms); if (rs && !sock_flag(rds_rs_to_sk(rs), SOCK_DEAD)) rds_sock_addref(rs); else rs = NULL; + rcu_read_unlock();
aiui, the rcu_read lock/unlock is only useful if the write side doing destructive operations does something to make sure readers are done before doing the destructive opertion. AFAIK, that does not exist for rds socket management today
Quoted 5 lines
Although sock release path is not a very hot path, but blocking it isn't a good idea either, especially when you can use
call_rcu(), which has the same effect.I don't see any reason we should prefer
synchronize_rcu()here.
Usually correctness (making sure all readers are done, before nuking a data structure) is a little bit more important than perforamance, aka "safety before speed" is what I've always been taught.
Clearly, your mileage varies. As you please.
--Sowmini
Quoted 13 lines · Sowmini Varadhan
Review
Quoted 10 lines · Cong Wang
__rds_create_bind_key(key, addr, port, scope_id);(quoted diff) +3
-1- rs = rhashtable_lookup_fast(&bind_hash_table, key, ht_parms); + rcu_read_lock(); + rs = rhashtable_lookup(&bind_hash_table, key, ht_parms); if (rs && !sock_flag(rds_rs_to_sk(rs), SOCK_DEAD)) rds_sock_addref(rs); else rs = NULL; + rcu_read_unlock();
Quoted 4 lines
aiui, the
rcu_readlock/unlock is only useful if the write side doing destructive operations does something to make sure readers are done before doing the destructive opertion. AFAIK, that does not exist for rds socket management today
That is exactly why we need it here, right?
As you mentioned previously, the sock could be freed after rhashtable_lookup_fast() but before rds_sock_addref(), extending the RCU read section after rds_sock_addref() exactly solves the problem here.
Or is there any other destructive problem you didn't mention? Mind to be specific?
Quoted 6 lines
Quoted 6 lines
Although sock release path is not a very hot path, but blocking it isn't a good idea either, especially when you can use
call_rcu(), which has the same effect.
Quoted 6 lines
Quoted 1 line
I don't see any reason we should prefer
synchronize_rcu()here.Usually correctness (making sure all readers are done, before nuking a data structure) is a little bit more important than perforamance, aka "safety before speed" is what I've always been taught.
Hmm, so you are saying synchronize_rcu() is kinda more correct than call_rcu()?? I never hear this before, would like to know why.
To my best knowledge, the only difference between them is the context, one is blocking, the other is non-blocking. Their correctness must be equivalent.
Quoted 13 lines · Cong Wang
Quoted 13 lines · Cong Wang
Quoted 10 lines
__rds_create_bind_key(key, addr, port, scope_id);(quoted diff) +3
-1- rs = rhashtable_lookup_fast(&bind_hash_table, key, ht_parms); + rcu_read_lock(); + rs = rhashtable_lookup(&bind_hash_table, key, ht_parms); if (rs && !sock_flag(rds_rs_to_sk(rs), SOCK_DEAD)) rds_sock_addref(rs); else rs = NULL; + rcu_read_unlock();
Quoted 6 lines
Quoted 4 lines
aiui, the
rcu_readlock/unlock is only useful if the write side doing destructive operations does something to make sure readers are done before doing the destructive opertion. AFAIK, that does not exist for rds socket management todayThat is exactly why we need it here, right?
Maybe I am confused, what exactly is the patch you are proposing?
Does it have the SOCK_RCU_FREE change?
Does it have the rcu_read_lock you have above? Where is the call_rcu?
Quoted 2 lines
Hmm, so you are saying
synchronize_rcu()is kinda more correct thancall_rcu()??
I'm not saying that, I'm asking "what exactly is the patch you are proposing?" The only one on record is
http://patchwork.ozlabs.org/patch/968282/
which does not have either synchronize_rcu or call_rcu.
Quoted 1 line
I never hear this before, would like to know why.
Please post precise patches first.
Thanks.
Quoted 20 lines · Sowmini Varadhan
Quoted 18 lines · Cong Wang
Quoted 17 lines · Cong Wang
Review
Quoted 10 lines
__rds_create_bind_key(key, addr, port, scope_id);(quoted diff) +3
-1- rs = rhashtable_lookup_fast(&bind_hash_table, key, ht_parms); + rcu_read_lock(); + rs = rhashtable_lookup(&bind_hash_table, key, ht_parms); if (rs && !sock_flag(rds_rs_to_sk(rs), SOCK_DEAD)) rds_sock_addref(rs); else rs = NULL; + rcu_read_unlock();aiui, the
rcu_readlock/unlock is only useful if the write side doing destructive operations does something to make sure readers are done before doing the destructive opertion. AFAIK, that does not exist for rds socket management today
Quoted 5 lines
Quoted 1 line
That is exactly why we need it here, right?
Maybe I am confused, what exactly is the patch you are proposing?
Does it have the
SOCK_RCU_FREEchange?
Yes, that patch is obviously on top of this patch.
Quoted 2 lines
Does it have the
rcu_read_lockyou have above?
Where is thecall_rcu?
Sure, as it is on top of this patch, the call_rcu() is here:
void sk_destruct(struct sock *sk)
{
if (sock_flag(sk, SOCK_RCU_FREE))
call_rcu(&sk->sk_rcu, __sk_destruct);
else
__sk_destruct(&sk->sk_rcu);
}Quoted 9 lines
Quoted 3 lines
Hmm, so you are saying
synchronize_rcu()is kinda more correct thancall_rcu()??I'm not saying that, I'm asking "what exactly is the patch you are proposing?" The only one on record is
http://patchwork.ozlabs.org/patch/968282/
which does not have eithersynchronize_rcuorcall_rcu.
It is very obviously on top of this patch ($subject).
Quoted 4 lines
Quoted 2 lines
I never hear this before, would like to know why.
Please post precise patches first.
Already showed you precisely what is is, just on top of this one.
Quoted 20 lines · Cong Wang
Quoted 13 lines · Sowmini Varadhan
Review
Quoted 10 lines · Cong Wang
__rds_create_bind_key(key, addr, port, scope_id);(quoted diff) +3
-1- rs = rhashtable_lookup_fast(&bind_hash_table, key, ht_parms); + rcu_read_lock(); + rs = rhashtable_lookup(&bind_hash_table, key, ht_parms); if (rs && !sock_flag(rds_rs_to_sk(rs), SOCK_DEAD)) rds_sock_addref(rs); else rs = NULL; + rcu_read_unlock();Quoted 4 lines
aiui, the
rcu_readlock/unlock is only useful if the write side doing destructive operations does something to make sure readers are done before doing the destructive opertion. AFAIK, that does not exist for rds socket management today
Quoted 7 lines
That is exactly why we need it here, right?
As you mentioned previously, the sock could be freed after
rhashtable_lookup_fast()but beforerds_sock_addref(), extending the RCU read section afterrds_sock_addref()exactly solves the problem here.
Thats the case really.
Quoted 17 lines
Or is there any other destructive problem you didn't mention? Mind to be specific?
Quoted 6 lines
Quoted 6 lines
Although sock release path is not a very hot path, but blocking it isn't a good idea either, especially when you can use
call_rcu(), which has the same effect.Quoted 6 lines
Quoted 1 line
I don't see any reason we should prefer
synchronize_rcu()here.Usually correctness (making sure all readers are done, before nuking a data structure) is a little bit more important than perforamance, aka "safety before speed" is what I've always been taught.
Quoted 7 lines
Hmm, so you are saying
synchronize_rcu()is kinda more correct thancall_rcu()?? I never hear this before, would like to know why.To my best knowledge, the only difference between them is the context, one is blocking, the other is non-blocking. Their correctness must be equivalent.
We have burn our hands with blocking synchronize_rcu() and that was actually the main reason we moved to rw locks from rcu to localise the cost. call_rcu()should be just fine as long as it plugs the hole. I don't want to add blocking in this path since this slows down the connection shutdown path and at times lead to soft dumps.
Would you mind posting an updated patch please with call_rcu and above extended RCU grace period with rcu_read_lock. Thanks !!
Regards,
Santosh
Regards,
Santosh
Quoted 2 lines · Santosh Shilimkar
Would you mind posting an updated patch please with
call_rcuand above extended RCU grace period withrcu_read_lock. Thanks !!
If you prefer to fix _two_ problems in one patch, sure.
For the record, the bug this patch fixes is NOT same with the one in rds_find_bound(), because there is no rds_find_bound() in the backtrace. If you want to see the full backtrace, here it is:
https://marc.info/?l=linux-netdev&m=153644444808962&w=2
This is why I believe they are two problems.
Whether fixing two problems in one patch or two patches is merely a preference, I leave it up to you.
Quoted 11 lines · Cong Wang
Quoted 2 lines · Santosh Shilimkar
Would you mind posting an updated patch please with
call_rcuand above extended RCU grace period withrcu_read_lock. Thanks !!If you prefer to fix _two_ problems in one patch, sure.
For the record, the bug this patch fixes is NOT same with the one in
rds_find_bound(), because there is nords_find_bound()in the backtrace. If you want to see the full backtrace, here it is:
Quoted 7 lines
https://marc.info/?l=linux-netdev&m=153644444808962&w=2
This is why I believe they are two problems.
Whether fixing two problems in one patch or two patches is merely a preference, I leave it up to you.
I had a partial fix as well in past but since it wasn't covering complete window, it was abandoned.
Since we know the other race, it would be best to address it together and hence requested a combine patch. Thanks for help !!
Regards,
Santosh
Quoted 17 lines · Santosh Shilimkar
Quoted 15 lines · Cong Wang
Quoted 2 lines · Santosh Shilimkar
Would you mind posting an updated patch please with
call_rcuand above extended RCU grace period withrcu_read_lock. Thanks !!If you prefer to fix _two_ problems in one patch, sure.
For the record, the bug this patch fixes is NOT same with the one in
rds_find_bound(), because there is nords_find_bound()in the backtrace. If you want to see the full backtrace, here it is:https://marc.info/?l=linux-netdev&m=153644444808962&w=2
This is why I believe they are two problems.
Quoted 8 lines
Quoted 3 lines
Whether fixing two problems in one patch or two patches is merely a preference, I leave it up to you.
I had a partial fix as well in past but since it wasn't covering complete window, it was abandoned.
Since we know the other race, it would be best to address it together and hence requested a combine patch. Thanks for help !!
No problem! v2 is coming shortly after passing syzbot test. :)