RDMA/rxe: Fix unsafe socket release during namespace cleanup
Subsystems (2)
PATCH 1/1 RDMA/rxe: Fix unsafe socket release during namespace cleanup
Since all the sockets are created in rdma link create command and destroyed in rdma link delete command, keeping udp_tunnel_sock_release in rxe_ns_exit risks a "double-free" if the namespace and the device are being cleaned up simultaneously.
--- drivers/infiniband/sw/rxe/rxe_ns.c | 20 -------------------- 1 file changed, 20 deletions(-)
drivers/infiniband/sw/rxe/rxe_ns.c +0 -20
@@ -39,26 +39,6 @@ static void rxe_ns_exit(struct net *net) {/* called when the network namespace is removed */ -struct rxe_ns_sock *ns_sk =net_generic (net, rxe_pernet_id); -struct sock *sk; - -rcu_read_lock (); - sk =rcu_dereference (ns_sk->rxe_sk4); -rcu_read_unlock (); -if (sk) { -rcu_assign_pointer (ns_sk->rxe_sk4,NULL ); -udp_tunnel_sock_release (sk->sk_socket); - } - -#if IS_ENABLED (CONFIG_IPV6 ) -rcu_read_lock (); - sk =rcu_dereference (ns_sk->rxe_sk6); -rcu_read_unlock (); -if (sk) { -rcu_assign_pointer (ns_sk->rxe_sk6,NULL ); -udp_tunnel_sock_release (sk->sk_socket); - } -#endif }/*
Signature · 1 line
2.43.0
Quoted 4 lines · Zhu Yanjun
Since all the sockets are created in rdma link create command and destroyed in rdma link delete command, keeping
udp_tunnel_sock_releaseinrxe_ns_exitrisks a "double-free" if the namespace and the device are being cleaned up simultaneously.
Please add a ladder diagram to clarify how it can be possible.
Thanks
Quoted 42 lines
Fixes 13f2a53c2a71 ("RDMA/rxe: Add net namespace support for IPv4/IPv6 sockets")Signed-off-by Zhu Yanjun <[email protected]>--- drivers/infiniband/sw/rxe/rxe_ns.c | 20 -------------------- 1 file changed, 20 deletions(-)drivers/infiniband/sw/rxe/rxe_ns.c +0
-20@@ -39,26 +39,6 @@ static void rxe_ns_exit(struct net *net) {/* called when the network namespace is removed */ -struct rxe_ns_sock *ns_sk =net_generic (net, rxe_pernet_id); -struct sock *sk; - -rcu_read_lock (); - sk =rcu_dereference (ns_sk->rxe_sk4); -rcu_read_unlock (); -if (sk) { -rcu_assign_pointer (ns_sk->rxe_sk4,NULL ); -udp_tunnel_sock_release (sk->sk_socket); - } - -#if IS_ENABLED (CONFIG_IPV6 ) -rcu_read_lock (); - sk =rcu_dereference (ns_sk->rxe_sk6); -rcu_read_unlock (); -if (sk) { -rcu_assign_pointer (ns_sk->rxe_sk6,NULL ); -udp_tunnel_sock_release (sk->sk_socket); - } -#endif }/* Signature · 1 line
2.43.0
Quoted 7 lines · Leon Romanovsky
Quoted 4 lines · Zhu Yanjun
Since all the sockets are created in rdma link create command and destroyed in rdma link delete command, keeping
udp_tunnel_sock_releaseinrxe_ns_exitrisks a "double-free" if the namespace and the device are being cleaned up simultaneously.Please add a ladder diagram to clarify how it can be possible.
Hi, Leon
The double-free occurs as follows:
CPU 0 (Net NameSpace cleanup) CPU 1 (RDMA device removal) --------------------- --------------------------- rxe_ns_exit() rxe_link_delete() (rdma link del )
-> sk = ns_sk->rxe_sk4 -> sk = ns_sk->rxe_sk4
-> udp_tunnel_sock_release(sk)
[Success: First Free] -> udp_tunnel_sock_release(sk)
[Crash: Double Free]
After removing the socket release logic from rxe_ns_exit(), we ensure that only the device destruction path (rxe_link_delete) is responsible for freeing the tunnel sockets, effectively eliminating the double-free problem.
I am not sure if I should put the above into the commit log.
Thanks a lot.
Zhu Yanjun
Quoted 45 lines
Thanks
Quoted 42 lines
Fixes 13f2a53c2a71 ("RDMA/rxe: Add net namespace support for IPv4/IPv6 sockets")Signed-off-by Zhu Yanjun <[email protected]>--- drivers/infiniband/sw/rxe/rxe_ns.c | 20 -------------------- 1 file changed, 20 deletions(-)drivers/infiniband/sw/rxe/rxe_ns.c +0
-20@@ -39,26 +39,6 @@ static void rxe_ns_exit(struct net *net) {/* called when the network namespace is removed */ -struct rxe_ns_sock *ns_sk =net_generic (net, rxe_pernet_id); -struct sock *sk; - -rcu_read_lock (); - sk =rcu_dereference (ns_sk->rxe_sk4); -rcu_read_unlock (); -if (sk) { -rcu_assign_pointer (ns_sk->rxe_sk4,NULL ); -udp_tunnel_sock_release (sk->sk_socket); - } - -#if IS_ENABLED (CONFIG_IPV6 ) -rcu_read_lock (); - sk =rcu_dereference (ns_sk->rxe_sk6); -rcu_read_unlock (); -if (sk) { -rcu_assign_pointer (ns_sk->rxe_sk6,NULL ); -udp_tunnel_sock_release (sk->sk_socket); - } -#endif }/* Signature · 1 line
2.43.0
Quoted 21 lines · yanjun.zhu
Quoted 7 lines · Leon Romanovsky
Quoted 4 lines · Zhu Yanjun
Since all the sockets are created in rdma link create command and destroyed in rdma link delete command, keeping
udp_tunnel_sock_releaseinrxe_ns_exitrisks a "double-free" if the namespace and the device are being cleaned up simultaneously.Please add a ladder diagram to clarify how it can be possible.
Hi, Leon
The double-free occurs as follows:
CPU 0 (Net NameSpace cleanup) CPU 1 (RDMA device removal) --------------------- --------------------------- rxe_ns_exit() rxe_link_delete() (rdma link del )-> sk =
ns_sk->rxe_sk4-> sk =ns_sk->rxe_sk4
->udp_tunnel_sock_release(sk)
[Success: First Free] ->udp_tunnel_sock_release(sk)
[Crash: Double Free]
Quoted 4 lines
After removing the socket release logic from
rxe_ns_exit(), we ensure that only the device destruction path (rxe_link_delete) is responsible for freeing the tunnel sockets, effectively eliminating the double-free problem.
I think it is possible to call rxe_ns_exit() without invoking rxe_link_delete(), and in that case the UDP socket will not be destroyed.
Thanks
Quoted 52 lines
I am not sure if I should put the above into the commit log.
Thanks a lot.
Zhu Yanjun
Quoted 44 lines
Thanks
Quoted 41 lines
Fixes 13f2a53c2a71 ("RDMA/rxe: Add net namespace support for IPv4/IPv6 sockets")Signed-off-by Zhu Yanjun <[email protected]>--- drivers/infiniband/sw/rxe/rxe_ns.c | 20 -------------------- 1 file changed, 20 deletions(-)drivers/infiniband/sw/rxe/rxe_ns.c +0
-20@@ -39,26 +39,6 @@ static void rxe_ns_exit(struct net *net) {/* called when the network namespace is removed */ -struct rxe_ns_sock *ns_sk =net_generic (net, rxe_pernet_id); -struct sock *sk; - -rcu_read_lock (); - sk =rcu_dereference (ns_sk->rxe_sk4); -rcu_read_unlock (); -if (sk) { -rcu_assign_pointer (ns_sk->rxe_sk4,NULL ); -udp_tunnel_sock_release (sk->sk_socket); - } - -#if IS_ENABLED (CONFIG_IPV6 ) -rcu_read_lock (); - sk =rcu_dereference (ns_sk->rxe_sk6); -rcu_read_unlock (); -if (sk) { -rcu_assign_pointer (ns_sk->rxe_sk6,NULL ); -udp_tunnel_sock_release (sk->sk_socket); - } -#endif }/* Signature · 1 line
2.43.0
Quoted 22 lines · Leon Romanovsky
Quoted 21 lines · yanjun.zhu
Quoted 7 lines · Leon Romanovsky
Quoted 4 lines · Zhu Yanjun
Since all the sockets are created in rdma link create command and destroyed in rdma link delete command, keeping
udp_tunnel_sock_releaseinrxe_ns_exitrisks a "double-free" if the namespace and the device are being cleaned up simultaneously.Please add a ladder diagram to clarify how it can be possible.
Hi, Leon
The double-free occurs as follows:
CPU 0 (Net NameSpace cleanup) CPU 1 (RDMA device removal) --------------------- --------------------------- rxe_ns_exit() rxe_link_delete() (rdma link del )-> sk =
ns_sk->rxe_sk4-> sk =ns_sk->rxe_sk4
->udp_tunnel_sock_release(sk)
[Success: First Free] ->udp_tunnel_sock_release(sk)
[Crash: Double Free]
Quoted 8 lines
Quoted 4 lines
After removing the socket release logic from
rxe_ns_exit(), we ensure that only the device destruction path (rxe_link_delete) is responsible for freeing the tunnel sockets, effectively eliminating the double-free problem.I think it is possible to call
rxe_ns_exit()without invokingrxe_link_delete(), and in that case the UDP socket will not be destroyed.
Thanks, my bad. I missed this scenario.
Zhu Yanjun
Quoted 55 lines
Thanks
Quoted 52 lines
I am not sure if I should put the above into the commit log.
Thanks a lot.
Zhu Yanjun
Quoted 44 lines
Thanks
Quoted 41 lines
Fixes 13f2a53c2a71 ("RDMA/rxe: Add net namespace support for IPv4/IPv6 sockets")Signed-off-by Zhu Yanjun <[email protected]>--- drivers/infiniband/sw/rxe/rxe_ns.c | 20 -------------------- 1 file changed, 20 deletions(-)drivers/infiniband/sw/rxe/rxe_ns.c +0
-20@@ -39,26 +39,6 @@ static void rxe_ns_exit(struct net *net) {/* called when the network namespace is removed */ -struct rxe_ns_sock *ns_sk =net_generic (net, rxe_pernet_id); -struct sock *sk; - -rcu_read_lock (); - sk =rcu_dereference (ns_sk->rxe_sk4); -rcu_read_unlock (); -if (sk) { -rcu_assign_pointer (ns_sk->rxe_sk4,NULL ); -udp_tunnel_sock_release (sk->sk_socket); - } - -#if IS_ENABLED (CONFIG_IPV6 ) -rcu_read_lock (); - sk =rcu_dereference (ns_sk->rxe_sk6); -rcu_read_unlock (); -if (sk) { -rcu_assign_pointer (ns_sk->rxe_sk6,NULL ); -udp_tunnel_sock_release (sk->sk_socket); - } -#endif }/* Signature · 1 line
2.43.0
Quoted 28 lines · Zhu Yanjun
Quoted 27 lines · Leon Romanovsky
Quoted 25 lines · yanjun.zhu
Quoted 7 lines · Leon Romanovsky
Quoted 4 lines · Zhu Yanjun
Since all the sockets are created in rdma link create command and destroyed in rdma link delete command, keeping
udp_tunnel_sock_releaseinrxe_ns_exitrisks a "double-free" if the namespace and the device are being cleaned up simultaneously.Please add a ladder diagram to clarify how it can be possible.
Hi, Leon
The double-free occurs as follows:
CPU 0 (Net NameSpace cleanup) CPU 1 (RDMA device removal) --------------------- --------------------------- rxe_ns_exit()
rxe_link_delete()(rdma link del ) -> sk = ns_sk->rxe_sk4 -> sk =ns_sk->rxe_sk4->udp_tunnel_sock_release(sk)
[Success: First Free] ->udp_tunnel_sock_release(sk)[Crash: Double Free]After removing the socket release logic from
rxe_ns_exit(), we ensure that only the device destruction path (rxe_link_delete) is responsible for freeing the tunnel sockets, effectively eliminating the double-free problem.
Quoted 15 lines
Quoted 3 lines
I think it is possible to call
rxe_ns_exit()without invokingrxe_link_delete(), and in that case the UDP socket will not be destroyed.Thanks, my bad. I missed this scenario.
Zhu Yanjun
Quoted 7 lines
Thanks
Quoted 4 lines
I am not sure if I should put the above into the commit log.
Thanks a lot.
Hi, Leon
I have performed further tests to verify the execution order and the necessity of the cleanup code in rxe_ns_exit().
My findings show that a double-free race condition is unlikely because of how the kernel manages namespace references:
Reference Dependency: The RXE RDMA link holds a reference to the network namespace.
Order of Execution: When a namespace is deleted while an RDMA link exists, rxe_ns_exit() is not invoked immediately. It is deferred until the RDMA link itself is deleted (e.g., via rdma link del), which drops the final reference count of the namespace.
Redundancy: Consequently, rxe_ns_exit() always follows the device cleanup path (rxe_link_delete). Since all tunnel sockets are already released during the device cleanup, the code in rxe_ns_exit() is redundant and does nothing.
Removing this code simplifies the driver by centralizing socket destruction in the device management path, where the sockets are originally created.
This ensures that we don't attempt to release the same resources twice, even if the destruction is technically serialized by the kernel's reference counting.
What are your thoughts on this observation?
Thanks,
Zhu Yanjun
Quoted 49 lines
Quoted 48 lines
Quoted 48 lines
Zhu Yanjun
Quoted 44 lines
Thanks
Quoted 41 lines
Fixes 13f2a53c2a71 ("RDMA/rxe: Add net namespace support for IPv4/ IPv6 sockets")Signed-off-by Zhu Yanjun <[email protected]>---
drivers/infiniband/sw/rxe/rxe_ns.c| 20 --------------------1 file changed, 20 deletions(-)drivers/infiniband/sw/rxe/rxe_ns.c +0
-20@@ -39,26 +39,6 @@ static void rxe_ns_exit(struct net *net) { /* called when the network namespace is removed */ - struct rxe_ns_sock *ns_sk = net_generic(net, rxe_pernet_id); - struct sock *sk; - - rcu_read_lock(); - sk = rcu_dereference(ns_sk->rxe_sk4); - rcu_read_unlock(); - if (sk) { - rcu_assign_pointer(ns_sk->rxe_sk4, NULL); - udp_tunnel_sock_release(sk->sk_socket); - } - -#if IS_ENABLED (CONFIG_IPV6 ) - rcu_read_lock(); - sk = rcu_dereference(ns_sk->rxe_sk6); - rcu_read_unlock(); - if (sk) { - rcu_assign_pointer(ns_sk->rxe_sk6, NULL); - udp_tunnel_sock_release(sk->sk_socket); - } -#endif } /*Signature · 1 line
2.43.0
Quoted 53 lines · yanjun.zhu
Quoted 28 lines · Zhu Yanjun
Quoted 27 lines · Leon Romanovsky
Quoted 25 lines · yanjun.zhu
Quoted 7 lines · Leon Romanovsky
Quoted 4 lines · Zhu Yanjun
Since all the sockets are created in rdma link create command and destroyed in rdma link delete command, keeping
udp_tunnel_sock_releaseinrxe_ns_exitrisks a "double-free" if the namespace and the device are being cleaned up simultaneously.Please add a ladder diagram to clarify how it can be possible.
Hi, Leon
The double-free occurs as follows:
CPU 0 (Net NameSpace cleanup) CPU 1 (RDMA device removal) --------------------- --------------------------- rxe_ns_exit()
rxe_link_delete()(rdma link del ) -> sk = ns_sk->rxe_sk4 -> sk =ns_sk->rxe_sk4->udp_tunnel_sock_release(sk)
[Success: First Free] ->udp_tunnel_sock_release(sk)[Crash: Double Free]After removing the socket release logic from
rxe_ns_exit(), we ensure that only the device destruction path (rxe_link_delete) is responsible for freeing the tunnel sockets, effectively eliminating the double-free problem.Quoted 15 lines
Quoted 3 lines
I think it is possible to call
rxe_ns_exit()without invokingrxe_link_delete(), and in that case the UDP socket will not be destroyed.Thanks, my bad. I missed this scenario.
Zhu Yanjun
Quoted 7 lines
Thanks
Quoted 4 lines
I am not sure if I should put the above into the commit log.
Thanks a lot.
Hi, Leon
I have performed further tests to verify the execution order and the necessity of the cleanup code in
rxe_ns_exit().My findings show that a double-free race condition is unlikely because of how the kernel manages namespace references:
Quoted 7 lines
Reference Dependency: The RXE RDMA link holds a reference to the network namespace.
Order of Execution: When a namespace is deleted while an RDMA link exists,
rxe_ns_exit()is not invoked immediately. It is deferred until the RDMA link itself is deleted (e.g., via rdma link del), which drops the final reference count of the namespace.
AFAIC, we've seen syzkaller reports where "rdma link del" was never invoked, yet RXE was removed regardless. Is it possible?
Thanks
Quoted 12 lines · Leon Romanovsky
Quoted 11 lines · yanjun.zhu
Hi, Leon
I have performed further tests to verify the execution order and the necessity of the cleanup code in
rxe_ns_exit().My findings show that a double-free race condition is unlikely because of how the kernel manages namespace references:
Reference Dependency: The RXE RDMA link holds a reference to the network namespace.
Quoted 7 lines
Quoted 4 lines
Order of Execution: When a namespace is deleted while an RDMA link exists,
rxe_ns_exit()is not invoked immediately. It is deferred until the RDMA link itself is deleted (e.g., via rdma link del), which drops the final reference count of the namespace.AFAIC, we've seen syzkaller reports where "rdma link del" was never invoked, yet RXE was removed regardless. Is it possible?
Hi Leon,
Thanks for your feedback.
Regarding the case where "rdma link del" is not invoked, I’d like to share my observations on the namespace cleanup flow for RXE:
In my tests, when a network namespace is deleted (e.g., ip netns del), the RXE device (and its underlying net_device) is typically moved back to the init_net rather than being destroyed immediately. This is why the RDMA links still show up in init_net and the namespace reference count remains held.
As long as the device exists (even if moved), the resource cleanup is managed by the device's lifecycle. The rxe_ns_exit() only gets called when the last reference to that netns is dropped, which usually happens after the RXE device itself is finally deleted.
I haven't personally encountered the syzkaller reports where this logic fails or leads to a leak/crash. If possible, could you please share the specific syzkaller logs or the link to the report? It would be very helpful for me to understand if there is a specific corner case (e.g., driver unloading or abnormal netns teardown) where the per-net cleanup acts as a necessary "safety net."
If such a race exists, I will reconsider whether to keep the cleanup code or move it to a more robust location.
Best regards,
Yanjun Zhu
Quoted 2 lines
Thanks