Skip to contentSkip to conversation

RDMA/rxe: Fix unsafe socket release during namespace cleanup

PATCH In review Zhu Yanjun 2 participants 8 messages Apr 24 – May 12

Subsystems (2)
lore
Share & tools
lore

Zhu Yanjun

author
#0 lore raw

PATCH 1/1 RDMA/rxe: Fix unsafe socket release during namespace cleanup

Since all the sockets are created in rdma link create command and destroyed in rdma link delete command, keeping udp_tunnel_sock_release in rxe_ns_exit risks a "double-free" if the namespace and the device are being cleaned up simultaneously.

Fixes 13f2a53c2a71 ("RDMA/rxe: Add net namespace support for IPv4/IPv6 sockets")
---
 drivers/infiniband/sw/rxe/rxe_ns.c | 20 --------------------
 1 file changed, 20 deletions(-)
drivers/infiniband/sw/rxe/rxe_ns.c +0 -20
index 8b9d734229b2..53add78b8e3a 100644
@@ -39,26 +39,6 @@ static void rxe_ns_exit(struct net *net)
 {
 	/* called when the network namespace is removed
 	 */
-	struct rxe_ns_sock *ns_sk = net_generic(net, rxe_pernet_id);
-	struct sock *sk;
-
-	rcu_read_lock();
-	sk = rcu_dereference(ns_sk->rxe_sk4);
-	rcu_read_unlock();
-	if (sk) {
-		rcu_assign_pointer(ns_sk->rxe_sk4, NULL);
-		udp_tunnel_sock_release(sk->sk_socket);
-	}
-
-#if IS_ENABLED(CONFIG_IPV6)
-	rcu_read_lock();
-	sk = rcu_dereference(ns_sk->rxe_sk6);
-	rcu_read_unlock();
-	if (sk) {
-		rcu_assign_pointer(ns_sk->rxe_sk6, NULL);
-		udp_tunnel_sock_release(sk->sk_socket);
-	}
-#endif
 }
 
 /*
Signature · 1 line
2.43.0
#1 lore raw
Quoted 4 lines · Zhu Yanjun

Since all the sockets are created in rdma link create command and destroyed in rdma link delete command, keeping udp_tunnel_sock_release in rxe_ns_exit risks a "double-free" if the namespace and the device are being cleaned up simultaneously.

Please add a ladder diagram to clarify how it can be possible.

Thanks

Quoted 42 lines
Fixes 13f2a53c2a71 ("RDMA/rxe: Add net namespace support for IPv4/IPv6 sockets")
---
 drivers/infiniband/sw/rxe/rxe_ns.c | 20 --------------------
 1 file changed, 20 deletions(-)
drivers/infiniband/sw/rxe/rxe_ns.c +0 -20
index 8b9d734229b2..53add78b8e3a 100644
@@ -39,26 +39,6 @@ static void rxe_ns_exit(struct net *net)
 {
 	/* called when the network namespace is removed
 	 */
-	struct rxe_ns_sock *ns_sk = net_generic(net, rxe_pernet_id);
-	struct sock *sk;
-
-	rcu_read_lock();
-	sk = rcu_dereference(ns_sk->rxe_sk4);
-	rcu_read_unlock();
-	if (sk) {
-		rcu_assign_pointer(ns_sk->rxe_sk4, NULL);
-		udp_tunnel_sock_release(sk->sk_socket);
-	}
-
-#if IS_ENABLED(CONFIG_IPV6)
-	rcu_read_lock();
-	sk = rcu_dereference(ns_sk->rxe_sk6);
-	rcu_read_unlock();
-	if (sk) {
-		rcu_assign_pointer(ns_sk->rxe_sk6, NULL);
-		udp_tunnel_sock_release(sk->sk_socket);
-	}
-#endif
 }
 
 /*
Signature · 1 line
2.43.0
author
#2 lore raw
Quoted 7 lines · Leon Romanovsky
Quoted 4 lines · Zhu Yanjun

Since all the sockets are created in rdma link create command and destroyed in rdma link delete command, keeping udp_tunnel_sock_release in rxe_ns_exit risks a "double-free" if the namespace and the device are being cleaned up simultaneously.

Please add a ladder diagram to clarify how it can be possible.

Hi, Leon

The double-free occurs as follows:

CPU 0 (Net NameSpace cleanup)        CPU 1 (RDMA device removal)
---------------------                ---------------------------
rxe_ns_exit()                        rxe_link_delete() (rdma link del )

-> sk = ns_sk->rxe_sk4 -> sk = ns_sk->rxe_sk4
-> udp_tunnel_sock_release(sk)
[Success: First Free] -> udp_tunnel_sock_release(sk)
[Crash: Double Free]

After removing the socket release logic from rxe_ns_exit(), we ensure that only the device destruction path (rxe_link_delete) is responsible for freeing the tunnel sockets, effectively eliminating the double-free problem.

I am not sure if I should put the above into the commit log.

Thanks a lot.

Zhu Yanjun

Quoted 45 lines

Thanks

Quoted 42 lines
Fixes 13f2a53c2a71 ("RDMA/rxe: Add net namespace support for IPv4/IPv6 sockets")
---
  drivers/infiniband/sw/rxe/rxe_ns.c | 20 --------------------
  1 file changed, 20 deletions(-)
drivers/infiniband/sw/rxe/rxe_ns.c +0 -20
index 8b9d734229b2..53add78b8e3a 100644
@@ -39,26 +39,6 @@ static void rxe_ns_exit(struct net *net)
  {
  	/* called when the network namespace is removed
  	 */
-	struct rxe_ns_sock *ns_sk = net_generic(net, rxe_pernet_id);
-	struct sock *sk;
-
-	rcu_read_lock();
-	sk = rcu_dereference(ns_sk->rxe_sk4);
-	rcu_read_unlock();
-	if (sk) {
-		rcu_assign_pointer(ns_sk->rxe_sk4, NULL);
-		udp_tunnel_sock_release(sk->sk_socket);
-	}
-
-#if IS_ENABLED(CONFIG_IPV6)
-	rcu_read_lock();
-	sk = rcu_dereference(ns_sk->rxe_sk6);
-	rcu_read_unlock();
-	if (sk) {
-		rcu_assign_pointer(ns_sk->rxe_sk6, NULL);
-		udp_tunnel_sock_release(sk->sk_socket);
-	}
-#endif
  }
 
  /*
Signature · 1 line
2.43.0
#3 lore raw
Quoted 21 lines · yanjun.zhu
Quoted 7 lines · Leon Romanovsky
Quoted 4 lines · Zhu Yanjun

Since all the sockets are created in rdma link create command and destroyed in rdma link delete command, keeping udp_tunnel_sock_release in rxe_ns_exit risks a "double-free" if the namespace and the device are being cleaned up simultaneously.

Please add a ladder diagram to clarify how it can be possible.

Hi, Leon

The double-free occurs as follows:

CPU 0 (Net NameSpace cleanup)        CPU 1 (RDMA device removal)
---------------------                ---------------------------
rxe_ns_exit()                        rxe_link_delete() (rdma link del )

-> sk = ns_sk->rxe_sk4 -> sk = ns_sk->rxe_sk4
-> udp_tunnel_sock_release(sk)
[Success: First Free] -> udp_tunnel_sock_release(sk)
[Crash: Double Free]

Quoted 4 lines

After removing the socket release logic from rxe_ns_exit(), we ensure that only the device destruction path (rxe_link_delete) is responsible for freeing the tunnel sockets, effectively eliminating the double-free problem.

I think it is possible to call rxe_ns_exit() without invoking rxe_link_delete(), and in that case the UDP socket will not be destroyed.

Thanks

Quoted 52 lines

I am not sure if I should put the above into the commit log.

Thanks a lot.

Zhu Yanjun

Quoted 44 lines

Thanks

Quoted 41 lines
Fixes 13f2a53c2a71 ("RDMA/rxe: Add net namespace support for IPv4/IPv6 sockets")
---
  drivers/infiniband/sw/rxe/rxe_ns.c | 20 --------------------
  1 file changed, 20 deletions(-)
drivers/infiniband/sw/rxe/rxe_ns.c +0 -20
index 8b9d734229b2..53add78b8e3a 100644
@@ -39,26 +39,6 @@ static void rxe_ns_exit(struct net *net)
  {
  	/* called when the network namespace is removed
  	 */
-	struct rxe_ns_sock *ns_sk = net_generic(net, rxe_pernet_id);
-	struct sock *sk;
-
-	rcu_read_lock();
-	sk = rcu_dereference(ns_sk->rxe_sk4);
-	rcu_read_unlock();
-	if (sk) {
-		rcu_assign_pointer(ns_sk->rxe_sk4, NULL);
-		udp_tunnel_sock_release(sk->sk_socket);
-	}
-
-#if IS_ENABLED(CONFIG_IPV6)
-	rcu_read_lock();
-	sk = rcu_dereference(ns_sk->rxe_sk6);
-	rcu_read_unlock();
-	if (sk) {
-		rcu_assign_pointer(ns_sk->rxe_sk6, NULL);
-		udp_tunnel_sock_release(sk->sk_socket);
-	}
-#endif
  }
  /*
Signature · 1 line
2.43.0
author
#4 lore raw
Quoted 22 lines · Leon Romanovsky
Quoted 21 lines · yanjun.zhu
Quoted 7 lines · Leon Romanovsky
Quoted 4 lines · Zhu Yanjun

Since all the sockets are created in rdma link create command and destroyed in rdma link delete command, keeping udp_tunnel_sock_release in rxe_ns_exit risks a "double-free" if the namespace and the device are being cleaned up simultaneously.

Please add a ladder diagram to clarify how it can be possible.

Hi, Leon

The double-free occurs as follows:

CPU 0 (Net NameSpace cleanup)        CPU 1 (RDMA device removal)
---------------------                ---------------------------
rxe_ns_exit()                        rxe_link_delete() (rdma link del )

-> sk = ns_sk->rxe_sk4 -> sk = ns_sk->rxe_sk4
-> udp_tunnel_sock_release(sk)
[Success: First Free] -> udp_tunnel_sock_release(sk)
[Crash: Double Free]

Quoted 8 lines
Quoted 4 lines

After removing the socket release logic from rxe_ns_exit(), we ensure that only the device destruction path (rxe_link_delete) is responsible for freeing the tunnel sockets, effectively eliminating the double-free problem.

I think it is possible to call rxe_ns_exit() without invoking rxe_link_delete(), and in that case the UDP socket will not be destroyed.

Thanks, my bad. I missed this scenario.

Zhu Yanjun

Quoted 55 lines

Thanks

Quoted 52 lines

I am not sure if I should put the above into the commit log.

Thanks a lot.

Zhu Yanjun

Quoted 44 lines

Thanks

Quoted 41 lines
Fixes 13f2a53c2a71 ("RDMA/rxe: Add net namespace support for IPv4/IPv6 sockets")
---
   drivers/infiniband/sw/rxe/rxe_ns.c | 20 --------------------
   1 file changed, 20 deletions(-)
drivers/infiniband/sw/rxe/rxe_ns.c +0 -20
index 8b9d734229b2..53add78b8e3a 100644
@@ -39,26 +39,6 @@ static void rxe_ns_exit(struct net *net)
   {
   	/* called when the network namespace is removed
   	 */
-	struct rxe_ns_sock *ns_sk = net_generic(net, rxe_pernet_id);
-	struct sock *sk;
-
-	rcu_read_lock();
-	sk = rcu_dereference(ns_sk->rxe_sk4);
-	rcu_read_unlock();
-	if (sk) {
-		rcu_assign_pointer(ns_sk->rxe_sk4, NULL);
-		udp_tunnel_sock_release(sk->sk_socket);
-	}
-
-#if IS_ENABLED(CONFIG_IPV6)
-	rcu_read_lock();
-	sk = rcu_dereference(ns_sk->rxe_sk6);
-	rcu_read_unlock();
-	if (sk) {
-		rcu_assign_pointer(ns_sk->rxe_sk6, NULL);
-		udp_tunnel_sock_release(sk->sk_socket);
-	}
-#endif
   }
   /*
Signature · 1 line
2.43.0
author
#5 lore raw
Quoted 28 lines · Zhu Yanjun
Quoted 27 lines · Leon Romanovsky
Quoted 25 lines · yanjun.zhu
Quoted 7 lines · Leon Romanovsky
Quoted 4 lines · Zhu Yanjun

Since all the sockets are created in rdma link create command and destroyed in rdma link delete command, keeping udp_tunnel_sock_release in rxe_ns_exit risks a "double-free" if the namespace and the device are being cleaned up simultaneously.

Please add a ladder diagram to clarify how it can be possible.

Hi, Leon

The double-free occurs as follows:

CPU 0 (Net NameSpace cleanup)        CPU 1 (RDMA device removal) ---------------------                --------------------------- rxe_ns_exit()                        rxe_link_delete() (rdma link del ) -> sk = ns_sk->rxe_sk4               -> sk = ns_sk->rxe_sk4 -> udp_tunnel_sock_release(sk)
      [Success: First Free]             -> udp_tunnel_sock_release(sk) [Crash: Double Free]

After removing the socket release logic from rxe_ns_exit(), we ensure that only the device destruction path (rxe_link_delete) is responsible for freeing the tunnel sockets, effectively eliminating the double-free problem.

Quoted 15 lines
Quoted 3 lines

I think it is possible to call rxe_ns_exit() without invoking rxe_link_delete(), and in that case the UDP socket will not be destroyed.

Thanks, my bad. I missed this scenario.

Zhu Yanjun

Quoted 7 lines

Thanks

Quoted 4 lines

I am not sure if I should put the above into the commit log.

Thanks a lot.

Hi, Leon

I have performed further tests to verify the execution order and the necessity of the cleanup code in rxe_ns_exit().

My findings show that a double-free race condition is unlikely because of how the kernel manages namespace references:

Reference Dependency: The RXE RDMA link holds a reference to the network namespace.

Order of Execution: When a namespace is deleted while an RDMA link exists, rxe_ns_exit() is not invoked immediately. It is deferred until the RDMA link itself is deleted (e.g., via rdma link del), which drops the final reference count of the namespace.

Redundancy: Consequently, rxe_ns_exit() always follows the device cleanup path (rxe_link_delete). Since all tunnel sockets are already released during the device cleanup, the code in rxe_ns_exit() is redundant and does nothing.

Removing this code simplifies the driver by centralizing socket destruction in the device management path, where the sockets are originally created.

This ensures that we don't attempt to release the same resources twice, even if the destruction is technically serialized by the kernel's reference counting.

What are your thoughts on this observation?

Thanks,

Zhu Yanjun

Quoted 49 lines
Quoted 48 lines
Quoted 48 lines

Zhu Yanjun

Quoted 44 lines

Thanks

Quoted 41 lines
Fixes 13f2a53c2a71 ("RDMA/rxe: Add net namespace support for IPv4/ IPv6 sockets")

---
   drivers/infiniband/sw/rxe/rxe_ns.c | 20 --------------------

   1 file changed, 20 deletions(-)
drivers/infiniband/sw/rxe/rxe_ns.c +0 -20
index 8b9d734229b2..53add78b8e3a 100644
@@ -39,26 +39,6 @@ static void rxe_ns_exit(struct net *net)
    {
        /* called when the network namespace is removed
         */
-    struct rxe_ns_sock *ns_sk = net_generic(net, rxe_pernet_id);
-    struct sock *sk;
-
-    rcu_read_lock();
-    sk = rcu_dereference(ns_sk->rxe_sk4);
-    rcu_read_unlock();
-    if (sk) {
-        rcu_assign_pointer(ns_sk->rxe_sk4, NULL);
-        udp_tunnel_sock_release(sk->sk_socket);
-    }
-
-#if IS_ENABLED(CONFIG_IPV6)
-    rcu_read_lock();
-    sk = rcu_dereference(ns_sk->rxe_sk6);
-    rcu_read_unlock();
-    if (sk) {
-        rcu_assign_pointer(ns_sk->rxe_sk6, NULL);
-        udp_tunnel_sock_release(sk->sk_socket);
-    }
-#endif
    }
    /*
Signature · 1 line
2.43.0
#6 lore raw
Quoted 53 lines · yanjun.zhu
Quoted 28 lines · Zhu Yanjun
Quoted 27 lines · Leon Romanovsky
Quoted 25 lines · yanjun.zhu
Quoted 7 lines · Leon Romanovsky
Quoted 4 lines · Zhu Yanjun

Since all the sockets are created in rdma link create command and destroyed in rdma link delete command, keeping udp_tunnel_sock_release in rxe_ns_exit risks a "double-free" if the namespace and the device are being cleaned up simultaneously.

Please add a ladder diagram to clarify how it can be possible.

Hi, Leon

The double-free occurs as follows:

CPU 0 (Net NameSpace cleanup)        CPU 1 (RDMA device removal) ---------------------                --------------------------- rxe_ns_exit()                        rxe_link_delete() (rdma link del ) -> sk = ns_sk->rxe_sk4               -> sk = ns_sk->rxe_sk4 -> udp_tunnel_sock_release(sk)
      [Success: First Free]             -> udp_tunnel_sock_release(sk) [Crash: Double Free]

After removing the socket release logic from rxe_ns_exit(), we ensure that only the device destruction path (rxe_link_delete) is responsible for freeing the tunnel sockets, effectively eliminating the double-free problem.

Quoted 15 lines
Quoted 3 lines

I think it is possible to call rxe_ns_exit() without invoking rxe_link_delete(), and in that case the UDP socket will not be destroyed.

Thanks, my bad. I missed this scenario.

Zhu Yanjun

Quoted 7 lines

Thanks

Quoted 4 lines

I am not sure if I should put the above into the commit log.

Thanks a lot.

Hi, Leon

I have performed further tests to verify the execution order and the necessity of the cleanup code in rxe_ns_exit().

My findings show that a double-free race condition is unlikely because of how the kernel manages namespace references:

Quoted 7 lines

Reference Dependency: The RXE RDMA link holds a reference to the network namespace.

Order of Execution: When a namespace is deleted while an RDMA link exists, rxe_ns_exit() is not invoked immediately. It is deferred until the RDMA link itself is deleted (e.g., via rdma link del), which drops the final reference count of the namespace.

AFAIC, we've seen syzkaller reports where "rdma link del" was never invoked, yet RXE was removed regardless. Is it possible?

Thanks

author
#7 lore raw
Quoted 12 lines · Leon Romanovsky
Quoted 11 lines · yanjun.zhu

Hi, Leon

I have performed further tests to verify the execution order and the necessity of the cleanup code in rxe_ns_exit().

My findings show that a double-free race condition is unlikely because of how the kernel manages namespace references:

Reference Dependency: The RXE RDMA link holds a reference to the network namespace.

Quoted 7 lines
Quoted 4 lines

Order of Execution: When a namespace is deleted while an RDMA link exists, rxe_ns_exit() is not invoked immediately. It is deferred until the RDMA link itself is deleted (e.g., via rdma link del), which drops the final reference count of the namespace.

AFAIC, we've seen syzkaller reports where "rdma link del" was never invoked, yet RXE was removed regardless. Is it possible?

Hi Leon,

Thanks for your feedback.

Regarding the case where "rdma link del" is not invoked, I’d like to share my observations on the namespace cleanup flow for RXE:

In my tests, when a network namespace is deleted (e.g., ip netns del), the RXE device (and its underlying net_device) is typically moved back to the init_net rather than being destroyed immediately. This is why the RDMA links still show up in init_net and the namespace reference count remains held.

As long as the device exists (even if moved), the resource cleanup is managed by the device's lifecycle. The rxe_ns_exit() only gets called when the last reference to that netns is dropped, which usually happens after the RXE device itself is finally deleted.

I haven't personally encountered the syzkaller reports where this logic fails or leads to a leak/crash. If possible, could you please share the specific syzkaller logs or the link to the report? It would be very helpful for me to understand if there is a specific corner case (e.g., driver unloading or abnormal netns teardown) where the per-net cleanup acts as a necessary "safety net."

If such a race exists, I will reconsider whether to keep the cleanup code or move it to a more robust location.

Best regards,
Yanjun Zhu

Quoted 2 lines

Thanks