Skip to contentSkip to conversation

SUNRPC: close a rare race in xs_tcp_setup_socket.

PATCH In review NeilBrown 2 participants 2 messages Oct 31, 2013

Subsystems (3)
lore
Share & tools
lore

NeilBrown

author
#0 lore raw

PATCH 1/1 SUNRPC: close a rare race in xs_tcp_setup_socket.

We have one report of a crash in xs_tcp_setup_socket. The call path to the crash is:

xs_tcp_setup_socket -> inet_stream_connect -> lock_sock_nested.

The 'sock' passed to that last function is NULL.

The only way I can see this happening is a concurrent call to xs_close:

xs_close -> xs_reset_transport -> sock_release -> inet_release

inet_release sets:
sock->sk = NULL;
inet_stream_connect calls
lock_sock(sock->sk);
which gets NULL.

All calls to xs_close are protected by XPRT_LOCKED as are most activations of the workqueue which runs xs_tcp_setup_socket. The exception is xs_tcp_schedule_linger_timeout.

So presumably the timeout queued by the later fires exactly when some other code runs xs_close().

To protect against this we can move the cancel_delayed_work_sync() call from xs_destory() to xs_close().

As xs_close is never called from the worker scheduled on ->connect_worker, this can never deadlock.

net/sunrpc/xprtsock.c +2 -4
index ee03d35677d9..b19ba535ae1a 100644
@@ -835,6 +835,8 @@ static void xs_close(struct rpc_xprt *xprt)
 
 	dprintk("RPC:       xs_close xprt %p\n", xprt);
 
+	cancel_delayed_work_sync(&transport->connect_worker);
+
 	xs_reset_transport(transport);
 	xprt->reestablish_timeout = 0;
 
@@ -869,12 +871,8 @@ static void xs_local_destroy(struct rpc_xprt *xprt)
  */
 static void xs_destroy(struct rpc_xprt *xprt)
 {
-	struct sock_xprt *transport = container_of(xprt, struct sock_xprt, xprt);
-
 	dprintk("RPC:       xs_destroy xprt %p\n", xprt);
 
-	cancel_delayed_work_sync(&transport->connect_worker);
-
 	xs_local_destroy(xprt);
 }
 
#1 lore raw
Quoted 52 lines · NeilBrown

We have one report of a crash in xs_tcp_setup_socket. The call path to the crash is:

xs_tcp_setup_socket -> inet_stream_connect -> lock_sock_nested.

The 'sock' passed to that last function is NULL.

The only way I can see this happening is a concurrent call to xs_close:

xs_close -> xs_reset_transport -> sock_release -> inet_release

inet_release sets:

   sock->sk = NULL;

inet_stream_connect calls
lock_sock(sock->sk);
which gets NULL.

All calls to xs_close are protected by XPRT_LOCKED as are most activations of the workqueue which runs xs_tcp_setup_socket. The exception is xs_tcp_schedule_linger_timeout.

So presumably the timeout queued by the later fires exactly when some other code runs xs_close().

To protect against this we can move the cancel_delayed_work_sync() call from xs_destory() to xs_close().

As xs_close is never called from the worker scheduled on ->connect_worker, this can never deadlock.

net/sunrpc/xprtsock.c +1 -1
index ee03d35677d9..b19ba535ae1a 100644
--- a/net/sunrpc/xprtsock.c
+-+-+- b/net/sunrpc/xprtsock.c

@@ -835,6 +-835,8 @@ static void xs_close(struct rpc_xprt *xprt)

net/sunrpc/xprtsock.c +2 -0
 	dprintk("RPC:       xs_close xprt %p\n", xprt);
 
+-	cancel_delayed_work_sync(&transport->connect_worker);
+-
 	xs_reset_transport(transport);
 	xprt->reestablish_timeout = 0;

@@ -869,12 +-871,8 @@ static void xs_local_destroy(struct rpc_xprt *xprt)
*/
static void xs_destroy(struct rpc_xprt *xprt)
{

net/sunrpc/xprtsock.c static void xs_local_destroy(struct rpc_xprt *xprt) · line 877

Quoted 8 lines
net/sunrpc/xprtsock.c +0 -4
-	struct sock_xprt *transport = container_of(xprt, struct sock_xprt, xprt);
-
 	dprintk("RPC:       xs_destroy xprt %p\n", xprt);
 
-	cancel_delayed_work_sync(&transport->connect_worker);
-
 	xs_local_destroy(xprt);
 }

Hi Neil,

I noticed one small issue when applying: transport->connect_worker is not initialised for the AF_LOCAL case. I therefore added a dummy initialiser to xs_setup_local().

Cheers
Trond

Signature · 6 lines
Trond Myklebust
Linux NFS client maintainer

NetApp
[email protected]
www.netapp.com