Fix SUNRPC wakeup/execute race condition
Subsystems (3)
A message that is not in the archive (sent off-list or not yet received): 1157576316.3292.13.camel@dyn9047022153
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): 1157731084.3292.25.camel@dyn9047022153
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): [email protected]
PATCH 1/1 Fix SUNRPC wakeup/execute race condition
The sunrpc scheduler contains a race condition that can let an RPC task end up being neither running nor on any wait queue. The race takes place between rpc_make_runnable (called from rpc_wake_up_task) and __rpc_execute under the following condition:
First __rpc_execute calls tk_action which puts the task on some wait queue. The task is dequeued by another process before __rpc_execute continues its execution. While executing rpc_make_runnable exactly after setting the task running' bit and before clearing the queued' bit __rpc_execute picks up execution, clears `running' and subsequently both functions fall through, both under the false assumption somebody else took the job.
Swapping rpc_test_and_set_running with rpc_clear_queued in rpc_make_runnable fixes that hole. The reordering hopefully doesn't introduce some new race condition, in fact the only possible one is already correctly detected and handled in __rpc_execute.
Bug noticed on a 4-way x86_64 system under XEN with an NFSv4 server on the same physical machine, apparently one of the few ways to hit this race condition at all.
net/sunrpc/sched.c +1 -4
@@ -302,12 +302,9 @@ EXPORT_SYMBOL(__rpc_wait_for_completion_*/ static void rpc_make_runnable (struct rpc_task *task) { -int do_ret; -BUG_ON (task->tk_timeout_fn); - do_ret =rpc_test_and_set_running (task);rpc_clear_queued (task); -if (do_ret) +if (rpc_test_and_set_running (task))return ;if (RPC_IS_ASYNC (task)) {int status;
Quoted 33 lines · Christophe Saout
The sunrpc scheduler contains a race condition that can let an RPC task end up being neither running nor on any wait queue. The race takes place between
rpc_make_runnable(called fromrpc_wake_up_task) and__rpc_executeunder the following condition:First
__rpc_executecallstk_actionwhich puts the task on some wait queue. The task is dequeued by another process before__rpc_executecontinues its execution. While executingrpc_make_runnableexactly after setting the taskrunning' bit and before clearing thequeued' bit__rpc_executepicks up execution, clears `running' and subsequently both functions fall through, both under the false assumption somebody else took the job.Swapping
rpc_test_and_set_runningwithrpc_clear_queuedinrpc_make_runnablefixes that hole. The reordering hopefully doesn't introduce some new race condition, in fact the only possible one is already correctly detected and handled in__rpc_execute.Bug noticed on a 4-way
x86_64system under XEN with an NFSv4 server on the same physical machine, apparently one of the few ways to hit this race condition at all.Cc Trond Myklebust <[email protected]>Cc J. Bruce Fields <[email protected]>Cc Olaf Kirch <[email protected]>Signed-off-by Christophe Saout <[email protected]>net/sunrpc/sched.c +0
-0@@ -302,12 +302,9 @@ EXPORT_SYMBOL(__rpc_wait_for_completion_*/ static void rpc_make_runnable (struct rpc_task *task) {
net/sunrpc/sched.c EXPORT_SYMBOL(__rpc_wait_for_completion_ · line 310
Quoted 10 lines
net/sunrpc/sched.c +1
-4-int do_ret; -BUG_ON (task->tk_timeout_fn); - do_ret =rpc_test_and_set_running (task);rpc_clear_queued (task); -if (do_ret) +if (rpc_test_and_set_running (task))return ;if (RPC_IS_ASYNC (task)) {int status;
This fix looks wrong to me. If we've made it to 'rpc_make_runnable', then the rpc_task will have already been removed from the rpc_wait_queue.
The only question left is "who is responsible for waking up the synchronous task / setting up the asynchronous workqueue item?".
Cheers,
Trond
Am Sonntag, den 05.11.2006, 01:50 -0500 schrieb Trond Myklebust:
Quoted 12 lines
Quoted 12 lines
net/sunrpc/sched.c +0
-4@@ -302,12 +302,9 @@ EXPORT_SYMBOL(__rpc_wait_for_completion_*/ static void rpc_make_runnable (struct rpc_task *task) { -int do_ret; -BUG_ON (task->tk_timeout_fn); - do_ret =rpc_test_and_set_running (task);rpc_clear_queued (task); -if (do_ret)
Quoted 8 lines
net/sunrpc/sched.c EXPORT_SYMBOL(__rpc_wait_for_completion_ · line 310
Quoted 4 lines
net/sunrpc/sched.c +1
-0+if (rpc_test_and_set_running (task))return ;if (RPC_IS_ASYNC (task)) {int status;This fix looks wrong to me. If we've made it to '
rpc_make_runnable', then therpc_taskwill have already been removed from therpc_wait_queue.
I just flipped the two lines, changed nothing else. Why exactly do you think that's wrong, I don't see anything particular that could be broken by chaning the ordering. Anyway, the fsstress has been running for 18 hours straight now without showing any signs of problems.
Quoted 23 lines · Christophe Saout
Am Sonntag, den 05.11.2006, 01:50 -0500 schrieb Trond Myklebust:
Quoted 12 lines
Quoted 12 lines
net/sunrpc/sched.c +0
-4@@ -302,12 +302,9 @@ EXPORT_SYMBOL(__rpc_wait_for_completion_*/ static void rpc_make_runnable (struct rpc_task *task) { -int do_ret; -BUG_ON (task->tk_timeout_fn); - do_ret =rpc_test_and_set_running (task);rpc_clear_queued (task); -if (do_ret)Quoted 8 lines
net/sunrpc/sched.c EXPORT_SYMBOL(__rpc_wait_for_completion_ · line 310
Quoted 4 lines
net/sunrpc/sched.c +1
-0+if (rpc_test_and_set_running (task))return ;if (RPC_IS_ASYNC (task)) {int status;This fix looks wrong to me. If we've made it to '
rpc_make_runnable', then therpc_taskwill have already been removed from therpc_wait_queue.
Quoted 4 lines
I just flipped the two lines, changed nothing else. Why exactly do you think that's wrong, I don't see anything particular that could be broken by chaning the ordering. Anyway, the fsstress has been running for 18 hours straight now without showing any signs of problems.
OK. I finally see the bug that you've spotted. The problem occurs when __rpc_execute clears RPC_TASK_RUNNING after rpc_make_runnable has called rpc_test_and_set_running, but before it has called rpc_clear_queued.
However if you just swap the two lines, you run into a new race:__rpc_execute() may just put the rpc_task back to sleep before your call to rpc_test_and_set_running() finishes executing.
We therefore need an extra test for RPC_IS_QUEUED() in rpc_make_runnable().
Cheers,
Trond
Quoted 24 lines · Trond Myklebust
Quoted 23 lines · Christophe Saout
Am Sonntag, den 05.11.2006, 01:50 -0500 schrieb Trond Myklebust:
Quoted 20 lines
Quoted 6 lines
net/sunrpc/sched.c +0
-0@@ -302,12 +302,9 @@ EXPORT_SYMBOL(__rpc_wait_for_completion_*/ static void rpc_make_runnable (struct rpc_task *task) {net/sunrpc/sched.c EXPORT_SYMBOL(__rpc_wait_for_completion_ · line 310
Quoted 10 lines
net/sunrpc/sched.c +1
-4-int do_ret; -BUG_ON (task->tk_timeout_fn); - do_ret =rpc_test_and_set_running (task);rpc_clear_queued (task); -if (do_ret) +if (rpc_test_and_set_running (task))return ;if (RPC_IS_ASYNC (task)) {int status;This fix looks wrong to me. If we've made it to '
rpc_make_runnable', then therpc_taskwill have already been removed from therpc_wait_queue.
Quoted 8 lines
Quoted 4 lines
I just flipped the two lines, changed nothing else. Why exactly do you think that's wrong, I don't see anything particular that could be broken by chaning the ordering. Anyway, the fsstress has been running for 18 hours straight now without showing any signs of problems.
OK. I finally see the bug that you've spotted. The problem occurs when
__rpc_executeclearsRPC_TASK_RUNNINGafterrpc_make_runnablehas calledrpc_test_and_set_running, but before it has calledrpc_clear_queued.
Yes, exactly.
Quoted 5 lines
However if you just swap the two lines, you run into a new race:
__rpc_execute()may just put therpc_taskback to sleep before your call torpc_test_and_set_running()finishes executing.
We therefore need an extra test forRPC_IS_QUEUED()inrpc_make_runnable().
Damn, you're right. I missed that one. What about that:
----
The sunrpc scheduler contains a race condition that can let an RPC task end up being neither running nor on any wait queue. The race takes place between rpc_make_runnable (called from rpc_wake_up_task) and __rpc_execute under the following condition:
First __rpc_execute calls tk_action which puts the task on some wait queue. The task is dequeued by another process before __rpc_execute continues its execution. While executing rpc_make_runnable exactly after setting the task running' bit and before clearing the queued' bit __rpc_execute picks up execution, clears `running' and subsequently both functions fall through, both under the false assumption somebody else took the job.
Swapping rpc_test_and_set_running with rpc_clear_queued in rpc_make_runnable fixes that hole. This introduces another possible race condition that can be handled by checking for queued' after setting the running' bit.
Bug noticed on a 4-way x86_64 system under XEN with an NFSv4 server on the same physical machine, apparently one of the few ways to hit this race condition at all.
net/sunrpc/sched.c +6 -4
@@ -302,12 +302,15 @@ EXPORT_SYMBOL(__rpc_wait_for_completion_*/ static void rpc_make_runnable (struct rpc_task *task) { -int do_ret; -BUG_ON (task->tk_timeout_fn); - do_ret =rpc_test_and_set_running (task);rpc_clear_queued (task); -if (do_ret) +if (rpc_test_and_set_running (task))return ; +/* We might have raced */ +if (RPC_IS_QUEUED (task)) { +rpc_clear_running (task); +return ; + }if (RPC_IS_ASYNC (task)) {int status;
- int do_ret;
-
BUG_ON(task->tk_timeout_fn);
- do_ret = rpc_test_and_set_running(task);
rpc_clear_queued(task);
- if (do_ret)
+ if (rpc_test_and_set_running(task))
return;
+ /* We might have raced with __rpc_execute */
+ if (RPC_IS_QUEUED(task)) {
+ rpc_clear_running(task);
+ return;
+ }
if (RPC_IS_ASYNC(task)) {
int status;
Quoted 82 lines · Christophe Saout
Quoted 24 lines · Trond Myklebust
Quoted 23 lines · Christophe Saout
Am Sonntag, den 05.11.2006, 01:50 -0500 schrieb Trond Myklebust:
Quoted 20 lines
Quoted 6 lines
net/sunrpc/sched.c +0
-0@@ -302,12 +302,9 @@ EXPORT_SYMBOL(__rpc_wait_for_completion_*/ static void rpc_make_runnable (struct rpc_task *task) {net/sunrpc/sched.c EXPORT_SYMBOL(__rpc_wait_for_completion_ · line 310
Quoted 10 lines
net/sunrpc/sched.c +1
-4-int do_ret; -BUG_ON (task->tk_timeout_fn); - do_ret =rpc_test_and_set_running (task);rpc_clear_queued (task); -if (do_ret) +if (rpc_test_and_set_running (task))return ;if (RPC_IS_ASYNC (task)) {int status;This fix looks wrong to me. If we've made it to '
rpc_make_runnable', then therpc_taskwill have already been removed from therpc_wait_queue.Quoted 8 lines
Quoted 4 lines
I just flipped the two lines, changed nothing else. Why exactly do you think that's wrong, I don't see anything particular that could be broken by chaning the ordering. Anyway, the fsstress has been running for 18 hours straight now without showing any signs of problems.
OK. I finally see the bug that you've spotted. The problem occurs when
__rpc_executeclearsRPC_TASK_RUNNINGafterrpc_make_runnablehas calledrpc_test_and_set_running, but before it has calledrpc_clear_queued.Yes, exactly.
Quoted 5 lines
However if you just swap the two lines, you run into a new race:
__rpc_execute()may just put therpc_taskback to sleep before your call torpc_test_and_set_running()finishes executing.
We therefore need an extra test forRPC_IS_QUEUED()inrpc_make_runnable().Damn, you're right. I missed that one. What about that:
----
The sunrpc scheduler contains a race condition that can let an RPC task end up being neither running nor on any wait queue. The race takes place betweenrpc_make_runnable(called fromrpc_wake_up_task) and__rpc_executeunder the following condition:First
__rpc_executecallstk_actionwhich puts the task on some wait queue. The task is dequeued by another process before__rpc_executecontinues its execution. While executingrpc_make_runnableexactly after setting the taskrunning' bit and before clearing thequeued' bit__rpc_executepicks up execution, clears `running' and subsequently both functions fall through, both under the false assumption somebody else took the job.Swapping
rpc_test_and_set_runningwithrpc_clear_queuedinrpc_make_runnablefixes that hole. This introduces another possible race condition that can be handled by checking forqueued' after setting therunning' bit.Bug noticed on a 4-way
x86_64system under XEN with an NFSv4 server on the same physical machine, apparently one of the few ways to hit this race condition at all.Cc Trond Myklebust <[email protected]>Cc J. Bruce Fields <[email protected]>Signed-off-by Christophe Saout <[email protected]>net/sunrpc/sched.c +0
-3@@ -302,12 +302,15 @@ EXPORT_SYMBOL(__rpc_wait_for_completion_*/ static void rpc_make_runnable (struct rpc_task *task) { -int do_ret; -BUG_ON (task->tk_timeout_fn); - do_ret =rpc_test_and_set_running (task);rpc_clear_queued (task);
net/sunrpc/sched.c EXPORT_SYMBOL(__rpc_wait_for_completion_ · line 315
Quoted 10 lines
net/sunrpc/sched.c +6
-1-if (do_ret) +if (rpc_test_and_set_running (task))return ; +/* We might have raced */ +if (RPC_IS_QUEUED (task)) { +rpc_clear_running (task); +return ; + }if (RPC_IS_ASYNC (task)) {int status;
This looks OK.
Quoted 5 lines
net/sunrpc/sched.c +0
-3-int do_ret; -BUG_ON (task->tk_timeout_fn); - do_ret =rpc_test_and_set_running (task);rpc_clear_queued (task);
net/sunrpc/sched.c EXPORT_SYMBOL(__rpc_wait_for_completion_ · line 315
Quoted 10 lines
net/sunrpc/sched.c +6
-1-if (do_ret) +if (rpc_test_and_set_running (task))return ; +/* We might have raced with __rpc_execute */ +if (RPC_IS_QUEUED (task)) { +rpc_clear_running (task); +return ; + }if (RPC_IS_ASYNC (task)) {int status;
A cut'n paste error?
Cheers,
Trond