Fix two bugs CB_NOTIFY_LOCK failing to wake a water
Subsystems (2)
v2 cover Fix two bugs CB_NOTIFY_LOCK failing to wake a water
This patch set fix bugs related to CB_NOTIFY_LOCK. These bugs cause poor performance in locking operation. And this patchset should also fix the failure when running xfstests generic/089 on a NFSv4.1 filesystem.
Yihao Wu (2):
NFSv4.1: Again fix a race where CB_NOTIFY_LOCK fails to wake a waiter NFSv4.1: Fix bug only first CB_NOTIFY_LOCK is handled
fs/nfs/nfs4proc.c | 31 ++++++++++++------------------- 1 file changed, 12 insertions(+), 19 deletions(-)
Signature · 1 line
1.8.3.1
PATCH v2 1/2 NFSv4.1: Again fix a race where CB_NOTIFY_LOCK fails to wake a waiter
Commit b7dbcc0e433f "NFSv4.1: Fix a race where CB_NOTIFY_LOCK fails to wake a waiter" found this bug. However it didn't fix it.
This commit replaces schedule_timeout() with wait_woken() and default_wake_function() with woken_wake_function() in function nfs4_retry_setlk() and nfs4_wake_lock_waiter(). wait_woken() uses memory barriers in its implementation to avoid potential race condition when putting a process into sleeping state and then waking it up.
--- fs/nfs/nfs4proc.c | 23 +++++++---------------- 1 file changed, 7 insertions(+), 16 deletions(-)
fs/nfs/nfs4proc.c +7 -16
@@ -6932,7 +6932,6 @@ struct nfs4_lock_waiter {struct task_struct *task;struct inode *inode;struct nfs_lowner *owner; -bool notified; }; static int
@@ -6954,13 +6953,13 @@ struct nfs4_lock_waiter {/* Make sure it's for the right inode */ if (nfs_compare_fh (NFS_FH (waiter->inode), &cbnl->cbnl_fh))return 0 ; - - waiter->notified = true; }/* override "private" so we can use default_wake_function */ wait->private = waiter->task; - ret =autoremove_wake_function (wait, mode, flags, key); + ret =woken_wake_function (wait, mode, flags, key); +if (ret) +list_del_init (&wait->entry); wait->private = waiter;return ret; }
@@ -6979,8 +6978,7 @@ struct nfs4_lock_waiter { .s_dev = server->s_dev };struct nfs4_lock_waiter waiter = { .task = current, .inode = state->inode, - .owner = &owner, - .notified = false }; + .owner = &owner};wait_queue_entry_t wait;/* Don't bother with waitqueue if we don't expect a callback */
@@ -6993,21 +6991,14 @@ struct nfs4_lock_waiter {add_wait_queue (q, &wait);while (!signalled ()) { - waiter.notified = false; status =nfs4_proc_setlk (state, cmd, request);if ((status != -EAGAIN ) ||IS_SETLK (cmd))break ; status = -ERESTARTSYS ; -spin_lock_irqsave (&q->lock, flags); -if (waiter.notified) { -spin_unlock_irqrestore (&q->lock, flags); -continue ; - } -set_current_state (TASK_INTERRUPTIBLE ); -spin_unlock_irqrestore (&q->lock, flags); - -freezable_schedule_timeout (NFS4_LOCK_MAXTIMEOUT ); +freezer_do_not_count (); +wait_woken (&wait,TASK_INTERRUPTIBLE ,NFS4_LOCK_MAXTIMEOUT ); +freezer_count (); }finish_wait (q, &wait);
Signature · 1 line
1.8.3.1
Re: [PATCH v2 1/2] NFSv4.1: Again fix a race where CB_NOTIFY_LOCK fails to wake a waiter
Hi Yihao,
Quoted 67 lines · Yihao Wu
Commit
b7dbcc0e433f"NFSv4.1: Fix a race whereCB_NOTIFY_LOCKfails to wake a waiter" found this bug. However it didn't fix it.This commit replaces
schedule_timeout()withwait_woken()anddefault_wake_function()withwoken_wake_function()in functionnfs4_retry_setlk()andnfs4_wake_lock_waiter().wait_woken()uses memory barriers in its implementation to avoid potential race condition when putting a process into sleeping state and then waking it up.Fixes a1d617d8f134 ("nfs: allow blocking locks to be awoken by lock callbacks")Cc [email protected] #4.9+Signed-off-by Yihao Wu <[email protected]>--- fs/nfs/nfs4proc.c | 23 +++++++---------------- 1 file changed, 7 insertions(+), 16 deletions(-)fs/nfs/nfs4proc.c +4
-9@@ -6932,7 +6932,6 @@ struct nfs4_lock_waiter {struct task_struct *task;struct inode *inode;struct nfs_lowner *owner; -bool notified; }; static int@@ -6954,13 +6953,13 @@ struct nfs4_lock_waiter {/* Make sure it's for the right inode */ if (nfs_compare_fh (NFS_FH (waiter->inode), &cbnl->cbnl_fh))return 0 ; - - waiter->notified = true; }/* override "private" so we can use default_wake_function */ wait->private = waiter->task; - ret =autoremove_wake_function (wait, mode, flags, key); + ret =woken_wake_function (wait, mode, flags, key); +if (ret) +list_del_init (&wait->entry); wait->private = waiter;return ret; }@@ -6979,8 +6978,7 @@ struct nfs4_lock_waiter { .s_dev = server->s_dev };struct nfs4_lock_waiter waiter = { .task = current, .inode = state->inode, - .owner = &owner, - .notified = false }; + .owner = &owner};wait_queue_entry_t wait;/* Don't bother with waitqueue if we don't expect a callback */ @@ -6993,21 +6991,14 @@ struct nfs4_lock_waiter {add_wait_queue (q, &wait);while (!signalled ()) { - waiter.notified = false; status =nfs4_proc_setlk (state, cmd, request);if ((status != -EAGAIN ) ||IS_SETLK (cmd))break ; status = -ERESTARTSYS ; -spin_lock_irqsave (&q->lock, flags); -if (waiter.notified) {
fs/nfs/nfs4proc.c struct nfs4_lock_waiter { · line 7001
Quoted 10 lines
fs/nfs/nfs4proc.c +3
-7-spin_unlock_irqrestore (&q->lock, flags); -continue ; - } -set_current_state (TASK_INTERRUPTIBLE ); -spin_unlock_irqrestore (&q->lock, flags); - -freezable_schedule_timeout (NFS4_LOCK_MAXTIMEOUT ); +freezer_do_not_count (); +wait_woken (&wait,TASK_INTERRUPTIBLE ,NFS4_LOCK_MAXTIMEOUT ); +freezer_count ();
Since now variable 'flags' is not used anymore, we have to delete it as well. Otherwise there is a compile warning “unused variable ‘flags’”.
Thanks,
Joseph
Quoted 4 lines
fs/nfs/nfs4proc.c +0
-0}finish_wait (q, &wait);
Re: [PATCH v2 1/2] NFSv4.1: Again fix a race where CB_NOTIFY_LOCK fails to wake a waiter
Quoted 84 lines · Joseph Qi
Hi Yihao,
Quoted 67 lines · Yihao Wu
Commit
b7dbcc0e433f"NFSv4.1: Fix a race whereCB_NOTIFY_LOCKfails to wake a waiter" found this bug. However it didn't fix it.This commit replaces
schedule_timeout()withwait_woken()anddefault_wake_function()withwoken_wake_function()in functionnfs4_retry_setlk()andnfs4_wake_lock_waiter().wait_woken()uses memory barriers in its implementation to avoid potential race condition when putting a process into sleeping state and then waking it up.Fixes a1d617d8f134 ("nfs: allow blocking locks to be awoken by lock callbacks")Cc [email protected] #4.9+Signed-off-by Yihao Wu <[email protected]>--- fs/nfs/nfs4proc.c | 23 +++++++---------------- 1 file changed, 7 insertions(+), 16 deletions(-)fs/nfs/nfs4proc.c +4
-9@@ -6932,7 +6932,6 @@ struct nfs4_lock_waiter {struct task_struct *task;struct inode *inode;struct nfs_lowner *owner; -bool notified; }; static int@@ -6954,13 +6953,13 @@ struct nfs4_lock_waiter {/* Make sure it's for the right inode */ if (nfs_compare_fh (NFS_FH (waiter->inode), &cbnl->cbnl_fh))return 0 ; - - waiter->notified = true; }/* override "private" so we can use default_wake_function */ wait->private = waiter->task; - ret =autoremove_wake_function (wait, mode, flags, key); + ret =woken_wake_function (wait, mode, flags, key); +if (ret) +list_del_init (&wait->entry); wait->private = waiter;return ret; }@@ -6979,8 +6978,7 @@ struct nfs4_lock_waiter { .s_dev = server->s_dev };struct nfs4_lock_waiter waiter = { .task = current, .inode = state->inode, - .owner = &owner, - .notified = false }; + .owner = &owner};wait_queue_entry_t wait;/* Don't bother with waitqueue if we don't expect a callback */ @@ -6993,21 +6991,14 @@ struct nfs4_lock_waiter {add_wait_queue (q, &wait);while (!signalled ()) { - waiter.notified = false; status =nfs4_proc_setlk (state, cmd, request);if ((status != -EAGAIN ) ||IS_SETLK (cmd))break ; status = -ERESTARTSYS ; -spin_lock_irqsave (&q->lock, flags); -if (waiter.notified) {fs/nfs/nfs4proc.c struct nfs4_lock_waiter { · line 7001
Quoted 10 lines
fs/nfs/nfs4proc.c +3
-7-spin_unlock_irqrestore (&q->lock, flags); -continue ; - } -set_current_state (TASK_INTERRUPTIBLE ); -spin_unlock_irqrestore (&q->lock, flags); - -freezable_schedule_timeout (NFS4_LOCK_MAXTIMEOUT ); +freezer_do_not_count (); +wait_woken (&wait,TASK_INTERRUPTIBLE ,NFS4_LOCK_MAXTIMEOUT ); +freezer_count ();Since now variable 'flags' is not used anymore, we have to delete it as well. Otherwise there is a compile warning “unused variable ‘flags’”.
Quoted 2 lines
Thanks,
Joseph
Thank you Joseph. I'll remove unused 'flags' in PATCH v3.
Thanks,
Yihao Wu
Quoted 5 lines
Quoted 5 lines
fs/nfs/nfs4proc.c +0
-0}finish_wait (q, &wait);
PATCH v3 1/2 NFSv4.1: Again fix a race where CB_NOTIFY_LOCK fails to wake a waiter
In reply to Yihao WuCommit b7dbcc0e433f "NFSv4.1: Fix a race where CB_NOTIFY_LOCK fails to wake a waiter" found this bug. However it didn't fix it.
This commit replaces schedule_timeout() with wait_woken() and default_wake_function() with woken_wake_function() in function nfs4_retry_setlk() and nfs4_wake_lock_waiter(). wait_woken() uses memory barriers in its implementation to avoid potential race condition when putting a process into sleeping state and then waking it up.
---v2->v3: remove unused variable flags in nfs4_retry_setlk()
fs/nfs/nfs4proc.c | 24 +++++++----------------- 1 file changed, 7 insertions(+), 17 deletions(-)
fs/nfs/nfs4proc.c +7 -17
@@ -6932,7 +6932,6 @@ struct nfs4_lock_waiter {struct task_struct *task;struct inode *inode;struct nfs_lowner *owner; -bool notified; }; static int
@@ -6954,13 +6953,13 @@ struct nfs4_lock_waiter {/* Make sure it's for the right inode */ if (nfs_compare_fh (NFS_FH (waiter->inode), &cbnl->cbnl_fh))return 0 ; - - waiter->notified = true; }/* override "private" so we can use default_wake_function */ wait->private = waiter->task; - ret =autoremove_wake_function (wait, mode, flags, key); + ret =woken_wake_function (wait, mode, flags, key); +if (ret) +list_del_init (&wait->entry); wait->private = waiter;return ret; }
@@ -6969,7 +6968,6 @@ struct nfs4_lock_waiter { nfs4_retry_setlk(struct nfs4_state *state, int cmd, struct file_lock *request) { int status = -ERESTARTSYS; -unsigned long flags;struct nfs4_lock_state *lsp = request->fl_u.nfs4_fl.owner;struct nfs_server *server =NFS_SERVER (state->inode);struct nfs_client *clp = server->nfs_client;
@@ -6979,8 +6977,7 @@ struct nfs4_lock_waiter { .s_dev = server->s_dev };struct nfs4_lock_waiter waiter = { .task = current, .inode = state->inode, - .owner = &owner, - .notified = false }; + .owner = &owner};wait_queue_entry_t wait;/* Don't bother with waitqueue if we don't expect a callback */
@@ -6993,21 +6990,14 @@ struct nfs4_lock_waiter {add_wait_queue (q, &wait);while (!signalled ()) { - waiter.notified = false; status =nfs4_proc_setlk (state, cmd, request);if ((status != -EAGAIN ) ||IS_SETLK (cmd))break ; status = -ERESTARTSYS ; -spin_lock_irqsave (&q->lock, flags); -if (waiter.notified) { -spin_unlock_irqrestore (&q->lock, flags); -continue ; - } -set_current_state (TASK_INTERRUPTIBLE ); -spin_unlock_irqrestore (&q->lock, flags); - -freezable_schedule_timeout (NFS4_LOCK_MAXTIMEOUT ); +freezer_do_not_count (); +wait_woken (&wait,TASK_INTERRUPTIBLE ,NFS4_LOCK_MAXTIMEOUT ); +freezer_count (); }finish_wait (q, &wait);
Signature · 1 line
1.8.3.1
PATCH v2 2/2 NFSv4.1: Fix bug only first CB_NOTIFY_LOCK is handled
In reply to Yihao WuWhen a waiter is waked by CB_NOTIFY_LOCK, it will retry nfs4_proc_setlk(). The waiter may fail to nfs4_proc_setlk() and sleep again. However, the waiter is already removed from clp->cl_lock_waitq when handling CB_NOTIFY_LOCK in nfs4_wake_lock_waiter(). So any subsequent CB_NOTIFY_LOCK won't wake this waiter anymore. We should put the waiter back to clp->cl_lock_waitq before retrying.
--- fs/nfs/nfs4proc.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-)
fs/nfs/nfs4proc.c +5 -3
@@ -6988,20 +6988,22 @@ struct nfs4_lock_waiter {init_wait (&wait); wait.private = &waiter; wait.func = nfs4_wake_lock_waiter; -add_wait_queue (q, &wait);while (!signalled ()) { +add_wait_queue (q, &wait); status =nfs4_proc_setlk (state, cmd, request); -if ((status != -EAGAIN ) ||IS_SETLK (cmd)) +if ((status != -EAGAIN ) ||IS_SETLK (cmd)) { +finish_wait (q, &wait);break ; + } status = -ERESTARTSYS ;freezer_do_not_count ();wait_woken (&wait,TASK_INTERRUPTIBLE ,NFS4_LOCK_MAXTIMEOUT );freezer_count (); +finish_wait (q, &wait); } -finish_wait (q, &wait); return status; }#else /* !CONFIG_NFS_V4_1 */
Signature · 1 line
1.8.3.1
Quoted 11 lines · Yihao Wu
This patch set fix bugs related to
CB_NOTIFY_LOCK. These bugs cause poor performance in locking operation. And this patchset should also fix the failure when running xfstests generic/089 on a NFSv4.1 filesystem.Yihao Wu (2):
NFSv4.1: Again fix a race whereCB_NOTIFY_LOCKfails to wake a waiter NFSv4.1: Fix bug only firstCB_NOTIFY_LOCKis handledfs/nfs/nfs4proc.c | 31 ++++++++++++------------------- 1 file changed, 12 insertions(+), 19 deletions(-)
Looks good to me. Nice catch, btw!
Quoted 9 lines · Jeff Layton
Quoted 8 lines · Yihao Wu
This patch set fix bugs related to
CB_NOTIFY_LOCK. These bugs cause poor performance in locking operation. And this patchset should also fix the failure when running xfstests generic/089 on a NFSv4.1 filesystem.Yihao Wu (2):
NFSv4.1: Again fix a race whereCB_NOTIFY_LOCKfails to wake a waiter NFSv4.1: Fix bug only firstCB_NOTIFY_LOCKis handled
Quoted 8 lines
Quoted 3 lines
fs/nfs/nfs4proc.c | 31 ++++++++++++------------------- 1 file changed, 12 insertions(+), 19 deletions(-)Looks good to me. Nice catch, btw!
Reviewed-by Jeff Layton <[email protected]>
Thank you for your reviewing, Jeff!
And it seems I forgot to CC maintainers of fs/nfs. So I added you to the CC list, Trond and Anna. Does this patchset need further reviewing?
Sorry to bother you.
Thanks,
Yihao Wu
Jeff Layton
maintainerQuoted 19 lines · Yihao Wu
Quoted 9 lines · Jeff Layton
Quoted 8 lines · Yihao Wu
This patch set fix bugs related to
CB_NOTIFY_LOCK. These bugs cause poor performance in locking operation. And this patchset should also fix the failure when running xfstests generic/089 on a NFSv4.1 filesystem.Yihao Wu (2):
NFSv4.1: Again fix a race whereCB_NOTIFY_LOCKfails to wake a waiter NFSv4.1: Fix bug only firstCB_NOTIFY_LOCKis handledQuoted 8 lines
Quoted 3 lines
fs/nfs/nfs4proc.c | 31 ++++++++++++------------------- 1 file changed, 12 insertions(+), 19 deletions(-)Looks good to me. Nice catch, btw!
Reviewed-by Jeff Layton <[email protected]>
Quoted 7 lines
Thank you for your reviewing, Jeff!
And it seems I forgot to CC maintainers of fs/nfs. So I added you to the CC list, Trond and Anna. Does this patchset need further reviewing?
Sorry to bother you.
This should probably go through Anna's tree, and I assume she'll pick it up once she and/or Trond have had a chance to review it.
Thanks,
Signature · 1 line
Jeff Layton <[email protected]>