nfsd: more delegation fixes to prepare for client_mutex removal
Subsystems (4)
v2 cover nfsd: more delegation fixes to prepare for client_mutex removal
Here's a respin of the delegation fixes that I sent yesterday. The main change is to move the remove_stid calls into the caller in the second patch instead of keeping it in nfs4_free_stid, per Christoph's suggestion.
Unfortunately, that causes a pile of merge conflicts in the later patches, so I need to resend the whole set.
I also include the patch that converts the num_delegations counter to an atomic_long_t, as I missed that in the earlier series.
There is one more patch that fixes up handling of the cl_revoked list that comes later, but it can't easily be moved here as it depends on some locking changes that come later.
Jeff Layton (6):
nfsd: eliminate nfsd4_init_callback
nfsd: Avoid taking state_lock while holding inode lock in
nfsd_break_one_deleg
nfsd: nfs4_alloc_init_lease should take a nfs4_file arg
locks: add file_has_lease to prevent delegation break races
nfsd: Protect the nfs4_file delegation fields using the fi_lock
nfsd: Fix delegation revocation
Trond Myklebust (4):
nfsd: Ensure stateids remain unique until they are freed
nfsd: Move the delegation reference counter into the struct nfs4_stid
nfsd: Simplify stateid management
nfsd: Convert delegation counter to an atomic_long_t type
fs/locks.c | 26 +++++++ fs/nfsd/nfs4callback.c | 23 ++++-- fs/nfsd/nfs4state.c | 197 +++++++++++++++++++++++++++++++------------------ fs/nfsd/state.h | 7 +- include/linux/fs.h | 6 ++ 5 files changed, 179 insertions(+), 80 deletions(-)
Signature · 1 line
1.9.3
PATCH v2 1/10 nfsd: eliminate nfsd4_init_callback
It's just an obfuscated INIT_WORK call. Just make the work_func_t a non-static symbol and use a normal INIT_WORK call.
--- fs/nfsd/nfs4callback.c | 8 ++------ fs/nfsd/nfs4state.c | 4 ++-- fs/nfsd/state.h | 2 +- 3 files changed, 5 insertions(+), 9 deletions(-)
fs/nfsd/nfs4callback.c +2 -6
@@ -1011,7 +1011,8 @@ static void nfsd4_process_cb_update(struct nfsd4_callback *cb)run_nfsd4_cb (cb); } -static void nfsd4_do_callback_rpc (struct work_struct *w) +void +nfsd4_do_callback_rpc (struct work_struct *w) { struct nfsd4_callback *cb = container_of(w, struct nfsd4_callback, cb_work);struct nfs4_client *clp = cb->cb_clp;
@@ -1031,11 +1032,6 @@ static void nfsd4_do_callback_rpc(struct work_struct *w) cb->cb_ops, cb); } -void nfsd4_init_callback (struct nfsd4_callback *cb) -{ -INIT_WORK (&cb->cb_work, nfsd4_do_callback_rpc); -} -void nfsd4_cb_recall (struct nfs4_delegation *dp) {struct nfsd4_callback *cb = &dp->dl_recall;
fs/nfsd/nfs4state.c +2 -2
@@ -592,7 +592,7 @@ alloc_init_deleg(struct nfs4_client *clp, struct nfs4_ol_stateid *stp, struct svfh_copy_shallow (&dp->dl_fh, ¤t_fh->fh_handle); dp->dl_time =0 ;atomic_set (&dp->dl_count,1 ); -nfsd4_init_callback (&dp->dl_recall); +INIT_WORK (&dp->dl_recall.cb_work, nfsd4_do_callback_rpc);return dp; }
@@ -1677,7 +1677,7 @@ static struct nfs4_client *create_client(struct xdr_netobj name,spin_unlock (&nn->client_lock);return NULL ; } -nfsd4_init_callback (&clp->cl_cb_null); +INIT_WORK (&clp->cl_cb_null.cb_work, nfsd4_do_callback_rpc); clp->cl_time =get_seconds ();clear_bit (0 , &clp->cl_cb_slot_busy);copy_verf (clp, verf);
fs/nfsd/state.h +1 -1
@@ -436,7 +436,7 @@ extern struct nfs4_client_reclaim *nfsd4_find_reclaim_client(const char *recdir,extern __be32 nfs4_check_open_reclaim (clientid_t *clid,struct nfsd4_compound_state *cstate,struct nfsd_net *nn);extern int set_callback_cred (void ); -extern void nfsd4_init_callback (struct nfsd4_callback *); +void nfsd4_do_callback_rpc (struct work_struct *w);extern void nfsd4_probe_callback (struct nfs4_client *clp);extern void nfsd4_probe_callback_sync (struct nfs4_client *clp);extern void nfsd4_change_callback (struct nfs4_client *clp,struct nfs4_cb_conn *);
Signature · 1 line
1.9.3
PATCH v2 2/10 nfsd: Avoid taking state_lock while holding inode lock in nfsd_break_one_deleg
In reply to Jeff Laytonstate_lock is a heavily contended global lock. We don't want to grab that while simultaneously holding the inode->i_lock.
Add a new per-nfs4_file lock that we can use to protect the per-nfs4_file delegation list. Hold that while walking the list in the break_deleg callback and queue the workqueue job for each one.
The workqueue job can then take the state_lock and do the list manipulations without the i_lock being held prior to starting the rpc call.
--- fs/nfsd/nfs4callback.c | 25 ++++++++++++++++++---- fs/nfsd/nfs4state.c | 58 +++++++++++++++++++++++++++++++++----------------- fs/nfsd/state.h | 4 +++- 3 files changed, 62 insertions(+), 25 deletions(-)
fs/nfsd/nfs4callback.c +21 -4
@@ -933,7 +933,7 @@ void nfsd4_shutdown_callback(struct nfs4_client *clp)set_bit (NFSD4_CLIENT_CB_KILL , &clp->cl_flags);/* * Note this won't actually result in a null callback; -* instead, nfsd4_do_callback_rpc() will detect the killed +* instead, nfsd4_run_cb_null() will detect the killed * client, destroy the rpc client, and stop: */ do_probe_callback (clp);
@@ -1011,10 +1011,9 @@ static void nfsd4_process_cb_update(struct nfsd4_callback *cb)run_nfsd4_cb (cb); } -void -nfsd4_do_callback_rpc (struct work_struct *w) +static void +nfsd4_run_callback_rpc (struct nfsd4_callback *cb) { - struct nfsd4_callback *cb = container_of(w, struct nfsd4_callback, cb_work);struct nfs4_client *clp = cb->cb_clp;struct rpc_clnt *clnt;
@@ -1032,6 +1031,24 @@ nfsd4_do_callback_rpc(struct work_struct *w) cb->cb_ops, cb); } +void +nfsd4_run_cb_null (struct work_struct *w) +{ + struct nfsd4_callback *cb = container_of(w, struct nfsd4_callback, + cb_work); +nfsd4_run_callback_rpc (cb); +} + +void +nfsd4_run_cb_recall (struct work_struct *w) +{ + struct nfsd4_callback *cb = container_of(w, struct nfsd4_callback, + cb_work); + +nfsd4_prepare_cb_recall (cb->cb_op); +nfsd4_run_callback_rpc (cb); +} +void nfsd4_cb_recall (struct nfs4_delegation *dp) {struct nfsd4_callback *cb = &dp->dl_recall;
fs/nfsd/nfs4state.c +38 -20
126 lines not on this page · raw message
fs/nfsd/state.h +3 -1
@@ -436,7 +436,8 @@ extern struct nfs4_client_reclaim *nfsd4_find_reclaim_client(const char *recdir,extern __be32 nfs4_check_open_reclaim (clientid_t *clid,struct nfsd4_compound_state *cstate,struct nfsd_net *nn);extern int set_callback_cred (void ); -void nfsd4_do_callback_rpc (struct work_struct *w); +void nfsd4_run_cb_null (struct work_struct *w); +void nfsd4_run_cb_recall (struct work_struct *w);extern void nfsd4_probe_callback (struct nfs4_client *clp);extern void nfsd4_probe_callback_sync (struct nfs4_client *clp);extern void nfsd4_change_callback (struct nfs4_client *clp,struct nfs4_cb_conn *);
@@ -444,6 +445,7 @@ extern void nfsd4_cb_recall(struct nfs4_delegation *dp);extern int nfsd4_create_callback_queue (void );extern void nfsd4_destroy_callback_queue (void );extern void nfsd4_shutdown_callback (struct nfs4_client *); +extern void nfsd4_prepare_cb_recall (struct nfs4_delegation *dp);extern void nfs4_put_delegation (struct nfs4_delegation *dp);extern struct nfs4_client_reclaim *nfs4_client_to_reclaim (const char *name,struct nfsd_net *nn);
Signature · 1 line
1.9.3
Re: [PATCH v2 02/10] nfsd: Avoid taking state_lock while holding inode lock in nfsd_break_one_deleg
Quoted 10 lines · Jeff Layton
state_lockis a heavily contended global lock. We don't want to grab that while simultaneously holding theinode->i_lock.Add a new per-nfs4_file lock that we can use to protect the per-nfs4_file delegation list. Hold that while walking the list in the
break_delegcallback and queue the workqueue job for each one.The workqueue job can then take the
state_lockand do the list manipulations without thei_lockbeing held prior to starting the rpc call.
Looks fine.
As a separate issue: does nfs4_state_shutdown net have the same problem as destroy_client? It seems like there should be a cp->dl_time++ there too.
Though I'm not sure what the point of that del_recall_lru loop is at all. nfs4_state_destroy_net() destroys each client and its associated delegations. Maybe that function should just be
nfs4_lock_state(); nfs4_client_tracking_exit(net); nfs4_state_destroy_net(net); WARN_ON(!list_empty(&nn->del_recall_lru); nfs4_unlock_state();
--b.
Quoted 224 lines
Signed-off-by Trond Myklebust <[email protected]>Signed-off-by Jeff Layton <[email protected]>Reviewed-by Christoph Hellwig <[email protected]>--- fs/nfsd/nfs4callback.c | 25 ++++++++++++++++++---- fs/nfsd/nfs4state.c | 58 +++++++++++++++++++++++++++++++++----------------- fs/nfsd/state.h | 4 +++- 3 files changed, 62 insertions(+), 25 deletions(-)fs/nfsd/nfs4callback.c +21
-4@@ -933,7 +933,7 @@ void nfsd4_shutdown_callback(struct nfs4_client *clp)set_bit (NFSD4_CLIENT_CB_KILL , &clp->cl_flags);/* * Note this won't actually result in a null callback; -* instead, nfsd4_do_callback_rpc() will detect the killed +* instead, nfsd4_run_cb_null() will detect the killed * client, destroy the rpc client, and stop: */ do_probe_callback (clp);@@ -1011,10 +1011,9 @@ static void nfsd4_process_cb_update(struct nfsd4_callback *cb)run_nfsd4_cb (cb); } -void -nfsd4_do_callback_rpc (struct work_struct *w) +static void +nfsd4_run_callback_rpc (struct nfsd4_callback *cb) { - struct nfsd4_callback *cb = container_of(w, struct nfsd4_callback, cb_work);struct nfs4_client *clp = cb->cb_clp;struct rpc_clnt *clnt;@@ -1032,6 +1031,24 @@ nfsd4_do_callback_rpc(struct work_struct *w) cb->cb_ops, cb); } +void +nfsd4_run_cb_null (struct work_struct *w) +{ + struct nfsd4_callback *cb = container_of(w, struct nfsd4_callback, + cb_work); +nfsd4_run_callback_rpc (cb); +} + +void +nfsd4_run_cb_recall (struct work_struct *w) +{ + struct nfsd4_callback *cb = container_of(w, struct nfsd4_callback, + cb_work); + +nfsd4_prepare_cb_recall (cb->cb_op); +nfsd4_run_callback_rpc (cb); +} +void nfsd4_cb_recall (struct nfs4_delegation *dp) {struct nfsd4_callback *cb = &dp->dl_recall;fs/nfsd/nfs4state.c +38
-20@@ -254,6 +254,8 @@ static void nfsd4_free_file(struct nfs4_file *f)static inline void put_nfs4_file (struct nfs4_file *fi) { +might_lock (&state_lock); +if (atomic_dec_and_lock (&fi->fi_ref, &state_lock)) {hlist_del (&fi->fi_hash);spin_unlock (&state_lock);@@ -554,6 +556,8 @@ static void block_delegations(struct knfsd_fh *fh)u32 hash;struct bloom_pair *bd = &blocked_delegations; +lockdep_assert_held (&state_lock); + hash =arch_fast_hash (&fh->fh_base, fh->fh_size,0 );__set_bit (hash&255 , bd->set[bd->new]);@@ -592,7 +596,7 @@ alloc_init_deleg(struct nfs4_client *clp, struct nfs4_ol_stateid *stp, struct svfh_copy_shallow (&dp->dl_fh, ¤t_fh->fh_handle); dp->dl_time =0 ;atomic_set (&dp->dl_count,1 ); -INIT_WORK (&dp->dl_recall.cb_work, nfsd4_do_callback_rpc); +INIT_WORK (&dp->dl_recall.cb_work, nfsd4_run_cb_recall);return dp; }@@ -640,7 +644,9 @@ hash_delegation_locked(struct nfs4_delegation *dp, struct nfs4_file *fp)lockdep_assert_held (&state_lock); dp->dl_stid.sc_type =NFS4_DELEG_STID ; +spin_lock (&fp->fi_lock);list_add (&dp->dl_perfile, &fp->fi_delegations); +spin_unlock (&fp->fi_lock);list_add (&dp->dl_perclnt, &dp->dl_stid.sc_client->cl_delegations); }@@ -648,14 +654,18 @@ hash_delegation_locked(struct nfs4_delegation *dp, struct nfs4_file *fp)static void unhash_delegation (struct nfs4_delegation *dp) { +struct nfs4_file *fp = dp->dl_file; +spin_lock (&state_lock);list_del_init (&dp->dl_perclnt); -list_del_init (&dp->dl_perfile);list_del_init (&dp->dl_recall_lru); +spin_lock (&fp->fi_lock); +list_del_init (&dp->dl_perfile); +spin_unlock (&fp->fi_lock);spin_unlock (&state_lock); -if (dp->dl_file) { -nfs4_put_deleg_lease (dp->dl_file); -put_nfs4_file (dp->dl_file); +if (fp) { +nfs4_put_deleg_lease (fp); +put_nfs4_file (fp); dp->dl_file =NULL ; } }@@ -1677,7 +1687,7 @@ static struct nfs4_client *create_client(struct xdr_netobj name,spin_unlock (&nn->client_lock);return NULL ; } -INIT_WORK (&clp->cl_cb_null.cb_work, nfsd4_do_callback_rpc); +INIT_WORK (&clp->cl_cb_null.cb_work, nfsd4_run_cb_null); clp->cl_time =get_seconds ();clear_bit (0 , &clp->cl_cb_slot_busy);copy_verf (clp, verf);@@ -3079,30 +3089,38 @@ nfs4_share_conflict(struct svc_fh *current_fh, unsigned int deny_type)return ret; } -static void nfsd_break_one_deleg (struct nfs4_delegation *dp) +void nfsd4_prepare_cb_recall (struct nfs4_delegation *dp) {struct nfs4_client *clp = dp->dl_stid.sc_client;struct nfsd_net *nn =net_generic (clp->net, nfsd_net_id); -lockdep_assert_held (&state_lock); -/* We're assuming the state code never drops its reference -* without first removing the lease. Since we're in this lease -* callback (and since the lease code is serialized by the kernel -* lock) we know the server hasn't removed the lease yet, we know -* it's safe to take a reference: */ -atomic_inc (&dp->dl_count); - +/* +* We can't do this in nfsd_break_deleg_cb because it is +* already holding inode->i_lock +*/ +spin_lock (&state_lock); +block_delegations (&dp->dl_fh);/* * If the dl_time != 0, then we know that it has already been * queued for a lease break. Don't queue it again. */ if (dp->dl_time ==0 ) { -list_add_tail (&dp->dl_recall_lru, &nn->del_recall_lru); dp->dl_time =get_seconds (); +list_add_tail (&dp->dl_recall_lru, &nn->del_recall_lru); } +spin_unlock (&state_lock); +} -block_delegations (&dp->dl_fh); - +static void nfsd_break_one_deleg (struct nfs4_delegation *dp) +{ +/* +* We're assuming the state code never drops its reference +* without first removing the lease. Since we're in this lease +* callback (and since the lease code is serialized by the kernel +* lock) we know the server hasn't removed the lease yet, we know +* it's safe to take a reference. +*/ +atomic_inc (&dp->dl_count);nfsd4_cb_recall (dp); }@@ -3127,11 +3145,11 @@ static void nfsd_break_deleg_cb(struct file_lock *fl)*/ fl->fl_break_time =0 ; -spin_lock (&state_lock); fp->fi_had_conflict = true; +spin_lock (&fp->fi_lock);list_for_each_entry (dp, &fp->fi_delegations, dl_perfile)nfsd_break_one_deleg (dp); -spin_unlock (&state_lock); +spin_unlock (&fp->fi_lock); } staticfs/nfsd/state.h +3
-1@@ -436,7 +436,8 @@ extern struct nfs4_client_reclaim *nfsd4_find_reclaim_client(const char *recdir,extern __be32 nfs4_check_open_reclaim (clientid_t *clid,struct nfsd4_compound_state *cstate,struct nfsd_net *nn);extern int set_callback_cred (void ); -void nfsd4_do_callback_rpc (struct work_struct *w); +void nfsd4_run_cb_null (struct work_struct *w); +void nfsd4_run_cb_recall (struct work_struct *w);extern void nfsd4_probe_callback (struct nfs4_client *clp);extern void nfsd4_probe_callback_sync (struct nfs4_client *clp);extern void nfsd4_change_callback (struct nfs4_client *clp,struct nfs4_cb_conn *);@@ -444,6 +445,7 @@ extern void nfsd4_cb_recall(struct nfs4_delegation *dp);extern int nfsd4_create_callback_queue (void );extern void nfsd4_destroy_callback_queue (void );extern void nfsd4_shutdown_callback (struct nfs4_client *); +extern void nfsd4_prepare_cb_recall (struct nfs4_delegation *dp);extern void nfs4_put_delegation (struct nfs4_delegation *dp);extern struct nfs4_client_reclaim *nfs4_client_to_reclaim (const char *name,struct nfsd_net *nn);Signature · 1 line
1.9.3
Re: [PATCH v2 02/10] nfsd: Avoid taking state_lock while holding inode lock in nfsd_break_one_deleg
Quoted 12 lines · On Wed, 16 Jul 2014 21:34:41 -0400 "J. Bruce Fields" <[email protected]>…
Quoted 10 lines · Jeff Layton
state_lockis a heavily contended global lock. We don't want to grab that while simultaneously holding theinode->i_lock.Add a new per-nfs4_file lock that we can use to protect the per-nfs4_file delegation list. Hold that while walking the list in the
break_delegcallback and queue the workqueue job for each one.The workqueue job can then take the
state_lockand do the list manipulations without thei_lockbeing held prior to starting the rpc call.
Quoted 6 lines
Looks fine.
As a separate issue: does
nfs4_state_shutdownnet have the same problem asdestroy_client? It seems like there should be a cp->dl_time++ there too.
I had looked at that before and convinced myself that it wasn't needed, but now I can't recall why. I think you're correct though -- we ought to bump it there too. It certainly won't hurt to do so and it might prevent the same race there.
Quoted 4 lines
Though I'm not sure what the point of that
del_recall_lruloop is at all.nfs4_state_destroy_net()destroys each client and its associated delegations. Maybe that function should just be
Quoted 8 lines
nfs4_lock_state(); nfs4_client_tracking_exit(net); nfs4_state_destroy_net(net); WARN_ON(!list_empty(&nn->del_recall_lru); nfs4_unlock_state();--b.
Yeah, it is odd. I end up relying on that for the "recall_delegations" fault injector in a later patch in the series, so I'd probably prefer to leave it as-is for now. We should be able to get rid of it eventually though.
Quoted 234 lines
Quoted 234 lines
Signed-off-by Trond Myklebust <[email protected]>Signed-off-by Jeff Layton <[email protected]>Reviewed-by Christoph Hellwig <[email protected]>--- fs/nfsd/nfs4callback.c | 25 ++++++++++++++++++---- fs/nfsd/nfs4state.c | 58+++++++++++++++++++++++++++++++++-----------------
fs/nfsd/state.h| 4 +++- 3 files changed, 62 insertions(+),
25 deletions(-)fs/nfsd/nfs4callback.c +21
-4@@ -933,7 +933,7 @@ void nfsd4_shutdown_callback(struct nfs4_client *clp) set_bit(NFSD4_CLIENT_CB_KILL, &clp->cl_flags);/* * Note this won't actually result in a null callback; -* instead, nfsd4_do_callback_rpc() will detect the killed +* instead, nfsd4_run_cb_null() will detect the killed * client, destroy the rpc client, and stop: */ do_probe_callback (clp);@@ -1011,10 +1011,9 @@ static void nfsd4_process_cb_update(struct nfsd4_callback *cb) run_nfsd4_cb(cb); } -void -nfsd4_do_callback_rpc (struct work_struct *w) +static void +nfsd4_run_callback_rpc (struct nfsd4_callback *cb) { - struct nfsd4_callback *cb = container_of(w, struct nfsd4_callback, cb_work); struct nfs4_client *clp = cb->cb_clp;struct rpc_clnt *clnt;@@ -1032,6 +1031,24 @@ nfsd4_do_callback_rpc(struct work_struct *w) cb->cb_ops, cb); } +void +nfsd4_run_cb_null (struct work_struct *w) +{ + struct nfsd4_callback *cb = container_of(w, struct nfsd4_callback, + cb_work); +nfsd4_run_callback_rpc (cb); +} + +void +nfsd4_run_cb_recall (struct work_struct *w) +{ + struct nfsd4_callback *cb = container_of(w, struct nfsd4_callback, + cb_work); + +nfsd4_prepare_cb_recall (cb->cb_op); +nfsd4_run_callback_rpc (cb); +} +void nfsd4_cb_recall (struct nfs4_delegation *dp){
fs/nfsd/nfs4callback.c +0
-0struct nfsd4_callback *cb = &dp->dl_recall;fs/nfsd/nfs4state.c +17
-10@@ -254,6 +254,8 @@ static void nfsd4_free_file(struct nfs4_file *f)static inline void put_nfs4_file (struct nfs4_file *fi) { +might_lock (&state_lock); +if (atomic_dec_and_lock (&fi->fi_ref, &state_lock)) {hlist_del (&fi->fi_hash);spin_unlock (&state_lock);@@ -554,6 +556,8 @@ static void block_delegations(struct knfsd_fh *fh) u32 hash;struct bloom_pair *bd = &blocked_delegations; +lockdep_assert_held (&state_lock); + hash =arch_fast_hash (&fh->fh_base, fh->fh_size,0 );__set_bit (hash&255 , bd->set[bd->new]);@@ -592,7 +596,7 @@ alloc_init_deleg(struct nfs4_client *clp, struct nfs4_ol_stateid *stp, struct sv fh_copy_shallow(&dp->dl_fh, ¤t_fh->fh_handle); dp->dl_time = 0;atomic_set (&dp->dl_count,1 ); -INIT_WORK (&dp->dl_recall.cb_work, nfsd4_do_callback_rpc); +INIT_WORK (&dp->dl_recall.cb_work, nfsd4_run_cb_recall);return dp; }@@ -640,7 +644,9 @@ hash_delegation_locked(struct nfs4_delegation *dp, struct nfs4_file *fp) lockdep_assert_held(&state_lock); dp->dl_stid.sc_type =NFS4_DELEG_STID ; +spin_lock (&fp->fi_lock);list_add (&dp->dl_perfile, &fp->fi_delegations); +spin_unlock (&fp->fi_lock);list_add (&dp->dl_perclnt, &dp->dl_stid.sc_client->cl_delegations); }@@ -648,14 +654,18 @@ hash_delegation_locked(struct nfs4_delegation *dp, struct nfs4_file *fp) static voidunhash_delegation (struct nfs4_delegation *dp) { +struct nfs4_file *fp = dp->dl_file; +spin_lock (&state_lock);list_del_init (&dp->dl_perclnt); -list_del_init (&dp->dl_perfile);list_del_init (&dp->dl_recall_lru); +spin_lock (&fp->fi_lock); +list_del_init (&dp->dl_perfile); +spin_unlock (&fp->fi_lock);spin_unlock (&state_lock); -if (dp->dl_file) { -nfs4_put_deleg_lease (dp->dl_file); -put_nfs4_file (dp->dl_file); +if (fp) { +nfs4_put_deleg_lease (fp); +put_nfs4_file (fp); dp->dl_file =NULL ; } }@@ -1677,7 +1687,7 @@ static struct nfs4_client *create_client(struct xdr_netobj name, spin_unlock(&nn->client_lock); return NULL; } -INIT_WORK (&clp->cl_cb_null.cb_work, nfsd4_do_callback_rpc); +INIT_WORK (&clp->cl_cb_null.cb_work, nfsd4_run_cb_null); clp->cl_time =get_seconds ();clear_bit (0 , &clp->cl_cb_slot_busy);copy_verf (clp, verf);@@ -3079,30 +3089,38 @@ nfs4_share_conflict(struct svc_fh *current_fh, unsigned int deny_type) return ret; } -static void nfsd_break_one_deleg (struct nfs4_delegation *dp) +void nfsd4_prepare_cb_recall (struct nfs4_delegation *dp) {struct nfs4_client *clp = dp->dl_stid.sc_client;struct nfsd_net *nn =net_generic (clp->net, nfsd_net_id); -lockdep_assert_held (&state_lock); -/* We're assuming the state code never drops its reference -* without first removing the lease. Since we're in this lease
fs/nfsd/nfs4state.c +0
-1-* callback (and since the lease code is serialized by the kernel
fs/nfsd/nfs4state.c +0
-1-* lock) we know the server hasn't removed the lease yet, we know
fs/nfsd/nfs4state.c +6
-3-* it's safe to take a reference: */ -atomic_inc (&dp->dl_count); - +/* +* We can't do this in nfsd_break_deleg_cb because it is +* already holding inode->i_lock +*/ +spin_lock (&state_lock); +block_delegations (&dp->dl_fh);/* * If the dl_time != 0, then we know that it has already been
fs/nfsd/nfs4state.c +0
-1* queued for a lease break. Don't queue it again. */ if (dp->dl_time ==0 ) { - list_add_tail(&dp->dl_recall_lru,
&nn->del_recall_lru);dp->dl_time=get_seconds();fs/nfsd/nfs4state.c +1
-0+ list_add_tail(&dp->dl_recall_lru,
&nn->del_recall_lru); }fs/nfsd/nfs4state.c +7
-2+spin_unlock (&state_lock); +} -block_delegations (&dp->dl_fh); - +static void nfsd_break_one_deleg (struct nfs4_delegation *dp) +{ +/* +* We're assuming the state code never drops its reference +* without first removing the lease. Since we're in this lease
fs/nfsd/nfs4state.c +1
-0+* callback (and since the lease code is serialized by the kernel
fs/nfsd/nfs4state.c +1
-0+* lock) we know the server hasn't removed the lease yet, we know
fs/nfsd/nfs4state.c +3
-0+* it's safe to take a reference. +*/ +atomic_inc (&dp->dl_count);nfsd4_cb_recall (dp); }@@ -3127,11 +3145,11 @@ static void nfsd_break_deleg_cb(struct
file_lock*fl) */fs/nfsd/nfs4state.c +2
-2fl->fl_break_time =0 ; -spin_lock (&state_lock); fp->fi_had_conflict = true; +spin_lock (&fp->fi_lock);list_for_each_entry (dp, &fp->fi_delegations, dl_perfile)nfsd_break_one_deleg (dp); -spin_unlock (&state_lock); +spin_unlock (&fp->fi_lock); } staticfs/nfsd/state.h +0
-0@@ -436,7 +436,8 @@ extern struct nfs4_client_reclaim *nfsd4_find_reclaim_client(const char *recdir, extern __be32
nfs4_check_open_reclaim(clientid_t *clid, struct nfsd4_compound_state *cstate, struct nfsd_net *nn); extern intset_callback_cred(void); -voidnfsd4_do_callback_rpc(struct work_struct *w); +voidnfsd4_run_cb_null(struct work_struct *w); +voidnfsd4_run_cb_recall(struct work_struct *w);
extern voidnfsd4_probe_callback(struct nfs4_client *clp); extern voidnfsd4_probe_callback_sync(struct nfs4_client *clp); extern void nfsd4_change_callback(structnfs4_client*clp, structnfs4_cb_conn*); @@ -444,6 +445,7 @@ extern voidnfsd4_cb_recall(struct nfs4_delegation *dp); extern intnfsd4_create_callback_queue(void); extern voidnfsd4_destroy_callback_queue(void); extern voidnfsd4_shutdown_callback(struct nfs4_client *); +extern voidnfsd4_prepare_cb_recall(struct nfs4_delegation *dp); extern voidnfs4_put_delegation(struct nfs4_delegation *dp); extern structnfs4_client_reclaim*nfs4_client_to_reclaim(const char *name, structnfsd_net*nn); --
1.9.3
Signature · 1 line
Jeff Layton <[email protected]>
PATCH v2 3/10 nfsd: nfs4_alloc_init_lease should take a nfs4_file arg
In reply to Jeff LaytonNo need to pass the delegation pointer in here as it's only used to get the nfs4_file pointer.
--- fs/nfsd/nfs4state.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
fs/nfsd/nfs4state.c +3 -3
@@ -3474,7 +3474,7 @@ static bool nfsd4_cb_channel_good(struct nfs4_client *clp)return clp->cl_minorversion && clp->cl_cb_state ==NFSD4_CB_UNKNOWN ; } -static struct file_lock *nfs4_alloc_init_lease (struct nfs4_delegation *dp,int flag) +static struct file_lock *nfs4_alloc_init_lease (struct nfs4_file *fp,int flag) {struct file_lock *fl;
@@ -3486,7 +3486,7 @@ static struct file_lock *nfs4_alloc_init_lease(struct nfs4_delegation *dp, int f fl->fl_flags =FL_DELEG ; fl->fl_type = flag ==NFS4_OPEN_DELEGATE_READ ?F_RDLCK :F_WRLCK ; fl->fl_end =OFFSET_MAX ; - fl->fl_owner = (fl_owner_t)(dp->dl_file); + fl->fl_owner = (fl_owner_t )fp; fl->fl_pid = current->tgid;return fl; }
@@ -3497,7 +3497,7 @@ static int nfs4_setlease(struct nfs4_delegation *dp)struct file_lock *fl;int status; - fl =nfs4_alloc_init_lease (dp,NFS4_OPEN_DELEGATE_READ ); + fl =nfs4_alloc_init_lease (fp,NFS4_OPEN_DELEGATE_READ );if (!fl)return -ENOMEM ; fl->fl_file =find_readable_file (fp);
Signature · 1 line
1.9.3
PATCH v2 4/10 nfsd: Ensure stateids remain unique until they are freed
In reply to Jeff LaytonFrom: Trond Myklebust <[email protected]>
Add an extra delegation state to allow the stateid to remain in the idr tree until the last reference has been released. This will be necessary to ensure uniqueness once the client_mutex is removed.
[jlayton: reset the sc_type under the state_lock in unhash_delegation]
--- fs/nfsd/nfs4state.c | 8 ++++---- fs/nfsd/state.h | 1 + 2 files changed, 5 insertions(+), 4 deletions(-)
fs/nfsd/nfs4state.c +4 -4
@@ -616,6 +616,7 @@ void nfs4_put_delegation(struct nfs4_delegation *dp) {if (atomic_dec_and_test (&dp->dl_count)) { +remove_stid (&dp->dl_stid);nfs4_free_stid (deleg_slab, &dp->dl_stid); num_delegations--; }
@@ -657,6 +658,7 @@ unhash_delegation(struct nfs4_delegation *dp)struct nfs4_file *fp = dp->dl_file;spin_lock (&state_lock); + dp->dl_stid.sc_type =NFS4_CLOSED_DELEG_STID ;list_del_init (&dp->dl_perclnt);list_del_init (&dp->dl_recall_lru);spin_lock (&fp->fi_lock);
@@ -670,19 +672,15 @@ unhash_delegation(struct nfs4_delegation *dp) } } - -static void destroy_revoked_delegation (struct nfs4_delegation *dp) {list_del_init (&dp->dl_recall_lru); -remove_stid (&dp->dl_stid);nfs4_put_delegation (dp); }static void destroy_delegation (struct nfs4_delegation *dp) {unhash_delegation (dp); -remove_stid (&dp->dl_stid);nfs4_put_delegation (dp); }
@@ -4036,7 +4034,9 @@ static __be32 nfsd4_validate_stateid(struct nfs4_client *cl, stateid_t *stateid)return nfs_ok;default :printk ("unknown stateid type %x\n" , s->sc_type); +/* Fallthrough */ case NFS4_CLOSED_STID : +case NFS4_CLOSED_DELEG_STID :return nfserr_bad_stateid; } }
fs/nfsd/state.h +1 -0
@@ -80,6 +80,7 @@ struct nfs4_stid {#define NFS4_CLOSED_STID 8/* For a deleg stateid kept around only to process free_stateid's: */ #define NFS4_REVOKED_DELEG_STID 16 +#define NFS4_CLOSED_DELEG_STID 32unsigned char sc_type;stateid_t sc_stateid;struct nfs4_client *sc_client;
Signature · 1 line
1.9.3
Re: [PATCH v2 04/10] nfsd: Ensure stateids remain unique until they are freed
Quoted 5 lines · Jeff Layton
From: Trond Myklebust <[email protected]>
Add an extra delegation state to allow the stateid to remain in the idr tree until the last reference has been released. This will be necessary to ensure uniqueness once the
client_mutexis removed.
Cyclic idr allocation should make reuse extremely unlikely, shouldn't it?
Probably needs to be fixed anyway, applying.
--b.
Quoted 76 lines
[jlayton: reset the
sc_typeunder thestate_lockinunhash_delegation]Signed-off-by Trond Myklebust <[email protected]>Signed-off-by Jeff Layton <[email protected]>Reviewed-by Christoph Hellwig <[email protected]>--- fs/nfsd/nfs4state.c | 8 ++++---- fs/nfsd/state.h | 1 + 2 files changed, 5 insertions(+), 4 deletions(-)fs/nfsd/nfs4state.c +4
-4@@ -616,6 +616,7 @@ void nfs4_put_delegation(struct nfs4_delegation *dp) {if (atomic_dec_and_test (&dp->dl_count)) { +remove_stid (&dp->dl_stid);nfs4_free_stid (deleg_slab, &dp->dl_stid); num_delegations--; }@@ -657,6 +658,7 @@ unhash_delegation(struct nfs4_delegation *dp)struct nfs4_file *fp = dp->dl_file;spin_lock (&state_lock); + dp->dl_stid.sc_type =NFS4_CLOSED_DELEG_STID ;list_del_init (&dp->dl_perclnt);list_del_init (&dp->dl_recall_lru);spin_lock (&fp->fi_lock);@@ -670,19 +672,15 @@ unhash_delegation(struct nfs4_delegation *dp) } } - -static void destroy_revoked_delegation (struct nfs4_delegation *dp) {list_del_init (&dp->dl_recall_lru); -remove_stid (&dp->dl_stid);nfs4_put_delegation (dp); }static void destroy_delegation (struct nfs4_delegation *dp) {unhash_delegation (dp); -remove_stid (&dp->dl_stid);nfs4_put_delegation (dp); }@@ -4036,7 +4034,9 @@ static __be32 nfsd4_validate_stateid(struct nfs4_client *cl, stateid_t *stateid)return nfs_ok;default :printk ("unknown stateid type %x\n" , s->sc_type); +/* Fallthrough */ case NFS4_CLOSED_STID : +case NFS4_CLOSED_DELEG_STID :return nfserr_bad_stateid; } }fs/nfsd/state.h +1
-0@@ -80,6 +80,7 @@ struct nfs4_stid {#define NFS4_CLOSED_STID 8/* For a deleg stateid kept around only to process free_stateid's: */ #define NFS4_REVOKED_DELEG_STID 16 +#define NFS4_CLOSED_DELEG_STID 32unsigned char sc_type;stateid_t sc_stateid;struct nfs4_client *sc_client;Signature · 1 line
1.9.3
Re: [PATCH v2 04/10] nfsd: Ensure stateids remain unique until they are freed
Quoted 7 lines · On Thu, 17 Jul 2014 14:44:17 -0400 "J. Bruce Fields" <[email protected]>…
Quoted 5 lines · Jeff Layton
From: Trond Myklebust <[email protected]>
Add an extra delegation state to allow the stateid to remain in the idr tree until the last reference has been released. This will be necessary to ensure uniqueness once the
client_mutexis removed.
Quoted 7 lines
Cyclic idr allocation should make reuse extremely unlikely, shouldn't it?
Probably needs to be fixed anyway, applying.
--b.
Yes, for the most part, but reuse is still possible once the counter wraps if you had a long-lived stateid (for instance). Not a likely problem, but we might as well fix it while we're in here.
Quoted 77 lines
Quoted 77 lines
[jlayton: reset the
sc_typeunder thestate_lockinunhash_delegation]Signed-off-by Trond Myklebust <[email protected]>Signed-off-by Jeff Layton <[email protected]>Reviewed-by Christoph Hellwig <[email protected]>--- fs/nfsd/nfs4state.c | 8 ++++---- fs/nfsd/state.h | 1 + 2 files changed, 5 insertions(+), 4 deletions(-)fs/nfsd/nfs4state.c +4
-4@@ -616,6 +616,7 @@ void nfs4_put_delegation(struct nfs4_delegation *dp) {if (atomic_dec_and_test (&dp->dl_count)) { +remove_stid (&dp->dl_stid);nfs4_free_stid (deleg_slab, &dp->dl_stid); num_delegations--; }@@ -657,6 +658,7 @@ unhash_delegation(struct nfs4_delegation *dp)struct nfs4_file *fp = dp->dl_file;spin_lock (&state_lock); + dp->dl_stid.sc_type =NFS4_CLOSED_DELEG_STID ;list_del_init (&dp->dl_perclnt);list_del_init (&dp->dl_recall_lru);spin_lock (&fp->fi_lock);@@ -670,19 +672,15 @@ unhash_delegation(struct nfs4_delegation *dp) } } - -static void destroy_revoked_delegation (struct nfs4_delegation *dp) {list_del_init (&dp->dl_recall_lru); -remove_stid (&dp->dl_stid);nfs4_put_delegation (dp); }static void destroy_delegation (struct nfs4_delegation *dp) {unhash_delegation (dp); -remove_stid (&dp->dl_stid);nfs4_put_delegation (dp); }@@ -4036,7 +4034,9 @@ static __be32 nfsd4_validate_stateid(struct nfs4_client *cl, stateid_t *stateid) return nfs_ok;default :printk ("unknown stateid type %x\n" , s->sc_type); +/* Fallthrough */ case NFS4_CLOSED_STID : +case NFS4_CLOSED_DELEG_STID :return nfserr_bad_stateid; } }fs/nfsd/state.h +1
-0@@ -80,6 +80,7 @@ struct nfs4_stid {#define NFS4_CLOSED_STID 8/* For a deleg stateid kept around only to process free_stateid's: */ #define NFS4_REVOKED_DELEG_STID 16 +#define NFS4_CLOSED_DELEG_STID 32unsigned char sc_type;stateid_t sc_stateid;struct nfs4_client *sc_client;Signature · 1 line
1.9.3
Signature · 1 line
Jeff Layton <[email protected]>
PATCH v2 5/10 locks: add file_has_lease to prevent delegation break races
In reply to Jeff LaytonOnce we remove the client_mutex, we'll have a potential race between setting a lease on a file and breaking the delegation. We may set the lease, but by the time we go to hash it, it may have already been broken. Currently, we know that this won't happen as the nfs4_laundromat takes the client_mutex, but we want to remove that.
As part of that fix, add a function that can tell us whether a particular file has a lease set on it. In a later nfsd patch, we'll use that to close the potential race window.
--- fs/locks.c | 26 ++++++++++++++++++++++++++ include/linux/fs.h | 6 ++++++ 2 files changed, 32 insertions(+)
fs/locks.c +26 -0
@@ -1308,6 +1308,32 @@ static bool leases_conflict(struct file_lock *lease, struct file_lock *breaker) }/** +* file_has_lease - does the given file have a lease set on it? +* @file: struct file on which we want to check the lease +* +* Returns true if a lease was is set on the given file description, +* false otherwise. +*/ +bool +file_has_lease (struct file *file) +{ +bool ret = false; +struct inode *inode =file_inode (file); +struct file_lock *fl; + +spin_lock (&inode->i_lock); +for (fl = inode->i_flock; fl &&IS_LEASE (fl); fl = fl->fl_next) { +if (fl->fl_file == file) { + ret = true; +break ; + } + } +spin_unlock (&inode->i_lock); +return ret; +} +EXPORT_SYMBOL (file_has_lease); + +/** * __break_lease - revoke all outstanding leases on file * @inode: the inode of the file to return * @mode: O_RDONLY: break only write leases; O_WRONLY or O_RDWR:
include/linux/fs.h +6 -0
@@ -953,6 +953,7 @@ extern int vfs_test_lock(struct file *, struct file_lock *);extern int vfs_lock_file (struct file *,unsigned int ,struct file_lock *,struct file_lock *);extern int vfs_cancel_lock (struct file *filp,struct file_lock *fl);extern int flock_lock_file_wait (struct file *filp,struct file_lock *fl); +extern bool file_has_lease (struct file *file);extern int __break_lease (struct inode *inode,unsigned int flags,unsigned int type);extern void lease_get_mtime (struct inode *,struct timespec *time);extern int generic_setlease (struct file *,long ,struct file_lock **);
@@ -1064,6 +1065,11 @@ static inline int flock_lock_file_wait(struct file *filp,return -ENOLCK ; } +static inline bool file_has_lease (struct file *file) +{ +return false; +} +static inline int __break_lease (struct inode *inode,unsigned int mode,unsigned int type) {return 0 ;
Signature · 1 line
1.9.3
PATCH v2 6/10 nfsd: Protect the nfs4_file delegation fields using the fi_lock
In reply to Jeff LaytonMove more of the delegation fields to be protected by the fi_lock. It's more granular than the state_lock and in later patches we'll want to be able to rely on it in addition to the state_lock.
Also, the current code in nfs4_setlease calls vfs_setlease and uses the client_mutex to ensure that it doesn't disappear before we can hash the delegation. With the client_mutex gone, we'll have a potential race condition.
It's possible that the delegation could be recalled after we acquire the lease but before we ever get around to hashing it. If that happens, then we'd have a nfs4_file that *thinks* it has a delegation, when it actually has none.
Attempt to acquire a delegation. If that succeeds, take the state_lock and recheck to make sure the lease is still there. If it is, then take the fi_lock and set up the rest of the delegation fields. This prevents the race because the delegation break workqueue job must also take the state_lock.
--- fs/nfsd/nfs4state.c | 51 ++++++++++++++++++++++++++++++++++++++------------- 1 file changed, 38 insertions(+), 13 deletions(-)
fs/nfsd/nfs4state.c +38 -13
@@ -624,6 +624,8 @@ nfs4_put_delegation(struct nfs4_delegation *dp)static void nfs4_put_deleg_lease (struct nfs4_file *fp) { +lockdep_assert_held (&state_lock); +if (!fp->fi_lease)return ;if (atomic_dec_and_test (&fp->fi_delegees)) {
@@ -643,11 +645,10 @@ static void hash_delegation_locked(struct nfs4_delegation *dp, struct nfs4_file *fp) { lockdep_assert_held(&state_lock); +lockdep_assert_held (&fp->fi_lock); dp->dl_stid.sc_type =NFS4_DELEG_STID ; -spin_lock (&fp->fi_lock);list_add (&dp->dl_perfile, &fp->fi_delegations); -spin_unlock (&fp->fi_lock);list_add (&dp->dl_perclnt, &dp->dl_stid.sc_client->cl_delegations); }
@@ -659,17 +660,17 @@ unhash_delegation(struct nfs4_delegation *dp)spin_lock (&state_lock); dp->dl_stid.sc_type =NFS4_CLOSED_DELEG_STID ; +spin_lock (&fp->fi_lock);list_del_init (&dp->dl_perclnt);list_del_init (&dp->dl_recall_lru); -spin_lock (&fp->fi_lock);list_del_init (&dp->dl_perfile);spin_unlock (&fp->fi_lock); -spin_unlock (&state_lock);if (fp) {nfs4_put_deleg_lease (fp); -put_nfs4_file (fp); dp->dl_file =NULL ; } +spin_unlock (&state_lock); +put_nfs4_file (fp); }static void destroy_revoked_delegation (struct nfs4_delegation *dp)
@@ -3493,7 +3494,7 @@ static int nfs4_setlease(struct nfs4_delegation *dp) {struct nfs4_file *fp = dp->dl_file;struct file_lock *fl; -int status; +int status =0 ; fl =nfs4_alloc_init_lease (fp,NFS4_OPEN_DELEGATE_READ );if (!fl)
@@ -3501,15 +3502,31 @@ static int nfs4_setlease(struct nfs4_delegation *dp) fl->fl_file =find_readable_file (fp); status =vfs_setlease (fl->fl_file, fl->fl_type, &fl);if (status) - goto out_free; + goto out_fput; +spin_lock (&state_lock); +/* Did the lease get broken before we took the lock? */ + status = -EAGAIN ; +if (!file_has_lease (fl->fl_file)) + goto out_unlock; +spin_lock (&fp->fi_lock); +/* Race breaker */ +if (fp->fi_lease) { + status =0 ; +atomic_inc (&fp->fi_delegees); +hash_delegation_locked (dp, fp); +spin_unlock (&fp->fi_lock); + goto out_unlock; + } fp->fi_lease = fl; fp->fi_deleg_file = fl->fl_file;atomic_set (&fp->fi_delegees,1 ); -spin_lock (&state_lock);hash_delegation_locked (dp, fp); +spin_unlock (&fp->fi_lock);spin_unlock (&state_lock);return 0 ; -out_free: +out_unlock: +spin_unlock (&state_lock); +out_fput:if (fl->fl_file)fput (fl->fl_file);locks_free_lock (fl);
@@ -3518,19 +3535,27 @@ out_free:static int nfs4_set_delegation (struct nfs4_delegation *dp,struct nfs4_file *fp) { +int status =0 ; +if (fp->fi_had_conflict)return -EAGAIN ;get_nfs4_file (fp); +spin_lock (&state_lock); +spin_lock (&fp->fi_lock); dp->dl_file = fp; -if (!fp->fi_lease) +if (!fp->fi_lease) { +spin_unlock (&fp->fi_lock); +spin_unlock (&state_lock);return nfs4_setlease (dp); -spin_lock (&state_lock); + }atomic_inc (&fp->fi_delegees);if (fp->fi_had_conflict) { -spin_unlock (&state_lock); -return -EAGAIN ; + status = -EAGAIN ; + goto out_unlock; }hash_delegation_locked (dp, fp); +out_unlock: +spin_unlock (&fp->fi_lock);spin_unlock (&state_lock);return 0 ; }
Signature · 1 line
1.9.3
Re: [PATCH v2 06/10] nfsd: Protect the nfs4_file delegation fields using the fi_lock
Quoted 6 lines
fs/nfsd/nfs4state.c +1
-1dp->dl_stid.sc_type =NFS4_CLOSED_DELEG_STID ; +spin_lock (&fp->fi_lock);list_del_init (&dp->dl_perclnt);list_del_init (&dp->dl_recall_lru); -spin_lock (&fp->fi_lock);list_del_init (&dp->dl_perfile);
fs/nfsd/nfs4state.c unhash_delegation(struct nfs4_delegation *dp) · line 674
Quoted 10 lines
fs/nfsd/nfs4state.c +2
-2spin_unlock (&fp->fi_lock); -spin_unlock (&state_lock);if (fp) {nfs4_put_deleg_lease (fp); -put_nfs4_file (fp); dp->dl_file =NULL ; } +spin_unlock (&state_lock); +put_nfs4_file (fp); }
Unless I've missed something put_nfs4_file does not handle a NULL pointer argument in the current tree, so this needs a NULL check for fp.
fs/nfsd/nfs4state.c static int nfs4_setlease(struct nfs4_delegation *dp) · line 3510
Quoted 5 lines
fs/nfsd/nfs4state.c +5
-0+spin_lock (&state_lock); +/* Did the lease get broken before we took the lock? */ + status = -EAGAIN ; +if (!file_has_lease (fl->fl_file)) + goto out_unlock;
So if the delegation we just tried to add got broken, but someone else managed to add one this would also return true. But could that happen either in real life or in theory? Shouldn't we instead have an atomic flag on the delegation that it got broken which we could check here?
Re: [PATCH v2 06/10] nfsd: Protect the nfs4_file delegation fields using the fi_lock
Quoted 12 lines · On Wed, 16 Jul 2014 11:09:43 -0700 Christoph Hellwig <[email protected]> wrote:
Quoted 12 lines
fs/nfsd/nfs4state.c +1
-3dp->dl_stid.sc_type =NFS4_CLOSED_DELEG_STID ; +spin_lock (&fp->fi_lock);list_del_init (&dp->dl_perclnt);list_del_init (&dp->dl_recall_lru); -spin_lock (&fp->fi_lock);list_del_init (&dp->dl_perfile);spin_unlock (&fp->fi_lock); -spin_unlock (&state_lock);if (fp) {nfs4_put_deleg_lease (fp); -put_nfs4_file (fp); dp->dl_file =NULL ;
Quoted 9 lines
fs/nfsd/nfs4state.c unhash_delegation(struct nfs4_delegation *dp) · line 674
Quoted 4 lines
fs/nfsd/nfs4state.c +2
-0} +spin_unlock (&state_lock); +put_nfs4_file (fp); }Unless I've missed something
put_nfs4_filedoes not handle a NULL pointer argument in the current tree, so this needs a NULL check for fp.
Well spotted, I'll fix that...
Quoted 12 lines
fs/nfsd/nfs4state.c static int nfs4_setlease(struct nfs4_delegation *dp) · line 3510
Quoted 5 lines
fs/nfsd/nfs4state.c +5
-0+spin_lock (&state_lock); +/* Did the lease get broken before we took the lock? */ + status = -EAGAIN ; +if (!file_has_lease (fl->fl_file)) + goto out_unlock;So if the delegation we just tried to add got broken, but someone else managed to add one this would also return true. But could that happen either in real life or in theory? Shouldn't we instead have an atomic flag on the delegation that it got broken which we could check here?
Hmm...maybe. It's a very unlikely race though, and I think the same sort of race exists today. In fact, it's worse today since we don't do any checking of the validity of the lease after acquiring it now.
The flag sounds like a good idea, but the code is structured completely wrong for it currently. The delegation is only hashed after we get a lease, so the lease break wouldn't find anything to set a flag on.
Quite frankly, I _really_ do not want to have to rework the locking in the delegation code yet again. I think this scheme is an improvement over what we have now, even if it's not 100% perfect.
Once we get the scalability set done, I'd like to go back and overhaul the delegation code. There are a lot of ugly warts here, but fixing them is really a separate project in its own right.
Signature · 1 line
Jeff Layton <[email protected]>
Re: [PATCH v2 06/10] nfsd: Protect the nfs4_file delegation fields using the fi_lock
Quoted 8 lines · Jeff Layton
Hmm...maybe. It's a very unlikely race though, and I think the same sort of race exists today. In fact, it's worse today since we don't do any checking of the validity of the lease after acquiring it now.
The flag sounds like a good idea, but the code is structured completely wrong for it currently. The delegation is only hashed after we get a lease, so the lease break wouldn't find anything to set a flag on.
Quoted 7 lines
Quite frankly, I _really_ do not want to have to rework the locking in the delegation code yet again. I think this scheme is an improvement over what we have now, even if it's not 100% perfect.
Once we get the scalability set done, I'd like to go back and overhaul the delegation code. There are a lot of ugly warts here, but fixing them is really a separate project in its own right.
But in the old code we had the client lock over all of this, right?
Anyway, if Bruce and you are fine with this I'm not going to block it, although it seems a little incomplete to me.
Re: [PATCH v2 06/10] nfsd: Protect the nfs4_file delegation fields using the fi_lock
Quoted 13 lines · On Thu, 17 Jul 2014 07:55:34 -0700 Christoph Hellwig <[email protected]> wrote:
Quoted 12 lines · Jeff Layton
Hmm...maybe. It's a very unlikely race though, and I think the same sort of race exists today. In fact, it's worse today since we don't do any checking of the validity of the lease after acquiring it now.
The flag sounds like a good idea, but the code is structured completely wrong for it currently. The delegation is only hashed after we get a lease, so the lease break wouldn't find anything to set a flag on.
Quite frankly, I _really_ do not want to have to rework the locking in the delegation code yet again. I think this scheme is an improvement over what we have now, even if it's not 100% perfect.
Quoted 6 lines
Quoted 3 lines
Once we get the scalability set done, I'd like to go back and overhaul the delegation code. There are a lot of ugly warts here, but fixing them is really a separate project in its own right.
But in the old code we had the client lock over all of this, right?
Yes, but it's still possible for the lease to get broken after setting it but before the delegation is hashed in the existing code. You're correct though that another nfsd wouldn't be able to set a lease on the same file though.
One possibility to ensure that can't happen is to check whether fp->fi_had_conflict became true after vfs_setlease returns. That would also obviate the need for file_has_lease. I'm still trying to figure out whether there are potential races there with that approach though.
Quoted 2 lines
Anyway, if Bruce and you are fine with this I'm not going to block it, although it seems a little incomplete to me.
Thanks, and I agree. I think we're going to need to revisit this area, but I suspect we need to do some surgery to the generic lease handling code too.
For one thing, it's a little disturbing to me that vfs_setlease returns a pointer to the file_lock that's sitting on the i_flock list. I'm not sure that it's a problem today, but it seems like a caller could end up trying to use that pointer after time_out_leases has been called, or after the lease has been otherwise deleted.
Signature · 1 line
Jeff Layton <[email protected]>
PATCH v2 7/10 nfsd: Move the delegation reference counter into the struct nfs4_stid
In reply to Jeff LaytonFrom: Trond Myklebust <[email protected]>
We will want to add reference counting to the lock stateid and open stateids too in later patches.
--- fs/nfsd/nfs4state.c | 6 +++--- fs/nfsd/state.h | 2 +- 2 files changed, 4 insertions(+), 4 deletions(-)
fs/nfsd/nfs4state.c +3 -3
@@ -472,6 +472,7 @@ kmem_cache *slab) stid->sc_stateid.si_opaque.so_clid = cl->cl_clientid;/* Will be incremented before return to client: */ stid->sc_stateid.si_generation =0 ; +atomic_set (&stid->sc_count,1 );/* * It shouldn't be a problem to reuse an opaque stateid value.
@@ -595,7 +596,6 @@ alloc_init_deleg(struct nfs4_client *clp, struct nfs4_ol_stateid *stp, struct sv dp->dl_type =NFS4_OPEN_DELEGATE_READ ;fh_copy_shallow (&dp->dl_fh, ¤t_fh->fh_handle); dp->dl_time =0 ; -atomic_set (&dp->dl_count,1 );INIT_WORK (&dp->dl_recall.cb_work, nfsd4_run_cb_recall);return dp; }
@@ -615,7 +615,7 @@ static void nfs4_free_stid(struct kmem_cache *slab, struct nfs4_stid *s)void nfs4_put_delegation (struct nfs4_delegation *dp) { -if (atomic_dec_and_test (&dp->dl_count)) { +if (atomic_dec_and_test (&dp->dl_stid.sc_count)) {remove_stid (&dp->dl_stid);nfs4_free_stid (deleg_slab, &dp->dl_stid); num_delegations--;
@@ -3119,7 +3119,7 @@ static void nfsd_break_one_deleg(struct nfs4_delegation *dp)* lock) we know the server hasn't removed the lease yet, we know * it's safe to take a reference. */ -atomic_inc (&dp->dl_count); +atomic_inc (&dp->dl_stid.sc_count);nfsd4_cb_recall (dp); }
fs/nfsd/state.h +1 -1
@@ -73,6 +73,7 @@ struct nfsd4_callback { };struct nfs4_stid { +atomic_t sc_count;#define NFS4_OPEN_STID 1#define NFS4_LOCK_STID 2#define NFS4_DELEG_STID 4
@@ -91,7 +92,6 @@ struct nfs4_delegation {struct list_head dl_perfile;struct list_head dl_perclnt;struct list_head dl_recall_lru;/* delegation recalled */ -atomic_t dl_count;/* ref count */ struct nfs4_file *dl_file;u32 dl_type;time_t dl_time;
Signature · 1 line
1.9.3
PATCH v2 8/10 nfsd: Simplify stateid management
In reply to Jeff LaytonFrom: Trond Myklebust <[email protected]>
Don't allow stateids to clear the open file pointer until they are being destroyed. Also, move to kzalloc and get rid of explicit zeroing of fields.
--- fs/nfsd/nfs4state.c | 21 ++++++++++----------- 1 file changed, 10 insertions(+), 11 deletions(-)
fs/nfsd/nfs4state.c +10 -11
@@ -459,7 +459,7 @@ kmem_cache *slab)struct nfs4_stid *stid;int new_id; - stid =kmem_cache_alloc (slab,GFP_KERNEL ); + stid =kmem_cache_zalloc (slab,GFP_KERNEL );if (!stid)return NULL ;
@@ -467,11 +467,9 @@ kmem_cache *slab)if (new_id <0 ) goto out_free; stid->sc_client = cl; - stid->sc_type =0 ; stid->sc_stateid.si_opaque.so_id = new_id; stid->sc_stateid.si_opaque.so_clid = cl->cl_clientid;/* Will be incremented before return to client: */ - stid->sc_stateid.si_generation =0 ;atomic_set (&stid->sc_count,1 );/*
@@ -592,10 +590,8 @@ alloc_init_deleg(struct nfs4_client *clp, struct nfs4_ol_stateid *stp, struct svINIT_LIST_HEAD (&dp->dl_perfile);INIT_LIST_HEAD (&dp->dl_perclnt);INIT_LIST_HEAD (&dp->dl_recall_lru); - dp->dl_file =NULL ; dp->dl_type =NFS4_OPEN_DELEGATE_READ ;fh_copy_shallow (&dp->dl_fh, ¤t_fh->fh_handle); - dp->dl_time =0 ;INIT_WORK (&dp->dl_recall.cb_work, nfsd4_run_cb_recall);return dp; }
@@ -616,6 +612,8 @@ void nfs4_put_delegation(struct nfs4_delegation *dp) {if (atomic_dec_and_test (&dp->dl_stid.sc_count)) { +if (dp->dl_file) +put_nfs4_file (dp->dl_file);remove_stid (&dp->dl_stid);nfs4_free_stid (deleg_slab, &dp->dl_stid); num_delegations--;
@@ -665,12 +663,9 @@ unhash_delegation(struct nfs4_delegation *dp)list_del_init (&dp->dl_recall_lru);list_del_init (&dp->dl_perfile);spin_unlock (&fp->fi_lock); -if (fp) { +if (fp)nfs4_put_deleg_lease (fp); - dp->dl_file =NULL ; - }spin_unlock (&state_lock); -put_nfs4_file (fp); }static void destroy_revoked_delegation (struct nfs4_delegation *dp)
@@ -878,12 +873,12 @@ static void unhash_generic_stateid(struct nfs4_ol_stateid *stp)static void close_generic_stateid (struct nfs4_ol_stateid *stp) {release_all_access (stp); -put_nfs4_file (stp->st_file); - stp->st_file =NULL ; }static void free_generic_stateid (struct nfs4_ol_stateid *stp) { +if (stp->st_file) +put_nfs4_file (stp->st_file);remove_stid (&stp->st_stid);nfs4_free_stid (stateid_slab, &stp->st_stid); }
@@ -4446,6 +4441,10 @@ static void nfsd4_close_open_stateid(struct nfs4_ol_stateid *s)if (list_empty (&oo->oo_owner.so_stateids))release_openowner (oo); }else { +if (s->st_file) { +put_nfs4_file (s->st_file); + s->st_file =NULL ; + } oo->oo_last_closed_stid = s;/* * In the 4.0 case we need to keep the owners around a
Signature · 1 line
1.9.3
Re: [PATCH v2 08/10] nfsd: Simplify stateid management
Quoted 5 lines · Jeff Layton
From: Trond Myklebust <[email protected]>
Don't allow stateids to clear the open file pointer until they are being destroyed. Also, move to kzalloc and get rid of explicit zeroing of fields.
This doesn't explain what's actually simplified as the subject line claims, nor does it explain why this move is safe. It generally looks good to me, but a little more explanation wouldn't hurt.
PATCH v2 9/10 nfsd: Fix delegation revocation
In reply to Jeff LaytonEnsure that the delegations cannot be found by the laundromat etc once we add them to the various 'revoke' lists.
--- fs/nfsd/nfs4state.c | 45 +++++++++++++++++++++++++-------------------- 1 file changed, 25 insertions(+), 20 deletions(-)
fs/nfsd/nfs4state.c +25 -20
@@ -650,13 +650,13 @@ hash_delegation_locked(struct nfs4_delegation *dp, struct nfs4_file *fp)list_add (&dp->dl_perclnt, &dp->dl_stid.sc_client->cl_delegations); } -/* Called under the state lock. */ static void -unhash_delegation (struct nfs4_delegation *dp) +unhash_delegation_locked (struct nfs4_delegation *dp) {struct nfs4_file *fp = dp->dl_file; -spin_lock (&state_lock); +lockdep_assert_held (&state_lock); + dp->dl_stid.sc_type =NFS4_CLOSED_DELEG_STID ;spin_lock (&fp->fi_lock);list_del_init (&dp->dl_perclnt);
@@ -665,18 +665,19 @@ unhash_delegation(struct nfs4_delegation *dp)spin_unlock (&fp->fi_lock);if (fp)nfs4_put_deleg_lease (fp); -spin_unlock (&state_lock); } -static void destroy_revoked_delegation (struct nfs4_delegation *dp) +static void unhash_and_destroy_delegation (struct nfs4_delegation *dp) { -list_del_init (&dp->dl_recall_lru); +spin_lock (&state_lock); +unhash_delegation_locked (dp); +spin_unlock (&state_lock);nfs4_put_delegation (dp); } -static void destroy_delegation (struct nfs4_delegation *dp) +static void destroy_revoked_delegation (struct nfs4_delegation *dp) { -unhash_delegation (dp); +list_del_init (&dp->dl_recall_lru);nfs4_put_delegation (dp); }
@@ -685,11 +686,10 @@ static void revoke_delegation(struct nfs4_delegation *dp)struct nfs4_client *clp = dp->dl_stid.sc_client;if (clp->cl_minorversion ==0 ) -destroy_delegation (dp); +destroy_revoked_delegation (dp);else { -unhash_delegation (dp); dp->dl_stid.sc_type =NFS4_REVOKED_DELEG_STID ; -list_add (&dp->dl_recall_lru, &clp->cl_revoked); +list_move (&dp->dl_recall_lru, &clp->cl_revoked); } }
@@ -1447,15 +1447,16 @@ destroy_client(struct nfs4_client *clp)spin_lock (&state_lock);while (!list_empty (&clp->cl_delegations)) { dp = list_entry(clp->cl_delegations.next, struct nfs4_delegation, dl_perclnt); -list_del_init (&dp->dl_perclnt); +unhash_delegation_locked (dp);/* Ensure that deleg break won't try to requeue it */ ++dp->dl_time; -list_move (&dp->dl_recall_lru, &reaplist); +list_add (&dp->dl_recall_lru, &reaplist); }spin_unlock (&state_lock);while (!list_empty (&reaplist)) { dp = list_entry(reaplist.next, struct nfs4_delegation, dl_recall_lru); -destroy_delegation (dp); +list_del_init (&dp->dl_recall_lru); +nfs4_put_delegation (dp); }list_splice_init (&clp->cl_revoked, &reaplist);while (!list_empty (&reaplist)) {
@@ -3639,7 +3640,7 @@ nfs4_open_delegation(struct net *net, struct svc_fh *fh, open->op_delegate_type =NFS4_OPEN_DELEGATE_READ ;return ; out_free: -destroy_delegation (dp); +nfs4_put_delegation (dp); out_no_deleg: open->op_delegate_type =NFS4_OPEN_DELEGATE_NONE ; if (open->op_claim_type == NFS4_OPEN_CLAIM_PREVIOUS &&
@@ -3878,7 +3879,8 @@ nfs4_laundromat(struct nfsd_net *nn) new_timeo =min (new_timeo, t);break ; } -list_move (&dp->dl_recall_lru, &reaplist); +unhash_delegation_locked (dp); +list_add (&dp->dl_recall_lru, &reaplist); }spin_unlock (&state_lock);list_for_each_safe (pos, next, &reaplist) {
@@ -4510,7 +4512,7 @@ nfsd4_delegreturn(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,if (status) goto out; -destroy_delegation (dp); +unhash_and_destroy_delegation (dp); out:nfs4_unlock_state ();
@@ -5353,7 +5355,8 @@ static u64 nfsd_find_all_delegations(struct nfs4_client *clp, u64 max,* don't monkey with it now that we are. */ ++dp->dl_time; -list_move (&dp->dl_recall_lru, victims); +unhash_delegation_locked (dp); +list_add (&dp->dl_recall_lru, victims); }if (++count == max)break ;
@@ -5608,12 +5611,14 @@ nfs4_state_shutdown_net(struct net *net)spin_lock (&state_lock);list_for_each_safe (pos, next, &nn->del_recall_lru) { dp = list_entry (pos, struct nfs4_delegation, dl_recall_lru); -list_move (&dp->dl_recall_lru, &reaplist); +unhash_delegation_locked (dp); +list_add (&dp->dl_recall_lru, &reaplist); }spin_unlock (&state_lock);list_for_each_safe (pos, next, &reaplist) { dp = list_entry (pos, struct nfs4_delegation, dl_recall_lru); -destroy_delegation (dp); +list_del_init (&dp->dl_recall_lru); +nfs4_put_delegation (dp); }nfsd4_client_tracking_exit (net);
Signature · 1 line
1.9.3
Re: [PATCH v2 09/10] nfsd: Fix delegation revocation
Quoted 1 line
-/* Called under the state lock. */
You're removing a comment hat only becomes true after your patch :)
But I agree that it's obsolete, the lockdep_assert_held is better than the comment.
fs/nfsd/nfs4state.c static void revoke_delegation(struct nfs4_delegation *dp) · line 689
Quoted 6 lines
fs/nfsd/nfs4state.c +1
-1@@ -685,11 +686,10 @@ static void revoke_delegation(struct nfs4_delegation *dp)struct nfs4_client *clp = dp->dl_stid.sc_client;if (clp->cl_minorversion ==0 ) -destroy_delegation (dp); +destroy_revoked_delegation (dp);
Seems a little confusing as we didn't turn the code into a REVOKED_DELEG stateid type yet, but given that destroy_revoked_delegation doesn't care this is probably fine.
fs/nfsd/nfs4state.c static void revoke_delegation(struct nfs4_delegation *dp) · line 692
Quoted 5 lines
fs/nfsd/nfs4state.c +1
-2else { -unhash_delegation (dp); dp->dl_stid.sc_type =NFS4_REVOKED_DELEG_STID ; -list_add (&dp->dl_recall_lru, &clp->cl_revoked); +list_move (&dp->dl_recall_lru, &clp->cl_revoked);
What protects access to cl_revoked, btw?
Otherwise this looks fine to me,
(would have been a little nicer to read if you had kept destroy_revoked_delegation, otherwise diff obsfucates the fact that it didn't change at all..
Re: [PATCH v2 09/10] nfsd: Fix delegation revocation
Quoted 7 lines · On Wed, 16 Jul 2014 11:30:47 -0700 Christoph Hellwig <[email protected]> wrote:
Quoted 1 line
-/* Called under the state lock. */
You're removing a comment hat only becomes true after your patch :)
But I agree that it's obsolete, the
lockdep_assert_heldis better than the comment.
Yeah, the other problem is that that comment is ambiguous. Does it mean the state_lock or the client_mutex (which is also sometimes termed "state lock" in this code).
Quoted 11 lines
fs/nfsd/nfs4state.c static void revoke_delegation(struct nfs4_delegation *dp) · line 689
Quoted 6 lines
fs/nfsd/nfs4state.c +1
-1@@ -685,11 +686,10 @@ static void revoke_delegation(struct nfs4_delegation *dp)struct nfs4_client *clp = dp->dl_stid.sc_client;if (clp->cl_minorversion ==0 ) -destroy_delegation (dp); +destroy_revoked_delegation (dp);Seems a little confusing as we didn't turn the code into a
REVOKED_DELEGstateid type yet, but given thatdestroy_revoked_delegationdoesn't care this is probably fine.
True, but since we're destroying it anyway we don't need to revoke it. Still it is less confusing and since I'm respinning for other reasons, I can change that too...
Quoted 8 lines
fs/nfsd/nfs4state.c static void revoke_delegation(struct nfs4_delegation *dp) · line 692
Quoted 5 lines
fs/nfsd/nfs4state.c +1
-2else { -unhash_delegation (dp); dp->dl_stid.sc_type =NFS4_REVOKED_DELEG_STID ; -list_add (&dp->dl_recall_lru, &clp->cl_revoked); +list_move (&dp->dl_recall_lru, &clp->cl_revoked);What protects access to
cl_revoked, btw?
At this point in the series, the client_mutex does. There's a patch later that protects the cl_revoked list with the cl_lock, but it depends on some other locking changes and so it's harder to move it to the front of the series just yet.
Quoted 7 lines
Otherwise this looks fine to me,
Reviewed-by Christoph Hellwig <[email protected]>(would have been a little nicer to read if you had kept
destroy_revoked_delegation, otherwise diff obsfucates the fact that it didn't change at all..
Thanks. Do you mean destroy_delegation? destroy_revoked_delegation still exists -- destroy_delegation is removed, but it's just a wrapper around nfs4_put_delegation at this point...
Signature · 1 line
Jeff Layton <[email protected]>
Re: [PATCH v2 09/10] nfsd: Fix delegation revocation
Quoted 9 lines · Jeff Layton
Quoted 5 lines
Seems a little confusing as we didn't turn the code into a
REVOKED_DELEGstateid type yet, but given thatdestroy_revoked_delegationdoesn't care this is probably fine.True, but since we're destroying it anyway we don't need to revoke it. Still it is less confusing and since I'm respinning for other reasons, I can change that too...
It might make sense to just rename it to destroy_delegation and use it for both types. There's another opencoded instance later on as well.
Quoted 9 lines
Quoted 5 lines
Reviewed-by Christoph Hellwig <[email protected]>(would have been a little nicer to read if you had kept
destroy_revoked_delegation, otherwise diff obsfucates the fact that it didn't change at all..Thanks. Do you mean
destroy_delegation?destroy_revoked_delegationstill exists --destroy_delegationis removed, but it's just a wrapper aroundnfs4_put_delegationat this point...
I meant keept in the same place. You added a function above it, and removed one below it, so diff makes a mess of it despite the function not changing at all.
PATCH v2 10/10 nfsd: Convert delegation counter to an atomic_long_t type
In reply to Jeff LaytonFrom: Trond Myklebust <[email protected]>
We want to convert to an atomic type so that we don't need to lock across the call to alloc_init_deleg(). Then convert to a long type so that we match the size of 'max_delegations'.
None of this is a problem today, but it will be once we remove client_mutex protection.
--- fs/nfsd/nfs4state.c | 18 +++++++++++------- 1 file changed, 11 insertions(+), 7 deletions(-)
fs/nfsd/nfs4state.c +11 -7
@@ -343,7 +343,7 @@ find_any_file(struct nfs4_file *f)return ret; } -static int num_delegations; +static atomic_long_t num_delegations;unsigned long max_delegations;/*
@@ -571,22 +571,23 @@ static struct nfs4_delegation * alloc_init_deleg(struct nfs4_client *clp, struct nfs4_ol_stateid *stp, struct svc_fh *current_fh) { struct nfs4_delegation *dp; +long n;dprintk ("NFSD alloc_init_deleg\n" ); -if (num_delegations > max_delegations) -return NULL ; + n =atomic_long_inc_return (&num_delegations); +if (n <0 || n > max_delegations) + goto out_dec;if (delegation_blocked (¤t_fh->fh_handle)) -return NULL ; + goto out_dec; dp =delegstateid (nfs4_alloc_stid (clp, deleg_slab));if (dp ==NULL ) -return dp; + goto out_dec;/* * delegation seqid's are never incremented. The 4.1 special * meaning of seqid 0 isn't meaningful, really, but let's avoid * 0 anyway just for consistency and use 1: */ dp->dl_stid.sc_stateid.si_generation =1 ; - num_delegations++;INIT_LIST_HEAD (&dp->dl_perfile);INIT_LIST_HEAD (&dp->dl_perclnt);INIT_LIST_HEAD (&dp->dl_recall_lru);
@@ -594,6 +595,9 @@ alloc_init_deleg(struct nfs4_client *clp, struct nfs4_ol_stateid *stp, struct svfh_copy_shallow (&dp->dl_fh, ¤t_fh->fh_handle);INIT_WORK (&dp->dl_recall.cb_work, nfsd4_run_cb_recall);return dp; +out_dec: +atomic_long_dec (&num_delegations); +return NULL ; }static void remove_stid (struct nfs4_stid *s)
@@ -616,7 +620,7 @@ nfs4_put_delegation(struct nfs4_delegation *dp)put_nfs4_file (dp->dl_file);remove_stid (&dp->dl_stid);nfs4_free_stid (deleg_slab, &dp->dl_stid); - num_delegations--; +atomic_long_dec (&num_delegations); } }
Signature · 1 line
1.9.3
Re: [PATCH v2 10/10] nfsd: Convert delegation counter to an atomic_long_t type
Quoted 10 lines · Jeff Layton
From: Trond Myklebust <[email protected]>
We want to convert to an atomic type so that we don't need to lock across the call to
alloc_init_deleg(). Then convert to a long type so that we match the size of 'max_delegations'.None of this is a problem today, but it will be once we remove
client_mutexprotection.Signed-off-by Trond Myklebust <[email protected]>
Looks good,
Quoted 30 lines · On Wed, 16 Jul 2014 10:31:55 -0400 Jeff Layton <[email protected]> wrote:
Here's a respin of the delegation fixes that I sent yesterday. The main change is to move the
remove_stidcalls into the caller in the second patch instead of keeping it innfs4_free_stid, per Christoph's suggestion.Unfortunately, that causes a pile of merge conflicts in the later patches, so I need to resend the whole set.
I also include the patch that converts the
num_delegationscounter to anatomic_long_t, as I missed that in the earlier series.There is one more patch that fixes up handling of the
cl_revokedlist that comes later, but it can't easily be moved here as it depends on some locking changes that come later.Jeff Layton (6):
nfsd: eliminatenfsd4_init_callback
nfsd: Avoid takingstate_lockwhile holding inode lock in
nfsd_break_one_deleg
nfsd:nfs4_alloc_init_leaseshould take anfs4_filearg
locks: addfile_has_leaseto prevent delegation break races
nfsd: Protect thenfs4_filedelegation fields using thefi_lock
nfsd: Fix delegation revocationTrond Myklebust (4):
nfsd: Ensure stateids remain unique until they are freed
nfsd: Move the delegation reference counter into the structnfs4_stid
nfsd: Simplify stateid management
nfsd: Convert delegation counter to anatomic_long_ttype
Quoted 7 lines
fs/locks.c | 26 +++++++ fs/nfsd/nfs4callback.c | 23 ++++-- fs/nfsd/nfs4state.c | 197 +++++++++++++++++++++++++++++++------------------ fs/nfsd/state.h | 7 +- include/linux/fs.h | 6 ++ 5 files changed, 179 insertions(+), 80 deletions(-)
Sorry...obviously the subject here should have read "0/10" and not "0/8".
Signature · 1 line
Jeff Layton <[email protected]>