nfsd: COPY refcounting fix and cleanup
Subsystems (2)
Jeff Layton
maintainercover nfsd: COPY refcounting fix and cleanup
Just a couple of fixes for refcounting issues in the COPY codepaths. I sort of doubt either of these are the cause of the more recent crash reports around nfsd_file refcount underflows, but they do seem like real (and potential) bugs.
Jeff Layton (2):
nfsd: zero out pointers after putting nfsd_files on COPY setup error
nfsd: clean up potential nfsd_file refcount leaks in COPY codepath
fs/nfsd/nfs4proc.c | 25 ++++++++++++------------- 1 file changed, 12 insertions(+), 13 deletions(-)
Signature · 1 line
2.39.0
Jeff Layton
maintainerPATCH 1/2 nfsd: zero out pointers after putting nfsd_files on COPY setup error
At first, I thought this might be a source of nfsd_file overputs, but the current callers seem to avoid an extra put when nfsd4_verify_copy returns an error.
Still, it's "bad form" to leave the pointers filled out when we don't have a reference to them anymore, and that might lead to bugs later. Zero them out as a defensive coding measure.
--- fs/nfsd/nfs4proc.c | 2 ++ 1 file changed, 2 insertions(+)
fs/nfsd/nfs4proc.c +2 -0
@@ -1214,8 +1214,10 @@ nfsd4_verify_copy(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,return status; out_put_dst:nfsd_file_put (*dst); + *dst =NULL ; out_put_src:nfsd_file_put (*src); + *src =NULL ; goto out; }
Signature · 1 line
2.39.0
Jeff Layton
maintainerPATCH 2/2 nfsd: clean up potential nfsd_file refcount leaks in COPY codepath
In reply to Jeff LaytonThere are two different flavors of the nfsd4_copy struct. One is embedded in the compound and is used directly in synchronous copies. The other is dynamically allocated, refcounted and tracked in the client struture. For the embedded one, the cleanup just involves releasing any nfsd_files held on its behalf. For the async one, the cleanup is a bit more involved, and we need to dequeue it from lists, unhash it, etc.
There is at least one potential refcount leak in this code now. If the kthread_create call fails, then both the src and dst nfsd_files in the original nfsd4_copy object are leaked.
The cleanup in this codepath is also sort of weird. In the async copy case, we'll have up to four nfsd_file references (src and dst for both flavors of copy structure). They are both put at the end of nfsd4_do_async_copy, even though the ones held on behalf of the embedded one outlive that structure.
Change it so that we always clean up the nfsd_file refs held by the embedded copy structure before nfsd4_copy returns. Rework cleanup_async_copy to handle both inter and intra copies. Eliminate nfsd4_cleanup_intra_ssc since it now becomes a no-op.
--- fs/nfsd/nfs4proc.c | 23 ++++++++++------------- 1 file changed, 10 insertions(+), 13 deletions(-)
fs/nfsd/nfs4proc.c +10 -13
@@ -1512,7 +1512,6 @@ nfsd4_cleanup_inter_ssc(struct nfsd4_ssc_umount_item *nsui, struct file *filp,long timeout =msecs_to_jiffies (nfsd4_ssc_umount_timeout);nfs42_ssc_close (filp); -nfsd_file_put (dst);fput (filp);spin_lock (&nn->nfsd_ssc_lock);
@@ -1562,13 +1561,6 @@ nfsd4_setup_intra_ssc(struct svc_rqst *rqstp, ©->nf_dst); } -static void -nfsd4_cleanup_intra_ssc (struct nfsd_file *src,struct nfsd_file *dst) -{ -nfsd_file_put (src); -nfsd_file_put (dst); -} -static void nfsd4_cb_offload_release (struct nfsd4_callback *cb) { struct nfsd4_cb_offload *cbo =
@@ -1683,12 +1675,18 @@ static void dup_copy_fields(struct nfsd4_copy *src, struct nfsd4_copy *dst) dst->ss_nsui = src->ss_nsui; } +static void release_copy_files (struct nfsd4_copy *copy) +{ +if (copy->nf_src) +nfsd_file_put (copy->nf_src); +if (copy->nf_dst) +nfsd_file_put (copy->nf_dst); +} +static void cleanup_async_copy (struct nfsd4_copy *copy) {nfs4_free_copy_state (copy); -nfsd_file_put (copy->nf_dst); -if (!nfsd4_ssc_is_inter (copy)) -nfsd_file_put (copy->nf_src); +release_copy_files (copy);spin_lock (©->cp_clp->async_lock);list_del (©->copies);spin_unlock (©->cp_clp->async_lock);
@@ -1748,7 +1746,6 @@ static int nfsd4_do_async_copy(void *data) }else { nfserr =nfsd4_do_copy (copy, copy->nf_src->nf_file, copy->nf_dst->nf_file, false); -nfsd4_cleanup_intra_ssc (copy->nf_src, copy->nf_dst); } do_callback:
@@ -1811,9 +1808,9 @@ nfsd4_copy(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate, }else { status =nfsd4_do_copy (copy, copy->nf_src->nf_file, copy->nf_dst->nf_file, true); -nfsd4_cleanup_intra_ssc (copy->nf_src, copy->nf_dst); } out: +release_copy_files (copy);return status; out_err:if (async_copy)
Signature · 1 line
2.39.0
Re: [PATCH 2/2] nfsd: clean up potential nfsd_file refcount leaks in COPY codepath
Quoted 11 lines · Jeff Layton
There are two different flavors of the
nfsd4_copystruct. One is embedded in the compound and is used directly in synchronous copies. The other is dynamically allocated, refcounted and tracked in the client struture. For the embedded one, the cleanup just involves releasing anynfsd_filesheld on its behalf. For the async one, the cleanup is a bit more involved, and we need to dequeue it from lists, unhash it, etc.There is at least one potential refcount leak in this code now. If the
kthread_createcall fails, then both the src and dstnfsd_filesin the originalnfsd4_copyobject are leaked.
I don't believe that's true. If kthread_create thread fails we call cleanup_async_copy() that does a put on the file descriptors.
Quoted 3 lines
The cleanup in this codepath is also sort of weird. In the async copy case, we'll have up to four
nfsd_filereferences (src and dst for both flavors of copy structure).
That's not true. There is a careful distinction between intra -- which had 2 valid file pointers and does a get on both as they both point to something that's opened on this server--- but inter -- only does a get on the dst file descriptor, the src doesn't exit. And yes I realize the code checks for nfs_src being null which it should be but it makes the code less clear and at some point somebody might want to decide to really do a put on it.
Quoted 8 lines
They are both put at the end of
nfsd4_do_async_copy, even though the ones held on behalf of the embedded one outlive that structure.Change it so that we always clean up the
nfsd_filerefs held by the embedded copy structure beforenfsd4_copyreturns. Reworkcleanup_async_copyto handle both inter and intra copies. Eliminatenfsd4_cleanup_intra_sscsince it now becomes a no-op.
I feel by combining the cleanup for both it obscures a very important destication that src filehandle doesn't exist for inter.
Quoted 75 lines
Signed-off-by Jeff Layton <[email protected]>--- fs/nfsd/nfs4proc.c | 23 ++++++++++------------- 1 file changed, 10 insertions(+), 13 deletions(-)fs/nfsd/nfs4proc.c +10
-14@@ -1512,7 +1512,6 @@ nfsd4_cleanup_inter_ssc(struct nfsd4_ssc_umount_item *nsui, struct file *filp,long timeout =msecs_to_jiffies (nfsd4_ssc_umount_timeout);nfs42_ssc_close (filp); -nfsd_file_put (dst);fput (filp);spin_lock (&nn->nfsd_ssc_lock);@@ -1562,13 +1561,6 @@ nfsd4_setup_intra_ssc(struct svc_rqst *rqstp, ©->nf_dst); } -static void -nfsd4_cleanup_intra_ssc (struct nfsd_file *src,struct nfsd_file *dst) -{ -nfsd_file_put (src); -nfsd_file_put (dst); -} -static void nfsd4_cb_offload_release (struct nfsd4_callback *cb) { struct nfsd4_cb_offload *cbo =@@ -1683,12 +1675,18 @@ static void dup_copy_fields(struct nfsd4_copy *src, struct nfsd4_copy *dst) dst->ss_nsui = src->ss_nsui; } +static void release_copy_files (struct nfsd4_copy *copy) +{ +if (copy->nf_src) +nfsd_file_put (copy->nf_src); +if (copy->nf_dst) +nfsd_file_put (copy->nf_dst); +} +static void cleanup_async_copy (struct nfsd4_copy *copy) {nfs4_free_copy_state (copy); -nfsd_file_put (copy->nf_dst); -if (!nfsd4_ssc_is_inter (copy)) -nfsd_file_put (copy->nf_src); +release_copy_files (copy);spin_lock (©->cp_clp->async_lock);list_del (©->copies);spin_unlock (©->cp_clp->async_lock);@@ -1748,7 +1746,6 @@ static int nfsd4_do_async_copy(void *data) }else { nfserr =nfsd4_do_copy (copy, copy->nf_src->nf_file, copy->nf_dst->nf_file, false); -nfsd4_cleanup_intra_ssc (copy->nf_src, copy->nf_dst); } do_callback:@@ -1811,9 +1808,9 @@ nfsd4_copy(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate, }else { status =nfsd4_do_copy (copy, copy->nf_src->nf_file, copy->nf_dst->nf_file, true); -nfsd4_cleanup_intra_ssc (copy->nf_src, copy->nf_dst); } out: +release_copy_files (copy);return status; out_err:if (async_copy) --2.39.0
Jeff Layton
maintainerRe: [PATCH 2/2] nfsd: clean up potential nfsd_file refcount leaks in COPY codepath
Quoted 9 lines · Olga Kornievskaia
Quoted 8 lines · Jeff Layton
There are two different flavors of the
nfsd4_copystruct. One is embedded in the compound and is used directly in synchronous copies. The other is dynamically allocated, refcounted and tracked in the client struture. For the embedded one, the cleanup just involves releasing anynfsd_filesheld on its behalf. For the async one, the cleanup is a bit more involved, and we need to dequeue it from lists, unhash it, etc.
Quoted 7 lines
Quoted 3 lines
There is at least one potential refcount leak in this code now. If the
kthread_createcall fails, then both the src and dstnfsd_filesin the originalnfsd4_copyobject are leaked.I don't believe that's true. If
kthread_createthread fails we callcleanup_async_copy()that does a put on the file descriptors.
You mean this?
out_err:
if (async_copy)
cleanup_async_copy(async_copy);That puts the references that were taken in dup_copy_fields, but the original (embedded) nfsd4_copy also holds references and those are not being put in this codepath.
Quoted 4 lines
Quoted 3 lines
The cleanup in this codepath is also sort of weird. In the async copy case, we'll have up to four
nfsd_filereferences (src and dst for both flavors of copy structure).
Quoted 8 lines
That's not true. There is a careful distinction between intra -- which had 2 valid file pointers and does a get on both as they both point to something that's opened on this server--- but inter -- only does a get on the dst file descriptor, the src doesn't exit. And yes I realize the code checks for
nfs_srcbeing null which it should be but it makes the code less clear and at some point somebody might want to decide to really do a put on it.
This is part of the problem here. We have a nfsd4_copy structure, and depending on what has been done to it, you need to call different methods to clean it up. That seems like a real antipattern to me.
Quoted 11 lines
Quoted 8 lines
They are both put at the end of
nfsd4_do_async_copy, even though the ones held on behalf of the embedded one outlive that structure.Change it so that we always clean up the
nfsd_filerefs held by the embedded copy structure beforenfsd4_copyreturns. Reworkcleanup_async_copyto handle both inter and intra copies. Eliminatenfsd4_cleanup_intra_sscsince it now becomes a no-op.I feel by combining the cleanup for both it obscures a very important destication that src filehandle doesn't exist for inter.
If the src filehandle doesn't exist, then the pointer to it will be NULL. I don't see what we gain by keeping these two distinct, other than avoiding a NULL pointer check.
Quoted 76 lines
Quoted 76 lines
Signed-off-by Jeff Layton <[email protected]>--- fs/nfsd/nfs4proc.c | 23 ++++++++++------------- 1 file changed, 10 insertions(+), 13 deletions(-)fs/nfsd/nfs4proc.c +10
-14@@ -1512,7 +1512,6 @@ nfsd4_cleanup_inter_ssc(struct nfsd4_ssc_umount_item *nsui, struct file *filp,long timeout =msecs_to_jiffies (nfsd4_ssc_umount_timeout);nfs42_ssc_close (filp); -nfsd_file_put (dst);fput (filp);spin_lock (&nn->nfsd_ssc_lock);@@ -1562,13 +1561,6 @@ nfsd4_setup_intra_ssc(struct svc_rqst *rqstp, ©->nf_dst); } -static void -nfsd4_cleanup_intra_ssc (struct nfsd_file *src,struct nfsd_file *dst) -{ -nfsd_file_put (src); -nfsd_file_put (dst); -} -static void nfsd4_cb_offload_release (struct nfsd4_callback *cb) { struct nfsd4_cb_offload *cbo =@@ -1683,12 +1675,18 @@ static void dup_copy_fields(struct nfsd4_copy *src, struct nfsd4_copy *dst) dst->ss_nsui = src->ss_nsui; } +static void release_copy_files (struct nfsd4_copy *copy) +{ +if (copy->nf_src) +nfsd_file_put (copy->nf_src); +if (copy->nf_dst) +nfsd_file_put (copy->nf_dst); +} +static void cleanup_async_copy (struct nfsd4_copy *copy) {nfs4_free_copy_state (copy); -nfsd_file_put (copy->nf_dst); -if (!nfsd4_ssc_is_inter (copy)) -nfsd_file_put (copy->nf_src); +release_copy_files (copy);spin_lock (©->cp_clp->async_lock);list_del (©->copies);spin_unlock (©->cp_clp->async_lock);@@ -1748,7 +1746,6 @@ static int nfsd4_do_async_copy(void *data) }else { nfserr =nfsd4_do_copy (copy, copy->nf_src->nf_file, copy->nf_dst->nf_file, false); -nfsd4_cleanup_intra_ssc (copy->nf_src, copy->nf_dst); } do_callback:@@ -1811,9 +1808,9 @@ nfsd4_copy(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate, }else { status =nfsd4_do_copy (copy, copy->nf_src->nf_file, copy->nf_dst->nf_file, true); -nfsd4_cleanup_intra_ssc (copy->nf_src, copy->nf_dst); } out: +release_copy_files (copy);return status; out_err:if (async_copy) --2.39.0
Signature · 1 line
Jeff Layton <[email protected]>
Re: [PATCH 2/2] nfsd: clean up potential nfsd_file refcount leaks in COPY codepath
Quoted 21 lines · Jeff Layton
Quoted 9 lines · Olga Kornievskaia
Quoted 8 lines · Jeff Layton
There are two different flavors of the
nfsd4_copystruct. One is embedded in the compound and is used directly in synchronous copies. The other is dynamically allocated, refcounted and tracked in the client struture. For the embedded one, the cleanup just involves releasing anynfsd_filesheld on its behalf. For the async one, the cleanup is a bit more involved, and we need to dequeue it from lists, unhash it, etc.Quoted 7 lines
Quoted 3 lines
There is at least one potential refcount leak in this code now. If the
kthread_createcall fails, then both the src and dstnfsd_filesin the originalnfsd4_copyobject are leaked.I don't believe that's true. If
kthread_createthread fails we callcleanup_async_copy()that does a put on the file descriptors.You mean this?
Quoted 7 lines
out_err:if (async_copy) cleanup_async_copy(async_copy);That puts the references that were taken in
dup_copy_fields, but the original (embedded)nfsd4_copyalso holds references and those are not being put in this codepath.
Can you please point out where do we take a reference on the original copy?
Quoted 13 lines
Quoted 4 lines
Quoted 3 lines
The cleanup in this codepath is also sort of weird. In the async copy case, we'll have up to four
nfsd_filereferences (src and dst for both flavors of copy structure).Quoted 8 lines
That's not true. There is a careful distinction between intra -- which had 2 valid file pointers and does a get on both as they both point to something that's opened on this server--- but inter -- only does a get on the dst file descriptor, the src doesn't exit. And yes I realize the code checks for
nfs_srcbeing null which it should be but it makes the code less clear and at some point somebody might want to decide to really do a put on it.
Quoted 3 lines
This is part of the problem here. We have a
nfsd4_copystructure, and depending on what has been done to it, you need to call different methods to clean it up. That seems like a real antipattern to me.
But they call different methods because different things need to be done there and it makes it clear what needs to be for what type of copy.
Quoted 9 lines
Quoted 9 lines
Quoted 8 lines
They are both put at the end of
nfsd4_do_async_copy, even though the ones held on behalf of the embedded one outlive that structure.Change it so that we always clean up the
nfsd_filerefs held by the embedded copy structure beforenfsd4_copyreturns. Reworkcleanup_async_copyto handle both inter and intra copies. Eliminatenfsd4_cleanup_intra_sscsince it now becomes a no-op.
Quoted 6 lines
Quoted 2 lines
I feel by combining the cleanup for both it obscures a very important destication that src filehandle doesn't exist for inter.
If the src filehandle doesn't exist, then the pointer to it will be NULL. I don't see what we gain by keeping these two distinct, other than avoiding a NULL pointer check.
My reason would be for code clarity because different things are supposed to happen for intra and inter. Difference of opinion it seems.
Quoted 80 lines
Quoted 62 lines
Quoted 62 lines
Signed-off-by Jeff Layton <[email protected]>--- fs/nfsd/nfs4proc.c | 23 ++++++++++------------- 1 file changed, 10 insertions(+), 13 deletions(-)fs/nfsd/nfs4proc.c +9
-12@@ -1512,7 +1512,6 @@ nfsd4_cleanup_inter_ssc(struct nfsd4_ssc_umount_item *nsui, struct file *filp,long timeout =msecs_to_jiffies (nfsd4_ssc_umount_timeout);nfs42_ssc_close (filp); -nfsd_file_put (dst);fput (filp);spin_lock (&nn->nfsd_ssc_lock);@@ -1562,13 +1561,6 @@ nfsd4_setup_intra_ssc(struct svc_rqst *rqstp, ©->nf_dst); } -static void -nfsd4_cleanup_intra_ssc (struct nfsd_file *src,struct nfsd_file *dst) -{ -nfsd_file_put (src); -nfsd_file_put (dst); -} -static void nfsd4_cb_offload_release (struct nfsd4_callback *cb) { struct nfsd4_cb_offload *cbo =@@ -1683,12 +1675,18 @@ static void dup_copy_fields(struct nfsd4_copy *src, struct nfsd4_copy *dst) dst->ss_nsui = src->ss_nsui; } +static void release_copy_files (struct nfsd4_copy *copy) +{ +if (copy->nf_src) +nfsd_file_put (copy->nf_src); +if (copy->nf_dst) +nfsd_file_put (copy->nf_dst); +} +static void cleanup_async_copy (struct nfsd4_copy *copy) {nfs4_free_copy_state (copy); -nfsd_file_put (copy->nf_dst); -if (!nfsd4_ssc_is_inter (copy)) -nfsd_file_put (copy->nf_src); +release_copy_files (copy);spin_lock (©->cp_clp->async_lock);list_del (©->copies);spin_unlock (©->cp_clp->async_lock);@@ -1748,7 +1746,6 @@ static int nfsd4_do_async_copy(void *data) }else { nfserr =nfsd4_do_copy (copy, copy->nf_src->nf_file, copy->nf_dst->nf_file, false); -nfsd4_cleanup_intra_ssc (copy->nf_src, copy->nf_dst); }Quoted 15 lines
Quoted 13 lines
fs/nfsd/nfs4proc.c +1
-2do_callback:@@ -1811,9 +1808,9 @@ nfsd4_copy(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate, }else { status =nfsd4_do_copy (copy, copy->nf_src->nf_file, copy->nf_dst->nf_file, true); -nfsd4_cleanup_intra_ssc (copy->nf_src, copy->nf_dst); } out: +release_copy_files (copy);return status; out_err:if (async_copy) --Quoted 2 lines
2.39.0
--
Jeff Layton <[email protected]>
Re: [PATCH 2/2] nfsd: clean up potential nfsd_file refcount leaks in COPY codepath
Quoted 49 lines · Olga Kornievskaia
Quoted 21 lines · Jeff Layton
Quoted 9 lines · Olga Kornievskaia
Quoted 8 lines · Jeff Layton
There are two different flavors of the
nfsd4_copystruct. One is embedded in the compound and is used directly in synchronous copies. The other is dynamically allocated, refcounted and tracked in the client struture. For the embedded one, the cleanup just involves releasing anynfsd_filesheld on its behalf. For the async one, the cleanup is a bit more involved, and we need to dequeue it from lists, unhash it, etc.Quoted 7 lines
Quoted 3 lines
There is at least one potential refcount leak in this code now. If the
kthread_createcall fails, then both the src and dstnfsd_filesin the originalnfsd4_copyobject are leaked.I don't believe that's true. If
kthread_createthread fails we callcleanup_async_copy()that does a put on the file descriptors.You mean this?
Quoted 7 lines
out_err:if (async_copy) cleanup_async_copy(async_copy);That puts the references that were taken in
dup_copy_fields, but the original (embedded)nfsd4_copyalso holds references and those are not being put in this codepath.Can you please point out where do we take a reference on the original copy?
Quoted 13 lines
Quoted 12 lines
Quoted 3 lines
The cleanup in this codepath is also sort of weird. In the async copy case, we'll have up to four
nfsd_filereferences (src and dst for both flavors of copy structure).That's not true. There is a careful distinction between intra -- which had 2 valid file pointers and does a get on both as they both point to something that's opened on this server--- but inter -- only does a get on the dst file descriptor, the src doesn't exit. And yes I realize the code checks for
nfs_srcbeing null which it should be but it makes the code less clear and at some point somebody might want to decide to really do a put on it.
Quoted 7 lines
Quoted 3 lines
This is part of the problem here. We have a
nfsd4_copystructure, and depending on what has been done to it, you need to call different methods to clean it up. That seems like a real antipattern to me.But they call different methods because different things need to be done there and it makes it clear what needs to be for what type of copy.
In cases like this, it makes sense to consider using types to ensure the code can't do the wrong thing. So you might want to have a struct nfs4_copy_A for the inter code to use, and a struct nfs4_copy_B for the intra code to use. Sharing the same struct for both use cases is probably what's confusing to human readers.
I've never been a stickler for removing every last ounce of code duplication. Here, it might help to have a little duplication just to make it easier to reason about the reference counting in the two use cases.
That's my view from the mountain top, worth every penny you paid for it.
Quoted 12 lines
Quoted 12 lines
Quoted 11 lines
Quoted 8 lines
They are both put at the end of
nfsd4_do_async_copy, even though the ones held on behalf of the embedded one outlive that structure.Change it so that we always clean up the
nfsd_filerefs held by the embedded copy structure beforenfsd4_copyreturns. Reworkcleanup_async_copyto handle both inter and intra copies. Eliminatenfsd4_cleanup_intra_sscsince it now becomes a no-op.I feel by combining the cleanup for both it obscures a very important destication that src filehandle doesn't exist for inter.
Quoted 7 lines
Quoted 3 lines
If the src filehandle doesn't exist, then the pointer to it will be NULL. I don't see what we gain by keeping these two distinct, other than avoiding a NULL pointer check.
My reason would be for code clarity because different things are supposed to happen for intra and inter. Difference of opinion it seems.
--
Chuck Lever
Jeff Layton
maintainerRe: [PATCH 2/2] nfsd: clean up potential nfsd_file refcount leaks in COPY codepath
Quoted 62 lines · Chuck Lever III
Quoted 48 lines · Olga Kornievskaia
Quoted 21 lines · Jeff Layton
Quoted 9 lines · Olga Kornievskaia
Quoted 8 lines · Jeff Layton
There are two different flavors of the
nfsd4_copystruct. One is embedded in the compound and is used directly in synchronous copies. The other is dynamically allocated, refcounted and tracked in the client struture. For the embedded one, the cleanup just involves releasing anynfsd_filesheld on its behalf. For the async one, the cleanup is a bit more involved, and we need to dequeue it from lists, unhash it, etc.Quoted 7 lines
Quoted 3 lines
There is at least one potential refcount leak in this code now. If the
kthread_createcall fails, then both the src and dstnfsd_filesin the originalnfsd4_copyobject are leaked.I don't believe that's true. If
kthread_createthread fails we callcleanup_async_copy()that does a put on the file descriptors.You mean this?
Quoted 7 lines
out_err:if (async_copy) cleanup_async_copy(async_copy);That puts the references that were taken in
dup_copy_fields, but the original (embedded)nfsd4_copyalso holds references and those are not being put in this codepath.Can you please point out where do we take a reference on the original copy?
Quoted 13 lines
Quoted 12 lines
Quoted 3 lines
The cleanup in this codepath is also sort of weird. In the async copy case, we'll have up to four
nfsd_filereferences (src and dst for both flavors of copy structure).That's not true. There is a careful distinction between intra -- which had 2 valid file pointers and does a get on both as they both point to something that's opened on this server--- but inter -- only does a get on the dst file descriptor, the src doesn't exit. And yes I realize the code checks for
nfs_srcbeing null which it should be but it makes the code less clear and at some point somebody might want to decide to really do a put on it.Quoted 7 lines
Quoted 3 lines
This is part of the problem here. We have a
nfsd4_copystructure, and depending on what has been done to it, you need to call different methods to clean it up. That seems like a real antipattern to me.But they call different methods because different things need to be done there and it makes it clear what needs to be for what type of copy.
In cases like this, it makes sense to consider using types to ensure the code can't do the wrong thing. So you might want to have a struct
nfs4_copy_Afor the inter code to use, and a structnfs4_copy_Bfor the intra code to use. Sharing the same struct for both use cases is probably what's confusing to human readers.
Quoted 8 lines
I've never been a stickler for removing every last ounce of code duplication. Here, it might help to have a little duplication just to make it easier to reason about the reference counting in the two use cases.
That's my view from the mountain top, worth every penny you paid for it.
+1
The nfsd4_copy structure has a lot of fields in it that only matter for the async copy case. ISTM that nfsd4_copy (the function) should dynamically allocate a struct nfsd4_async_copy that contains a nfsd4_copy and whatever other fields are needed.
Then, we could trim down struct nfsd4_copy to just the info needed.
For instance, the nf_src and nf_dst fields really don't need to be in nfsd4_copy. For the synchronous copy case, we can just keep those pointers on the stack, and for the async case they would be inside the larger structure.
That would allow us to trim down the footprint of the compound union too.
Signature · 1 line
Jeff Layton <[email protected]>
Re: [PATCH 2/2] nfsd: clean up potential nfsd_file refcount leaks in COPY codepath
Quoted 76 lines · Jeff Layton
Quoted 62 lines · Chuck Lever III
Quoted 48 lines · Olga Kornievskaia
Quoted 21 lines · Jeff Layton
Quoted 9 lines · Olga Kornievskaia
Quoted 8 lines · Jeff Layton
There are two different flavors of the
nfsd4_copystruct. One is embedded in the compound and is used directly in synchronous copies. The other is dynamically allocated, refcounted and tracked in the client struture. For the embedded one, the cleanup just involves releasing anynfsd_filesheld on its behalf. For the async one, the cleanup is a bit more involved, and we need to dequeue it from lists, unhash it, etc.Quoted 7 lines
Quoted 3 lines
There is at least one potential refcount leak in this code now. If the
kthread_createcall fails, then both the src and dstnfsd_filesin the originalnfsd4_copyobject are leaked.I don't believe that's true. If
kthread_createthread fails we callcleanup_async_copy()that does a put on the file descriptors.You mean this?
Quoted 7 lines
out_err:if (async_copy) cleanup_async_copy(async_copy);That puts the references that were taken in
dup_copy_fields, but the original (embedded)nfsd4_copyalso holds references and those are not being put in this codepath.Can you please point out where do we take a reference on the original copy?
Quoted 13 lines
Quoted 12 lines
Quoted 3 lines
The cleanup in this codepath is also sort of weird. In the async copy case, we'll have up to four
nfsd_filereferences (src and dst for both flavors of copy structure).That's not true. There is a careful distinction between intra -- which had 2 valid file pointers and does a get on both as they both point to something that's opened on this server--- but inter -- only does a get on the dst file descriptor, the src doesn't exit. And yes I realize the code checks for
nfs_srcbeing null which it should be but it makes the code less clear and at some point somebody might want to decide to really do a put on it.Quoted 7 lines
Quoted 3 lines
This is part of the problem here. We have a
nfsd4_copystructure, and depending on what has been done to it, you need to call different methods to clean it up. That seems like a real antipattern to me.But they call different methods because different things need to be done there and it makes it clear what needs to be for what type of copy.
In cases like this, it makes sense to consider using types to ensure the code can't do the wrong thing. So you might want to have a struct
nfs4_copy_Afor the inter code to use, and a structnfs4_copy_Bfor the intra code to use. Sharing the same struct for both use cases is probably what's confusing to human readers.Quoted 8 lines
I've never been a stickler for removing every last ounce of code duplication. Here, it might help to have a little duplication just to make it easier to reason about the reference counting in the two use cases.
That's my view from the mountain top, worth every penny you paid for it.
Quoted 8 lines
+1
The
nfsd4_copystructure has a lot of fields in it that only matter for the async copy case. ISTM thatnfsd4_copy(the function) should dynamically allocate a structnfsd4_async_copythat contains anfsd4_copyand whatever other fields are needed.Then, we could trim down struct
nfsd4_copyto just the info needed.
Yeah, some of those fields are actually quite large, like filehandles.
Quoted 7 lines
For instance, the
nf_srcandnf_dstfields really don't need to be innfsd4_copy. For the synchronous copy case, we can just keep those pointers on the stack, and for the async case they would be inside the larger structure.That would allow us to trim down the footprint of the compound union too.
That seems sensible. Do you feel like redriving this clean-up series with the changes you describe above?
--
Chuck Lever
Jeff Layton
maintainerRe: [PATCH 2/2] nfsd: clean up potential nfsd_file refcount leaks in COPY codepath
Quoted 92 lines · Chuck Lever III
Quoted 75 lines · Jeff Layton
Quoted 62 lines · Chuck Lever III
Quoted 48 lines · Olga Kornievskaia
Quoted 21 lines · Jeff Layton
Quoted 9 lines · Olga Kornievskaia
Quoted 8 lines · Jeff Layton
There are two different flavors of the
nfsd4_copystruct. One is embedded in the compound and is used directly in synchronous copies. The other is dynamically allocated, refcounted and tracked in the client struture. For the embedded one, the cleanup just involves releasing anynfsd_filesheld on its behalf. For the async one, the cleanup is a bit more involved, and we need to dequeue it from lists, unhash it, etc.Quoted 7 lines
Quoted 3 lines
There is at least one potential refcount leak in this code now. If the
kthread_createcall fails, then both the src and dstnfsd_filesin the originalnfsd4_copyobject are leaked.I don't believe that's true. If
kthread_createthread fails we callcleanup_async_copy()that does a put on the file descriptors.You mean this?
Quoted 7 lines
out_err:if (async_copy) cleanup_async_copy(async_copy);That puts the references that were taken in
dup_copy_fields, but the original (embedded)nfsd4_copyalso holds references and those are not being put in this codepath.Can you please point out where do we take a reference on the original copy?
Quoted 13 lines
Quoted 12 lines
Quoted 3 lines
The cleanup in this codepath is also sort of weird. In the async copy case, we'll have up to four
nfsd_filereferences (src and dst for both flavors of copy structure).That's not true. There is a careful distinction between intra -- which had 2 valid file pointers and does a get on both as they both point to something that's opened on this server--- but inter -- only does a get on the dst file descriptor, the src doesn't exit. And yes I realize the code checks for
nfs_srcbeing null which it should be but it makes the code less clear and at some point somebody might want to decide to really do a put on it.Quoted 7 lines
Quoted 3 lines
This is part of the problem here. We have a
nfsd4_copystructure, and depending on what has been done to it, you need to call different methods to clean it up. That seems like a real antipattern to me.But they call different methods because different things need to be done there and it makes it clear what needs to be for what type of copy.
In cases like this, it makes sense to consider using types to ensure the code can't do the wrong thing. So you might want to have a struct
nfs4_copy_Afor the inter code to use, and a structnfs4_copy_Bfor the intra code to use. Sharing the same struct for both use cases is probably what's confusing to human readers.Quoted 8 lines
I've never been a stickler for removing every last ounce of code duplication. Here, it might help to have a little duplication just to make it easier to reason about the reference counting in the two use cases.
That's my view from the mountain top, worth every penny you paid for it.
Quoted 8 lines
+1
The
nfsd4_copystructure has a lot of fields in it that only matter for the async copy case. ISTM thatnfsd4_copy(the function) should dynamically allocate a structnfsd4_async_copythat contains anfsd4_copyand whatever other fields are needed.Then, we could trim down struct
nfsd4_copyto just the info needed.Yeah, some of those fields are actually quite large, like filehandles.
Quoted 5 lines
For instance, the
nf_srcandnf_dstfields really don't need to be innfsd4_copy. For the synchronous copy case, we can just keep those pointers on the stack, and for the async case they would be inside the larger structure.
Quoted 6 lines
Quoted 2 lines
That would allow us to trim down the footprint of the compound union too.
That seems sensible. Do you feel like redriving this clean-up series with the changes you describe above?
I can, unless Olga, Dai or someone else would rather do it. Not sure how soon I can get to it though.
Signature · 1 line
Jeff Layton <[email protected]>
Re: [PATCH 2/2] nfsd: clean up potential nfsd_file refcount leaks in COPY codepath
Quoted 97 lines · Jeff Layton
Quoted 95 lines · Chuck Lever III
Quoted 75 lines · Jeff Layton
Quoted 62 lines · Chuck Lever III
Quoted 48 lines · Olga Kornievskaia
Quoted 21 lines · Jeff Layton
Quoted 9 lines · Olga Kornievskaia
Quoted 8 lines · Jeff Layton
There are two different flavors of the
nfsd4_copystruct. One is embedded in the compound and is used directly in synchronous copies. The other is dynamically allocated, refcounted and tracked in the client struture. For the embedded one, the cleanup just involves releasing anynfsd_filesheld on its behalf. For the async one, the cleanup is a bit more involved, and we need to dequeue it from lists, unhash it, etc.Quoted 7 lines
Quoted 3 lines
There is at least one potential refcount leak in this code now. If the
kthread_createcall fails, then both the src and dstnfsd_filesin the originalnfsd4_copyobject are leaked.I don't believe that's true. If
kthread_createthread fails we callcleanup_async_copy()that does a put on the file descriptors.You mean this?
Quoted 7 lines
out_err:if (async_copy) cleanup_async_copy(async_copy);That puts the references that were taken in
dup_copy_fields, but the original (embedded)nfsd4_copyalso holds references and those are not being put in this codepath.Can you please point out where do we take a reference on the original copy?
Quoted 13 lines
Quoted 12 lines
Quoted 3 lines
The cleanup in this codepath is also sort of weird. In the async copy case, we'll have up to four
nfsd_filereferences (src and dst for both flavors of copy structure).That's not true. There is a careful distinction between intra -- which had 2 valid file pointers and does a get on both as they both point to something that's opened on this server--- but inter -- only does a get on the dst file descriptor, the src doesn't exit. And yes I realize the code checks for
nfs_srcbeing null which it should be but it makes the code less clear and at some point somebody might want to decide to really do a put on it.Quoted 7 lines
Quoted 3 lines
This is part of the problem here. We have a
nfsd4_copystructure, and depending on what has been done to it, you need to call different methods to clean it up. That seems like a real antipattern to me.But they call different methods because different things need to be done there and it makes it clear what needs to be for what type of copy.
In cases like this, it makes sense to consider using types to ensure the code can't do the wrong thing. So you might want to have a struct
nfs4_copy_Afor the inter code to use, and a structnfs4_copy_Bfor the intra code to use. Sharing the same struct for both use cases is probably what's confusing to human readers.Quoted 8 lines
I've never been a stickler for removing every last ounce of code duplication. Here, it might help to have a little duplication just to make it easier to reason about the reference counting in the two use cases.
That's my view from the mountain top, worth every penny you paid for it.
Quoted 8 lines
+1
The
nfsd4_copystructure has a lot of fields in it that only matter for the async copy case. ISTM thatnfsd4_copy(the function) should dynamically allocate a structnfsd4_async_copythat contains anfsd4_copyand whatever other fields are needed.Then, we could trim down struct
nfsd4_copyto just the info needed.Yeah, some of those fields are actually quite large, like filehandles.
Quoted 7 lines
For instance, the
nf_srcandnf_dstfields really don't need to be innfsd4_copy. For the synchronous copy case, we can just keep those pointers on the stack, and for the async case they would be inside the larger structure.That would allow us to trim down the footprint of the compound union too.
Quoted 6 lines
Quoted 3 lines
That seems sensible. Do you feel like redriving this clean-up series with the changes you describe above?
I can, unless Olga, Dai or someone else would rather do it. Not sure how soon I can get to it though.
I'm not going to volunteer as I don't believe in the suggested change.
I think there is a performance advantage to having this structure preallocated and ready for use.
Quoted 2 lines
--
Jeff Layton <[email protected]>
Jeff Layton
maintainerRe: [PATCH 2/2] nfsd: clean up potential nfsd_file refcount leaks in COPY codepath
In reply to Olga KornievskaiaQuoted 26 lines · Olga Kornievskaia
Quoted 25 lines · Jeff Layton
Quoted 9 lines · Olga Kornievskaia
Quoted 8 lines · Jeff Layton
There are two different flavors of the
nfsd4_copystruct. One is embedded in the compound and is used directly in synchronous copies. The other is dynamically allocated, refcounted and tracked in the client struture. For the embedded one, the cleanup just involves releasing anynfsd_filesheld on its behalf. For the async one, the cleanup is a bit more involved, and we need to dequeue it from lists, unhash it, etc.Quoted 7 lines
Quoted 3 lines
There is at least one potential refcount leak in this code now. If the
kthread_createcall fails, then both the src and dstnfsd_filesin the originalnfsd4_copyobject are leaked.I don't believe that's true. If
kthread_createthread fails we callcleanup_async_copy()that does a put on the file descriptors.You mean this?
out_err:if (async_copy) cleanup_async_copy(async_copy);
Quoted 6 lines
Quoted 3 lines
That puts the references that were taken in
dup_copy_fields, but the original (embedded)nfsd4_copyalso holds references and those are not being put in this codepath.Can you please point out where do we take a reference on the original copy?
In the case of an inter-server copy, nf_dst is set in nfsd4_setup_inter_ssc. For intraserver copy, both pointers are set via the call to nfsd4_verify_copy. Both functions call nfs4_preprocess_stateid_op, which returns a reference to the nfsd_file in the second to last arg.
Quoted 13 lines
Quoted 13 lines
Quoted 12 lines
Quoted 3 lines
The cleanup in this codepath is also sort of weird. In the async copy case, we'll have up to four
nfsd_filereferences (src and dst for both flavors of copy structure).That's not true. There is a careful distinction between intra -- which had 2 valid file pointers and does a get on both as they both point to something that's opened on this server--- but inter -- only does a get on the dst file descriptor, the src doesn't exit. And yes I realize the code checks for
nfs_srcbeing null which it should be but it makes the code less clear and at some point somebody might want to decide to really do a put on it.
Quoted 8 lines
Quoted 3 lines
This is part of the problem here. We have a
nfsd4_copystructure, and depending on what has been done to it, you need to call different methods to clean it up. That seems like a real antipattern to me.But they call different methods because different things need to be done there and it makes it clear what needs to be for what type of copy.
I sure as hell had a hard time dissecting how all of that was supposed to work. There is clear bug here, and I think this patch makes the result clearer and more robust in the face of changes.
There are actually 4 different cases here: sync vs. async, alongside intra vs. interserver copy. These are all overloaded onto a nfsd4_copy structure, seemingly for no good reason.
The cleanup, in particular seems quite fragile to me, and there is a dearth of defensive coding measures. If you subtly call the "wrong" cleanup function at the wrong point in time, then things may go awry.
I'll leave it up to Chuck to make the final determination, but I see this patch as an improvement.
Quoted 100 lines
Quoted 9 lines
Quoted 9 lines
Quoted 8 lines
They are both put at the end of
nfsd4_do_async_copy, even though the ones held on behalf of the embedded one outlive that structure.Change it so that we always clean up the
nfsd_filerefs held by the embedded copy structure beforenfsd4_copyreturns. Reworkcleanup_async_copyto handle both inter and intra copies. Eliminatenfsd4_cleanup_intra_sscsince it now becomes a no-op.Quoted 6 lines
Quoted 2 lines
I feel by combining the cleanup for both it obscures a very important destication that src filehandle doesn't exist for inter.
If the src filehandle doesn't exist, then the pointer to it will be NULL. I don't see what we gain by keeping these two distinct, other than avoiding a NULL pointer check.
My reason would be for code clarity because different things are supposed to happen for intra and inter. Difference of opinion it seems.
Quoted 80 lines
Quoted 62 lines
Quoted 62 lines
Signed-off-by Jeff Layton <[email protected]>--- fs/nfsd/nfs4proc.c | 23 ++++++++++------------- 1 file changed, 10 insertions(+), 13 deletions(-)fs/nfsd/nfs4proc.c +9
-12@@ -1512,7 +1512,6 @@ nfsd4_cleanup_inter_ssc(struct nfsd4_ssc_umount_item *nsui, struct file *filp,long timeout =msecs_to_jiffies (nfsd4_ssc_umount_timeout);nfs42_ssc_close (filp); -nfsd_file_put (dst);fput (filp);spin_lock (&nn->nfsd_ssc_lock);@@ -1562,13 +1561,6 @@ nfsd4_setup_intra_ssc(struct svc_rqst *rqstp, ©->nf_dst); } -static void -nfsd4_cleanup_intra_ssc (struct nfsd_file *src,struct nfsd_file *dst) -{ -nfsd_file_put (src); -nfsd_file_put (dst); -} -static void nfsd4_cb_offload_release (struct nfsd4_callback *cb) { struct nfsd4_cb_offload *cbo =@@ -1683,12 +1675,18 @@ static void dup_copy_fields(struct nfsd4_copy *src, struct nfsd4_copy *dst) dst->ss_nsui = src->ss_nsui; } +static void release_copy_files (struct nfsd4_copy *copy) +{ +if (copy->nf_src) +nfsd_file_put (copy->nf_src); +if (copy->nf_dst) +nfsd_file_put (copy->nf_dst); +} +static void cleanup_async_copy (struct nfsd4_copy *copy) {nfs4_free_copy_state (copy); -nfsd_file_put (copy->nf_dst); -if (!nfsd4_ssc_is_inter (copy)) -nfsd_file_put (copy->nf_src); +release_copy_files (copy);spin_lock (©->cp_clp->async_lock);list_del (©->copies);spin_unlock (©->cp_clp->async_lock);@@ -1748,7 +1746,6 @@ static int nfsd4_do_async_copy(void *data) }else { nfserr =nfsd4_do_copy (copy, copy->nf_src->nf_file, copy->nf_dst->nf_file, false); -nfsd4_cleanup_intra_ssc (copy->nf_src, copy->nf_dst); }Quoted 15 lines
Quoted 13 lines
fs/nfsd/nfs4proc.c +1
-2do_callback:@@ -1811,9 +1808,9 @@ nfsd4_copy(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate, }else { status =nfsd4_do_copy (copy, copy->nf_src->nf_file, copy->nf_dst->nf_file, true); -nfsd4_cleanup_intra_ssc (copy->nf_src, copy->nf_dst); } out: +release_copy_files (copy);return status; out_err:if (async_copy) --Quoted 2 lines
2.39.0
--
Jeff Layton <[email protected]>
Signature · 1 line
Jeff Layton <[email protected]>
Re: [PATCH 2/2] nfsd: clean up potential nfsd_file refcount leaks in COPY codepath
Quoted 32 lines · Jeff Layton
Quoted 30 lines · Olga Kornievskaia
Quoted 28 lines · Jeff Layton
Quoted 9 lines · Olga Kornievskaia
Quoted 8 lines · Jeff Layton
There are two different flavors of the
nfsd4_copystruct. One is embedded in the compound and is used directly in synchronous copies. The other is dynamically allocated, refcounted and tracked in the client struture. For the embedded one, the cleanup just involves releasing anynfsd_filesheld on its behalf. For the async one, the cleanup is a bit more involved, and we need to dequeue it from lists, unhash it, etc.Quoted 7 lines
Quoted 3 lines
There is at least one potential refcount leak in this code now. If the
kthread_createcall fails, then both the src and dstnfsd_filesin the originalnfsd4_copyobject are leaked.I don't believe that's true. If
kthread_createthread fails we callcleanup_async_copy()that does a put on the file descriptors.You mean this?
out_err:if (async_copy) cleanup_async_copy(async_copy);That puts the references that were taken in
dup_copy_fields, but the original (embedded)nfsd4_copyalso holds references and those are not being put in this codepath.
Quoted 8 lines
Quoted 2 lines
Can you please point out where do we take a reference on the original copy?
In the case of an inter-server copy,
nf_dstis set innfsd4_setup_inter_ssc. For intraserver copy, both pointers are set via the call tonfsd4_verify_copy. Both functions callnfs4_preprocess_stateid_op, which returns a reference to thenfsd_filein the second to last arg.
Ah. Thank you. I didn't know that nfs4_preprocess_stateid_op() takes a reference on it's 5th argument. I think I was previously looking at nfsd4_read() function which calls nfs4_preprocess_stateid_op() and gets back read->rd_nf but it never does a put on it when it returns. However, I now looked at other functions that call nfs4_preproess_stateid_op() such as nfsd4_fallocate() and I see that it does a put.
Quoted 142 lines
Quoted 14 lines
Quoted 14 lines
Quoted 13 lines
Quoted 4 lines
The cleanup in this codepath is also sort of weird. In the async copy case, we'll have up to four
nfsd_filereferences (src and dst for both flavors of copy structure).That's not true. There is a careful distinction between intra -- which had 2 valid file pointers and does a get on both as they both point to something that's opened on this server--- but inter -- only does a get on the dst file descriptor, the src doesn't exit. And yes I realize the code checks for
nfs_srcbeing null which it should be but it makes the code less clear and at some point somebody might want to decide to really do a put on it.Quoted 8 lines
Quoted 3 lines
This is part of the problem here. We have a
nfsd4_copystructure, and depending on what has been done to it, you need to call different methods to clean it up. That seems like a real antipattern to me.But they call different methods because different things need to be done there and it makes it clear what needs to be for what type of copy.
I sure as hell had a hard time dissecting how all of that was supposed to work. There is clear bug here, and I think this patch makes the result clearer and more robust in the face of changes.
There are actually 4 different cases here: sync vs. async, alongside intra vs. interserver copy. These are all overloaded onto a
nfsd4_copystructure, seemingly for no good reason.The cleanup, in particular seems quite fragile to me, and there is a dearth of defensive coding measures. If you subtly call the "wrong" cleanup function at the wrong point in time, then things may go awry.
I'll leave it up to Chuck to make the final determination, but I see this patch as an improvement.
Quoted 12 lines
Quoted 12 lines
Quoted 11 lines
Quoted 8 lines
They are both put at the end of
nfsd4_do_async_copy, even though the ones held on behalf of the embedded one outlive that structure.Change it so that we always clean up the
nfsd_filerefs held by the embedded copy structure beforenfsd4_copyreturns. Reworkcleanup_async_copyto handle both inter and intra copies. Eliminatenfsd4_cleanup_intra_sscsince it now becomes a no-op.I feel by combining the cleanup for both it obscures a very important destication that src filehandle doesn't exist for inter.
Quoted 88 lines
Quoted 3 lines
If the src filehandle doesn't exist, then the pointer to it will be NULL. I don't see what we gain by keeping these two distinct, other than avoiding a NULL pointer check.
My reason would be for code clarity because different things are supposed to happen for intra and inter. Difference of opinion it seems.
Quoted 80 lines
Quoted 62 lines
Quoted 62 lines
Signed-off-by Jeff Layton <[email protected]>--- fs/nfsd/nfs4proc.c | 23 ++++++++++------------- 1 file changed, 10 insertions(+), 13 deletions(-)fs/nfsd/nfs4proc.c +9
-12@@ -1512,7 +1512,6 @@ nfsd4_cleanup_inter_ssc(struct nfsd4_ssc_umount_item *nsui, struct file *filp,long timeout =msecs_to_jiffies (nfsd4_ssc_umount_timeout);nfs42_ssc_close (filp); -nfsd_file_put (dst);fput (filp);spin_lock (&nn->nfsd_ssc_lock);@@ -1562,13 +1561,6 @@ nfsd4_setup_intra_ssc(struct svc_rqst *rqstp, ©->nf_dst); } -static void -nfsd4_cleanup_intra_ssc (struct nfsd_file *src,struct nfsd_file *dst) -{ -nfsd_file_put (src); -nfsd_file_put (dst); -} -static void nfsd4_cb_offload_release (struct nfsd4_callback *cb) { struct nfsd4_cb_offload *cbo =@@ -1683,12 +1675,18 @@ static void dup_copy_fields(struct nfsd4_copy *src, struct nfsd4_copy *dst) dst->ss_nsui = src->ss_nsui; } +static void release_copy_files (struct nfsd4_copy *copy) +{ +if (copy->nf_src) +nfsd_file_put (copy->nf_src); +if (copy->nf_dst) +nfsd_file_put (copy->nf_dst); +} +static void cleanup_async_copy (struct nfsd4_copy *copy) {nfs4_free_copy_state (copy); -nfsd_file_put (copy->nf_dst); -if (!nfsd4_ssc_is_inter (copy)) -nfsd_file_put (copy->nf_src); +release_copy_files (copy);spin_lock (©->cp_clp->async_lock);list_del (©->copies);spin_unlock (©->cp_clp->async_lock);@@ -1748,7 +1746,6 @@ static int nfsd4_do_async_copy(void *data) }else { nfserr =nfsd4_do_copy (copy, copy->nf_src->nf_file, copy->nf_dst->nf_file, false); -nfsd4_cleanup_intra_ssc (copy->nf_src, copy->nf_dst); }Quoted 15 lines
Quoted 13 lines
fs/nfsd/nfs4proc.c +1
-2do_callback:@@ -1811,9 +1808,9 @@ nfsd4_copy(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate, }else { status =nfsd4_do_copy (copy, copy->nf_src->nf_file, copy->nf_dst->nf_file, true); -nfsd4_cleanup_intra_ssc (copy->nf_src, copy->nf_dst); } out: +release_copy_files (copy);return status; out_err:if (async_copy) --Quoted 2 lines
2.39.0
--
Jeff Layton <[email protected]>--
Jeff Layton <[email protected]>
Re: [PATCH 2/2] nfsd: clean up potential nfsd_file refcount leaks in COPY codepath
Quoted 43 lines · Olga Kornievskaia
Quoted 32 lines · Jeff Layton
Quoted 30 lines · Olga Kornievskaia
Quoted 28 lines · Jeff Layton
Quoted 9 lines · Olga Kornievskaia
Quoted 8 lines · Jeff Layton
There are two different flavors of the
nfsd4_copystruct. One is embedded in the compound and is used directly in synchronous copies. The other is dynamically allocated, refcounted and tracked in the client struture. For the embedded one, the cleanup just involves releasing anynfsd_filesheld on its behalf. For the async one, the cleanup is a bit more involved, and we need to dequeue it from lists, unhash it, etc.Quoted 7 lines
Quoted 3 lines
There is at least one potential refcount leak in this code now. If the
kthread_createcall fails, then both the src and dstnfsd_filesin the originalnfsd4_copyobject are leaked.I don't believe that's true. If
kthread_createthread fails we callcleanup_async_copy()that does a put on the file descriptors.You mean this?
out_err:if (async_copy) cleanup_async_copy(async_copy);That puts the references that were taken in
dup_copy_fields, but the original (embedded)nfsd4_copyalso holds references and those are not being put in this codepath.Quoted 8 lines
Quoted 2 lines
Can you please point out where do we take a reference on the original copy?
In the case of an inter-server copy,
nf_dstis set innfsd4_setup_inter_ssc. For intraserver copy, both pointers are set via the call tonfsd4_verify_copy. Both functions callnfs4_preprocess_stateid_op, which returns a reference to thenfsd_filein the second to last arg.
Quoted 7 lines
Ah. Thank you. I didn't know that
nfs4_preprocess_stateid_op()takes a reference on it's 5th argument. I think I was previously looking atnfsd4_read()function which callsnfs4_preprocess_stateid_op()and gets backread->rd_nfbut it never does a put on it when it returns. However, I now looked at other functions that callnfs4_preproess_stateid_op()such asnfsd4_fallocate()and I see that it does a put.
So is there a refcount leak in the nfsd4_read() then since it doesn't do a put? Or the internals obscure and that even though it calls the same function and passes that parameter no refcount was increased. Is it based on the "WR_STATE, RD_STATE" parameter. I see that nfsd4_write() does do a put. For copy, we call the src_fd with RD_STATE and dst_fd with WR_STATE. If I were to follow the logic of nfsd4_read/nfsd4_write, the the copy doesn't need to do a put for src but will need it for the dst. The proposed patch does it for both.
So I'm still confused if this patch is the correct solution.
Quoted 142 lines
Quoted 142 lines
Quoted 14 lines
Quoted 14 lines
Quoted 13 lines
Quoted 4 lines
The cleanup in this codepath is also sort of weird. In the async copy case, we'll have up to four
nfsd_filereferences (src and dst for both flavors of copy structure).That's not true. There is a careful distinction between intra -- which had 2 valid file pointers and does a get on both as they both point to something that's opened on this server--- but inter -- only does a get on the dst file descriptor, the src doesn't exit. And yes I realize the code checks for
nfs_srcbeing null which it should be but it makes the code less clear and at some point somebody might want to decide to really do a put on it.Quoted 8 lines
Quoted 3 lines
This is part of the problem here. We have a
nfsd4_copystructure, and depending on what has been done to it, you need to call different methods to clean it up. That seems like a real antipattern to me.But they call different methods because different things need to be done there and it makes it clear what needs to be for what type of copy.
I sure as hell had a hard time dissecting how all of that was supposed to work. There is clear bug here, and I think this patch makes the result clearer and more robust in the face of changes.
There are actually 4 different cases here: sync vs. async, alongside intra vs. interserver copy. These are all overloaded onto a
nfsd4_copystructure, seemingly for no good reason.The cleanup, in particular seems quite fragile to me, and there is a dearth of defensive coding measures. If you subtly call the "wrong" cleanup function at the wrong point in time, then things may go awry.
I'll leave it up to Chuck to make the final determination, but I see this patch as an improvement.
Quoted 12 lines
Quoted 12 lines
Quoted 11 lines
Quoted 8 lines
They are both put at the end of
nfsd4_do_async_copy, even though the ones held on behalf of the embedded one outlive that structure.Change it so that we always clean up the
nfsd_filerefs held by the embedded copy structure beforenfsd4_copyreturns. Reworkcleanup_async_copyto handle both inter and intra copies. Eliminatenfsd4_cleanup_intra_sscsince it now becomes a no-op.I feel by combining the cleanup for both it obscures a very important destication that src filehandle doesn't exist for inter.
Quoted 88 lines
Quoted 3 lines
If the src filehandle doesn't exist, then the pointer to it will be NULL. I don't see what we gain by keeping these two distinct, other than avoiding a NULL pointer check.
My reason would be for code clarity because different things are supposed to happen for intra and inter. Difference of opinion it seems.
Quoted 80 lines
Quoted 62 lines
Quoted 62 lines
Signed-off-by Jeff Layton <[email protected]>--- fs/nfsd/nfs4proc.c | 23 ++++++++++------------- 1 file changed, 10 insertions(+), 13 deletions(-)fs/nfsd/nfs4proc.c +9
-12@@ -1512,7 +1512,6 @@ nfsd4_cleanup_inter_ssc(struct nfsd4_ssc_umount_item *nsui, struct file *filp,long timeout =msecs_to_jiffies (nfsd4_ssc_umount_timeout);nfs42_ssc_close (filp); -nfsd_file_put (dst);fput (filp);spin_lock (&nn->nfsd_ssc_lock);@@ -1562,13 +1561,6 @@ nfsd4_setup_intra_ssc(struct svc_rqst *rqstp, ©->nf_dst); } -static void -nfsd4_cleanup_intra_ssc (struct nfsd_file *src,struct nfsd_file *dst) -{ -nfsd_file_put (src); -nfsd_file_put (dst); -} -static void nfsd4_cb_offload_release (struct nfsd4_callback *cb) { struct nfsd4_cb_offload *cbo =@@ -1683,12 +1675,18 @@ static void dup_copy_fields(struct nfsd4_copy *src, struct nfsd4_copy *dst) dst->ss_nsui = src->ss_nsui; } +static void release_copy_files (struct nfsd4_copy *copy) +{ +if (copy->nf_src) +nfsd_file_put (copy->nf_src); +if (copy->nf_dst) +nfsd_file_put (copy->nf_dst); +} +static void cleanup_async_copy (struct nfsd4_copy *copy) {nfs4_free_copy_state (copy); -nfsd_file_put (copy->nf_dst); -if (!nfsd4_ssc_is_inter (copy)) -nfsd_file_put (copy->nf_src); +release_copy_files (copy);spin_lock (©->cp_clp->async_lock);list_del (©->copies);spin_unlock (©->cp_clp->async_lock);@@ -1748,7 +1746,6 @@ static int nfsd4_do_async_copy(void *data) }else { nfserr =nfsd4_do_copy (copy, copy->nf_src->nf_file, copy->nf_dst->nf_file, false); -nfsd4_cleanup_intra_ssc (copy->nf_src, copy->nf_dst); }Quoted 15 lines
Quoted 13 lines
fs/nfsd/nfs4proc.c +1
-2do_callback:@@ -1811,9 +1808,9 @@ nfsd4_copy(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate, }else { status =nfsd4_do_copy (copy, copy->nf_src->nf_file, copy->nf_dst->nf_file, true); -nfsd4_cleanup_intra_ssc (copy->nf_src, copy->nf_dst); } out: +release_copy_files (copy);return status; out_err:if (async_copy) --Quoted 2 lines
2.39.0
--
Jeff Layton <[email protected]>--
Jeff Layton <[email protected]>
Jeff Layton
maintainerRe: [PATCH 2/2] nfsd: clean up potential nfsd_file refcount leaks in COPY codepath
Quoted 52 lines · Olga Kornievskaia
Quoted 43 lines · Olga Kornievskaia
Quoted 32 lines · Jeff Layton
Quoted 30 lines · Olga Kornievskaia
Quoted 28 lines · Jeff Layton
Quoted 9 lines · Olga Kornievskaia
Quoted 8 lines · Jeff Layton
There are two different flavors of the
nfsd4_copystruct. One is embedded in the compound and is used directly in synchronous copies. The other is dynamically allocated, refcounted and tracked in the client struture. For the embedded one, the cleanup just involves releasing anynfsd_filesheld on its behalf. For the async one, the cleanup is a bit more involved, and we need to dequeue it from lists, unhash it, etc.Quoted 7 lines
Quoted 3 lines
There is at least one potential refcount leak in this code now. If the
kthread_createcall fails, then both the src and dstnfsd_filesin the originalnfsd4_copyobject are leaked.I don't believe that's true. If
kthread_createthread fails we callcleanup_async_copy()that does a put on the file descriptors.You mean this?
out_err:if (async_copy) cleanup_async_copy(async_copy);That puts the references that were taken in
dup_copy_fields, but the original (embedded)nfsd4_copyalso holds references and those are not being put in this codepath.Quoted 8 lines
Quoted 2 lines
Can you please point out where do we take a reference on the original copy?
In the case of an inter-server copy,
nf_dstis set innfsd4_setup_inter_ssc. For intraserver copy, both pointers are set via the call tonfsd4_verify_copy. Both functions callnfs4_preprocess_stateid_op, which returns a reference to thenfsd_filein the second to last arg.Quoted 7 lines
Ah. Thank you. I didn't know that
nfs4_preprocess_stateid_op()takes a reference on it's 5th argument. I think I was previously looking atnfsd4_read()function which callsnfs4_preprocess_stateid_op()and gets backread->rd_nfbut it never does a put on it when it returns. However, I now looked at other functions that callnfs4_preproess_stateid_op()such asnfsd4_fallocate()and I see that it does a put.
Quoted 5 lines
So is there a refcount leak in the
nfsd4_read()then since it doesn't do a put? Or the internals obscure and that even though it calls the same function and passes that parameter no refcount was increased. Is it based on the "WR_STATE,RD_STATE" parameter.
I don't think so. The put is done just below there, in nfsd4_read_release:
if (u->read.rd_nf)
nfsd_file_put(u->read.rd_nf);That said, I am hunting a refcount overput with nfsd_files that I've not been able to nail down yet (which is why I've been auditing the nfsd_file refcounting). If you see anything that looks hinky, please do point it out.
Quoted 6 lines
I see that
nfsd4_write()does do a put. For copy, we call thesrc_fdwithRD_STATEanddst_fdwithWR_STATE. If I were to follow the logic of nfsd4_read/nfsd4_write, the the copy doesn't need to do a put for src but will need it for the dst. The proposed patch does it for both.
That'd be wrong. READs have to hold a ref to the open file while the reply is being marshalled. A WRITE can release it once the data has been written to the file. Maybe that's worth a comment.
Note that I just sent a patch to the list to add a comment that (hopefully) makes it clear that nfs4_preprocess_stateid_op returns a reference in that field.
FWIW, it wouldn't be safe for it to do anything else. Returning a pointer to a refcounted object without taking a reference would be very problematic.
Quoted 2 lines
So I'm still confused if this patch is the correct solution.
Fair enough. I'm not sure I understand the pushback, as the result seems clearer to me. If you want to propose an alternate fix, I'm happy to take a look.
Quoted 142 lines
Quoted 142 lines
Quoted 142 lines
Quoted 14 lines
Quoted 14 lines
Quoted 13 lines
Quoted 4 lines
The cleanup in this codepath is also sort of weird. In the async copy case, we'll have up to four
nfsd_filereferences (src and dst for both flavors of copy structure).That's not true. There is a careful distinction between intra -- which had 2 valid file pointers and does a get on both as they both point to something that's opened on this server--- but inter -- only does a get on the dst file descriptor, the src doesn't exit. And yes I realize the code checks for
nfs_srcbeing null which it should be but it makes the code less clear and at some point somebody might want to decide to really do a put on it.Quoted 8 lines
Quoted 3 lines
This is part of the problem here. We have a
nfsd4_copystructure, and depending on what has been done to it, you need to call different methods to clean it up. That seems like a real antipattern to me.But they call different methods because different things need to be done there and it makes it clear what needs to be for what type of copy.
I sure as hell had a hard time dissecting how all of that was supposed to work. There is clear bug here, and I think this patch makes the result clearer and more robust in the face of changes.
There are actually 4 different cases here: sync vs. async, alongside intra vs. interserver copy. These are all overloaded onto a
nfsd4_copystructure, seemingly for no good reason.The cleanup, in particular seems quite fragile to me, and there is a dearth of defensive coding measures. If you subtly call the "wrong" cleanup function at the wrong point in time, then things may go awry.
I'll leave it up to Chuck to make the final determination, but I see this patch as an improvement.
Quoted 12 lines
Quoted 12 lines
Quoted 11 lines
Quoted 8 lines
They are both put at the end of
nfsd4_do_async_copy, even though the ones held on behalf of the embedded one outlive that structure.Change it so that we always clean up the
nfsd_filerefs held by the embedded copy structure beforenfsd4_copyreturns. Reworkcleanup_async_copyto handle both inter and intra copies. Eliminatenfsd4_cleanup_intra_sscsince it now becomes a no-op.I feel by combining the cleanup for both it obscures a very important destication that src filehandle doesn't exist for inter.
Quoted 88 lines
Quoted 3 lines
If the src filehandle doesn't exist, then the pointer to it will be NULL. I don't see what we gain by keeping these two distinct, other than avoiding a NULL pointer check.
My reason would be for code clarity because different things are supposed to happen for intra and inter. Difference of opinion it seems.
Quoted 80 lines
Quoted 62 lines
Quoted 62 lines
Signed-off-by Jeff Layton <[email protected]>--- fs/nfsd/nfs4proc.c | 23 ++++++++++------------- 1 file changed, 10 insertions(+), 13 deletions(-)fs/nfsd/nfs4proc.c +9
-12@@ -1512,7 +1512,6 @@ nfsd4_cleanup_inter_ssc(struct nfsd4_ssc_umount_item *nsui, struct file *filp,long timeout =msecs_to_jiffies (nfsd4_ssc_umount_timeout);nfs42_ssc_close (filp); -nfsd_file_put (dst);fput (filp);spin_lock (&nn->nfsd_ssc_lock);@@ -1562,13 +1561,6 @@ nfsd4_setup_intra_ssc(struct svc_rqst *rqstp, ©->nf_dst); } -static void -nfsd4_cleanup_intra_ssc (struct nfsd_file *src,struct nfsd_file *dst) -{ -nfsd_file_put (src); -nfsd_file_put (dst); -} -static void nfsd4_cb_offload_release (struct nfsd4_callback *cb) { struct nfsd4_cb_offload *cbo =@@ -1683,12 +1675,18 @@ static void dup_copy_fields(struct nfsd4_copy *src, struct nfsd4_copy *dst) dst->ss_nsui = src->ss_nsui; } +static void release_copy_files (struct nfsd4_copy *copy) +{ +if (copy->nf_src) +nfsd_file_put (copy->nf_src); +if (copy->nf_dst) +nfsd_file_put (copy->nf_dst); +} +static void cleanup_async_copy (struct nfsd4_copy *copy) {nfs4_free_copy_state (copy); -nfsd_file_put (copy->nf_dst); -if (!nfsd4_ssc_is_inter (copy)) -nfsd_file_put (copy->nf_src); +release_copy_files (copy);spin_lock (©->cp_clp->async_lock);list_del (©->copies);spin_unlock (©->cp_clp->async_lock);@@ -1748,7 +1746,6 @@ static int nfsd4_do_async_copy(void *data) }else { nfserr =nfsd4_do_copy (copy, copy->nf_src->nf_file, copy->nf_dst->nf_file, false); -nfsd4_cleanup_intra_ssc (copy->nf_src, copy->nf_dst); }Quoted 15 lines
Quoted 13 lines
fs/nfsd/nfs4proc.c +1
-2do_callback:@@ -1811,9 +1808,9 @@ nfsd4_copy(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate, }else { status =nfsd4_do_copy (copy, copy->nf_src->nf_file, copy->nf_dst->nf_file, true); -nfsd4_cleanup_intra_ssc (copy->nf_src, copy->nf_dst); } out: +release_copy_files (copy);return status; out_err:if (async_copy) --Quoted 2 lines
2.39.0
--
Jeff Layton <[email protected]>--
Jeff Layton <[email protected]>
Signature · 1 line
Jeff Layton <[email protected]>
Re: [PATCH 2/2] nfsd: clean up potential nfsd_file refcount leaks in COPY codepath
Quoted 63 lines · Jeff Layton
Quoted 52 lines · Olga Kornievskaia
Quoted 43 lines · Olga Kornievskaia
Quoted 32 lines · Jeff Layton
Quoted 30 lines · Olga Kornievskaia
Quoted 28 lines · Jeff Layton
Quoted 9 lines · Olga Kornievskaia
Quoted 8 lines · Jeff Layton
There are two different flavors of the
nfsd4_copystruct. One is embedded in the compound and is used directly in synchronous copies. The other is dynamically allocated, refcounted and tracked in the client struture. For the embedded one, the cleanup just involves releasing anynfsd_filesheld on its behalf. For the async one, the cleanup is a bit more involved, and we need to dequeue it from lists, unhash it, etc.Quoted 7 lines
Quoted 3 lines
There is at least one potential refcount leak in this code now. If the
kthread_createcall fails, then both the src and dstnfsd_filesin the originalnfsd4_copyobject are leaked.I don't believe that's true. If
kthread_createthread fails we callcleanup_async_copy()that does a put on the file descriptors.You mean this?
out_err:if (async_copy) cleanup_async_copy(async_copy);That puts the references that were taken in
dup_copy_fields, but the original (embedded)nfsd4_copyalso holds references and those are not being put in this codepath.Quoted 8 lines
Quoted 2 lines
Can you please point out where do we take a reference on the original copy?
In the case of an inter-server copy,
nf_dstis set innfsd4_setup_inter_ssc. For intraserver copy, both pointers are set via the call tonfsd4_verify_copy. Both functions callnfs4_preprocess_stateid_op, which returns a reference to thenfsd_filein the second to last arg.Quoted 7 lines
Ah. Thank you. I didn't know that
nfs4_preprocess_stateid_op()takes a reference on it's 5th argument. I think I was previously looking atnfsd4_read()function which callsnfs4_preprocess_stateid_op()and gets backread->rd_nfbut it never does a put on it when it returns. However, I now looked at other functions that callnfs4_preproess_stateid_op()such asnfsd4_fallocate()and I see that it does a put.Quoted 5 lines
So is there a refcount leak in the
nfsd4_read()then since it doesn't do a put? Or the internals obscure and that even though it calls the same function and passes that parameter no refcount was increased. Is it based on the "WR_STATE,RD_STATE" parameter.I don't think so. The put is done just below there, in
nfsd4_read_release:
Quoted 7 lines
if (u->read.rd_nf) nfsd_file_put(u->read.rd_nf);That said, I am hunting a refcount overput with
nfsd_filesthat I've not been able to nail down yet (which is why I've been auditing thenfsd_filerefcounting). If you see anything that looks hinky, please do point it out.
Thank you for the explanation about the read. I didn't know about the op_release.
Quoted 19 lines
Quoted 6 lines
I see that
nfsd4_write()does do a put. For copy, we call thesrc_fdwithRD_STATEanddst_fdwithWR_STATE. If I were to follow the logic of nfsd4_read/nfsd4_write, the the copy doesn't need to do a put for src but will need it for the dst. The proposed patch does it for both.That'd be wrong. READs have to hold a ref to the open file while the reply is being marshalled. A WRITE can release it once the data has been written to the file. Maybe that's worth a comment.
Note that I just sent a patch to the list to add a comment that (hopefully) makes it clear that
nfs4_preprocess_stateid_opreturns a reference in that field.FWIW, it wouldn't be safe for it to do anything else. Returning a pointer to a refcounted object without taking a reference would be very problematic.
Quoted 6 lines
Quoted 2 lines
So I'm still confused if this patch is the correct solution.
Fair enough. I'm not sure I understand the pushback, as the result seems clearer to me. If you want to propose an alternate fix, I'm happy to take a look.
After your explanation about the read, things make sense to me so, no pushback.
Quoted 145 lines
Quoted 140 lines
Quoted 140 lines
Quoted 140 lines
Quoted 14 lines
Quoted 14 lines
Quoted 13 lines
Quoted 4 lines
The cleanup in this codepath is also sort of weird. In the async copy case, we'll have up to four
nfsd_filereferences (src and dst for both flavors of copy structure).That's not true. There is a careful distinction between intra -- which had 2 valid file pointers and does a get on both as they both point to something that's opened on this server--- but inter -- only does a get on the dst file descriptor, the src doesn't exit. And yes I realize the code checks for
nfs_srcbeing null which it should be but it makes the code less clear and at some point somebody might want to decide to really do a put on it.Quoted 8 lines
Quoted 3 lines
This is part of the problem here. We have a
nfsd4_copystructure, and depending on what has been done to it, you need to call different methods to clean it up. That seems like a real antipattern to me.But they call different methods because different things need to be done there and it makes it clear what needs to be for what type of copy.
I sure as hell had a hard time dissecting how all of that was supposed to work. There is clear bug here, and I think this patch makes the result clearer and more robust in the face of changes.
There are actually 4 different cases here: sync vs. async, alongside intra vs. interserver copy. These are all overloaded onto a
nfsd4_copystructure, seemingly for no good reason.The cleanup, in particular seems quite fragile to me, and there is a dearth of defensive coding measures. If you subtly call the "wrong" cleanup function at the wrong point in time, then things may go awry.
I'll leave it up to Chuck to make the final determination, but I see this patch as an improvement.
Quoted 100 lines
Quoted 9 lines
Quoted 9 lines
Quoted 8 lines
They are both put at the end of
nfsd4_do_async_copy, even though the ones held on behalf of the embedded one outlive that structure.Change it so that we always clean up the
nfsd_filerefs held by the embedded copy structure beforenfsd4_copyreturns. Reworkcleanup_async_copyto handle both inter and intra copies. Eliminatenfsd4_cleanup_intra_sscsince it now becomes a no-op.Quoted 6 lines
Quoted 2 lines
I feel by combining the cleanup for both it obscures a very important destication that src filehandle doesn't exist for inter.
If the src filehandle doesn't exist, then the pointer to it will be NULL. I don't see what we gain by keeping these two distinct, other than avoiding a NULL pointer check.
My reason would be for code clarity because different things are supposed to happen for intra and inter. Difference of opinion it seems.
Quoted 80 lines
Quoted 62 lines
Quoted 62 lines
Signed-off-by Jeff Layton <[email protected]>--- fs/nfsd/nfs4proc.c | 23 ++++++++++------------- 1 file changed, 10 insertions(+), 13 deletions(-)fs/nfsd/nfs4proc.c +9
-12@@ -1512,7 +1512,6 @@ nfsd4_cleanup_inter_ssc(struct nfsd4_ssc_umount_item *nsui, struct file *filp,long timeout =msecs_to_jiffies (nfsd4_ssc_umount_timeout);nfs42_ssc_close (filp); -nfsd_file_put (dst);fput (filp);spin_lock (&nn->nfsd_ssc_lock);@@ -1562,13 +1561,6 @@ nfsd4_setup_intra_ssc(struct svc_rqst *rqstp, ©->nf_dst); } -static void -nfsd4_cleanup_intra_ssc (struct nfsd_file *src,struct nfsd_file *dst) -{ -nfsd_file_put (src); -nfsd_file_put (dst); -} -static void nfsd4_cb_offload_release (struct nfsd4_callback *cb) { struct nfsd4_cb_offload *cbo =@@ -1683,12 +1675,18 @@ static void dup_copy_fields(struct nfsd4_copy *src, struct nfsd4_copy *dst) dst->ss_nsui = src->ss_nsui; } +static void release_copy_files (struct nfsd4_copy *copy) +{ +if (copy->nf_src) +nfsd_file_put (copy->nf_src); +if (copy->nf_dst) +nfsd_file_put (copy->nf_dst); +} +static void cleanup_async_copy (struct nfsd4_copy *copy) {nfs4_free_copy_state (copy); -nfsd_file_put (copy->nf_dst); -if (!nfsd4_ssc_is_inter (copy)) -nfsd_file_put (copy->nf_src); +release_copy_files (copy);spin_lock (©->cp_clp->async_lock);list_del (©->copies);spin_unlock (©->cp_clp->async_lock);@@ -1748,7 +1746,6 @@ static int nfsd4_do_async_copy(void *data) }else { nfserr =nfsd4_do_copy (copy, copy->nf_src->nf_file, copy->nf_dst->nf_file, false); -nfsd4_cleanup_intra_ssc (copy->nf_src, copy->nf_dst); }Quoted 15 lines
Quoted 13 lines
fs/nfsd/nfs4proc.c +1
-2do_callback:@@ -1811,9 +1808,9 @@ nfsd4_copy(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate, }else { status =nfsd4_do_copy (copy, copy->nf_src->nf_file, copy->nf_dst->nf_file, true); -nfsd4_cleanup_intra_ssc (copy->nf_src, copy->nf_dst); } out: +release_copy_files (copy);return status; out_err:if (async_copy) --Quoted 2 lines
2.39.0
--
Jeff Layton <[email protected]>Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
--
Jeff Layton <[email protected]>--
Jeff Layton <[email protected]>
[email protected]
reviewerRe: [PATCH 2/2] nfsd: clean up potential nfsd_file refcount leaks in COPY codepath
In reply to Jeff LaytonQuoted 32 lines · Jeff Layton
There are two different flavors of the
nfsd4_copystruct. One is embedded in the compound and is used directly in synchronous copies. The other is dynamically allocated, refcounted and tracked in the client struture. For the embedded one, the cleanup just involves releasing anynfsd_filesheld on its behalf. For the async one, the cleanup is a bit more involved, and we need to dequeue it from lists, unhash it, etc.There is at least one potential refcount leak in this code now. If the
kthread_createcall fails, then both the src and dstnfsd_filesin the originalnfsd4_copyobject are leaked.The cleanup in this codepath is also sort of weird. In the async copy case, we'll have up to four
nfsd_filereferences (src and dst for both flavors of copy structure). They are both put at the end ofnfsd4_do_async_copy, even though the ones held on behalf of the embedded one outlive that structure.Change it so that we always clean up the
nfsd_filerefs held by the embedded copy structure beforenfsd4_copyreturns. Reworkcleanup_async_copyto handle both inter and intra copies. Eliminatenfsd4_cleanup_intra_sscsince it now becomes a no-op.Signed-off-by Jeff Layton <[email protected]>--- fs/nfsd/nfs4proc.c | 23 ++++++++++------------- 1 file changed, 10 insertions(+), 13 deletions(-)fs/nfsd/nfs4proc.c +0
-0
fs/nfsd/nfs4proc.c nfsd4_cleanup_inter_ssc(struct nfsd4_ssc_umount_item *nsui, struct file *filp, · line 1515
Quoted 4 lines
fs/nfsd/nfs4proc.c +0
-1@@ -1512,7 +1512,6 @@ nfsd4_cleanup_inter_ssc(struct nfsd4_ssc_umount_item *nsui, struct file *filp,long timeout =msecs_to_jiffies (nfsd4_ssc_umount_timeout);nfs42_ssc_close (filp); -nfsd_file_put (dst);
I think we still need this, in addition to release_copy_files called from cleanup_async_copy. For async inter-copy, there are 2 reference count added to the destination file, one from nfsd4_setup_inter_ssc and the other one from dup_copy_fields. The above nfsd_file_put is for the count added by dup_copy_fields.
Quoted 48 lines
fs/nfsd/nfs4proc.c +9
-11fput (filp);spin_lock (&nn->nfsd_ssc_lock);@@ -1562,13 +1561,6 @@ nfsd4_setup_intra_ssc(struct svc_rqst *rqstp, ©->nf_dst); } -static void -nfsd4_cleanup_intra_ssc (struct nfsd_file *src,struct nfsd_file *dst) -{ -nfsd_file_put (src); -nfsd_file_put (dst); -} -static void nfsd4_cb_offload_release (struct nfsd4_callback *cb) { struct nfsd4_cb_offload *cbo =@@ -1683,12 +1675,18 @@ static void dup_copy_fields(struct nfsd4_copy *src, struct nfsd4_copy *dst) dst->ss_nsui = src->ss_nsui; } +static void release_copy_files (struct nfsd4_copy *copy) +{ +if (copy->nf_src) +nfsd_file_put (copy->nf_src); +if (copy->nf_dst) +nfsd_file_put (copy->nf_dst); +} +static void cleanup_async_copy (struct nfsd4_copy *copy) {nfs4_free_copy_state (copy); -nfsd_file_put (copy->nf_dst); -if (!nfsd4_ssc_is_inter (copy)) -nfsd_file_put (copy->nf_src); +release_copy_files (copy);spin_lock (©->cp_clp->async_lock);list_del (©->copies);spin_unlock (©->cp_clp->async_lock);@@ -1748,7 +1746,6 @@ static int nfsd4_do_async_copy(void *data) }else { nfserr =nfsd4_do_copy (copy, copy->nf_src->nf_file, copy->nf_dst->nf_file, false); -nfsd4_cleanup_intra_ssc (copy->nf_src, copy->nf_dst); } do_callback:
fs/nfsd/nfs4proc.c nfsd4_copy(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate, · line 1815
Quoted 9 lines
fs/nfsd/nfs4proc.c +1
-1@@ -1811,9 +1808,9 @@ nfsd4_copy(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate, }else { status =nfsd4_do_copy (copy, copy->nf_src->nf_file, copy->nf_dst->nf_file, true); -nfsd4_cleanup_intra_ssc (copy->nf_src, copy->nf_dst); } out: +release_copy_files (copy);return status; out_err:
This is unrelated to the reference count issue.
Here if this is an inter-copy then we need to decrement the reference count of the nfsd4_ssc_umount_item so that the vfsmount can be unmounted later.
-Dai
Quoted 1 line
fs/nfsd/nfs4proc.c +0
-0if (async_copy)
Jeff Layton
maintainerRe: [PATCH 2/2] nfsd: clean up potential nfsd_file refcount leaks in COPY codepath
Quoted 35 lines · On Wed, 2023-01-18 at 21:05 -0800, [email protected] wrote:
Quoted 34 lines · Jeff Layton
There are two different flavors of the
nfsd4_copystruct. One is embedded in the compound and is used directly in synchronous copies. The other is dynamically allocated, refcounted and tracked in the client struture. For the embedded one, the cleanup just involves releasing anynfsd_filesheld on its behalf. For the async one, the cleanup is a bit more involved, and we need to dequeue it from lists, unhash it, etc.There is at least one potential refcount leak in this code now. If the
kthread_createcall fails, then both the src and dstnfsd_filesin the originalnfsd4_copyobject are leaked.The cleanup in this codepath is also sort of weird. In the async copy case, we'll have up to four
nfsd_filereferences (src and dst for both flavors of copy structure). They are both put at the end ofnfsd4_do_async_copy, even though the ones held on behalf of the embedded one outlive that structure.Change it so that we always clean up the
nfsd_filerefs held by the embedded copy structure beforenfsd4_copyreturns. Reworkcleanup_async_copyto handle both inter and intra copies. Eliminatenfsd4_cleanup_intra_sscsince it now becomes a no-op.Signed-off-by Jeff Layton <[email protected]>--- fs/nfsd/nfs4proc.c | 23 ++++++++++------------- 1 file changed, 10 insertions(+), 13 deletions(-)fs/nfsd/nfs4proc.c +0
-0@@ -1512,7 +1512,6 @@ nfsd4_cleanup_inter_ssc(struct nfsd4_ssc_umount_item *nsui, struct file *filp,long timeout =msecs_to_jiffies (nfsd4_ssc_umount_timeout);
Quoted 9 lines
fs/nfsd/nfs4proc.c nfsd4_cleanup_inter_ssc(struct nfsd4_ssc_umount_item *nsui, struct file *filp, · line 1515
Quoted 2 lines
fs/nfsd/nfs4proc.c +0
-1nfs42_ssc_close (filp); -nfsd_file_put (dst);I think we still need this, in addition to
release_copy_filescalled fromcleanup_async_copy. For async inter-copy, there are 2 reference count added to the destination file, one fromnfsd4_setup_inter_sscand the other one fromdup_copy_fields. The abovenfsd_file_putis for the count added bydup_copy_fields.
With this patch, the references held by the original copy structure are put by the call to release_copy_files at the end of nfsd4_copy. That means that the kthread task is only responsible for putting the references held by the (kmalloc'ed) async_copy structure. So, I think this gets the nfsd_file refcounting right.
Quoted 60 lines
Quoted 48 lines
fs/nfsd/nfs4proc.c +9
-11fput (filp);spin_lock (&nn->nfsd_ssc_lock);@@ -1562,13 +1561,6 @@ nfsd4_setup_intra_ssc(struct svc_rqst *rqstp, ©->nf_dst); } -static void -nfsd4_cleanup_intra_ssc (struct nfsd_file *src,struct nfsd_file *dst) -{ -nfsd_file_put (src); -nfsd_file_put (dst); -} -static void nfsd4_cb_offload_release (struct nfsd4_callback *cb) { struct nfsd4_cb_offload *cbo =@@ -1683,12 +1675,18 @@ static void dup_copy_fields(struct nfsd4_copy *src, struct nfsd4_copy *dst) dst->ss_nsui = src->ss_nsui; } +static void release_copy_files (struct nfsd4_copy *copy) +{ +if (copy->nf_src) +nfsd_file_put (copy->nf_src); +if (copy->nf_dst) +nfsd_file_put (copy->nf_dst); +} +static void cleanup_async_copy (struct nfsd4_copy *copy) {nfs4_free_copy_state (copy); -nfsd_file_put (copy->nf_dst); -if (!nfsd4_ssc_is_inter (copy)) -nfsd_file_put (copy->nf_src); +release_copy_files (copy);spin_lock (©->cp_clp->async_lock);list_del (©->copies);spin_unlock (©->cp_clp->async_lock);@@ -1748,7 +1746,6 @@ static int nfsd4_do_async_copy(void *data) }else { nfserr =nfsd4_do_copy (copy, copy->nf_src->nf_file, copy->nf_dst->nf_file, false); -nfsd4_cleanup_intra_ssc (copy->nf_src, copy->nf_dst); } do_callback:fs/nfsd/nfs4proc.c nfsd4_copy(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate, · line 1815
Quoted 9 lines
fs/nfsd/nfs4proc.c +1
-1@@ -1811,9 +1808,9 @@ nfsd4_copy(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate, }else { status =nfsd4_do_copy (copy, copy->nf_src->nf_file, copy->nf_dst->nf_file, true); -nfsd4_cleanup_intra_ssc (copy->nf_src, copy->nf_dst); } out: +release_copy_files (copy);return status; out_err:This is unrelated to the reference count issue.
Quoted 4 lines
Here if this is an inter-copy then we need to decrement the reference count of the
nfsd4_ssc_umount_itemso that the vfsmount can be unmounted later.
Oh, I think I see what you mean. Maybe something like the (untested) patch below on top of the original patch would fix that?
fs/nfsd/nfs4proc.c +10 -3
@@ -1511,8 +1511,10 @@ nfsd4_cleanup_inter_ssc(struct nfsd4_ssc_umount_item *nsui, struct file *filp,struct nfsd_net *nn =net_generic (dst->nf_net, nfsd_net_id);long timeout =msecs_to_jiffies (nfsd4_ssc_umount_timeout); -nfs42_ssc_close (filp); -fput (filp); +if (filp) { +nfs42_ssc_close (filp); +fput (filp); + } spin_lock(&nn->nfsd_ssc_lo list_del(&nsui->nsui_list);
@@ -1813,8 +1815,13 @@ nfsd4_copy(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,release_copy_files (copy);return status; out_err: -if (async_copy) +if (async_copy) {cleanup_async_copy (async_copy); +if (nfsd4_ssc_is_inter (async_copy)) +nfsd4_cleanup_inter_ssc (copy->ss_nsui,NULL , + copy->nf_dst); + + } status =nfserrno (-ENOMEM );/* * source's vfsmount of inter-copy will be unmounted
Signature · 1 line
Jeff Layton <[email protected]>
[email protected]
reviewerRe: [PATCH 2/2] nfsd: clean up potential nfsd_file refcount leaks in COPY codepath
Quoted 44 lines · Jeff Layton
Quoted 35 lines · On Wed, 2023-01-18 at 21:05 -0800, [email protected] wrote:
Quoted 34 lines · Jeff Layton
There are two different flavors of the
nfsd4_copystruct. One is embedded in the compound and is used directly in synchronous copies. The other is dynamically allocated, refcounted and tracked in the client struture. For the embedded one, the cleanup just involves releasing anynfsd_filesheld on its behalf. For the async one, the cleanup is a bit more involved, and we need to dequeue it from lists, unhash it, etc.There is at least one potential refcount leak in this code now. If the
kthread_createcall fails, then both the src and dstnfsd_filesin the originalnfsd4_copyobject are leaked.The cleanup in this codepath is also sort of weird. In the async copy case, we'll have up to four
nfsd_filereferences (src and dst for both flavors of copy structure). They are both put at the end ofnfsd4_do_async_copy, even though the ones held on behalf of the embedded one outlive that structure.Change it so that we always clean up the
nfsd_filerefs held by the embedded copy structure beforenfsd4_copyreturns. Reworkcleanup_async_copyto handle both inter and intra copies. Eliminatenfsd4_cleanup_intra_sscsince it now becomes a no-op.Signed-off-by Jeff Layton <[email protected]>--- fs/nfsd/nfs4proc.c | 23 ++++++++++------------- 1 file changed, 10 insertions(+), 13 deletions(-)fs/nfsd/nfs4proc.c +0
-0@@ -1512,7 +1512,6 @@ nfsd4_cleanup_inter_ssc(struct nfsd4_ssc_umount_item *nsui, struct file *filp,long timeout =msecs_to_jiffies (nfsd4_ssc_umount_timeout);Quoted 8 lines
fs/nfsd/nfs4proc.c nfsd4_cleanup_inter_ssc(struct nfsd4_ssc_umount_item *nsui, struct file *filp, · line 1515
Quoted 2 lines
fs/nfsd/nfs4proc.c +0
-1nfs42_ssc_close (filp); -nfsd_file_put (dst);I think we still need this, in addition to
release_copy_filescalled fromcleanup_async_copy. For async inter-copy, there are 2 reference count added to the destination file, one fromnfsd4_setup_inter_sscand the other one fromdup_copy_fields. The abovenfsd_file_putis for the count added bydup_copy_fields.
Quoted 5 lines
With this patch, the references held by the original copy structure are put by the call to
release_copy_filesat the end ofnfsd4_copy. That means that the kthread task is only responsible for putting the references held by the (kmalloc'ed)async_copystructure. So, I think this gets thenfsd_filerefcounting right.
Yes, I see. One refcount is decremented by release_copy_files at end of nfsd4_copy and another is decremented by release_copy_files in cleanup_async_copy.
Quoted 87 lines
Quoted 61 lines
Quoted 50 lines
fs/nfsd/nfs4proc.c +9
-11fput (filp);spin_lock (&nn->nfsd_ssc_lock);@@ -1562,13 +1561,6 @@ nfsd4_setup_intra_ssc(struct svc_rqst *rqstp, ©->nf_dst); } -static void -nfsd4_cleanup_intra_ssc (struct nfsd_file *src,struct nfsd_file *dst) -{ -nfsd_file_put (src); -nfsd_file_put (dst); -} -static void nfsd4_cb_offload_release (struct nfsd4_callback *cb) { struct nfsd4_cb_offload *cbo =@@ -1683,12 +1675,18 @@ static void dup_copy_fields(struct nfsd4_copy *src, struct nfsd4_copy *dst) dst->ss_nsui = src->ss_nsui; } +static void release_copy_files (struct nfsd4_copy *copy) +{ +if (copy->nf_src) +nfsd_file_put (copy->nf_src); +if (copy->nf_dst) +nfsd_file_put (copy->nf_dst); +} +static void cleanup_async_copy (struct nfsd4_copy *copy) {nfs4_free_copy_state (copy); -nfsd_file_put (copy->nf_dst); -if (!nfsd4_ssc_is_inter (copy)) -nfsd_file_put (copy->nf_src); +release_copy_files (copy);spin_lock (©->cp_clp->async_lock);list_del (©->copies);spin_unlock (©->cp_clp->async_lock);@@ -1748,7 +1746,6 @@ static int nfsd4_do_async_copy(void *data) }else { nfserr =nfsd4_do_copy (copy, copy->nf_src->nf_file, copy->nf_dst->nf_file, false); -nfsd4_cleanup_intra_ssc (copy->nf_src, copy->nf_dst); } do_callback:fs/nfsd/nfs4proc.c nfsd4_copy(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate, · line 1815
Quoted 9 lines
fs/nfsd/nfs4proc.c +1
-1@@ -1811,9 +1808,9 @@ nfsd4_copy(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate, }else { status =nfsd4_do_copy (copy, copy->nf_src->nf_file, copy->nf_dst->nf_file, true); -nfsd4_cleanup_intra_ssc (copy->nf_src, copy->nf_dst); } out: +release_copy_files (copy);return status; out_err:This is unrelated to the reference count issue.
Quoted 4 lines
Here if this is an inter-copy then we need to decrement the reference count of the
nfsd4_ssc_umount_itemso that the vfsmount can be unmounted later.Oh, I think I see what you mean. Maybe something like the (untested) patch below on top of the original patch would fix that?
fs/nfsd/nfs4proc.c +4
-2@@ -1511,8 +1511,10 @@ nfsd4_cleanup_inter_ssc(struct nfsd4_ssc_umount_item *nsui, struct file *filp,struct nfsd_net *nn =net_generic (dst->nf_net, nfsd_net_id);long timeout =msecs_to_jiffies (nfsd4_ssc_umount_timeout); -nfs42_ssc_close (filp); -fput (filp); +if (filp) { +nfs42_ssc_close (filp); +fput (filp); + } spin_lock(&nn->nfsd_ssc_lo list_del(&nsui->nsui_list);
fs/nfsd/nfs4proc.c nfsd4_copy(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate, · line 1820
Quoted 7 lines
fs/nfsd/nfs4proc.c +2
-1@@ -1813,8 +1815,13 @@ nfsd4_copy(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,release_copy_files (copy);return status; out_err: -if (async_copy) +if (async_copy) {cleanup_async_copy (async_copy); +if (nfsd4_ssc_is_inter (async_copy))
We don't need to call nfsd4_cleanup_inter_ssc since the thread nfsd4_do_async_copy has not started yet so the file is not opened. We just need to do refcount_dec(©->ss_nsui->nsui_refcnt), unless you want to change nfsd4_cleanup_inter_ssc to detect this error condition and only decrement the reference count.
-Dai
Quoted 8 lines
fs/nfsd/nfs4proc.c +4
-0+nfsd4_cleanup_inter_ssc (copy->ss_nsui,NULL , + copy->nf_dst); + + } status =nfserrno (-ENOMEM );/* * source's vfsmount of inter-copy will be unmounted
Jeff Layton
maintainerRe: [PATCH 2/2] nfsd: clean up potential nfsd_file refcount leaks in COPY codepath
Quoted 139 lines · On Thu, 2023-01-19 at 10:38 -0800, [email protected] wrote:
Quoted 44 lines · Jeff Layton
Quoted 35 lines · On Wed, 2023-01-18 at 21:05 -0800, [email protected] wrote:
Quoted 34 lines · Jeff Layton
There are two different flavors of the
nfsd4_copystruct. One is embedded in the compound and is used directly in synchronous copies. The other is dynamically allocated, refcounted and tracked in the client struture. For the embedded one, the cleanup just involves releasing anynfsd_filesheld on its behalf. For the async one, the cleanup is a bit more involved, and we need to dequeue it from lists, unhash it, etc.There is at least one potential refcount leak in this code now. If the
kthread_createcall fails, then both the src and dstnfsd_filesin the originalnfsd4_copyobject are leaked.The cleanup in this codepath is also sort of weird. In the async copy case, we'll have up to four
nfsd_filereferences (src and dst for both flavors of copy structure). They are both put at the end ofnfsd4_do_async_copy, even though the ones held on behalf of the embedded one outlive that structure.Change it so that we always clean up the
nfsd_filerefs held by the embedded copy structure beforenfsd4_copyreturns. Reworkcleanup_async_copyto handle both inter and intra copies. Eliminatenfsd4_cleanup_intra_sscsince it now becomes a no-op.Signed-off-by Jeff Layton <[email protected]>--- fs/nfsd/nfs4proc.c | 23 ++++++++++------------- 1 file changed, 10 insertions(+), 13 deletions(-)fs/nfsd/nfs4proc.c +0
-0@@ -1512,7 +1512,6 @@ nfsd4_cleanup_inter_ssc(struct nfsd4_ssc_umount_item *nsui, struct file *filp,long timeout =msecs_to_jiffies (nfsd4_ssc_umount_timeout);Quoted 8 lines
fs/nfsd/nfs4proc.c nfsd4_cleanup_inter_ssc(struct nfsd4_ssc_umount_item *nsui, struct file *filp, · line 1515
Quoted 2 lines
fs/nfsd/nfs4proc.c +0
-1nfs42_ssc_close (filp); -nfsd_file_put (dst);I think we still need this, in addition to
release_copy_filescalled fromcleanup_async_copy. For async inter-copy, there are 2 reference count added to the destination file, one fromnfsd4_setup_inter_sscand the other one fromdup_copy_fields. The abovenfsd_file_putis for the count added bydup_copy_fields.Quoted 5 lines
With this patch, the references held by the original copy structure are put by the call to
release_copy_filesat the end ofnfsd4_copy. That means that the kthread task is only responsible for putting the references held by the (kmalloc'ed)async_copystructure. So, I think this gets thenfsd_filerefcounting right.Yes, I see. One refcount is decremented by
release_copy_filesat end ofnfsd4_copyand another is decremented byrelease_copy_filesincleanup_async_copy.Quoted 84 lines
Quoted 61 lines
Quoted 50 lines
fs/nfsd/nfs4proc.c +9
-11fput (filp);spin_lock (&nn->nfsd_ssc_lock);@@ -1562,13 +1561,6 @@ nfsd4_setup_intra_ssc(struct svc_rqst *rqstp, ©->nf_dst); } -static void -nfsd4_cleanup_intra_ssc (struct nfsd_file *src,struct nfsd_file *dst) -{ -nfsd_file_put (src); -nfsd_file_put (dst); -} -static void nfsd4_cb_offload_release (struct nfsd4_callback *cb) { struct nfsd4_cb_offload *cbo =@@ -1683,12 +1675,18 @@ static void dup_copy_fields(struct nfsd4_copy *src, struct nfsd4_copy *dst) dst->ss_nsui = src->ss_nsui; } +static void release_copy_files (struct nfsd4_copy *copy) +{ +if (copy->nf_src) +nfsd_file_put (copy->nf_src); +if (copy->nf_dst) +nfsd_file_put (copy->nf_dst); +} +static void cleanup_async_copy (struct nfsd4_copy *copy) {nfs4_free_copy_state (copy); -nfsd_file_put (copy->nf_dst); -if (!nfsd4_ssc_is_inter (copy)) -nfsd_file_put (copy->nf_src); +release_copy_files (copy);spin_lock (©->cp_clp->async_lock);list_del (©->copies);spin_unlock (©->cp_clp->async_lock);@@ -1748,7 +1746,6 @@ static int nfsd4_do_async_copy(void *data) }else { nfserr =nfsd4_do_copy (copy, copy->nf_src->nf_file, copy->nf_dst->nf_file, false); -nfsd4_cleanup_intra_ssc (copy->nf_src, copy->nf_dst); } do_callback:fs/nfsd/nfs4proc.c nfsd4_copy(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate, · line 1815
Quoted 9 lines
fs/nfsd/nfs4proc.c +1
-1@@ -1811,9 +1808,9 @@ nfsd4_copy(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate, }else { status =nfsd4_do_copy (copy, copy->nf_src->nf_file, copy->nf_dst->nf_file, true); -nfsd4_cleanup_intra_ssc (copy->nf_src, copy->nf_dst); } out: +release_copy_files (copy);return status; out_err:This is unrelated to the reference count issue.
Quoted 4 lines
Here if this is an inter-copy then we need to decrement the reference count of the
nfsd4_ssc_umount_itemso that the vfsmount can be unmounted later.Oh, I think I see what you mean. Maybe something like the (untested) patch below on top of the original patch would fix that?
fs/nfsd/nfs4proc.c +4
-2@@ -1511,8 +1511,10 @@ nfsd4_cleanup_inter_ssc(struct nfsd4_ssc_umount_item *nsui, struct file *filp,struct nfsd_net *nn =net_generic (dst->nf_net, nfsd_net_id);long timeout =msecs_to_jiffies (nfsd4_ssc_umount_timeout); -nfs42_ssc_close (filp); -fput (filp); +if (filp) { +nfs42_ssc_close (filp); +fput (filp); + }
Quoted 17 lines
fs/nfsd/nfs4proc.c nfsd4_copy(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate, · line 1820
Quoted 10 lines
fs/nfsd/nfs4proc.c +2
-1spin_lock(&nn->nfsd_ssc_lo list_del(&nsui->nsui_list);@@ -1813,8 +1815,13 @@ nfsd4_copy(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,release_copy_files (copy);return status; out_err: -if (async_copy) +if (async_copy) {cleanup_async_copy (async_copy); +if (nfsd4_ssc_is_inter (async_copy))We don't need to call
nfsd4_cleanup_inter_sscsince the threadnfsd4_do_async_copyhas not started yet so the file is not opened. We just need to dorefcount_dec(©->ss_nsui->nsui_refcnt), unless you want to changenfsd4_cleanup_inter_sscto detect this error condition and only decrement the reference count.
Oh yeah, and this would break anyway since the nsui_list head is not being initialized. Dai, would you mind spinning up a patch for this since you're more familiar with the cleanup here?
Signature · 1 line
Jeff Layton <[email protected]>
[email protected]
reviewerRe: [PATCH 2/2] nfsd: clean up potential nfsd_file refcount leaks in COPY codepath
Quoted 153 lines · Jeff Layton
Quoted 136 lines · On Thu, 2023-01-19 at 10:38 -0800, [email protected] wrote:
Quoted 44 lines · Jeff Layton
Quoted 35 lines · On Wed, 2023-01-18 at 21:05 -0800, [email protected] wrote:
Quoted 34 lines · Jeff Layton
There are two different flavors of the
nfsd4_copystruct. One is embedded in the compound and is used directly in synchronous copies. The other is dynamically allocated, refcounted and tracked in the client struture. For the embedded one, the cleanup just involves releasing anynfsd_filesheld on its behalf. For the async one, the cleanup is a bit more involved, and we need to dequeue it from lists, unhash it, etc.There is at least one potential refcount leak in this code now. If the
kthread_createcall fails, then both the src and dstnfsd_filesin the originalnfsd4_copyobject are leaked.The cleanup in this codepath is also sort of weird. In the async copy case, we'll have up to four
nfsd_filereferences (src and dst for both flavors of copy structure). They are both put at the end ofnfsd4_do_async_copy, even though the ones held on behalf of the embedded one outlive that structure.Change it so that we always clean up the
nfsd_filerefs held by the embedded copy structure beforenfsd4_copyreturns. Reworkcleanup_async_copyto handle both inter and intra copies. Eliminatenfsd4_cleanup_intra_sscsince it now becomes a no-op.Signed-off-by Jeff Layton <[email protected]>--- fs/nfsd/nfs4proc.c | 23 ++++++++++------------- 1 file changed, 10 insertions(+), 13 deletions(-)fs/nfsd/nfs4proc.c +0
-0@@ -1512,7 +1512,6 @@ nfsd4_cleanup_inter_ssc(struct nfsd4_ssc_umount_item *nsui, struct file *filp,long timeout =msecs_to_jiffies (nfsd4_ssc_umount_timeout);Quoted 8 lines
fs/nfsd/nfs4proc.c nfsd4_cleanup_inter_ssc(struct nfsd4_ssc_umount_item *nsui, struct file *filp, · line 1515
Quoted 2 lines
fs/nfsd/nfs4proc.c +0
-1nfs42_ssc_close (filp); -nfsd_file_put (dst);I think we still need this, in addition to
release_copy_filescalled fromcleanup_async_copy. For async inter-copy, there are 2 reference count added to the destination file, one fromnfsd4_setup_inter_sscand the other one fromdup_copy_fields. The abovenfsd_file_putis for the count added bydup_copy_fields.Quoted 5 lines
With this patch, the references held by the original copy structure are put by the call to
release_copy_filesat the end ofnfsd4_copy. That means that the kthread task is only responsible for putting the references held by the (kmalloc'ed)async_copystructure. So, I think this gets thenfsd_filerefcounting right.Yes, I see. One refcount is decremented by
release_copy_filesat end ofnfsd4_copyand another is decremented byrelease_copy_filesincleanup_async_copy.Quoted 82 lines
Quoted 60 lines
Quoted 49 lines
fs/nfsd/nfs4proc.c +9
-11fput (filp);spin_lock (&nn->nfsd_ssc_lock);@@ -1562,13 +1561,6 @@ nfsd4_setup_intra_ssc(struct svc_rqst *rqstp, ©->nf_dst); } -static void -nfsd4_cleanup_intra_ssc (struct nfsd_file *src,struct nfsd_file *dst) -{ -nfsd_file_put (src); -nfsd_file_put (dst); -} -static void nfsd4_cb_offload_release (struct nfsd4_callback *cb) { struct nfsd4_cb_offload *cbo =@@ -1683,12 +1675,18 @@ static void dup_copy_fields(struct nfsd4_copy *src, struct nfsd4_copy *dst) dst->ss_nsui = src->ss_nsui; } +static void release_copy_files (struct nfsd4_copy *copy) +{ +if (copy->nf_src) +nfsd_file_put (copy->nf_src); +if (copy->nf_dst) +nfsd_file_put (copy->nf_dst); +} +static void cleanup_async_copy (struct nfsd4_copy *copy) {nfs4_free_copy_state (copy); -nfsd_file_put (copy->nf_dst); -if (!nfsd4_ssc_is_inter (copy)) -nfsd_file_put (copy->nf_src); +release_copy_files (copy);spin_lock (©->cp_clp->async_lock);list_del (©->copies);spin_unlock (©->cp_clp->async_lock);@@ -1748,7 +1746,6 @@ static int nfsd4_do_async_copy(void *data) }else { nfserr =nfsd4_do_copy (copy, copy->nf_src->nf_file, copy->nf_dst->nf_file, false); -nfsd4_cleanup_intra_ssc (copy->nf_src, copy->nf_dst); } do_callback:fs/nfsd/nfs4proc.c nfsd4_copy(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate, · line 1815
Quoted 9 lines
fs/nfsd/nfs4proc.c +1
-1@@ -1811,9 +1808,9 @@ nfsd4_copy(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate, }else { status =nfsd4_do_copy (copy, copy->nf_src->nf_file, copy->nf_dst->nf_file, true); -nfsd4_cleanup_intra_ssc (copy->nf_src, copy->nf_dst); } out: +release_copy_files (copy);return status; out_err:This is unrelated to the reference count issue.
Quoted 4 lines
Here if this is an inter-copy then we need to decrement the reference count of the
nfsd4_ssc_umount_itemso that the vfsmount can be unmounted later.Oh, I think I see what you mean. Maybe something like the (untested) patch below on top of the original patch would fix that?
fs/nfsd/nfs4proc.c +4
-2@@ -1511,8 +1511,10 @@ nfsd4_cleanup_inter_ssc(struct nfsd4_ssc_umount_item *nsui, struct file *filp,struct nfsd_net *nn =net_generic (dst->nf_net, nfsd_net_id);long timeout =msecs_to_jiffies (nfsd4_ssc_umount_timeout); -nfs42_ssc_close (filp); -fput (filp); +if (filp) { +nfs42_ssc_close (filp); +fput (filp); + }Quoted 16 lines
fs/nfsd/nfs4proc.c nfsd4_copy(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate, · line 1820
Quoted 10 lines
fs/nfsd/nfs4proc.c +2
-1spin_lock(&nn->nfsd_ssc_lo list_del(&nsui->nsui_list);@@ -1813,8 +1815,13 @@ nfsd4_copy(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,release_copy_files (copy);return status; out_err: -if (async_copy) +if (async_copy) {cleanup_async_copy (async_copy); +if (nfsd4_ssc_is_inter (async_copy))We don't need to call
nfsd4_cleanup_inter_sscsince the threadnfsd4_do_async_copyhas not started yet so the file is not opened. We just need to dorefcount_dec(©->ss_nsui->nsui_refcnt), unless you want to changenfsd4_cleanup_inter_sscto detect this error condition and only decrement the reference count.
Quoted 3 lines
Oh yeah, and this would break anyway since the
nsui_listhead is not being initialized. Dai, would you mind spinning up a patch for this since you're more familiar with the cleanup here?
Will do. My patch will only fix the unmount issue. Your patch does the clean up potential nfsd_file refcount leaks in COPY codepath.
Thanks,
-Dai
Quoted 1 line
[email protected]
reviewerRe: [PATCH 2/2] nfsd: clean up potential nfsd_file refcount leaks in COPY codepath
Quoted 154 lines · On 1/21/23 10:56 AM, [email protected] wrote:
Quoted 152 lines · Jeff Layton
Quoted 151 lines · On Thu, 2023-01-19 at 10:38 -0800, [email protected] wrote:
Quoted 44 lines · Jeff Layton
Quoted 28 lines · On Wed, 2023-01-18 at 21:05 -0800, [email protected] wrote:
Quoted 27 lines · Jeff Layton
There are two different flavors of the
nfsd4_copystruct. One is embedded in the compound and is used directly in synchronous copies. The other is dynamically allocated, refcounted and tracked in the client struture. For the embedded one, the cleanup just involves releasing anynfsd_filesheld on its behalf. For the async one, the cleanup is a bit more involved, and we need to dequeue it from lists, unhash it, etc.There is at least one potential refcount leak in this code now. If the
kthread_createcall fails, then both the src and dstnfsd_filesin the originalnfsd4_copyobject are leaked.The cleanup in this codepath is also sort of weird. In the async copy case, we'll have up to four
nfsd_filereferences (src and dst for both flavors of copy structure). They are both put at the end ofnfsd4_do_async_copy, even though the ones held on behalf of the embedded one outlive that structure.Change it so that we always clean up the
nfsd_filerefs held by the embedded copy structure beforenfsd4_copyreturns. Reworkcleanup_async_copyto handle both inter and intra copies. Eliminatenfsd4_cleanup_intra_sscsince it now becomes a no-op.Signed-off-by Jeff Layton <[email protected]>---
fs/nfsd/nfs4proc.c| 23 ++++++++++-------------1 file changed, 10 insertions(+), 13 deletions(-)Quoted 15 lines
fs/nfsd/nfs4proc.c nfsd4_cleanup_inter_ssc(struct · line 1515
Quoted 9 lines
fs/nfsd/nfs4proc.c +0
-1@@ -1512,7 +1512,6 @@ nfsd4_cleanup_inter_ssc(struct nfsd4_ssc_umount_item *nsui, struct file *filp, long timeout = msecs_to_jiffies(nfsd4_ssc_umount_timeout); nfs42_ssc_close(filp); - nfsd_file_put(dst);I think we still need this, in addition to
release_copy_filescalled fromcleanup_async_copy. For async inter-copy, there are 2 reference count added to the destination file, one fromnfsd4_setup_inter_sscand the other one fromdup_copy_fields. The abovenfsd_file_putis for the count added bydup_copy_fields.Quoted 5 lines
With this patch, the references held by the original copy structure are put by the call to
release_copy_filesat the end ofnfsd4_copy. That means that the kthread task is only responsible for putting the references held by the (kmalloc'ed)async_copystructure. So, I think this gets thenfsd_filerefcounting right.Yes, I see. One refcount is decremented by
release_copy_filesat end ofnfsd4_copyand another is decremented byrelease_copy_filesincleanup_async_copy.Quoted 86 lines
Quoted 59 lines
Quoted 53 lines
fs/nfsd/nfs4proc.c +0
-0fput(filp); spin_lock(&nn->nfsd_ssc_lock);@@ -1562,13 +1561,6 @@ nfsd4_setup_intra_ssc(struct svc_rqst *rqstp, ©->nf_dst); }-static void
-nfsd4_cleanup_intra_ssc(structnfsd_file*src, structnfsd_file*dst)fs/nfsd/nfs4proc.c +0
-5-{ - nfsd_file_put(src); - nfsd_file_put(dst); -} -static void
nfsd4_cb_offload_release(struct nfsd4_callback *cb){
structnfsd4_cb_offload*cbo =fs/nfsd/nfs4proc.c +8
-3@@ -1683,12 +1675,18 @@ static void dup_copy_fields(struct nfsd4_copy *src, struct nfsd4_copy *dst) dst->ss_nsui = src->ss_nsui; } +static void release_copy_files(struct nfsd4_copy *copy) +{ + if (copy->nf_src) + nfsd_file_put(copy->nf_src); + if (copy->nf_dst) + nfsd_file_put(copy->nf_dst); +} + static void cleanup_async_copy(struct nfsd4_copy *copy) { nfs4_free_copy_state(copy); - nfsd_file_put(copy->nf_dst); - if (!nfsd4_ssc_is_inter(copy)) - nfsd_file_put(copy->nf_src); + release_copy_files(copy); spin_lock(©->cp_clp->async_lock); list_del(©->copies);
spin_unlock(©->cp_clp->async_lock);fs/nfsd/nfs4proc.c +0
-1@@ -1748,7 +1746,6 @@ static int nfsd4_do_async_copy(void *data) } else { nfserr = nfsd4_do_copy(copy, copy->nf_src->nf_file, copy->nf_dst->nf_file, false); - nfsd4_cleanup_intra_ssc(copy->nf_src, copy->nf_dst); }
do_callback:fs/nfsd/nfs4proc.c +0
-1@@ -1811,9 +1808,9 @@ nfsd4_copy(struct svc_rqst *rqstp, structnfsd4_compound_state *cstate, } else { status = nfsd4_do_copy(copy, copy->nf_src->nf_file, copy->nf_dst->nf_file, true); - nfsd4_cleanup_intra_ssc(copy->nf_src, copy->nf_dst); }Quoted 4 lines
out:
+release_copy_files(copy);
return status;
out_err:This is unrelated to the reference count issue.
Quoted 4 lines
Here if this is an inter-copy then we need to decrement the reference count of the
nfsd4_ssc_umount_itemso that the vfsmount can be unmounted later.Oh, I think I see what you mean. Maybe something like the (untested) patch below on top of the original patch would fix that?
fs/nfsd/nfs4proc.c +4
-1@@ -1511,8 +1511,10 @@ nfsd4_cleanup_inter_ssc(struct nfsd4_ssc_umount_item *nsui, struct file *filp, struct nfsd_net *nn = net_generic(dst->nf_net, nfsd_net_id); long timeout = msecs_to_jiffies(nfsd4_ssc_umount_timeout); - nfs42_ssc_close(filp); - fput(filp); + if (filp) { + nfs42_ssc_close(filp); + fput(filp); + }spin_lock(&nn->nfsd_ssc_lo
list_del(&nsui->nsui_list);fs/nfsd/nfs4proc.c +0
-0@@ -1813,8 +1815,13 @@ nfsd4_copy(struct svc_rqst *rqstp, structnfsd4_compound_state *cstate, release_copy_files(copy); return status;Quoted 5 lines
out_err:
- if (async_copy)
+ if (async_copy) {cleanup_async_copy(async_copy); + if (nfsd4_ssc_is_inter(async_copy))We don't need to call
nfsd4_cleanup_inter_sscsince the threadnfsd4_do_async_copyhas not started yet so the file is not opened. We just need to dorefcount_dec(©->ss_nsui->nsui_refcnt), unless you want to changenfsd4_cleanup_inter_sscto detect this error condition and only decrement the reference count.
Quoted 6 lines
Quoted 3 lines
Oh yeah, and this would break anyway since the
nsui_listhead is not being initialized. Dai, would you mind spinning up a patch for this since you're more familiar with the cleanup here?Will do. My patch will only fix the unmount issue. Your patch does the clean up potential
nfsd_filerefcount leaks in COPY codepath.
Or do you want me to merge your patch and mine into one?
I think we need a bit more cleanup in addition to your patch. When kmalloc(sizeof(*async_copy->cp_src), ..) or nfs4_init_copy_state fails, the async_copy is not initialized yet so calling cleanup_async_copy can be a problem.
-Dai
Jeff Layton
maintainerRe: [PATCH 2/2] nfsd: clean up potential nfsd_file refcount leaks in COPY codepath
Quoted 166 lines · On Sat, 2023-01-21 at 11:50 -0800, [email protected] wrote:
Quoted 165 lines · On 1/21/23 10:56 AM, [email protected] wrote:
Quoted 163 lines · Jeff Layton
Quoted 162 lines · On Thu, 2023-01-19 at 10:38 -0800, [email protected] wrote:
Quoted 53 lines · Jeff Layton
Quoted 36 lines · On Wed, 2023-01-18 at 21:05 -0800, [email protected] wrote:
Quoted 35 lines · Jeff Layton
There are two different flavors of the
nfsd4_copystruct. One is embedded in the compound and is used directly in synchronous copies. The
other is dynamically allocated, refcounted and tracked in the client struture. For the embedded one, the cleanup just involves releasing anynfsd_filesheld on its behalf. For the async one, the cleanup is a bit
more involved, and we need to dequeue it from lists, unhash it, etc.There is at least one potential refcount leak in this code now. If the
kthread_createcall fails, then both the src and dstnfsd_filesin the
originalnfsd4_copyobject are leaked.The cleanup in this codepath is also sort of weird. In the async copy
case, we'll have up to fournfsd_filereferences (src and dst for both
flavors of copy structure). They are both put at the end ofnfsd4_do_async_copy, even though the ones held on behalf of the embedded
one outlive that structure.Change it so that we always clean up the
nfsd_filerefs held by the embedded copy structure beforenfsd4_copyreturns. Reworkcleanup_async_copyto handle both inter and intra copies. Eliminatenfsd4_cleanup_intra_sscsince it now becomes a no-op.Signed-off-by Jeff Layton <[email protected]>---
fs/nfsd/nfs4proc.c| 23 ++++++++++-------------1 file changed, 10 insertions(+), 13 deletions(-)Quoted 16 lines
fs/nfsd/nfs4proc.c nfsd4_cleanup_inter_ssc(struct · line 1515
Quoted 9 lines
fs/nfsd/nfs4proc.c +0
-1@@ -1512,7 +1512,6 @@ nfsd4_cleanup_inter_ssc(struct nfsd4_ssc_umount_item *nsui, struct file *filp, long timeout = msecs_to_jiffies(nfsd4_ssc_umount_timeout); nfs42_ssc_close(filp); - nfsd_file_put(dst);I think we still need this, in addition to
release_copy_filescalled fromcleanup_async_copy. For async inter-copy, there are 2 reference count added to the destination file, one fromnfsd4_setup_inter_sscand the other one fromdup_copy_fields. The abovenfsd_file_putis for
the count added bydup_copy_fields.Quoted 6 lines
With this patch, the references held by the original copy structure are
put by the call torelease_copy_filesat the end ofnfsd4_copy. That means that the kthread task is only responsible for putting the references held by the (kmalloc'ed)async_copystructure. So, I think this gets thenfsd_filerefcounting right.Yes, I see. One refcount is decremented by
release_copy_filesat end ofnfsd4_copyand another is decremented byrelease_copy_filesincleanup_async_copy.Quoted 87 lines
Quoted 59 lines
Quoted 53 lines
fs/nfsd/nfs4proc.c +0
-0fput(filp); spin_lock(&nn->nfsd_ssc_lock);@@ -1562,13 +1561,6 @@ nfsd4_setup_intra_ssc(struct svc_rqst *rqstp, ©->nf_dst); }-static void
-nfsd4_cleanup_intra_ssc(structnfsd_file*src, structnfsd_file*dst)fs/nfsd/nfs4proc.c +0
-5-{ - nfsd_file_put(src); - nfsd_file_put(dst); -} -static void
nfsd4_cb_offload_release(struct nfsd4_callback *cb){
structnfsd4_cb_offload*cbo =fs/nfsd/nfs4proc.c +8
-3@@ -1683,12 +1675,18 @@ static void dup_copy_fields(struct nfsd4_copy *src, struct nfsd4_copy *dst) dst->ss_nsui = src->ss_nsui; } +static void release_copy_files(struct nfsd4_copy *copy) +{ + if (copy->nf_src) + nfsd_file_put(copy->nf_src); + if (copy->nf_dst) + nfsd_file_put(copy->nf_dst); +} + static void cleanup_async_copy(struct nfsd4_copy *copy) { nfs4_free_copy_state(copy); - nfsd_file_put(copy->nf_dst); - if (!nfsd4_ssc_is_inter(copy)) - nfsd_file_put(copy->nf_src); + release_copy_files(copy); spin_lock(©->cp_clp->async_lock); list_del(©->copies);
spin_unlock(©->cp_clp->async_lock);fs/nfsd/nfs4proc.c +0
-1@@ -1748,7 +1746,6 @@ static int nfsd4_do_async_copy(void *data) } else { nfserr = nfsd4_do_copy(copy, copy->nf_src->nf_file, copy->nf_dst->nf_file, false); - nfsd4_cleanup_intra_ssc(copy->nf_src, copy->nf_dst); }
do_callback:fs/nfsd/nfs4proc.c +0
-1@@ -1811,9 +1808,9 @@ nfsd4_copy(struct svc_rqst *rqstp, structnfsd4_compound_state *cstate, } else { status = nfsd4_do_copy(copy, copy->nf_src->nf_file, copy->nf_dst->nf_file, true); - nfsd4_cleanup_intra_ssc(copy->nf_src, copy->nf_dst); }Quoted 4 lines
out:
+release_copy_files(copy);
return status;
out_err:This is unrelated to the reference count issue.
Quoted 5 lines
Here if this is an inter-copy then we need to decrement the reference count of the
nfsd4_ssc_umount_itemso that the vfsmount can be unmounted
later.Oh, I think I see what you mean. Maybe something like the (untested) patch below on top of the original patch would fix that?
fs/nfsd/nfs4proc.c +4
-1@@ -1511,8 +1511,10 @@ nfsd4_cleanup_inter_ssc(struct nfsd4_ssc_umount_item *nsui, struct file *filp, struct nfsd_net *nn = net_generic(dst->nf_net, nfsd_net_id); long timeout = msecs_to_jiffies(nfsd4_ssc_umount_timeout); - nfs42_ssc_close(filp); - fput(filp); + if (filp) { + nfs42_ssc_close(filp); + fput(filp); + }spin_lock(&nn->nfsd_ssc_lo
list_del(&nsui->nsui_list);fs/nfsd/nfs4proc.c +0
-0@@ -1813,8 +1815,13 @@ nfsd4_copy(struct svc_rqst *rqstp, structnfsd4_compound_state *cstate, release_copy_files(copy); return status;Quoted 5 lines
out_err:
- if (async_copy)
+ if (async_copy) {cleanup_async_copy(async_copy); + if (nfsd4_ssc_is_inter(async_copy))We don't need to call
nfsd4_cleanup_inter_sscsince the threadnfsd4_do_async_copyhas not started yet so the file is not opened. We just need to dorefcount_dec(©->ss_nsui->nsui_refcnt), unless you want to changenfsd4_cleanup_inter_sscto detect this error condition and only decrement the reference count.
Quoted 9 lines
Quoted 6 lines
Quoted 3 lines
Oh yeah, and this would break anyway since the
nsui_listhead is not being initialized. Dai, would you mind spinning up a patch for this since you're more familiar with the cleanup here?Will do. My patch will only fix the unmount issue. Your patch does the clean up potential
nfsd_filerefcount leaks in COPY codepath.Or do you want me to merge your patch and mine into one?
It probably is best to merge them, since backporters will probably want both patches anyway. Just make yourself the patch author and keep my S- o-b line.
Quoted 5 lines
I think we need a bit more cleanup in addition to your patch. When
kmalloc(sizeof(*async_copy->cp_src), ..)ornfs4_init_copy_statefails, theasync_copyis not initialized yet so callingcleanup_async_copycan be a problem.
Yeah.
It may even be best to ensure that the list_head and such are fully initialized for both allocated and embedded struct nfsd4_copy's. You might shave off a few cpu cycles by not doing that, but it makes things more fragile.
Even better, we really ought to split a lot of the fields in nfsd4_copy into a different structure (maybe nfsd4_async_copy). Trimming down struct nfsd4_copy would cut down the size of nfsd4_compound as well since it has a union that contains it. I was planning on doing that eventually, but if you want to take that on, then that would be fine too.
Signature · 1 line
Jeff Layton <[email protected]>
Re: [PATCH 2/2] nfsd: clean up potential nfsd_file refcount leaks in COPY codepath
Quoted 175 lines · Jeff Layton
Quoted 170 lines · On Sat, 2023-01-21 at 11:50 -0800, [email protected] wrote:
Quoted 169 lines · On 1/21/23 10:56 AM, [email protected] wrote:
Quoted 166 lines · Jeff Layton
Quoted 153 lines · On Thu, 2023-01-19 at 10:38 -0800, [email protected] wrote:
Quoted 53 lines · Jeff Layton
Quoted 36 lines · On Wed, 2023-01-18 at 21:05 -0800, [email protected] wrote:
Quoted 35 lines · Jeff Layton
There are two different flavors of the
nfsd4_copystruct. One is embedded in the compound and is used directly in synchronous copies. The
other is dynamically allocated, refcounted and tracked in the client struture. For the embedded one, the cleanup just involves releasing anynfsd_filesheld on its behalf. For the async one, the cleanup is a bit
more involved, and we need to dequeue it from lists, unhash it, etc.There is at least one potential refcount leak in this code now. If the
kthread_createcall fails, then both the src and dstnfsd_filesin the
originalnfsd4_copyobject are leaked.The cleanup in this codepath is also sort of weird. In the async copy
case, we'll have up to fournfsd_filereferences (src and dst for both
flavors of copy structure). They are both put at the end ofnfsd4_do_async_copy, even though the ones held on behalf of the embedded
one outlive that structure.Change it so that we always clean up the
nfsd_filerefs held by the embedded copy structure beforenfsd4_copyreturns. Reworkcleanup_async_copyto handle both inter and intra copies. Eliminatenfsd4_cleanup_intra_sscsince it now becomes a no-op.Signed-off-by Jeff Layton <[email protected]>--- fs/nfsd/nfs4proc.c | 23 ++++++++++------------- 1 file changed, 10 insertions(+), 13 deletions(-)Quoted 16 lines
fs/nfsd/nfs4proc.c nfsd4_cleanup_inter_ssc(struct · line 1515
Quoted 9 lines
fs/nfsd/nfs4proc.c +0
-1@@ -1512,7 +1512,6 @@ nfsd4_cleanup_inter_ssc(struct nfsd4_ssc_umount_item *nsui, struct file *filp, long timeout = msecs_to_jiffies(nfsd4_ssc_umount_timeout);nfs42_ssc_close (filp); -nfsd_file_put (dst);I think we still need this, in addition to
release_copy_filescalled fromcleanup_async_copy. For async inter-copy, there are 2 reference count added to the destination file, one fromnfsd4_setup_inter_sscand the other one fromdup_copy_fields. The abovenfsd_file_putis for
the count added bydup_copy_fields.Quoted 6 lines
With this patch, the references held by the original copy structure are
put by the call torelease_copy_filesat the end ofnfsd4_copy. That means that the kthread task is only responsible for putting the references held by the (kmalloc'ed)async_copystructure. So, I think this gets thenfsd_filerefcounting right.Yes, I see. One refcount is decremented by
release_copy_filesat end ofnfsd4_copyand another is decremented byrelease_copy_filesincleanup_async_copy.Quoted 89 lines
Quoted 59 lines
Quoted 47 lines
fs/nfsd/nfs4proc.c +8
-9fput (filp);spin_lock (&nn->nfsd_ssc_lock);@@ -1562,13 +1561,6 @@ nfsd4_setup_intra_ssc(struct svc_rqst *rqstp, ©->nf_dst); } -static void -nfsd4_cleanup_intra_ssc (struct nfsd_file *src,struct nfsd_file *dst) -{ -nfsd_file_put (src); -nfsd_file_put (dst); -} -static void nfsd4_cb_offload_release (struct nfsd4_callback *cb) { struct nfsd4_cb_offload *cbo =@@ -1683,12 +1675,18 @@ static void dup_copy_fields(struct nfsd4_copy *src, struct nfsd4_copy *dst) dst->ss_nsui = src->ss_nsui; } +static void release_copy_files(struct nfsd4_copy *copy) +{ +if (copy->nf_src) +nfsd_file_put (copy->nf_src); +if (copy->nf_dst) +nfsd_file_put (copy->nf_dst); +} +static void cleanup_async_copy (struct nfsd4_copy *copy) {nfs4_free_copy_state (copy); -nfsd_file_put (copy->nf_dst); -if (!nfsd4_ssc_is_inter (copy)) -nfsd_file_put (copy->nf_src); +release_copy_files (copy);spin_lock (©->cp_clp->async_lock);list_del (©->copies);
spin_unlock(©->cp_clp->async_lock);fs/nfsd/nfs4proc.c +0
-1@@ -1748,7 +1746,6 @@ static int nfsd4_do_async_copy(void *data) }else { nfserr =nfsd4_do_copy (copy, copy->nf_src->nf_file, copy->nf_dst->nf_file, false); -nfsd4_cleanup_intra_ssc (copy->nf_src, copy->nf_dst); } do_callback:fs/nfsd/nfs4proc.c nfsd4_copy(struct svc_rqst *rqstp, struct · line 1816
Quoted 10 lines
fs/nfsd/nfs4proc.c +1
-1@@ -1811,9 +1808,9 @@ nfsd4_copy(struct svc_rqst *rqstp, structnfsd4_compound_state *cstate, } else { status =nfsd4_do_copy (copy, copy->nf_src->nf_file, copy->nf_dst->nf_file, true); -nfsd4_cleanup_intra_ssc (copy->nf_src, copy->nf_dst); } out: +release_copy_files (copy);return status; out_err:This is unrelated to the reference count issue.
Quoted 5 lines
Here if this is an inter-copy then we need to decrement the reference count of the
nfsd4_ssc_umount_itemso that the vfsmount can be unmounted
later.Oh, I think I see what you mean. Maybe something like the (untested) patch below on top of the original patch would fix that?
fs/nfsd/nfs4proc.c +4
-2@@ -1511,8 +1511,10 @@ nfsd4_cleanup_inter_ssc(struct nfsd4_ssc_umount_item *nsui, struct file *filp, struct nfsd_net *nn = net_generic(dst->nf_net, nfsd_net_id);long timeout =msecs_to_jiffies (nfsd4_ssc_umount_timeout); -nfs42_ssc_close (filp); -fput (filp); +if (filp) { +nfs42_ssc_close (filp); +fput (filp); + } spin_lock(&nn->nfsd_ssc_lo list_del(&nsui->nsui_list);@@ -1813,8 +1815,13 @@ nfsd4_copy(struct svc_rqst *rqstp, structnfsd4_compound_state *cstate,release_copy_files (copy );return status; out_err: -if (async_copy)Quoted 9 lines
fs/nfsd/nfs4proc.c nfsd4_copy(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate, · line 1820
Quoted 3 lines
fs/nfsd/nfs4proc.c +2
-0+if (async_copy) {cleanup_async_copy (async_copy); +if (nfsd4_ssc_is_inter (async_copy))We don't need to call
nfsd4_cleanup_inter_sscsince the threadnfsd4_do_async_copyhas not started yet so the file is not opened. We just need to dorefcount_dec(©->ss_nsui->nsui_refcnt), unless you want to changenfsd4_cleanup_inter_sscto detect this error condition and only decrement the reference count.Oh yeah, and this would break anyway since the
nsui_listhead is not being initialized. Dai, would you mind spinning up a patch for this since you're more familiar with the cleanup here?
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
Will do. My patch will only fix the unmount issue. Your patch does the clean up potential
nfsd_filerefcount leaks in COPY codepath.Or do you want me to merge your patch and mine into one?
It probably is best to merge them, since backporters will probably want both patches anyway.
Unless these two changes are somehow interdependent, I'd like to keep them separate. They address two separate issues, yes?
And -- narrow fixes need to go to nfsd-fixes, but clean-ups can wait for nfsd-next. I'd rather not mix the two types of change.
Quoted 15 lines
Just make yourself the patch author and keep my S-o-b line.
Quoted 5 lines
I think we need a bit more cleanup in addition to your patch. When
kmalloc(sizeof(*async_copy->cp_src), ..)ornfs4_init_copy_statefails, theasync_copyis not initialized yet so callingcleanup_async_copycan be a problem.Yeah.
It may even be best to ensure that the
list_headand such are fully initialized for both allocated and embedded struct nfsd4_copy's. You might shave off a few cpu cycles by not doing that, but it makes things more fragile.
Quoted 9 lines
Even better, we really ought to split a lot of the fields in
nfsd4_copyinto a different structure (maybenfsd4_async_copy). Trimming down structnfsd4_copywould cut down the size ofnfsd4_compoundas well since it has a union that contains it. I was planning on doing that eventually, but if you want to take that on, then that would be fine too.Signature · 1 line
Jeff Layton <[email protected]>
--
Chuck Lever
[email protected]
reviewerRe: [PATCH 2/2] nfsd: clean up potential nfsd_file refcount leaks in COPY codepath
Quoted 171 lines · Chuck Lever III
Quoted 171 lines · Jeff Layton
Quoted 167 lines · On Sat, 2023-01-21 at 11:50 -0800, [email protected] wrote:
Quoted 166 lines · On 1/21/23 10:56 AM, [email protected] wrote:
Quoted 165 lines · Jeff Layton
Quoted 152 lines · On Thu, 2023-01-19 at 10:38 -0800, [email protected] wrote:
Quoted 53 lines · Jeff Layton
Quoted 36 lines · On Wed, 2023-01-18 at 21:05 -0800, [email protected] wrote:
Quoted 35 lines · Jeff Layton
There are two different flavors of the
nfsd4_copystruct. One is embedded in the compound and is used directly in synchronous copies. The
other is dynamically allocated, refcounted and tracked in the client struture. For the embedded one, the cleanup just involves releasing anynfsd_filesheld on its behalf. For the async one, the cleanup is a bit
more involved, and we need to dequeue it from lists, unhash it, etc.There is at least one potential refcount leak in this code now. If the
kthread_createcall fails, then both the src and dstnfsd_filesin the
originalnfsd4_copyobject are leaked.The cleanup in this codepath is also sort of weird. In the async copy
case, we'll have up to fournfsd_filereferences (src and dst for both
flavors of copy structure). They are both put at the end ofnfsd4_do_async_copy, even though the ones held on behalf of the embedded
one outlive that structure.Change it so that we always clean up the
nfsd_filerefs held by the embedded copy structure beforenfsd4_copyreturns. Reworkcleanup_async_copyto handle both inter and intra copies. Eliminatenfsd4_cleanup_intra_sscsince it now becomes a no-op.Signed-off-by Jeff Layton <[email protected]>--- fs/nfsd/nfs4proc.c | 23 ++++++++++------------- 1 file changed, 10 insertions(+), 13 deletions(-)Quoted 16 lines
fs/nfsd/nfs4proc.c nfsd4_cleanup_inter_ssc(struct · line 1515
Quoted 9 lines
fs/nfsd/nfs4proc.c +0
-1@@ -1512,7 +1512,6 @@ nfsd4_cleanup_inter_ssc(struct nfsd4_ssc_umount_item *nsui, struct file *filp, long timeout = msecs_to_jiffies(nfsd4_ssc_umount_timeout);nfs42_ssc_close (filp); -nfsd_file_put (dst);I think we still need this, in addition to
release_copy_filescalled fromcleanup_async_copy. For async inter-copy, there are 2 reference count added to the destination file, one fromnfsd4_setup_inter_sscand the other one fromdup_copy_fields. The abovenfsd_file_putis for
the count added bydup_copy_fields.Quoted 6 lines
With this patch, the references held by the original copy structure are
put by the call torelease_copy_filesat the end ofnfsd4_copy. That means that the kthread task is only responsible for putting the references held by the (kmalloc'ed)async_copystructure. So, I think this gets thenfsd_filerefcounting right.Yes, I see. One refcount is decremented by
release_copy_filesat end ofnfsd4_copyand another is decremented byrelease_copy_filesincleanup_async_copy.Quoted 88 lines
Quoted 58 lines
Quoted 46 lines
fs/nfsd/nfs4proc.c +8
-9fput (filp);spin_lock (&nn->nfsd_ssc_lock);@@ -1562,13 +1561,6 @@ nfsd4_setup_intra_ssc(struct svc_rqst *rqstp, ©->nf_dst); } -static void -nfsd4_cleanup_intra_ssc (struct nfsd_file *src,struct nfsd_file *dst) -{ -nfsd_file_put (src); -nfsd_file_put (dst); -} -static void nfsd4_cb_offload_release (struct nfsd4_callback *cb) { struct nfsd4_cb_offload *cbo =@@ -1683,12 +1675,18 @@ static void dup_copy_fields(struct nfsd4_copy *src, struct nfsd4_copy *dst) dst->ss_nsui = src->ss_nsui; } +static void release_copy_files(struct nfsd4_copy *copy) +{ +if (copy->nf_src) +nfsd_file_put (copy->nf_src); +if (copy->nf_dst) +nfsd_file_put (copy->nf_dst); +} +static void cleanup_async_copy (struct nfsd4_copy *copy) {nfs4_free_copy_state (copy); -nfsd_file_put (copy->nf_dst); -if (!nfsd4_ssc_is_inter (copy)) -nfsd_file_put (copy->nf_src); +release_copy_files (copy);spin_lock (©->cp_clp->async_lock);list_del (©->copies);
spin_unlock(©->cp_clp->async_lock);fs/nfsd/nfs4proc.c +0
-1@@ -1748,7 +1746,6 @@ static int nfsd4_do_async_copy(void *data) }else { nfserr =nfsd4_do_copy (copy, copy->nf_src->nf_file, copy->nf_dst->nf_file, false); -nfsd4_cleanup_intra_ssc (copy->nf_src, copy->nf_dst); } do_callback:fs/nfsd/nfs4proc.c nfsd4_copy(struct svc_rqst *rqstp, struct · line 1816
Quoted 10 lines
fs/nfsd/nfs4proc.c +1
-1@@ -1811,9 +1808,9 @@ nfsd4_copy(struct svc_rqst *rqstp, structnfsd4_compound_state *cstate, } else { status =nfsd4_do_copy (copy, copy->nf_src->nf_file, copy->nf_dst->nf_file, true); -nfsd4_cleanup_intra_ssc (copy->nf_src, copy->nf_dst); } out: +release_copy_files (copy);return status; out_err:This is unrelated to the reference count issue.
Quoted 5 lines
Here if this is an inter-copy then we need to decrement the reference count of the
nfsd4_ssc_umount_itemso that the vfsmount can be unmounted
later.Oh, I think I see what you mean. Maybe something like the (untested) patch below on top of the original patch would fix that?
fs/nfsd/nfs4proc.c +4
-2@@ -1511,8 +1511,10 @@ nfsd4_cleanup_inter_ssc(struct nfsd4_ssc_umount_item *nsui, struct file *filp, struct nfsd_net *nn = net_generic(dst->nf_net, nfsd_net_id);long timeout =msecs_to_jiffies (nfsd4_ssc_umount_timeout); -nfs42_ssc_close (filp); -fput (filp); +if (filp) { +nfs42_ssc_close (filp); +fput (filp); + } spin_lock(&nn->nfsd_ssc_lo list_del(&nsui->nsui_list);@@ -1813,8 +1815,13 @@ nfsd4_copy(struct svc_rqst *rqstp, structnfsd4_compound_state *cstate,release_copy_files (copy );return status; out_err: -if (async_copy)Quoted 9 lines
fs/nfsd/nfs4proc.c nfsd4_copy(struct svc_rqst *rqstp, struct · line 1821
Quoted 3 lines
fs/nfsd/nfs4proc.c +2
-0+if (async_copy) {cleanup_async_copy (async_copy); +if (nfsd4_ssc_is_inter (async_copy))We don't need to call
nfsd4_cleanup_inter_sscsince the threadnfsd4_do_async_copyhas not started yet so the file is not opened. We just need to dorefcount_dec(©->ss_nsui->nsui_refcnt), unless you want to changenfsd4_cleanup_inter_sscto detect this error condition and only decrement the reference count.Oh yeah, and this would break anyway since the
nsui_listhead is not being initialized. Dai, would you mind spinning up a patch for this since you're more familiar with the cleanup here?
Quoted 8 lines
Quoted 6 lines
Quoted 4 lines
Quoted 2 lines
Will do. My patch will only fix the unmount issue. Your patch does the clean up potential
nfsd_filerefcount leaks in COPY codepath.Or do you want me to merge your patch and mine into one?
It probably is best to merge them, since backporters will probably want both patches anyway.
Unless these two changes are somehow interdependent, I'd like to keep them separate. They address two separate issues, yes?
Yes.
Quoted 3 lines
And -- narrow fixes need to go to nfsd-fixes, but clean-ups can wait for nfsd-next. I'd rather not mix the two types of change.
Ok. Can we do this:
1. Jeff's patch goes to nfsd-fixes since it has the fix for missing reference count.
2. My fix for the cleanup of allocated memory goes to nfsd-fixes.
3. I will do the optimization Jeff proposed about list_head and nfsd4_compound in a separate patch that goes into nfsd-next.
-Dai
Quoted 28 lines
Quoted 14 lines
Just make yourself the patch author and keep my S-o-b line.
Quoted 5 lines
I think we need a bit more cleanup in addition to your patch. When
kmalloc(sizeof(*async_copy->cp_src), ..)ornfs4_init_copy_statefails, theasync_copyis not initialized yet so callingcleanup_async_copycan be a problem.Yeah.
It may even be best to ensure that the
list_headand such are fully initialized for both allocated and embedded struct nfsd4_copy's. You might shave off a few cpu cycles by not doing that, but it makes things more fragile.Quoted 9 lines
Even better, we really ought to split a lot of the fields in
nfsd4_copyinto a different structure (maybenfsd4_async_copy). Trimming down structnfsd4_copywould cut down the size ofnfsd4_compoundas well since it has a union that contains it. I was planning on doing that eventually, but if you want to take that on, then that would be fine too.Signature · 1 line
Jeff Layton <[email protected]>--
Chuck Lever
Re: [PATCH 2/2] nfsd: clean up potential nfsd_file refcount leaks in COPY codepath
Quoted 189 lines · Dai Ngo
Quoted 171 lines · Chuck Lever III
Quoted 171 lines · Jeff Layton
Quoted 167 lines · On Sat, 2023-01-21 at 11:50 -0800, [email protected] wrote:
Quoted 166 lines · On 1/21/23 10:56 AM, [email protected] wrote:
Quoted 165 lines · Jeff Layton
Quoted 152 lines · On Thu, 2023-01-19 at 10:38 -0800, [email protected] wrote:
Quoted 53 lines · Jeff Layton
Quoted 36 lines · On Wed, 2023-01-18 at 21:05 -0800, [email protected] wrote:
Quoted 35 lines · Jeff Layton
There are two different flavors of the
nfsd4_copystruct. One is embedded in the compound and is used directly in synchronous copies. The
other is dynamically allocated, refcounted and tracked in the client struture. For the embedded one, the cleanup just involves releasing anynfsd_filesheld on its behalf. For the async one, the cleanup is a bit
more involved, and we need to dequeue it from lists, unhash it, etc.There is at least one potential refcount leak in this code now. If the
kthread_createcall fails, then both the src and dstnfsd_filesin the
originalnfsd4_copyobject are leaked.The cleanup in this codepath is also sort of weird. In the async copy
case, we'll have up to fournfsd_filereferences (src and dst for both
flavors of copy structure). They are both put at the end ofnfsd4_do_async_copy, even though the ones held on behalf of the embedded
one outlive that structure.Change it so that we always clean up the
nfsd_filerefs held by the embedded copy structure beforenfsd4_copyreturns. Reworkcleanup_async_copyto handle both inter and intra copies. Eliminatenfsd4_cleanup_intra_sscsince it now becomes a no-op.Signed-off-by Jeff Layton <[email protected]>--- fs/nfsd/nfs4proc.c | 23 ++++++++++------------- 1 file changed, 10 insertions(+), 13 deletions(-)Quoted 16 lines
fs/nfsd/nfs4proc.c nfsd4_cleanup_inter_ssc(struct · line 1515
Quoted 9 lines
fs/nfsd/nfs4proc.c +0
-1@@ -1512,7 +1512,6 @@ nfsd4_cleanup_inter_ssc(struct nfsd4_ssc_umount_item *nsui, struct file *filp, long timeout = msecs_to_jiffies(nfsd4_ssc_umount_timeout);nfs42_ssc_close (filp); -nfsd_file_put (dst);I think we still need this, in addition to
release_copy_filescalled fromcleanup_async_copy. For async inter-copy, there are 2 reference count added to the destination file, one fromnfsd4_setup_inter_sscand the other one fromdup_copy_fields. The abovenfsd_file_putis for
the count added bydup_copy_fields.Quoted 6 lines
With this patch, the references held by the original copy structure are
put by the call torelease_copy_filesat the end ofnfsd4_copy. That means that the kthread task is only responsible for putting the references held by the (kmalloc'ed)async_copystructure. So, I think this gets thenfsd_filerefcounting right.Yes, I see. One refcount is decremented by
release_copy_filesat end ofnfsd4_copyand another is decremented byrelease_copy_filesincleanup_async_copy.Quoted 88 lines
Quoted 58 lines
Quoted 46 lines
fs/nfsd/nfs4proc.c +8
-9fput (filp);spin_lock (&nn->nfsd_ssc_lock);@@ -1562,13 +1561,6 @@ nfsd4_setup_intra_ssc(struct svc_rqst *rqstp, ©->nf_dst); } -static void -nfsd4_cleanup_intra_ssc (struct nfsd_file *src,struct nfsd_file *dst) -{ -nfsd_file_put (src); -nfsd_file_put (dst); -} -static void nfsd4_cb_offload_release (struct nfsd4_callback *cb) { struct nfsd4_cb_offload *cbo =@@ -1683,12 +1675,18 @@ static void dup_copy_fields(struct nfsd4_copy *src, struct nfsd4_copy *dst) dst->ss_nsui = src->ss_nsui; } +static void release_copy_files(struct nfsd4_copy *copy) +{ +if (copy->nf_src) +nfsd_file_put (copy->nf_src); +if (copy->nf_dst) +nfsd_file_put (copy->nf_dst); +} +static void cleanup_async_copy (struct nfsd4_copy *copy) {nfs4_free_copy_state (copy); -nfsd_file_put (copy->nf_dst); -if (!nfsd4_ssc_is_inter (copy)) -nfsd_file_put (copy->nf_src); +release_copy_files (copy);spin_lock (©->cp_clp->async_lock);list_del (©->copies);
spin_unlock(©->cp_clp->async_lock);fs/nfsd/nfs4proc.c +0
-1@@ -1748,7 +1746,6 @@ static int nfsd4_do_async_copy(void *data) }else { nfserr =nfsd4_do_copy (copy, copy->nf_src->nf_file, copy->nf_dst->nf_file, false); -nfsd4_cleanup_intra_ssc (copy->nf_src, copy->nf_dst); } do_callback:fs/nfsd/nfs4proc.c nfsd4_copy(struct svc_rqst *rqstp, struct · line 1816
Quoted 10 lines
fs/nfsd/nfs4proc.c +1
-1@@ -1811,9 +1808,9 @@ nfsd4_copy(struct svc_rqst *rqstp, structnfsd4_compound_state *cstate, } else { status =nfsd4_do_copy (copy, copy->nf_src->nf_file, copy->nf_dst->nf_file, true); -nfsd4_cleanup_intra_ssc (copy->nf_src, copy->nf_dst); } out: +release_copy_files (copy);return status; out_err:This is unrelated to the reference count issue.
Quoted 5 lines
Here if this is an inter-copy then we need to decrement the reference count of the
nfsd4_ssc_umount_itemso that the vfsmount can be unmounted
later.Oh, I think I see what you mean. Maybe something like the (untested) patch below on top of the original patch would fix that?
fs/nfsd/nfs4proc.c +4
-2@@ -1511,8 +1511,10 @@ nfsd4_cleanup_inter_ssc(struct nfsd4_ssc_umount_item *nsui, struct file *filp, struct nfsd_net *nn = net_generic(dst->nf_net, nfsd_net_id);long timeout =msecs_to_jiffies (nfsd4_ssc_umount_timeout); -nfs42_ssc_close (filp); -fput (filp); +if (filp) { +nfs42_ssc_close (filp); +fput (filp); + } spin_lock(&nn->nfsd_ssc_lo list_del(&nsui->nsui_list);@@ -1813,8 +1815,13 @@ nfsd4_copy(struct svc_rqst *rqstp, structnfsd4_compound_state *cstate,release_copy_files (copy );return status; out_err: -if (async_copy)Quoted 9 lines
fs/nfsd/nfs4proc.c nfsd4_copy(struct svc_rqst *rqstp, struct · line 1821
Quoted 3 lines
fs/nfsd/nfs4proc.c +2
-0+if (async_copy) {cleanup_async_copy (async_copy); +if (nfsd4_ssc_is_inter (async_copy))We don't need to call
nfsd4_cleanup_inter_sscsince the threadnfsd4_do_async_copyhas not started yet so the file is not opened. We just need to dorefcount_dec(©->ss_nsui->nsui_refcnt), unless you want to changenfsd4_cleanup_inter_sscto detect this error condition and only decrement the reference count.Oh yeah, and this would break anyway since the
nsui_listhead is not being initialized. Dai, would you mind spinning up a patch for this since you're more familiar with the cleanup here?Quoted 8 lines
Quoted 6 lines
Quoted 4 lines
Quoted 2 lines
Will do. My patch will only fix the unmount issue. Your patch does the clean up potential
nfsd_filerefcount leaks in COPY codepath.Or do you want me to merge your patch and mine into one?
It probably is best to merge them, since backporters will probably want both patches anyway.
Unless these two changes are somehow interdependent, I'd like to keep them separate. They address two separate issues, yes?
Yes.
Quoted 1 line
Quoted 7 lines
Quoted 2 lines
And -- narrow fixes need to go to nfsd-fixes, but clean-ups can wait for nfsd-next. I'd rather not mix the two types of change.
Ok. Can we do this:
1. Jeff's patch goes to nfsd-fixes since it has the fix for missing reference count.
To make sure I haven't lost track of anything:
The patch you refer to here is this one:
https://lore.kernel.org/linux-nfs/[email protected]/
Correct?
(I was waiting for Jeff and Olga to come to consensus, and I think they have, so I can apply it to nfsd-fixes now).
Quoted 1 line
2. My fix for the cleanup of allocated memory goes to nfsd-fixes.
And this one hasn't been posted yet, right? Or did I miss it?
Quoted 2 lines
3. I will do the optimization Jeff proposed about
list_headandnfsd4_compoundin a separate patch that goes into nfsd-next.
That should be fine.
Quoted 27 lines
-Dai
Quoted 25 lines
Quoted 14 lines
Just make yourself the patch author and keep my S-o-b line.
Quoted 5 lines
I think we need a bit more cleanup in addition to your patch. When
kmalloc(sizeof(*async_copy->cp_src), ..)ornfs4_init_copy_statefails, theasync_copyis not initialized yet so callingcleanup_async_copycan be a problem.Yeah.
It may even be best to ensure that the
list_headand such are fully initialized for both allocated and embedded struct nfsd4_copy's. You might shave off a few cpu cycles by not doing that, but it makes things more fragile.Quoted 9 lines
Even better, we really ought to split a lot of the fields in
nfsd4_copyinto a different structure (maybenfsd4_async_copy). Trimming down structnfsd4_copywould cut down the size ofnfsd4_compoundas well since it has a union that contains it. I was planning on doing that eventually, but if you want to take that on, then that would be fine too.Signature · 1 line
Jeff Layton <[email protected]>--
Chuck Lever
--
Chuck Lever
Re: [PATCH 2/2] nfsd: clean up potential nfsd_file refcount leaks in COPY codepath
Quoted 203 lines · Chuck Lever III
Quoted 190 lines · Dai Ngo
Quoted 171 lines · Chuck Lever III
Quoted 171 lines · Jeff Layton
Quoted 167 lines · On Sat, 2023-01-21 at 11:50 -0800, [email protected] wrote:
Quoted 166 lines · On 1/21/23 10:56 AM, [email protected] wrote:
Quoted 165 lines · Jeff Layton
Quoted 152 lines · On Thu, 2023-01-19 at 10:38 -0800, [email protected] wrote:
Quoted 53 lines · Jeff Layton
Quoted 36 lines · On Wed, 2023-01-18 at 21:05 -0800, [email protected] wrote:
Quoted 35 lines · Jeff Layton
There are two different flavors of the
nfsd4_copystruct. One is embedded in the compound and is used directly in synchronous copies. The
other is dynamically allocated, refcounted and tracked in the client struture. For the embedded one, the cleanup just involves releasing anynfsd_filesheld on its behalf. For the async one, the cleanup is a bit
more involved, and we need to dequeue it from lists, unhash it, etc.There is at least one potential refcount leak in this code now. If the
kthread_createcall fails, then both the src and dstnfsd_filesin the
originalnfsd4_copyobject are leaked.The cleanup in this codepath is also sort of weird. In the async copy
case, we'll have up to fournfsd_filereferences (src and dst for both
flavors of copy structure). They are both put at the end ofnfsd4_do_async_copy, even though the ones held on behalf of the embedded
one outlive that structure.Change it so that we always clean up the
nfsd_filerefs held by the embedded copy structure beforenfsd4_copyreturns. Reworkcleanup_async_copyto handle both inter and intra copies. Eliminatenfsd4_cleanup_intra_sscsince it now becomes a no-op.Signed-off-by Jeff Layton <[email protected]>--- fs/nfsd/nfs4proc.c | 23 ++++++++++------------- 1 file changed, 10 insertions(+), 13 deletions(-)Quoted 16 lines
fs/nfsd/nfs4proc.c nfsd4_cleanup_inter_ssc(struct · line 1515
Quoted 9 lines
fs/nfsd/nfs4proc.c +0
-1@@ -1512,7 +1512,6 @@ nfsd4_cleanup_inter_ssc(struct nfsd4_ssc_umount_item *nsui, struct file *filp, long timeout = msecs_to_jiffies(nfsd4_ssc_umount_timeout);nfs42_ssc_close (filp); -nfsd_file_put (dst);I think we still need this, in addition to
release_copy_filescalled fromcleanup_async_copy. For async inter-copy, there are 2 reference count added to the destination file, one fromnfsd4_setup_inter_sscand the other one fromdup_copy_fields. The abovenfsd_file_putis for
the count added bydup_copy_fields.Quoted 6 lines
With this patch, the references held by the original copy structure are
put by the call torelease_copy_filesat the end ofnfsd4_copy. That means that the kthread task is only responsible for putting the references held by the (kmalloc'ed)async_copystructure. So, I think this gets thenfsd_filerefcounting right.Yes, I see. One refcount is decremented by
release_copy_filesat end ofnfsd4_copyand another is decremented byrelease_copy_filesincleanup_async_copy.Quoted 88 lines
Quoted 58 lines
Quoted 46 lines
fs/nfsd/nfs4proc.c +8
-9fput (filp);spin_lock (&nn->nfsd_ssc_lock);@@ -1562,13 +1561,6 @@ nfsd4_setup_intra_ssc(struct svc_rqst *rqstp, ©->nf_dst); } -static void -nfsd4_cleanup_intra_ssc (struct nfsd_file *src,struct nfsd_file *dst) -{ -nfsd_file_put (src); -nfsd_file_put (dst); -} -static void nfsd4_cb_offload_release (struct nfsd4_callback *cb) { struct nfsd4_cb_offload *cbo =@@ -1683,12 +1675,18 @@ static void dup_copy_fields(struct nfsd4_copy *src, struct nfsd4_copy *dst) dst->ss_nsui = src->ss_nsui; } +static void release_copy_files(struct nfsd4_copy *copy) +{ +if (copy->nf_src) +nfsd_file_put (copy->nf_src); +if (copy->nf_dst) +nfsd_file_put (copy->nf_dst); +} +static void cleanup_async_copy (struct nfsd4_copy *copy) {nfs4_free_copy_state (copy); -nfsd_file_put (copy->nf_dst); -if (!nfsd4_ssc_is_inter (copy)) -nfsd_file_put (copy->nf_src); +release_copy_files (copy);spin_lock (©->cp_clp->async_lock);list_del (©->copies);
spin_unlock(©->cp_clp->async_lock);fs/nfsd/nfs4proc.c +0
-1@@ -1748,7 +1746,6 @@ static int nfsd4_do_async_copy(void *data) }else { nfserr =nfsd4_do_copy (copy, copy->nf_src->nf_file, copy->nf_dst->nf_file, false); -nfsd4_cleanup_intra_ssc (copy->nf_src, copy->nf_dst); } do_callback:fs/nfsd/nfs4proc.c nfsd4_copy(struct svc_rqst *rqstp, struct · line 1816
Quoted 10 lines
fs/nfsd/nfs4proc.c +1
-1@@ -1811,9 +1808,9 @@ nfsd4_copy(struct svc_rqst *rqstp, structnfsd4_compound_state *cstate, } else { status =nfsd4_do_copy (copy, copy->nf_src->nf_file, copy->nf_dst->nf_file, true); -nfsd4_cleanup_intra_ssc (copy->nf_src, copy->nf_dst); } out: +release_copy_files (copy);return status; out_err:This is unrelated to the reference count issue.
Quoted 5 lines
Here if this is an inter-copy then we need to decrement the reference count of the
nfsd4_ssc_umount_itemso that the vfsmount can be unmounted
later.Oh, I think I see what you mean. Maybe something like the (untested) patch below on top of the original patch would fix that?
fs/nfsd/nfs4proc.c +4
-2@@ -1511,8 +1511,10 @@ nfsd4_cleanup_inter_ssc(struct nfsd4_ssc_umount_item *nsui, struct file *filp, struct nfsd_net *nn = net_generic(dst->nf_net, nfsd_net_id);long timeout =msecs_to_jiffies (nfsd4_ssc_umount_timeout); -nfs42_ssc_close (filp); -fput (filp); +if (filp) { +nfs42_ssc_close (filp); +fput (filp); + } spin_lock(&nn->nfsd_ssc_lo list_del(&nsui->nsui_list);@@ -1813,8 +1815,13 @@ nfsd4_copy(struct svc_rqst *rqstp, structnfsd4_compound_state *cstate,release_copy_files (copy );return status; out_err: -if (async_copy)Quoted 9 lines
fs/nfsd/nfs4proc.c nfsd4_copy(struct svc_rqst *rqstp, struct · line 1821
Quoted 3 lines
fs/nfsd/nfs4proc.c +2
-0+if (async_copy) {cleanup_async_copy (async_copy); +if (nfsd4_ssc_is_inter (async_copy))We don't need to call
nfsd4_cleanup_inter_sscsince the threadnfsd4_do_async_copyhas not started yet so the file is not opened. We just need to dorefcount_dec(©->ss_nsui->nsui_refcnt), unless you want to changenfsd4_cleanup_inter_sscto detect this error condition and only decrement the reference count.Oh yeah, and this would break anyway since the
nsui_listhead is not being initialized. Dai, would you mind spinning up a patch for this since you're more familiar with the cleanup here?Quoted 8 lines
Quoted 6 lines
Quoted 4 lines
Quoted 2 lines
Will do. My patch will only fix the unmount issue. Your patch does the clean up potential
nfsd_filerefcount leaks in COPY codepath.Or do you want me to merge your patch and mine into one?
It probably is best to merge them, since backporters will probably want both patches anyway.
Unless these two changes are somehow interdependent, I'd like to keep them separate. They address two separate issues, yes?
Yes.
Quoted 1 line
Quoted 7 lines
Quoted 2 lines
And -- narrow fixes need to go to nfsd-fixes, but clean-ups can wait for nfsd-next. I'd rather not mix the two types of change.
Ok. Can we do this:
1. Jeff's patch goes to nfsd-fixes since it has the fix for missing reference count.
To make sure I haven't lost track of anything:
Quoted 8 lines
The patch you refer to here is this one:
https://lore.kernel.org/linux-nfs/[email protected]/
Correct?
(I was waiting for Jeff and Olga to come to consensus, and I think they have, so I can apply it to nfsd-fixes now).
Or not...
This one does not apply cleanly to nfsd-fixes, but does apply to nfsd-next. Also, the patch description says "clean up" and does not provide a Fixes:
tag. So, either:
- Jeff needs to test and redrive this patch against nfsd-fixes if we all agree that it fixes a real and urgent bug, not a potential one; or
- I will apply it as it stands to nfsd-next; or
- You were referring to something else in 1. above.
Let me know how you'd both like to proceed.
Quoted 34 lines
Quoted 1 line
2. My fix for the cleanup of allocated memory goes to nfsd-fixes.
And this one hasn't been posted yet, right? Or did I miss it?
Quoted 2 lines
3. I will do the optimization Jeff proposed about
list_headandnfsd4_compoundin a separate patch that goes into nfsd-next.That should be fine.
Quoted 23 lines
-Dai
Quoted 21 lines
Quoted 21 lines
Just make yourself the patch author and keep my S-o-b line.
Quoted 5 lines
I think we need a bit more cleanup in addition to your patch. When
kmalloc(sizeof(*async_copy->cp_src), ..)ornfs4_init_copy_statefails, theasync_copyis not initialized yet so callingcleanup_async_copycan be a problem.Yeah.
It may even be best to ensure that the
list_headand such are fully initialized for both allocated and embedded struct nfsd4_copy's. You might shave off a few cpu cycles by not doing that, but it makes things more fragile.Even better, we really ought to split a lot of the fields in
nfsd4_copyinto a different structure (maybenfsd4_async_copy). Trimming down structnfsd4_copywould cut down the size ofnfsd4_compoundas well since it has a union that contains it. I was planning on doing that eventually, but if you want to take that on, then that would be fine too.
Quoted 10 lines
Quoted 4 lines
Quoted 4 lines
Quoted 2 lines
Signature · 1 line
Jeff Layton <[email protected]>--
Chuck Lever--
Chuck Lever
--
Chuck Lever
Jeff Layton
maintainerRe: [PATCH 2/2] nfsd: clean up potential nfsd_file refcount leaks in COPY codepath
Quoted 217 lines · Chuck Lever III
Quoted 202 lines · Chuck Lever III
Quoted 190 lines · Dai Ngo
Quoted 171 lines · Chuck Lever III
Quoted 171 lines · Jeff Layton
Quoted 167 lines · On Sat, 2023-01-21 at 11:50 -0800, [email protected] wrote:
Quoted 166 lines · On 1/21/23 10:56 AM, [email protected] wrote:
Quoted 165 lines · Jeff Layton
Quoted 152 lines · On Thu, 2023-01-19 at 10:38 -0800, [email protected] wrote:
Quoted 53 lines · Jeff Layton
Quoted 36 lines · On Wed, 2023-01-18 at 21:05 -0800, [email protected] wrote:
Quoted 35 lines · Jeff Layton
There are two different flavors of the
nfsd4_copystruct. One is embedded in the compound and is used directly in synchronous copies. The
other is dynamically allocated, refcounted and tracked in the client struture. For the embedded one, the cleanup just involves releasing anynfsd_filesheld on its behalf. For the async one, the cleanup is a bit
more involved, and we need to dequeue it from lists, unhash it, etc.There is at least one potential refcount leak in this code now. If the
kthread_createcall fails, then both the src and dstnfsd_filesin the
originalnfsd4_copyobject are leaked.The cleanup in this codepath is also sort of weird. In the async copy
case, we'll have up to fournfsd_filereferences (src and dst for both
flavors of copy structure). They are both put at the end ofnfsd4_do_async_copy, even though the ones held on behalf of the embedded
one outlive that structure.Change it so that we always clean up the
nfsd_filerefs held by the embedded copy structure beforenfsd4_copyreturns. Reworkcleanup_async_copyto handle both inter and intra copies. Eliminatenfsd4_cleanup_intra_sscsince it now becomes a no-op.Signed-off-by Jeff Layton <[email protected]>--- fs/nfsd/nfs4proc.c | 23 ++++++++++------------- 1 file changed, 10 insertions(+), 13 deletions(-)Quoted 16 lines
fs/nfsd/nfs4proc.c nfsd4_cleanup_inter_ssc(struct · line 1515
Quoted 9 lines
fs/nfsd/nfs4proc.c +0
-1@@ -1512,7 +1512,6 @@ nfsd4_cleanup_inter_ssc(struct nfsd4_ssc_umount_item *nsui, struct file *filp, long timeout = msecs_to_jiffies(nfsd4_ssc_umount_timeout);nfs42_ssc_close (filp); -nfsd_file_put (dst);I think we still need this, in addition to
release_copy_filescalled fromcleanup_async_copy. For async inter-copy, there are 2 reference count added to the destination file, one fromnfsd4_setup_inter_sscand the other one fromdup_copy_fields. The abovenfsd_file_putis for
the count added bydup_copy_fields.Quoted 6 lines
With this patch, the references held by the original copy structure are
put by the call torelease_copy_filesat the end ofnfsd4_copy. That means that the kthread task is only responsible for putting the references held by the (kmalloc'ed)async_copystructure. So, I think this gets thenfsd_filerefcounting right.Yes, I see. One refcount is decremented by
release_copy_filesat end ofnfsd4_copyand another is decremented byrelease_copy_filesincleanup_async_copy.Quoted 88 lines
Quoted 58 lines
Quoted 46 lines
fs/nfsd/nfs4proc.c +8
-9fput (filp);spin_lock (&nn->nfsd_ssc_lock);@@ -1562,13 +1561,6 @@ nfsd4_setup_intra_ssc(struct svc_rqst *rqstp, ©->nf_dst); } -static void -nfsd4_cleanup_intra_ssc (struct nfsd_file *src,struct nfsd_file *dst) -{ -nfsd_file_put (src); -nfsd_file_put (dst); -} -static void nfsd4_cb_offload_release (struct nfsd4_callback *cb) { struct nfsd4_cb_offload *cbo =@@ -1683,12 +1675,18 @@ static void dup_copy_fields(struct nfsd4_copy *src, struct nfsd4_copy *dst) dst->ss_nsui = src->ss_nsui; } +static void release_copy_files(struct nfsd4_copy *copy) +{ +if (copy->nf_src) +nfsd_file_put (copy->nf_src); +if (copy->nf_dst) +nfsd_file_put (copy->nf_dst); +} +static void cleanup_async_copy (struct nfsd4_copy *copy) {nfs4_free_copy_state (copy); -nfsd_file_put (copy->nf_dst); -if (!nfsd4_ssc_is_inter (copy)) -nfsd_file_put (copy->nf_src); +release_copy_files (copy);spin_lock (©->cp_clp->async_lock);list_del (©->copies);
spin_unlock(©->cp_clp->async_lock);fs/nfsd/nfs4proc.c +0
-1@@ -1748,7 +1746,6 @@ static int nfsd4_do_async_copy(void *data) }else { nfserr =nfsd4_do_copy (copy, copy->nf_src->nf_file, copy->nf_dst->nf_file, false); -nfsd4_cleanup_intra_ssc (copy->nf_src, copy->nf_dst); } do_callback:fs/nfsd/nfs4proc.c nfsd4_copy(struct svc_rqst *rqstp, struct · line 1816
Quoted 10 lines
fs/nfsd/nfs4proc.c +1
-1@@ -1811,9 +1808,9 @@ nfsd4_copy(struct svc_rqst *rqstp, structnfsd4_compound_state *cstate, } else { status =nfsd4_do_copy (copy, copy->nf_src->nf_file, copy->nf_dst->nf_file, true); -nfsd4_cleanup_intra_ssc (copy->nf_src, copy->nf_dst); } out: +release_copy_files (copy);return status; out_err:This is unrelated to the reference count issue.
Quoted 5 lines
Here if this is an inter-copy then we need to decrement the reference count of the
nfsd4_ssc_umount_itemso that the vfsmount can be unmounted
later.Oh, I think I see what you mean. Maybe something like the (untested) patch below on top of the original patch would fix that?
fs/nfsd/nfs4proc.c +4
-2@@ -1511,8 +1511,10 @@ nfsd4_cleanup_inter_ssc(struct nfsd4_ssc_umount_item *nsui, struct file *filp, struct nfsd_net *nn = net_generic(dst->nf_net, nfsd_net_id);long timeout =msecs_to_jiffies (nfsd4_ssc_umount_timeout); -nfs42_ssc_close (filp); -fput (filp); +if (filp) { +nfs42_ssc_close (filp); +fput (filp); + } spin_lock(&nn->nfsd_ssc_lo list_del(&nsui->nsui_list);@@ -1813,8 +1815,13 @@ nfsd4_copy(struct svc_rqst *rqstp, structnfsd4_compound_state *cstate,release_copy_files (copy );return status; out_err: -if (async_copy)Quoted 9 lines
fs/nfsd/nfs4proc.c nfsd4_copy(struct svc_rqst *rqstp, struct · line 1821
Quoted 3 lines
fs/nfsd/nfs4proc.c +2
-0+if (async_copy) {cleanup_async_copy (async_copy); +if (nfsd4_ssc_is_inter (async_copy))We don't need to call
nfsd4_cleanup_inter_sscsince the threadnfsd4_do_async_copyhas not started yet so the file is not opened. We just need to dorefcount_dec(©->ss_nsui->nsui_refcnt), unless you want to changenfsd4_cleanup_inter_sscto detect this error condition and only decrement the reference count.Oh yeah, and this would break anyway since the
nsui_listhead is not being initialized. Dai, would you mind spinning up a patch for this since you're more familiar with the cleanup here?Quoted 8 lines
Quoted 6 lines
Quoted 4 lines
Quoted 2 lines
Will do. My patch will only fix the unmount issue. Your patch does the clean up potential
nfsd_filerefcount leaks in COPY codepath.Or do you want me to merge your patch and mine into one?
It probably is best to merge them, since backporters will probably want both patches anyway.
Unless these two changes are somehow interdependent, I'd like to keep them separate. They address two separate issues, yes?
Yes.
Quoted 1 line
Quoted 7 lines
Quoted 2 lines
And -- narrow fixes need to go to nfsd-fixes, but clean-ups can wait for nfsd-next. I'd rather not mix the two types of change.
Ok. Can we do this:
1. Jeff's patch goes to nfsd-fixes since it has the fix for missing reference count.
To make sure I haven't lost track of anything:
Quoted 8 lines
The patch you refer to here is this one:
https://lore.kernel.org/linux-nfs/[email protected]/
Correct?
(I was waiting for Jeff and Olga to come to consensus, and I think they have, so I can apply it to nfsd-fixes now).
Or not...
This one does not apply cleanly to nfsd-fixes, but does apply to nfsd-next. Also, the patch description says "clean up" and does not provide a Fixes:
tag. So, either:
Quoted 9 lines
- Jeff needs to test and redrive this patch against nfsd-fixes if we all agree that it fixes a real and urgent bug, not a potential one; or
- I will apply it as it stands to nfsd-next; or
- You were referring to something else in 1. above.
Let me know how you'd both like to proceed.
I'm fine with nfsd-next here. These are not a bugs that people are going to hit under normal circumstances. It's something we need to fix, but it's not urgent.
Signature · 1 line
Jeff Layton <[email protected]>
Re: [PATCH 2/2] nfsd: clean up potential nfsd_file refcount leaks in COPY codepath
In reply to Chuck Lever IIIQuoted 200 lines · Chuck Lever III
Quoted 190 lines · Dai Ngo
Quoted 171 lines · Chuck Lever III
Quoted 171 lines · Jeff Layton
Quoted 167 lines · On Sat, 2023-01-21 at 11:50 -0800, [email protected] wrote:
Quoted 166 lines · On 1/21/23 10:56 AM, [email protected] wrote:
Quoted 165 lines · Jeff Layton
Quoted 152 lines · On Thu, 2023-01-19 at 10:38 -0800, [email protected] wrote:
Quoted 53 lines · Jeff Layton
Quoted 36 lines · On Wed, 2023-01-18 at 21:05 -0800, [email protected] wrote:
Quoted 35 lines · Jeff Layton
There are two different flavors of the
nfsd4_copystruct. One is embedded in the compound and is used directly in synchronous copies. The
other is dynamically allocated, refcounted and tracked in the client struture. For the embedded one, the cleanup just involves releasing anynfsd_filesheld on its behalf. For the async one, the cleanup is a bit
more involved, and we need to dequeue it from lists, unhash it, etc.There is at least one potential refcount leak in this code now. If the
kthread_createcall fails, then both the src and dstnfsd_filesin the
originalnfsd4_copyobject are leaked.The cleanup in this codepath is also sort of weird. In the async copy
case, we'll have up to fournfsd_filereferences (src and dst for both
flavors of copy structure). They are both put at the end ofnfsd4_do_async_copy, even though the ones held on behalf of the embedded
one outlive that structure.Change it so that we always clean up the
nfsd_filerefs held by the embedded copy structure beforenfsd4_copyreturns. Reworkcleanup_async_copyto handle both inter and intra copies. Eliminatenfsd4_cleanup_intra_sscsince it now becomes a no-op.Signed-off-by Jeff Layton <[email protected]>--- fs/nfsd/nfs4proc.c | 23 ++++++++++------------- 1 file changed, 10 insertions(+), 13 deletions(-)Quoted 16 lines
fs/nfsd/nfs4proc.c nfsd4_cleanup_inter_ssc(struct · line 1515
Quoted 9 lines
fs/nfsd/nfs4proc.c +0
-1@@ -1512,7 +1512,6 @@ nfsd4_cleanup_inter_ssc(struct nfsd4_ssc_umount_item *nsui, struct file *filp, long timeout = msecs_to_jiffies(nfsd4_ssc_umount_timeout);nfs42_ssc_close (filp); -nfsd_file_put (dst);I think we still need this, in addition to
release_copy_filescalled fromcleanup_async_copy. For async inter-copy, there are 2 reference count added to the destination file, one fromnfsd4_setup_inter_sscand the other one fromdup_copy_fields. The abovenfsd_file_putis for
the count added bydup_copy_fields.Quoted 6 lines
With this patch, the references held by the original copy structure are
put by the call torelease_copy_filesat the end ofnfsd4_copy. That means that the kthread task is only responsible for putting the references held by the (kmalloc'ed)async_copystructure. So, I think this gets thenfsd_filerefcounting right.Yes, I see. One refcount is decremented by
release_copy_filesat end ofnfsd4_copyand another is decremented byrelease_copy_filesincleanup_async_copy.Quoted 88 lines
Quoted 58 lines
Quoted 46 lines
fs/nfsd/nfs4proc.c +8
-9fput (filp);spin_lock (&nn->nfsd_ssc_lock);@@ -1562,13 +1561,6 @@ nfsd4_setup_intra_ssc(struct svc_rqst *rqstp, ©->nf_dst); } -static void -nfsd4_cleanup_intra_ssc (struct nfsd_file *src,struct nfsd_file *dst) -{ -nfsd_file_put (src); -nfsd_file_put (dst); -} -static void nfsd4_cb_offload_release (struct nfsd4_callback *cb) { struct nfsd4_cb_offload *cbo =@@ -1683,12 +1675,18 @@ static void dup_copy_fields(struct nfsd4_copy *src, struct nfsd4_copy *dst) dst->ss_nsui = src->ss_nsui; } +static void release_copy_files(struct nfsd4_copy *copy) +{ +if (copy->nf_src) +nfsd_file_put (copy->nf_src); +if (copy->nf_dst) +nfsd_file_put (copy->nf_dst); +} +static void cleanup_async_copy (struct nfsd4_copy *copy) {nfs4_free_copy_state (copy); -nfsd_file_put (copy->nf_dst); -if (!nfsd4_ssc_is_inter (copy)) -nfsd_file_put (copy->nf_src); +release_copy_files (copy);spin_lock (©->cp_clp->async_lock);list_del (©->copies);
spin_unlock(©->cp_clp->async_lock);fs/nfsd/nfs4proc.c +0
-1@@ -1748,7 +1746,6 @@ static int nfsd4_do_async_copy(void *data) }else { nfserr =nfsd4_do_copy (copy, copy->nf_src->nf_file, copy->nf_dst->nf_file, false); -nfsd4_cleanup_intra_ssc (copy->nf_src, copy->nf_dst); } do_callback:fs/nfsd/nfs4proc.c nfsd4_copy(struct svc_rqst *rqstp, struct · line 1816
Quoted 10 lines
fs/nfsd/nfs4proc.c +1
-1@@ -1811,9 +1808,9 @@ nfsd4_copy(struct svc_rqst *rqstp, structnfsd4_compound_state *cstate, } else { status =nfsd4_do_copy (copy, copy->nf_src->nf_file, copy->nf_dst->nf_file, true); -nfsd4_cleanup_intra_ssc (copy->nf_src, copy->nf_dst); } out: +release_copy_files (copy);return status; out_err:This is unrelated to the reference count issue.
Quoted 5 lines
Here if this is an inter-copy then we need to decrement the reference count of the
nfsd4_ssc_umount_itemso that the vfsmount can be unmounted
later.Oh, I think I see what you mean. Maybe something like the (untested) patch below on top of the original patch would fix that?
fs/nfsd/nfs4proc.c +4
-2@@ -1511,8 +1511,10 @@ nfsd4_cleanup_inter_ssc(struct nfsd4_ssc_umount_item *nsui, struct file *filp, struct nfsd_net *nn = net_generic(dst->nf_net, nfsd_net_id);long timeout =msecs_to_jiffies (nfsd4_ssc_umount_timeout); -nfs42_ssc_close (filp); -fput (filp); +if (filp) { +nfs42_ssc_close (filp); +fput (filp); + } spin_lock(&nn->nfsd_ssc_lo list_del(&nsui->nsui_list);@@ -1813,8 +1815,13 @@ nfsd4_copy(struct svc_rqst *rqstp, structnfsd4_compound_state *cstate,release_copy_files (copy );return status; out_err: -if (async_copy)Quoted 9 lines
fs/nfsd/nfs4proc.c nfsd4_copy(struct svc_rqst *rqstp, struct · line 1821
Quoted 3 lines
fs/nfsd/nfs4proc.c +2
-0+if (async_copy) {cleanup_async_copy (async_copy); +if (nfsd4_ssc_is_inter (async_copy))We don't need to call
nfsd4_cleanup_inter_sscsince the threadnfsd4_do_async_copyhas not started yet so the file is not opened. We just need to dorefcount_dec(©->ss_nsui->nsui_refcnt), unless you want to changenfsd4_cleanup_inter_sscto detect this error condition and only decrement the reference count.Oh yeah, and this would break anyway since the
nsui_listhead is not being initialized. Dai, would you mind spinning up a patch for this since you're more familiar with the cleanup here?Quoted 8 lines
Quoted 6 lines
Quoted 4 lines
Quoted 2 lines
Will do. My patch will only fix the unmount issue. Your patch does the clean up potential
nfsd_filerefcount leaks in COPY codepath.Or do you want me to merge your patch and mine into one?
It probably is best to merge them, since backporters will probably want both patches anyway.
Unless these two changes are somehow interdependent, I'd like to keep them separate. They address two separate issues, yes?
Yes.
Quoted 1 line
Quoted 7 lines
Quoted 2 lines
And -- narrow fixes need to go to nfsd-fixes, but clean-ups can wait for nfsd-next. I'd rather not mix the two types of change.
Ok. Can we do this:
1. Jeff's patch goes to nfsd-fixes since it has the fix for missing reference count.
To make sure I haven't lost track of anything:
Quoted 8 lines
The patch you refer to here is this one:
https://lore.kernel.org/linux-nfs/[email protected]/
Correct?
(I was waiting for Jeff and Olga to come to consensus, and I think they have, so I can apply it to nfsd-fixes now).
Sorry folks but I got a bit lost in the thread. I thought Dai pointed out that we can't remove the put from the nfsd4_cleanup_inter_ssc() because that's the put for the copied structure and not the original file references which is what Jeff's patch is trying to address.
Quoted 44 lines
Quoted 1 line
2. My fix for the cleanup of allocated memory goes to nfsd-fixes.
And this one hasn't been posted yet, right? Or did I miss it?
Quoted 2 lines
3. I will do the optimization Jeff proposed about
list_headandnfsd4_compoundin a separate patch that goes into nfsd-next.That should be fine.
Quoted 27 lines
-Dai
Quoted 25 lines
Quoted 21 lines
Just make yourself the patch author and keep my S-o-b line.
Quoted 5 lines
I think we need a bit more cleanup in addition to your patch. When
kmalloc(sizeof(*async_copy->cp_src), ..)ornfs4_init_copy_statefails, theasync_copyis not initialized yet so callingcleanup_async_copycan be a problem.Yeah.
It may even be best to ensure that the
list_headand such are fully initialized for both allocated and embedded struct nfsd4_copy's. You might shave off a few cpu cycles by not doing that, but it makes things more fragile.Even better, we really ought to split a lot of the fields in
nfsd4_copyinto a different structure (maybenfsd4_async_copy). Trimming down structnfsd4_copywould cut down the size ofnfsd4_compoundas well since it has a union that contains it. I was planning on doing that eventually, but if you want to take that on, then that would be fine too.Quoted 2 lines
--
Jeff Layton <[email protected]>--
Chuck Lever--
Chuck Lever
Jeff Layton
maintainerRe: [PATCH 2/2] nfsd: clean up potential nfsd_file refcount leaks in COPY codepath
Quoted 207 lines · Olga Kornievskaia
Quoted 206 lines · Chuck Lever III
Quoted 190 lines · Dai Ngo
Quoted 171 lines · Chuck Lever III
Quoted 171 lines · Jeff Layton
Quoted 167 lines · On Sat, 2023-01-21 at 11:50 -0800, [email protected] wrote:
Quoted 166 lines · On 1/21/23 10:56 AM, [email protected] wrote:
Quoted 165 lines · Jeff Layton
Quoted 152 lines · On Thu, 2023-01-19 at 10:38 -0800, [email protected] wrote:
Quoted 53 lines · Jeff Layton
Quoted 36 lines · On Wed, 2023-01-18 at 21:05 -0800, [email protected] wrote:
Quoted 35 lines · Jeff Layton
There are two different flavors of the
nfsd4_copystruct. One is embedded in the compound and is used directly in synchronous copies. The
other is dynamically allocated, refcounted and tracked in the client struture. For the embedded one, the cleanup just involves releasing anynfsd_filesheld on its behalf. For the async one, the cleanup is a bit
more involved, and we need to dequeue it from lists, unhash it, etc.There is at least one potential refcount leak in this code now. If the
kthread_createcall fails, then both the src and dstnfsd_filesin the
originalnfsd4_copyobject are leaked.The cleanup in this codepath is also sort of weird. In the async copy
case, we'll have up to fournfsd_filereferences (src and dst for both
flavors of copy structure). They are both put at the end ofnfsd4_do_async_copy, even though the ones held on behalf of the embedded
one outlive that structure.Change it so that we always clean up the
nfsd_filerefs held by the embedded copy structure beforenfsd4_copyreturns. Reworkcleanup_async_copyto handle both inter and intra copies. Eliminatenfsd4_cleanup_intra_sscsince it now becomes a no-op.Signed-off-by Jeff Layton <[email protected]>--- fs/nfsd/nfs4proc.c | 23 ++++++++++------------- 1 file changed, 10 insertions(+), 13 deletions(-)Quoted 16 lines
fs/nfsd/nfs4proc.c nfsd4_cleanup_inter_ssc(struct · line 1515
Quoted 9 lines
fs/nfsd/nfs4proc.c +0
-1@@ -1512,7 +1512,6 @@ nfsd4_cleanup_inter_ssc(struct nfsd4_ssc_umount_item *nsui, struct file *filp, long timeout = msecs_to_jiffies(nfsd4_ssc_umount_timeout);nfs42_ssc_close (filp); -nfsd_file_put (dst);I think we still need this, in addition to
release_copy_filescalled fromcleanup_async_copy. For async inter-copy, there are 2 reference count added to the destination file, one fromnfsd4_setup_inter_sscand the other one fromdup_copy_fields. The abovenfsd_file_putis for
the count added bydup_copy_fields.Quoted 6 lines
With this patch, the references held by the original copy structure are
put by the call torelease_copy_filesat the end ofnfsd4_copy. That means that the kthread task is only responsible for putting the references held by the (kmalloc'ed)async_copystructure. So, I think this gets thenfsd_filerefcounting right.Yes, I see. One refcount is decremented by
release_copy_filesat end ofnfsd4_copyand another is decremented byrelease_copy_filesincleanup_async_copy.Quoted 88 lines
Quoted 58 lines
Quoted 46 lines
fs/nfsd/nfs4proc.c +8
-9fput (filp);spin_lock (&nn->nfsd_ssc_lock);@@ -1562,13 +1561,6 @@ nfsd4_setup_intra_ssc(struct svc_rqst *rqstp, ©->nf_dst); } -static void -nfsd4_cleanup_intra_ssc (struct nfsd_file *src,struct nfsd_file *dst) -{ -nfsd_file_put (src); -nfsd_file_put (dst); -} -static void nfsd4_cb_offload_release (struct nfsd4_callback *cb) { struct nfsd4_cb_offload *cbo =@@ -1683,12 +1675,18 @@ static void dup_copy_fields(struct nfsd4_copy *src, struct nfsd4_copy *dst) dst->ss_nsui = src->ss_nsui; } +static void release_copy_files(struct nfsd4_copy *copy) +{ +if (copy->nf_src) +nfsd_file_put (copy->nf_src); +if (copy->nf_dst) +nfsd_file_put (copy->nf_dst); +} +static void cleanup_async_copy (struct nfsd4_copy *copy) {nfs4_free_copy_state (copy); -nfsd_file_put (copy->nf_dst); -if (!nfsd4_ssc_is_inter (copy)) -nfsd_file_put (copy->nf_src); +release_copy_files (copy);spin_lock (©->cp_clp->async_lock);list_del (©->copies);
spin_unlock(©->cp_clp->async_lock);fs/nfsd/nfs4proc.c +0
-1@@ -1748,7 +1746,6 @@ static int nfsd4_do_async_copy(void *data) }else { nfserr =nfsd4_do_copy (copy, copy->nf_src->nf_file, copy->nf_dst->nf_file, false); -nfsd4_cleanup_intra_ssc (copy->nf_src, copy->nf_dst); } do_callback:fs/nfsd/nfs4proc.c nfsd4_copy(struct svc_rqst *rqstp, struct · line 1816
Quoted 10 lines
fs/nfsd/nfs4proc.c +1
-1@@ -1811,9 +1808,9 @@ nfsd4_copy(struct svc_rqst *rqstp, structnfsd4_compound_state *cstate, } else { status =nfsd4_do_copy (copy, copy->nf_src->nf_file, copy->nf_dst->nf_file, true); -nfsd4_cleanup_intra_ssc (copy->nf_src, copy->nf_dst); } out: +release_copy_files (copy);return status; out_err:This is unrelated to the reference count issue.
Quoted 5 lines
Here if this is an inter-copy then we need to decrement the reference count of the
nfsd4_ssc_umount_itemso that the vfsmount can be unmounted
later.Oh, I think I see what you mean. Maybe something like the (untested) patch below on top of the original patch would fix that?
fs/nfsd/nfs4proc.c +4
-2@@ -1511,8 +1511,10 @@ nfsd4_cleanup_inter_ssc(struct nfsd4_ssc_umount_item *nsui, struct file *filp, struct nfsd_net *nn = net_generic(dst->nf_net, nfsd_net_id);long timeout =msecs_to_jiffies (nfsd4_ssc_umount_timeout); -nfs42_ssc_close (filp); -fput (filp); +if (filp) { +nfs42_ssc_close (filp); +fput (filp); + } spin_lock(&nn->nfsd_ssc_lo list_del(&nsui->nsui_list);@@ -1813,8 +1815,13 @@ nfsd4_copy(struct svc_rqst *rqstp, structnfsd4_compound_state *cstate,release_copy_files (copy );return status; out_err: -if (async_copy)Quoted 9 lines
fs/nfsd/nfs4proc.c nfsd4_copy(struct svc_rqst *rqstp, struct · line 1821
Quoted 3 lines
fs/nfsd/nfs4proc.c +2
-0+if (async_copy) {cleanup_async_copy (async_copy); +if (nfsd4_ssc_is_inter (async_copy))We don't need to call
nfsd4_cleanup_inter_sscsince the threadnfsd4_do_async_copyhas not started yet so the file is not opened. We just need to dorefcount_dec(©->ss_nsui->nsui_refcnt), unless you want to changenfsd4_cleanup_inter_sscto detect this error condition and only decrement the reference count.Oh yeah, and this would break anyway since the
nsui_listhead is not being initialized. Dai, would you mind spinning up a patch for this since you're more familiar with the cleanup here?Quoted 8 lines
Quoted 6 lines
Quoted 4 lines
Quoted 2 lines
Will do. My patch will only fix the unmount issue. Your patch does the clean up potential
nfsd_filerefcount leaks in COPY codepath.Or do you want me to merge your patch and mine into one?
It probably is best to merge them, since backporters will probably want both patches anyway.
Unless these two changes are somehow interdependent, I'd like to keep them separate. They address two separate issues, yes?
Yes.
Quoted 1 line
Quoted 7 lines
Quoted 2 lines
And -- narrow fixes need to go to nfsd-fixes, but clean-ups can wait for nfsd-next. I'd rather not mix the two types of change.
Ok. Can we do this:
1. Jeff's patch goes to nfsd-fixes since it has the fix for missing reference count.
To make sure I haven't lost track of anything:
The patch you refer to here is this one:
https://lore.kernel.org/linux-nfs/[email protected]/
Correct?
Quoted 8 lines
Quoted 2 lines
(I was waiting for Jeff and Olga to come to consensus, and I think they have, so I can apply it to nfsd-fixes now).
Sorry folks but I got a bit lost in the thread. I thought Dai pointed out that we can't remove the put from the
nfsd4_cleanup_inter_ssc()because that's the put for the copied structure and not the original file references which is what Jeff's patch is trying to address.
...and then I replied:
With this patch, the references held by the original copy structure are put by the call to release_copy_files at the end of nfsd4_copy. That means that the kthread task is only responsible for putting the references held by the (kmalloc'ed) async_copy structure. So, I think this gets the nfsd_file refcounting right.
Quoted 45 lines
Quoted 45 lines
Quoted 2 lines
2. My fix for the cleanup of allocated memory goes to nfsd-fixes.
And this one hasn't been posted yet, right? Or did I miss it?
Quoted 2 lines
3. I will do the optimization Jeff proposed about
list_headandnfsd4_compoundin a separate patch that goes into nfsd-next.That should be fine.
Quoted 27 lines
-Dai
Quoted 25 lines
Quoted 21 lines
Just make yourself the patch author and keep my S-o-b line.
Quoted 5 lines
I think we need a bit more cleanup in addition to your patch. When
kmalloc(sizeof(*async_copy->cp_src), ..)ornfs4_init_copy_statefails, theasync_copyis not initialized yet so callingcleanup_async_copycan be a problem.Yeah.
It may even be best to ensure that the
list_headand such are fully initialized for both allocated and embedded struct nfsd4_copy's. You might shave off a few cpu cycles by not doing that, but it makes things more fragile.Even better, we really ought to split a lot of the fields in
nfsd4_copyinto a different structure (maybenfsd4_async_copy). Trimming down structnfsd4_copywould cut down the size ofnfsd4_compoundas well since it has a union that contains it. I was planning on doing that eventually, but if you want to take that on, then that would be fine too.Quoted 2 lines
--
Jeff Layton <[email protected]>--
Chuck Lever--
Chuck Lever
Signature · 1 line
Jeff Layton <[email protected]>
[email protected]
reviewerRe: [PATCH 2/2] nfsd: clean up potential nfsd_file refcount leaks in COPY codepath
In reply to Chuck Lever IIIQuoted 197 lines · Chuck Lever III
Quoted 183 lines · Dai Ngo
Quoted 170 lines · Chuck Lever III
Quoted 170 lines · Jeff Layton
Quoted 167 lines · On Sat, 2023-01-21 at 11:50 -0800, [email protected] wrote:
Quoted 166 lines · On 1/21/23 10:56 AM, [email protected] wrote:
Quoted 165 lines · Jeff Layton
Quoted 152 lines · On Thu, 2023-01-19 at 10:38 -0800, [email protected] wrote:
Quoted 53 lines · Jeff Layton
Quoted 36 lines · On Wed, 2023-01-18 at 21:05 -0800, [email protected] wrote:
Quoted 35 lines · Jeff Layton
There are two different flavors of the
nfsd4_copystruct. One is embedded in the compound and is used directly in synchronous copies. The
other is dynamically allocated, refcounted and tracked in the client struture. For the embedded one, the cleanup just involves releasing anynfsd_filesheld on its behalf. For the async one, the cleanup is a bit
more involved, and we need to dequeue it from lists, unhash it, etc.There is at least one potential refcount leak in this code now. If the
kthread_createcall fails, then both the src and dstnfsd_filesin the
originalnfsd4_copyobject are leaked.The cleanup in this codepath is also sort of weird. In the async copy
case, we'll have up to fournfsd_filereferences (src and dst for both
flavors of copy structure). They are both put at the end ofnfsd4_do_async_copy, even though the ones held on behalf of the embedded
one outlive that structure.Change it so that we always clean up the
nfsd_filerefs held by the embedded copy structure beforenfsd4_copyreturns. Reworkcleanup_async_copyto handle both inter and intra copies. Eliminatenfsd4_cleanup_intra_sscsince it now becomes a no-op.Signed-off-by Jeff Layton <[email protected]>--- fs/nfsd/nfs4proc.c | 23 ++++++++++------------- 1 file changed, 10 insertions(+), 13 deletions(-)Quoted 16 lines
fs/nfsd/nfs4proc.c nfsd4_cleanup_inter_ssc(struct · line 1515
Quoted 9 lines
fs/nfsd/nfs4proc.c +0
-1@@ -1512,7 +1512,6 @@ nfsd4_cleanup_inter_ssc(struct nfsd4_ssc_umount_item *nsui, struct file *filp, long timeout = msecs_to_jiffies(nfsd4_ssc_umount_timeout);nfs42_ssc_close (filp); -nfsd_file_put (dst);I think we still need this, in addition to
release_copy_filescalled fromcleanup_async_copy. For async inter-copy, there are 2 reference count added to the destination file, one fromnfsd4_setup_inter_sscand the other one fromdup_copy_fields. The abovenfsd_file_putis for
the count added bydup_copy_fields.Quoted 6 lines
With this patch, the references held by the original copy structure are
put by the call torelease_copy_filesat the end ofnfsd4_copy. That means that the kthread task is only responsible for putting the references held by the (kmalloc'ed)async_copystructure. So, I think this gets thenfsd_filerefcounting right.Yes, I see. One refcount is decremented by
release_copy_filesat end ofnfsd4_copyand another is decremented byrelease_copy_filesincleanup_async_copy.Quoted 88 lines
Quoted 58 lines
Quoted 46 lines
fs/nfsd/nfs4proc.c +8
-9fput (filp);spin_lock (&nn->nfsd_ssc_lock);@@ -1562,13 +1561,6 @@ nfsd4_setup_intra_ssc(struct svc_rqst *rqstp, ©->nf_dst); } -static void -nfsd4_cleanup_intra_ssc (struct nfsd_file *src,struct nfsd_file *dst) -{ -nfsd_file_put (src); -nfsd_file_put (dst); -} -static void nfsd4_cb_offload_release (struct nfsd4_callback *cb) { struct nfsd4_cb_offload *cbo =@@ -1683,12 +1675,18 @@ static void dup_copy_fields(struct nfsd4_copy *src, struct nfsd4_copy *dst) dst->ss_nsui = src->ss_nsui; } +static void release_copy_files(struct nfsd4_copy *copy) +{ +if (copy->nf_src) +nfsd_file_put (copy->nf_src); +if (copy->nf_dst) +nfsd_file_put (copy->nf_dst); +} +static void cleanup_async_copy (struct nfsd4_copy *copy) {nfs4_free_copy_state (copy); -nfsd_file_put (copy->nf_dst); -if (!nfsd4_ssc_is_inter (copy)) -nfsd_file_put (copy->nf_src); +release_copy_files (copy);spin_lock (©->cp_clp->async_lock);list_del (©->copies);
spin_unlock(©->cp_clp->async_lock);fs/nfsd/nfs4proc.c +0
-1@@ -1748,7 +1746,6 @@ static int nfsd4_do_async_copy(void *data) }else { nfserr =nfsd4_do_copy (copy, copy->nf_src->nf_file, copy->nf_dst->nf_file, false); -nfsd4_cleanup_intra_ssc (copy->nf_src, copy->nf_dst); } do_callback:fs/nfsd/nfs4proc.c nfsd4_copy(struct svc_rqst *rqstp, struct · line 1816
Quoted 10 lines
fs/nfsd/nfs4proc.c +1
-1@@ -1811,9 +1808,9 @@ nfsd4_copy(struct svc_rqst *rqstp, structnfsd4_compound_state *cstate, } else { status =nfsd4_do_copy (copy, copy->nf_src->nf_file, copy->nf_dst->nf_file, true); -nfsd4_cleanup_intra_ssc (copy->nf_src, copy->nf_dst); } out: +release_copy_files (copy);return status; out_err:This is unrelated to the reference count issue.
Quoted 5 lines
Here if this is an inter-copy then we need to decrement the reference count of the
nfsd4_ssc_umount_itemso that the vfsmount can be unmounted
later.Oh, I think I see what you mean. Maybe something like the (untested) patch below on top of the original patch would fix that?
fs/nfsd/nfs4proc.c +4
-2@@ -1511,8 +1511,10 @@ nfsd4_cleanup_inter_ssc(struct nfsd4_ssc_umount_item *nsui, struct file *filp, struct nfsd_net *nn = net_generic(dst->nf_net, nfsd_net_id);long timeout =msecs_to_jiffies (nfsd4_ssc_umount_timeout); -nfs42_ssc_close (filp); -fput (filp); +if (filp) { +nfs42_ssc_close (filp); +fput (filp); + } spin_lock(&nn->nfsd_ssc_lo list_del(&nsui->nsui_list);@@ -1813,8 +1815,13 @@ nfsd4_copy(struct svc_rqst *rqstp, structnfsd4_compound_state *cstate,release_copy_files (copy );return status; out_err: -if (async_copy)Quoted 9 lines
fs/nfsd/nfs4proc.c nfsd4_copy(struct svc_rqst *rqstp, struct · line 1821
Quoted 3 lines
fs/nfsd/nfs4proc.c +2
-0+if (async_copy) {cleanup_async_copy (async_copy); +if (nfsd4_ssc_is_inter (async_copy))We don't need to call
nfsd4_cleanup_inter_sscsince the threadnfsd4_do_async_copyhas not started yet so the file is not opened. We just need to dorefcount_dec(©->ss_nsui->nsui_refcnt), unless you want to changenfsd4_cleanup_inter_sscto detect this error condition and only decrement the reference count.Oh yeah, and this would break anyway since the
nsui_listhead is not being initialized. Dai, would you mind spinning up a patch for this since you're more familiar with the cleanup here?Quoted 8 lines
Quoted 6 lines
Quoted 4 lines
Quoted 2 lines
Will do. My patch will only fix the unmount issue. Your patch does the clean up potential
nfsd_filerefcount leaks in COPY codepath.Or do you want me to merge your patch and mine into one?
It probably is best to merge them, since backporters will probably want both patches anyway.
Unless these two changes are somehow interdependent, I'd like to keep them separate. They address two separate issues, yes?
Quoted 8 lines
Yes.
Quoted 2 lines
And -- narrow fixes need to go to nfsd-fixes, but clean-ups can wait for nfsd-next. I'd rather not mix the two types of change.
Ok. Can we do this:
1. Jeff's patch goes to nfsd-fixes since it has the fix for missing reference count.
To make sure I haven't lost track of anything:
The patch you refer to here is this one:
Quoted 8 lines
Correct?
(I was waiting for Jeff and Olga to come to consensus, and I think they have, so I can apply it to nfsd-fixes now).
Quoted 1 line
2. My fix for the cleanup of allocated memory goes to nfsd-fixes.
And this one hasn't been posted yet, right? Or did I miss it?
I will post this patch soon.
Quoted 5 lines
Quoted 4 lines
3. I will do the optimization Jeff proposed about
list_headandnfsd4_compoundin a separate patch that goes into nfsd-next.That should be fine.
Thanks,
-Dai
Quoted 34 lines
Quoted 29 lines
-Dai
Quoted 25 lines
Quoted 14 lines
Just make yourself the patch author and keep my S-o-b line.
Quoted 5 lines
I think we need a bit more cleanup in addition to your patch. When
kmalloc(sizeof(*async_copy->cp_src), ..)ornfs4_init_copy_statefails, theasync_copyis not initialized yet so callingcleanup_async_copycan be a problem.Yeah.
It may even be best to ensure that the
list_headand such are fully initialized for both allocated and embedded struct nfsd4_copy's. You might shave off a few cpu cycles by not doing that, but it makes things more fragile.Quoted 9 lines
Even better, we really ought to split a lot of the fields in
nfsd4_copyinto a different structure (maybenfsd4_async_copy). Trimming down structnfsd4_copywould cut down the size ofnfsd4_compoundas well since it has a union that contains it. I was planning on doing that eventually, but if you want to take that on, then that would be fine too.Signature · 1 line
Jeff Layton <[email protected]>--
Chuck Lever--
Chuck Lever