Address CVE-2024-49974

PATCH In review [email protected] 4 participants 10 messages Nov 18, 2024 – Nov 19, 2024

Subsystems (3)

Stable branch NFS client NFS server

lore

Share & tools

lore

[email protected]

author maintainer

2024-11-18 21:19 UTC #0 lore raw

cover Address CVE-2024-49974

From: Chuck Lever <[email protected]>

Backport the set of upstream patches that cap the number of concurrent background NFSv4.2 COPY operations.

Chuck Lever (4):
NFSD: Async COPY result needs to return a write verifier
NFSD: Limit the number of concurrent async COPY operations
NFSD: Initialize struct nfsd4_copy earlier
NFSD: Never decrement pending_async_copies on error

Dai Ngo (1):
NFSD: initialize copy->cp_clp early in nfsd4_copy for use by trace
point

 fs/nfsd/netns.h     |  1 +
 fs/nfsd/nfs4proc.c  | 36 +++++++++++++++++-------------------
 fs/nfsd/nfs4state.c |  1 +
 fs/nfsd/xdr4.h      |  1 +
 4 files changed, 20 insertions(+), 19 deletions(-)

Signature · 1 line

2.47.0

[email protected]

author maintainer

2024-11-18 21:19 UTC #1 lore raw

PATCH 1/5 NFSD: initialize copy->cp_clp early in nfsd4_copy for use by trace point

From: Dai Ngo <[email protected]>

[ Upstream commit 15d1975b7279693d6f09398e0e2e31aca2310275 ]

Prepare for adding server copy trace points.

Signed-off-by Dai Ngo <[email protected]>

Tested-by Chen Hanxiao <[email protected]>

Stable-dep-of: 9ed666eba4e0 ("NFSD: Async COPY result needs to return a write verifier")

Signed-off-by Chuck Lever <[email protected]>

---
 fs/nfsd/nfs4proc.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

fs/nfsd/nfs4proc.c +1 -1

index df9dbd93663e..50f17cee8bcf 100644
@@ -1768,6 +1768,7 @@ nfsd4_copy(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 	__be32 status;
 	struct nfsd4_copy *async_copy = NULL;
 
+	copy->cp_clp = cstate->clp;
 	if (nfsd4_ssc_is_inter(copy)) {
 		if (!inter_copy_offload_enable || nfsd4_copy_is_sync(copy)) {
 			status = nfserr_notsupp;

@@ -1782,7 +1783,6 @@ nfsd4_copy(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 			return status;
 	}
 
-	copy->cp_clp = cstate->clp;
 	memcpy(&copy->fh, &cstate->current_fh.fh_handle,
 		sizeof(struct knfsd_fh));
 	if (nfsd4_copy_is_async(copy)) {

Signature · 1 line

2.47.0

Chuck Lever

2024-11-19 14:06 UTC #2 lore raw

Re: [PATCH 6.1 1/5] NFSD: initialize copy->cp_clp early in nfsd4_copy for use by trace point

Quoted 9 lines · Sasha Levin

[ Sasha's backport helper bot ]
Hi,
The upstream commit SHA1 provided is correct: 15d1975b7279693d6f09398e0e2e31aca2310275
WARNING: Author mismatch between patch and upstream commit:
Backport author: [email protected]
Commit author: Dai Ngo <[email protected]>

Is this a bug in my backport script? Should patches backported to LTS retain the upstream patch author, or should they be From:
the backporter? If the former, I can adjust my scripts.

Quoted 58 lines

Commit in newer trees:

|-----------------|----------------------------------------------|
| 6.11.y          |  Present (exact SHA1)                        |
| 6.6.y           |  Not found                                   |
| 6.1.y           |  Not found                                   |
|-----------------|----------------------------------------------|

Note: The patch differs from the upstream commit:
---

tmp/tmp.6a2P3hH3a2 renamed from - +9 -4

@@ -1,17 +1,20 @@
+[ Upstream commit 15d1975b7279693d6f09398e0e2e31aca2310275 ]
+
 Prepare for adding server copy trace points.
 
 Signed-off-by: Dai Ngo <[email protected]>
 Tested-by: Chen Hanxiao <[email protected]>
+Stable-dep-of: 9ed666eba4e0 ("NFSD: Async COPY result needs to return a write verifier")
 Signed-off-by: Chuck Lever <[email protected]>
 ---
  fs/nfsd/nfs4proc.c | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
 
 diff --git a/fs/nfsd/nfs4proc.c b/fs/nfsd/nfs4proc.c
-index 4199ede0583c7..c27f2fdcea32c 100644
+index df9dbd93663e..50f17cee8bcf 100644
 --- a/fs/nfsd/nfs4proc.c
 +++ b/fs/nfsd/nfs4proc.c
-@@ -1798,6 +1798,7 @@ nfsd4_copy(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
+@@ -1768,6 +1768,7 @@ nfsd4_copy(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
  	__be32 status;
  	struct nfsd4_copy *async_copy = NULL;

@@ -19,7 +22,7 @@
  	if (nfsd4_ssc_is_inter(copy)) {
  		if (!inter_copy_offload_enable || nfsd4_copy_is_sync(copy)) {
  			status = nfserr_notsupp;
-@@ -1812,7 +1813,6 @@ nfsd4_copy(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
+@@ -1782,7 +1783,6 @@ nfsd4_copy(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
  			return status;
  	}

@@ -27,3 +30,6 @@
  	memcpy(&copy->fh, &cstate->current_fh.fh_handle,
  		sizeof(struct knfsd_fh));
  	if (nfsd4_copy_is_async(copy)) {
+-- 
+2.47.0
+
---

Results of testing on various branches:

| Branch                    | Patch Apply | Build Test |
|---------------------------|-------------|------------|
| stable/linux-6.1.y        |  Success    |  Success   |

Signature · 1 line

Chuck Lever

Greg KH

maintainer

2024-11-19 14:14 UTC #3 lore raw

Re: [PATCH 6.1 1/5] NFSD: initialize copy->cp_clp early in nfsd4_copy for use by trace point

Quoted 7 lines · Chuck Lever

Quoted 6 lines · Sasha Levin
[ Sasha's backport helper bot ]
Hi,
The upstream commit SHA1 provided is correct: 15d1975b7279693d6f09398e0e2e31aca2310275

Quoted 7 lines

Quoted 3 lines
WARNING: Author mismatch between patch and upstream commit:
Backport author: [email protected]
Commit author: Dai Ngo <[email protected]>
Is this a bug in my backport script? Should patches backported to LTS retain the upstream patch author, or should they be From:
the backporter? If the former, I can adjust my scripts.

No, this is correct, I think Sasha's scripts are a bit too sensitive here and in a few other places.

thanks,

greg k-h

Sasha Levin

maintainer

2024-11-19 15:30 UTC #4 lore raw

Re: [PATCH 6.1 1/5] NFSD: initialize copy->cp_clp early in nfsd4_copy for use by trace point

Quoted 12 lines · Greg KH

Quoted 11 lines · Chuck Lever
Quoted 9 lines · Sasha Levin
[ Sasha's backport helper bot ]
Hi,
The upstream commit SHA1 provided is correct: 15d1975b7279693d6f09398e0e2e31aca2310275
WARNING: Author mismatch between patch and upstream commit:
Backport author: [email protected]
Commit author: Dai Ngo <[email protected]>

Quoted 6 lines

Quoted 3 lines
Is this a bug in my backport script? Should patches backported to LTS retain the upstream patch author, or should they be From:
the backporter? If the former, I can adjust my scripts.
No, this is correct, I think Sasha's scripts are a bit too sensitive here and in a few other places.

Right, sorry - this is more of an indicator for me to say "this patch was backported by someone who's not the author, review it more carefully".

Signature · 2 lines

Thanks,
Sasha

Chuck Lever III

2024-11-19 15:31 UTC #5 lore raw

Re: [PATCH 6.1 1/5] NFSD: initialize copy->cp_clp early in nfsd4_copy for use by trace point

Quoted 21 lines · Sasha Levin

Quoted 16 lines · Greg KH
Quoted 14 lines · Chuck Lever
Quoted 9 lines · Sasha Levin
[ Sasha's backport helper bot ]
Hi,
The upstream commit SHA1 provided is correct: 15d1975b7279693d6f09398e0e2e31aca2310275
WARNING: Author mismatch between patch and upstream commit:
Backport author: [email protected]
Commit author: Dai Ngo <[email protected]>
Is this a bug in my backport script? Should patches backported to LTS retain the upstream patch author, or should they be From:
the backporter? If the former, I can adjust my scripts.

Quoted 6 lines

Quoted 2 lines
No, this is correct, I think Sasha's scripts are a bit too sensitive here and in a few other places.
Right, sorry - this is more of an indicator for me to say "this patch was backported by someone who's not the author, review it more carefully".

Got it. Sorry for the noise!

--
Chuck Lever

[email protected]

author maintainer

2024-11-18 21:19 UTC #6 lore raw

PATCH 2/5 NFSD: Async COPY result needs to return a write verifier

In reply to [email protected]

From: Chuck Lever <[email protected]>

[ Upstream commit 9ed666eba4e0a2bb8ffaa3739d830b64d4f2aaad ]

Currently, when NFSD handles an asynchronous COPY, it returns a zero write verifier, relying on the subsequent CB_OFFLOAD callback to pass the write verifier and a stable_how4 value to the client.

However, if the CB_OFFLOAD never arrives at the client (for example, if a network partition occurs just as the server sends the CB_OFFLOAD operation), the client will never receive this verifier. Thus, if the client sends a follow-up COMMIT, there is no way for the client to assess the COMMIT result.

The usual recovery for a missing CB_OFFLOAD is for the client to send an OFFLOAD_STATUS operation, but that operation does not carry a write verifier in its result. Neither does it carry a stable_how4 value, so the client /must/ send a COMMIT in this case -- which will always fail because currently there's still no write verifier in the COPY result.

Thus the server needs to return a normal write verifier in its COPY result even if the COPY operation is to be performed asynchronously.

If the server recognizes the callback stateid in subsequent OFFLOAD_STATUS operations, then obviously it has not restarted, and the write verifier the client received in the COPY result is still valid and can be used to assess a COMMIT of the copied data, if one is needed.

Reviewed-by Jeff Layton <[email protected]>

[ cel: adjusted to apply to origin/linux-6.1.y ]

Signed-off-by Chuck Lever <[email protected]>

---
 fs/nfsd/nfs4proc.c | 25 +++++++++----------------
 1 file changed, 9 insertions(+), 16 deletions(-)

fs/nfsd/nfs4proc.c +9 ~~-16~~

index 50f17cee8bcf..b8fb37cc59c4 100644
@@ -717,15 +717,6 @@ nfsd4_access(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 			   &access->ac_supported);
 }
 
-static void gen_boot_verifier(nfs4_verifier *verifier, struct net *net)
-{
-	__be32 *verf = (__be32 *)verifier->data;
-
-	BUILD_BUG_ON(2*sizeof(*verf) != sizeof(verifier->data));
-
-	nfsd_copy_write_verifier(verf, net_generic(net, nfsd_net_id));
-}
-
 static __be32
 nfsd4_commit(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 	     union nfsd4_op_u *u)

@@ -1593,7 +1584,6 @@ static void nfsd4_init_copy_res(struct nfsd4_copy *copy, bool sync)
 		test_bit(NFSD4_COPY_F_COMMITTED, &copy->cp_flags) ?
 			NFS_FILE_SYNC : NFS_UNSTABLE;
 	nfsd4_copy_set_sync(copy, sync);
-	gen_boot_verifier(&copy->cp_res.wr_verifier, copy->cp_clp->net);
 }
 
 static ssize_t _nfsd_copy_file_range(struct nfsd4_copy *copy,

@@ -1764,9 +1754,14 @@ static __be32
 nfsd4_copy(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 		union nfsd4_op_u *u)
 {
-	struct nfsd4_copy *copy = &u->copy;
-	__be32 status;
+	struct nfsd_net *nn = net_generic(SVC_NET(rqstp), nfsd_net_id);
 	struct nfsd4_copy *async_copy = NULL;
+	struct nfsd4_copy *copy = &u->copy;
+	struct nfsd42_write_res *result;
+	__be32 status;
+
+	result = &copy->cp_res;
+	nfsd_copy_write_verifier((__be32 *)&result->wr_verifier.data, nn);
 
 	copy->cp_clp = cstate->clp;
 	if (nfsd4_ssc_is_inter(copy)) {

@@ -1786,8 +1781,6 @@ nfsd4_copy(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 	memcpy(&copy->fh, &cstate->current_fh.fh_handle,
 		sizeof(struct knfsd_fh));
 	if (nfsd4_copy_is_async(copy)) {
-		struct nfsd_net *nn = net_generic(SVC_NET(rqstp), nfsd_net_id);
-
 		status = nfserrno(-ENOMEM);
 		async_copy = kzalloc(sizeof(struct nfsd4_copy), GFP_KERNEL);
 		if (!async_copy)

@@ -1799,8 +1792,8 @@ nfsd4_copy(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 			goto out_err;
 		if (!nfs4_init_copy_state(nn, copy))
 			goto out_err;
-		memcpy(&copy->cp_res.cb_stateid, &copy->cp_stateid.cs_stid,
-			sizeof(copy->cp_res.cb_stateid));
+		memcpy(&result->cb_stateid, &copy->cp_stateid.cs_stid,
+			sizeof(result->cb_stateid));
 		dup_copy_fields(copy, async_copy);
 		async_copy->copy_task = kthread_create(nfsd4_do_async_copy,
 				async_copy, "%s", "copy thread");

Signature · 1 line

2.47.0

[email protected]

author maintainer

2024-11-18 21:19 UTC #7 lore raw

PATCH 3/5 NFSD: Limit the number of concurrent async COPY operations

In reply to [email protected]

From: Chuck Lever <[email protected]>

[ Upstream commit aadc3bbea163b6caaaebfdd2b6c4667fbc726752 ]

Nothing appears to limit the number of concurrent async COPY operations that clients can start. In addition, AFAICT each async COPY can copy an unlimited number of 4MB chunks, so can run for a long time. Thus IMO async COPY can become a DoS vector.

Add a restriction mechanism that bounds the number of concurrent background COPY operations. Start simple and try to be fair -- this patch implements a per-namespace limit.

An async COPY request that occurs while this limit is exceeded gets NFS4ERR_DELAY. The requesting client can choose to send the request again after a delay or fall back to a traditional read/write style copy.

If there is need to make the mechanism more sophisticated, we can visit that in future patches.

Cc [email protected]

Reviewed-by Jeff Layton <[email protected]>

Link https://nvd.nist.gov/vuln/detail/CVE-2024-49974

Signed-off-by Chuck Lever <[email protected]>

---
 fs/nfsd/netns.h     |  1 +
 fs/nfsd/nfs4proc.c  | 11 +++++++++--
 fs/nfsd/nfs4state.c |  1 +
 fs/nfsd/xdr4.h      |  1 +
 4 files changed, 12 insertions(+), 2 deletions(-)

fs/nfsd/netns.h +1 -0

index 548422b24a7d..41c750f34473 100644
@@ -152,6 +152,7 @@ struct nfsd_net {
 	u32		s2s_cp_cl_id;
 	struct idr	s2s_cp_stateids;
 	spinlock_t	s2s_cp_lock;
+	atomic_t	pending_async_copies;
 
 	/*
 	 * Version information

fs/nfsd/nfs4proc.c +9 -2

index b8fb37cc59c4..6637a7ef974b 100644
@@ -1243,6 +1243,7 @@ static void nfs4_put_copy(struct nfsd4_copy *copy)
 {
 	if (!refcount_dec_and_test(&copy->refcount))
 		return;
+	atomic_dec(&copy->cp_nn->pending_async_copies);
 	kfree(copy->cp_src);
 	kfree(copy);
 }

@@ -1781,10 +1782,16 @@ nfsd4_copy(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 	memcpy(&copy->fh, &cstate->current_fh.fh_handle,
 		sizeof(struct knfsd_fh));
 	if (nfsd4_copy_is_async(copy)) {
-		status = nfserrno(-ENOMEM);
 		async_copy = kzalloc(sizeof(struct nfsd4_copy), GFP_KERNEL);
 		if (!async_copy)
 			goto out_err;
+		async_copy->cp_nn = nn;
+		/* Arbitrary cap on number of pending async copy operations */
+		if (atomic_inc_return(&nn->pending_async_copies) >
+				(int)rqstp->rq_pool->sp_nrthreads) {
+			atomic_dec(&nn->pending_async_copies);
+			goto out_err;
+		}
 		INIT_LIST_HEAD(&async_copy->copies);
 		refcount_set(&async_copy->refcount, 1);
 		async_copy->cp_src = kmalloc(sizeof(*async_copy->cp_src), GFP_KERNEL);

@@ -1823,7 +1830,7 @@ nfsd4_copy(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 	}
 	if (async_copy)
 		cleanup_async_copy(async_copy);
-	status = nfserrno(-ENOMEM);
+	status = nfserr_jukebox;
 	goto out;
 }

fs/nfsd/nfs4state.c +1 -0

index 893d099e0099..e195012db48e 100644
@@ -8076,6 +8076,7 @@ static int nfs4_state_create_net(struct net *net)
 	spin_lock_init(&nn->client_lock);
 	spin_lock_init(&nn->s2s_cp_lock);
 	idr_init(&nn->s2s_cp_stateids);
+	atomic_set(&nn->pending_async_copies, 0);
 
 	spin_lock_init(&nn->blocked_locks_lock);
 	INIT_LIST_HEAD(&nn->blocked_locks_lru);

fs/nfsd/xdr4.h +1 -0

index 510978e602da..9bd1ade6ba54 100644
@@ -574,6 +574,7 @@ struct nfsd4_copy {
 	struct nfsd4_ssc_umount_item *ss_nsui;
 	struct nfs_fh		c_fh;
 	nfs4_stateid		stateid;
+	struct nfsd_net		*cp_nn;
 };
 
 static inline void nfsd4_copy_set_sync(struct nfsd4_copy *copy, bool sync)

Signature · 1 line

2.47.0

[email protected]

author maintainer

2024-11-18 21:19 UTC #8 lore raw

PATCH 4/5 NFSD: Initialize struct nfsd4_copy earlier

In reply to [email protected]

From: Chuck Lever <[email protected]>

[ Upstream commit 63fab04cbd0f96191b6e5beedc3b643b01c15889 ]

Ensure the refcount and async_copies fields are initialized early. cleanup_async_copy() will reference these fields if an error occurs in nfsd4_copy(). If they are not correctly initialized, at the very least, a refcount underflow occurs.

Reported-by Olga Kornievskaia <[email protected]>

Fixes aadc3bbea163 ("NFSD: Limit the number of concurrent async COPY operations")

Reviewed-by Jeff Layton <[email protected]>

Tested-by Olga Kornievskaia <[email protected]>

Signed-off-by Chuck Lever <[email protected]>

---
 fs/nfsd/nfs4proc.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

fs/nfsd/nfs4proc.c +2 -2

index 6637a7ef974b..c0f13989bd24 100644
@@ -1786,14 +1786,14 @@ nfsd4_copy(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 		if (!async_copy)
 			goto out_err;
 		async_copy->cp_nn = nn;
+		INIT_LIST_HEAD(&async_copy->copies);
+		refcount_set(&async_copy->refcount, 1);
 		/* Arbitrary cap on number of pending async copy operations */
 		if (atomic_inc_return(&nn->pending_async_copies) >
 				(int)rqstp->rq_pool->sp_nrthreads) {
 			atomic_dec(&nn->pending_async_copies);
 			goto out_err;
 		}
-		INIT_LIST_HEAD(&async_copy->copies);
-		refcount_set(&async_copy->refcount, 1);
 		async_copy->cp_src = kmalloc(sizeof(*async_copy->cp_src), GFP_KERNEL);
 		if (!async_copy->cp_src)
 			goto out_err;

Signature · 1 line

2.47.0

[email protected]

author maintainer

2024-11-18 21:19 UTC #9 lore raw

PATCH 5/5 NFSD: Never decrement pending_async_copies on error

In reply to [email protected]

From: Chuck Lever <[email protected]>

[ Upstream commit 8286f8b622990194207df9ab852e0f87c60d35e9 ]

The error flow in nfsd4_copy() calls cleanup_async_copy(), which already decrements nn->pending_async_copies.

Reported-by Olga Kornievskaia <[email protected]>

Fixes aadc3bbea163 ("NFSD: Limit the number of concurrent async COPY operations")

Signed-off-by Chuck Lever <[email protected]>

---
 fs/nfsd/nfs4proc.c | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

fs/nfsd/nfs4proc.c +1 -3

index c0f13989bd24..0aebb2dc5776 100644
@@ -1790,10 +1790,8 @@ nfsd4_copy(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 		refcount_set(&async_copy->refcount, 1);
 		/* Arbitrary cap on number of pending async copy operations */
 		if (atomic_inc_return(&nn->pending_async_copies) >
-				(int)rqstp->rq_pool->sp_nrthreads) {
-			atomic_dec(&nn->pending_async_copies);
+				(int)rqstp->rq_pool->sp_nrthreads)
 			goto out_err;
-		}
 		async_copy->cp_src = kmalloc(sizeof(*async_copy->cp_src), GFP_KERNEL);
 		if (!async_copy->cp_src)
 			goto out_err;

Signature · 1 line

2.47.0