FAILED: patch "[PATCH] nfsd: fix use-after-free due to delegation race" failed to apply to 4.19-stable tree

PATCH [email protected] 3 participants 4 messages Dec 12, 2021 – Dec 17, 2021

Subsystems (2)

Stable branch NFS server

lore

Share & tools

lore

[email protected]

author maintainer

2021-12-12 13:11 UTC #0 lore raw

The patch below does not apply to the 4.19-stable tree.
If someone wants it applied there, or to any other stable or longterm tree, then please email the backport, including the original git commit id to <[email protected]>.

thanks,

greg k-h

------------------ original commit in Linus's tree ------------------

From 548ec0805c399c65ed66c6641be467f717833ab5 Mon Sep 17 00:00:00 2001
From: "J. Bruce Fields" <[email protected]>
Date: Mon, 29 Nov 2021 15:08:00 -0500
Subject: [PATCH] nfsd: fix use-after-free due to delegation race

A delegation break could arrive as soon as we've called vfs_setlease. A delegation break runs a callback which immediately (in nfsd4_cb_recall_prepare) adds the delegation to del_recall_lru. If we then exit nfs4_set_delegation without hashing the delegation, it will be freed as soon as the callback is done with it, without ever being removed from del_recall_lru.

Symptoms show up later as use-after-free or list corruption warnings, usually in the laundromat thread.

I suspect aba2072f4523 "nfsd: grant read delegations to clients holding writes" made this bug easier to hit, but I looked as far back as v3.0 and it looks to me it already had the same problem. So I'm not sure where the bug was introduced; it may have been there from the beginning.

Cc [email protected]

Signed-off-by J. Bruce Fields <[email protected]>

fs/nfsd/nfs4state.c +7 -2

index bfad94c70b84..1956d377d1a6 100644
@@ -1207,6 +1207,11 @@ hash_delegation_locked(struct nfs4_delegation *dp, struct nfs4_file *fp)
 	return 0;
 }
 
+static bool delegation_hashed(struct nfs4_delegation *dp)
+{
+	return !(list_empty(&dp->dl_perfile));
+}
+
 static bool
 unhash_delegation_locked(struct nfs4_delegation *dp)
 {

@@ -1214,7 +1219,7 @@ unhash_delegation_locked(struct nfs4_delegation *dp)
 
 	lockdep_assert_held(&state_lock);
 
-	if (list_empty(&dp->dl_perfile))
+	if (!delegation_hashed(dp))
 		return false;
 
 	dp->dl_stid.sc_type = NFS4_CLOSED_DELEG_STID;

@@ -4598,7 +4603,7 @@ static void nfsd4_cb_recall_prepare(struct nfsd4_callback *cb)
 	 * queued for a lease break. Don't queue it again.
 	 */
 	spin_lock(&state_lock);
-	if (dp->dl_time == 0) {
+	if (delegation_hashed(dp) && dp->dl_time == 0) {
 		dp->dl_time = ktime_get_boottime_seconds();
 		list_add_tail(&dp->dl_recall_lru, &nn->del_recall_lru);
 	}

Salvatore Bonaccorso

2021-12-16 20:05 UTC #1 lore raw

Hi Bruce, hi Greg,

Quoted 5 lines · On Sun, Dec 12, 2021 at 02:11:30PM +0100, [email protected] wrote:

The patch below does not apply to the 4.19-stable tree.
If someone wants it applied there, or to any other stable or longterm tree, then please email the backport, including the original git commit id to <[email protected]>.

Unless I'm completely wrong this only does not apply to the older series because in 5.6-rc1 we had 20b7d86f29d3 ("nfsd: use boottime for lease expiry calculation") soe there is a minor context change in one hunk.

Attached is a patch accounting for that, Bruce is this okay?

This would apply as well back on top of v4.9.293.

Regards,
Salvatore

From eeeaf376a2efabdad46ce03516552f4290949834 Mon Sep 17 00:00:00 2001
From: "J. Bruce Fields" <[email protected]>
Date: Mon, 29 Nov 2021 15:08:00 -0500
Subject: [PATCH] nfsd: fix use-after-free due to delegation race

commit 548ec0805c399c65ed66c6641be467f717833ab5 upstream.

Symptoms show up later as use-after-free or list corruption warnings, usually in the laundromat thread.

Cc [email protected]

Signed-off-by J. Bruce Fields <[email protected]>

Signed-off-by Greg Kroah-Hartman <[email protected]>

[Salvatore Bonaccorso: Backport for context changes to versions which do not have 20b7d86f29d3 ("nfsd: use boottime for lease expiry calculation")]

Signed-off-by Salvatore Bonaccorso <[email protected]>

---
 fs/nfsd/nfs4state.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

fs/nfsd/nfs4state.c +7 -2

index 655079ae1dd1..dfb2a790efc1 100644
@@ -975,6 +975,11 @@ hash_delegation_locked(struct nfs4_delegation *dp, struct nfs4_file *fp)
 	return 0;
 }
 
+static bool delegation_hashed(struct nfs4_delegation *dp)
+{
+	return !(list_empty(&dp->dl_perfile));
+}
+
 static bool
 unhash_delegation_locked(struct nfs4_delegation *dp)
 {

@@ -982,7 +987,7 @@ unhash_delegation_locked(struct nfs4_delegation *dp)
 
 	lockdep_assert_held(&state_lock);
 
-	if (list_empty(&dp->dl_perfile))
+	if (!delegation_hashed(dp))
 		return false;
 
 	dp->dl_stid.sc_type = NFS4_CLOSED_DELEG_STID;

@@ -3912,7 +3917,7 @@ static void nfsd4_cb_recall_prepare(struct nfsd4_callback *cb)
 	 * queued for a lease break. Don't queue it again.
 	 */
 	spin_lock(&state_lock);
-	if (dp->dl_time == 0) {
+	if (delegation_hashed(dp) && dp->dl_time == 0) {
 		dp->dl_time = get_seconds();
 		list_add_tail(&dp->dl_recall_lru, &nn->del_recall_lru);
 	}

Signature · 1 line

2.34.1

Bruce Fields

2021-12-17 02:42 UTC #2 lore raw

Quoted 6 lines · Salvatore Bonaccorso

Quoted 4 lines · On Sun, Dec 12, 2021 at 02:11:30PM +0100, [email protected] wrote:
The patch below does not apply to the 4.19-stable tree.
If someone wants it applied there, or to any other stable or longterm tree, then please email the backport, including the original git commit id to <[email protected]>.

Quoted 6 lines

Unless I'm completely wrong this only does not apply to the older series because in 5.6-rc1 we had 20b7d86f29d3 ("nfsd: use boottime for lease expiry calculation") soe there is a minor context change in one hunk.
Attached is a patch accounting for that, Bruce is this okay?

Yes, this looks fine to me. Thank you!

--b.

Quoted 75 lines

This would apply as well back on top of v4.9.293.
Regards,
Salvatore
From eeeaf376a2efabdad46ce03516552f4290949834 Mon Sep 17 00:00:00 2001
From: "J. Bruce Fields" <[email protected]>
Date: Mon, 29 Nov 2021 15:08:00 -0500
Subject: [PATCH] nfsd: fix use-after-free due to delegation race
commit 548ec0805c399c65ed66c6641be467f717833ab5 upstream.
A delegation break could arrive as soon as we've called vfs_setlease. A delegation break runs a callback which immediately (in nfsd4_cb_recall_prepare) adds the delegation to del_recall_lru. If we then exit nfs4_set_delegation without hashing the delegation, it will be freed as soon as the callback is done with it, without ever being removed from del_recall_lru.
Symptoms show up later as use-after-free or list corruption warnings, usually in the laundromat thread.
I suspect aba2072f4523 "nfsd: grant read delegations to clients holding writes" made this bug easier to hit, but I looked as far back as v3.0 and it looks to me it already had the same problem. So I'm not sure where the bug was introduced; it may have been there from the beginning.
Cc [email protected]
Signed-off-by J. Bruce Fields <[email protected]>
Signed-off-by Greg Kroah-Hartman <[email protected]>
[Salvatore Bonaccorso: Backport for context changes to versions which do not have 20b7d86f29d3 ("nfsd: use boottime for lease expiry calculation")]
Signed-off-by Salvatore Bonaccorso <[email protected]>
---
 fs/nfsd/nfs4state.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)
fs/nfsd/nfs4state.c +7 -3
index 655079ae1dd1..dfb2a790efc1 100644
@@ -975,6 +975,11 @@ hash_delegation_locked(struct nfs4_delegation *dp, struct nfs4_file *fp)
        return 0;
 }
 
+static bool delegation_hashed(struct nfs4_delegation *dp)
+{
+       return !(list_empty(&dp->dl_perfile));
+}
+
 static bool
 unhash_delegation_locked(struct nfs4_delegation *dp)
 {
@@ -982,7 +987,7 @@ unhash_delegation_locked(struct nfs4_delegation *dp)
 
        lockdep_assert_held(&state_lock);
 
-       if (list_empty(&dp->dl_perfile))
+       if (!delegation_hashed(dp))
                return false;
 
        dp->dl_stid.sc_type = NFS4_CLOSED_DELEG_STID;
@@ -3912,7 +3917,7 @@ static void nfsd4_cb_recall_prepare(struct nfsd4_callback *cb)
         * queued for a lease break. Don't queue it again.
         */
        spin_lock(&state_lock);
-       if (dp->dl_time == 0) {
+       if (delegation_hashed(dp) && dp->dl_time == 0) {
                dp->dl_time = get_seconds();
                list_add_tail(&dp->dl_recall_lru, &nn->del_recall_lru);
        }
--
2.34.1

Greg KH

author maintainer

2021-12-17 13:52 UTC #3 lore raw

Quoted 7 lines · Bruce Fields

Quoted 6 lines · Salvatore Bonaccorso
Quoted 4 lines · On Sun, Dec 12, 2021 at 02:11:30PM +0100, [email protected] wrote:
The patch below does not apply to the 4.19-stable tree.
If someone wants it applied there, or to any other stable or longterm tree, then please email the backport, including the original git commit id to <[email protected]>.

Quoted 8 lines

Quoted 6 lines
Unless I'm completely wrong this only does not apply to the older series because in 5.6-rc1 we had 20b7d86f29d3 ("nfsd: use boottime for lease expiry calculation") soe there is a minor context change in one hunk.
Attached is a patch accounting for that, Bruce is this okay?
Yes, this looks fine to me. Thank you!

Now queued up, thanks.

greg k-h