Skip to contentSkip to conversation

NFSD: CB_RECALL_ANY fixes and a meaningful keep count

PATCH New Chuck Lever 1 participant 9 messages Aug 18

Subsystems (2)
lore
Share & tools
lore

Chuck Lever

author maintainer
#0 lore raw

v5 cover NFSD: CB_RECALL_ANY fixes and a meaningful keep count

NFSD currently sends every CB_RECALL_ANY with craa_objects_to_keep set to zero, as RFC 8881 Section 20.6.3 does not mandate any particular way for a client to choose which delegations to return, if any.

NFSD intends CB_RECALL_ANY only as a signal to return currently unused delegations. To date, only the Linux NFS client has been deeply tested against it. That implementation ignores the craa_objects_to_keep value and returns only one unused delegation of the named types. Therefore the fixed zero craa_objects_to_keep value never produced visible misbehavior during our testing.

However, a zero value can result in non-Linux clients giving back more delegations than is necessary to relieve temporary memory pressure on the server, which needlessly punctures the clients' delegation working set.

Change NFSD so that each CB_RECALL_ANY asks a client to give up a sensible number of delegations instead of all of them at once. To handle the accounting correctly, it is necessary for this series to split the current state shrinker mechanism. The patch descriptions have those details.

---
Changes in v5:
- Exclude clients the sweep skips from the apportionment's divisor.
- Skip clients whose CB_RECALL_ANY callback is still running.
- Discount the delegation shrinker's count by its pending requests.
- Link to v4: https://patch.msgid.link/[email protected]

Changes in v4:
- Avoid a divide by zero when max_delegations is zero.
- Collect Jeff's Reviewed-by on 5-8, given on the v3 posting.
- Link to v3: https://patch.msgid.link/[email protected]

Changes in v3:
- Count delegations per namespace so each shrinker reports its own.
- Give courtesy clients and delegations separate state shrinkers.
- Record the shrinker's scan requests and discount the count by them.
- Scale each client's CB_RECALL_ANY by its share of the scan request.
- Reword the keep-count rationale in the commit message and comment.
- Link to v2: https://patch.msgid.link/[email protected]

Changes in v2:
- Drop the gate that skipped clients holding a single delegation.
- Cover letter rewritten to give performance rationale.
- Link to v1: https://patch.msgid.link/[email protected]

---
Chuck Lever (8):
NFSD: Do not send CB_RECALL_ANY to NFSv4.0 clients
NFSD: Count the delegations held by each client
NFSD: Name directory delegations in the CB_RECALL_ANY type mask
NFSD: Send a meaningful CB_RECALL_ANY keep count
NFSD: Count delegations per network namespace
NFSD: Give delegations their own state shrinker
NFSD: Pace the state shrinker's scan requests
NFSD: Apportion CB_RECALL_ANY recalls among clients

 fs/nfsd/netns.h     |  17 +++-
 fs/nfsd/nfs4state.c | 285 +++++++++++++++++++++++++++++++++++++++++++++-------
 fs/nfsd/state.h     |   2 +
 3 files changed, 267 insertions(+), 37 deletions(-)

---
base-commit: de7051270554f99559f9b6bcf3c4fb695c7c223d
change-id: 20260810-recall-any-keep-count-f50c2ae1b792

Best regards,

Signature · 1 line
Chuck Lever <[email protected]>
author maintainer
#1 lore raw

PATCH v5 1/8 NFSD: Do not send CB_RECALL_ANY to NFSv4.0 clients

deleg_reaper() sends CB_RECALL_ANY to every ACTIVE client holding delegations, but CB_RECALL_ANY is an NFSv4.1 operation. An NFSv4.0 client's callback service accepts only CB_GETATTR and CB_RECALL, so it replies OP_ILLEGAL. The decoder maps the unexpected opnum to -EIO, and nfsd4_cb_done() marks the client's callback channel down.

Nothing brings the channel back. nfsd4_run_cb_work() sets NFSD4_CB_UP only for a minor version above zero, and the only nfsd4_probe_callback() call site an NFSv4.0 client reaches is nfsd4_setclientid_confirm(). One visit from the reaper therefore leaves the channel marked down until the client re-establishes its clientid. RENEW then returns NFS4ERR_CB_PATH_DOWN for as long as the client holds delegations. nfsd4_cb_channel_good() stops returning true, so the client is granted no further delegations.

Skip clients at minor version zero.

Fixes 44df6f439a17 ("NFSD: add delegation reaper to react to low memory condition")
---
 fs/nfsd/nfs4state.c | 2 ++
 1 file changed, 2 insertions(+)
fs/nfsd/nfs4state.c +2 -0
index 1ba97e3f65eb..9a3574b853ad 100644
@@ -7945,6 +7945,8 @@ deleg_reaper(struct nfsd_net *nn)
 	list_for_each_safe(pos, next, &nn->client_lru) {
 		clp = list_entry(pos, struct nfs4_client, cl_lru);
 
+		if (clp->cl_minorversion == 0)
+			continue;
 		if (clp->cl_state != NFSD4_ACTIVE)
 			continue;
 		if (list_empty(&clp->cl_delegations))
Signature · 1 line
2.54.0
author maintainer
#2 lore raw

PATCH v5 2/8 NFSD: Count the delegations held by each client

In reply to Chuck Lever

struct nfs4_client records the delegations it holds on cl_delegations but keeps no count of them. deleg_reaper() walks nn->client_lru under nn->client_lock, but cl_delegations is serialized by nn->deleg_lock, which nests outside nn->client_lock. A caller there cannot take nn->deleg_lock to count the list. The cost tells against the walk as
well: an O(n) count per client, on a pass that already visits every client.

Add cl_deleg_count, maintained at the two sites that mutate cl_delegations. Both hold nn->deleg_lock, so the counter is already serialized against itself and needs no atomic of its own. The decrement sits below the delegation_hashed() test, next to the list_del_init it pairs with, so it runs only when the delegation really leaves the list.

A reader that holds only nn->client_lock is not synchronized against either update site, so it can see a count that does not match the list. Such a reader marks the access with data_race() and may not depend on the value for correctness.

No functional change.

---
 fs/nfsd/nfs4state.c | 2 ++
 fs/nfsd/state.h     | 2 ++
 2 files changed, 4 insertions(+)
fs/nfsd/nfs4state.c +2 -0
index 9a3574b853ad..fbe073ee5225 100644
@@ -1527,6 +1527,7 @@ hash_delegation_locked(struct nfs4_delegation *dp, struct nfs4_file *fp)
 	dp->dl_stid.sc_type = SC_TYPE_DELEG;
 	list_add(&dp->dl_perfile, &fp->fi_delegations);
 	list_add(&dp->dl_perclnt, &clp->cl_delegations);
+	clp->cl_deleg_count++;
 	return 0;
 }
 
@@ -1558,6 +1559,7 @@ unhash_delegation_locked(struct nfs4_delegation *dp, unsigned short statusmask)
 	++dp->dl_time;
 	spin_lock(&fp->fi_lock);
 	list_del_init(&dp->dl_perclnt);
+	dp->dl_stid.sc_client->cl_deleg_count--;
 	list_del_init(&dp->dl_recall_lru);
 	list_del_init(&dp->dl_perfile);
 	spin_unlock(&fp->fi_lock);
fs/nfsd/state.h +2 -0
index c65b604e29f1..cd9294f024bb 100644
@@ -633,6 +633,8 @@ struct nfs4_client {
 
 	unsigned int		cl_state;
 	atomic_t		cl_delegs_in_recall;
+	/* Length of cl_delegations, updated under nn->deleg_lock */
+	unsigned int		cl_deleg_count;
 
 	struct nfsd4_cb_recall_any	*cl_ra;
 	time64_t		cl_ra_time;
Signature · 1 line
2.54.0
author maintainer
#3 lore raw

PATCH v5 3/8 NFSD: Name directory delegations in the CB_RECALL_ANY type mask

In reply to Chuck Lever

RFC 8881 Section 20.6.3 distinguishes an NFSv4.1 server implementation that shares one pool among all classes of recallable objects from one that keeps separate pools per class. NFSD falls in the former category.

The CB_RECALL_ANY operation's craa_type_mask argument names the types of objects in the recallable resource pool, but NFSD's implementation does not name directory delegations, even though they are allocated through __alloc_init_deleg(), they are counted against the max_delegations budget, and the state shrinker reclaims them.

Add RCA4_TYPE_MASK_DIR_DLG to craa_type_mask so clients that implement directory delegations consider them when choosing which delegations to return.

---
 fs/nfsd/nfs4state.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
fs/nfsd/nfs4state.c +2 -1
index fbe073ee5225..2214b4d60c7e 100644
@@ -7967,7 +7967,8 @@ deleg_reaper(struct nfsd_net *nn)
 		clp->cl_ra_time = ktime_get_boottime_seconds();
 		clp->cl_ra->ra_keep = 0;
 		clp->cl_ra->ra_bmval[0] = BIT(RCA4_TYPE_MASK_RDATA_DLG) |
-						BIT(RCA4_TYPE_MASK_WDATA_DLG);
+						BIT(RCA4_TYPE_MASK_WDATA_DLG) |
+						BIT(RCA4_TYPE_MASK_DIR_DLG);
 		trace_nfsd_cb_recall_any(clp->cl_ra);
 		nfsd4_run_cb(&clp->cl_ra->ra_cb);
 	}
Signature · 1 line
2.54.0
author maintainer
#4 lore raw

PATCH v5 4/8 NFSD: Send a meaningful CB_RECALL_ANY keep count

In reply to Chuck Lever

deleg_reaper() sets craa_objects_to_keep to zero on every CB_RECALL_ANY. RFC 8881 Section 20.6.3 defines that field as the number of objects the client may keep, leaving the client to choose which of the excess to return, because the server cannot read lack of recent use as lack of usefulness. Zero asks for every delegation the client holds, including the ones backing files an application still has open.

There is also no reason NFSD has to reclaim the entire delegation working set on the first sign of memory pressure.

Derive the keep count from cl_deleg_count so that each callback asks for one delegation. Both the shrinker and the laundromat re-arm while their condition lasts, so a client with more to give is asked again on the next pass.

The Linux client ignores craa_objects_to_keep and returns unused delegations selected from the type mask alone, so the count changes nothing for it.

Fixes 44df6f439a17 ("NFSD: add delegation reaper to react to low memory condition")
---
 fs/nfsd/nfs4state.c | 19 ++++++++++++++++---
 1 file changed, 16 insertions(+), 3 deletions(-)
fs/nfsd/nfs4state.c +16 -3
index 2214b4d60c7e..5c2dd6657fac 100644
@@ -7942,6 +7942,7 @@ deleg_reaper(struct nfsd_net *nn)
 {
 	struct list_head *pos, *next;
 	struct nfs4_client *clp;
+	unsigned int count;
 
 	spin_lock(&nn->client_lock);
 	list_for_each_safe(pos, next, &nn->client_lru) {
@@ -7951,21 +7952,33 @@ deleg_reaper(struct nfsd_net *nn)
 			continue;
 		if (clp->cl_state != NFSD4_ACTIVE)
 			continue;
-		if (list_empty(&clp->cl_delegations))
-			continue;
 		if (atomic_read(&clp->cl_delegs_in_recall))
 			continue;
 		if (ktime_get_boottime_seconds() - clp->cl_ra_time < 5)
 			continue;
 		if (clp->cl_cb_state != NFSD4_CB_UP)
 			continue;
+		/*
+		 * This read races with hash_delegation_locked() and
+		 * unhash_delegation_locked() on other CPUs. A stale
+		 * count only skews the keep value; the next
+		 * laundromat pass sees a more current one.
+		 */
+		count = data_race(READ_ONCE(clp->cl_deleg_count));
+		if (!count)
+			continue;
 		if (test_and_set_bit(NFSD4_CALLBACK_RUNNING, &clp->cl_ra->ra_cb.cb_flags))
 			continue;
 
 		/* release in nfsd4_cb_recall_any_release */
 		kref_get(&clp->cl_nfsdfs.cl_ref);
 		clp->cl_ra_time = ktime_get_boottime_seconds();
-		clp->cl_ra->ra_keep = 0;
+		/*
+		 * Ask for a single delegation. Recalling one before it
+		 * is needed costs the client an OPEN when it next
+		 * touches the file.
+		 */
+		clp->cl_ra->ra_keep = count - 1;
 		clp->cl_ra->ra_bmval[0] = BIT(RCA4_TYPE_MASK_RDATA_DLG) |
 						BIT(RCA4_TYPE_MASK_WDATA_DLG) |
 						BIT(RCA4_TYPE_MASK_DIR_DLG);
Signature · 1 line
2.54.0
author maintainer
#5 lore raw

PATCH v5 5/8 NFSD: Count delegations per network namespace

In reply to Chuck Lever

The state shrinker is allocated per network namespace, but nfsd4_state_shrinker_count() reports num_delegations, which counts the delegations held by the whole host. Every namespace therefore reports every delegation on the server. Reclaim sees the population multiplied by the number of namespaces running NFSD. A namespace holding no delegations of its own still reports a nonzero count and queues its reaper, which then finds nothing to recall.

Count the delegations in each namespace and report that instead. num_delegations stays for the admission check in __alloc_init_deleg() and the ceiling check in nfs4_laundromat(). Both compare against max_delegations, which is sized from host memory and so remains a host-wide limit.

---
 fs/nfsd/netns.h     | 2 ++
 fs/nfsd/nfs4state.c | 7 ++++++-
 2 files changed, 8 insertions(+), 1 deletion(-)
fs/nfsd/netns.h +2 -0
index 71eebfea020d..bb62d19430bc 100644
@@ -238,6 +238,8 @@ struct nfsd_net {
 	int			nfs4_max_clients;
 
 	atomic_t		nfsd_courtesy_clients;
+	/* per-namespace; num_delegations in nfs4state.c is host-wide */
+	atomic_long_t		nfsd_delegations;
 	struct shrinker		*nfsd_client_shrinker;
 	struct work_struct	nfsd_shrinker_work;
 
fs/nfsd/nfs4state.c +6 -1
index 5c2dd6657fac..2c169bfabfbe 100644
@@ -1161,6 +1161,7 @@ static struct nfs4_ol_stateid * nfs4_alloc_open_stateid(struct nfs4_client *clp)
  */
 static void nfs4_free_deleg(struct nfs4_stid *stid)
 {
+	struct nfsd_net *nn = net_generic(stid->sc_client->net, nfsd_net_id);
 	struct nfs4_delegation *dp = delegstateid(stid);
 
 	WARN_ON_ONCE(!list_empty(&stid->sc_cp_list));
@@ -1171,6 +1172,7 @@ static void nfs4_free_deleg(struct nfs4_stid *stid)
 	nfsd41_cb_destroy_referring_call_list(&dp->dl_recall);
 	kmem_cache_free(deleg_slab, stid);
 	atomic_long_dec(&num_delegations);
+	atomic_long_dec(&nn->nfsd_delegations);
 }
 
 /*
@@ -1255,6 +1257,7 @@ __alloc_init_deleg(struct nfs4_client *clp, struct nfs4_file *fp,
 		   struct nfs4_clnt_odstate *odstate, u32 dl_type,
 		   void (*sc_free)(struct nfs4_stid *))
 {
+	struct nfsd_net *nn = net_generic(clp->net, nfsd_net_id);
 	struct nfs4_delegation *dp;
 	struct nfs4_stid *stid;
 	long n;
@@ -1263,6 +1266,7 @@ __alloc_init_deleg(struct nfs4_client *clp, struct nfs4_file *fp,
 		return NULL;
 
 	n = atomic_long_inc_return(&num_delegations);
+	atomic_long_inc(&nn->nfsd_delegations);
 	if (n < 0 || n > max_delegations)
 		goto out_dec;
 
@@ -1295,6 +1299,7 @@ __alloc_init_deleg(struct nfs4_client *clp, struct nfs4_file *fp,
 	return dp;
 out_dec:
 	atomic_long_dec(&num_delegations);
+	atomic_long_dec(&nn->nfsd_delegations);
 	return NULL;
 }
 
@@ -5564,7 +5569,7 @@ nfsd4_state_shrinker_count(struct shrinker *shrink, struct shrink_control *sc)
 
 	count = atomic_read(&nn->nfsd_courtesy_clients);
 	if (!count)
-		count = atomic_long_read(&num_delegations);
+		count = atomic_long_read(&nn->nfsd_delegations);
 	if (count)
 		queue_work(laundry_wq, &nn->nfsd_shrinker_work);
 	return (unsigned long)count;
Signature · 1 line
2.54.0
author maintainer
#6 lore raw

PATCH v5 6/8 NFSD: Give delegations their own state shrinker

In reply to Chuck Lever

Since commit 44df6f439a17 ("NFSD: add delegation reaper to react to low memory condition"), nfsd_client_shrinker has managed two unrelated populations of objects.

One population is courtesy clients. Shrinking that population can be done synchronously and without risk of deadlock. The shrinker callback could return a precise count of the number of objects that were released.

The other population is delegations. Shrinking that population requires sending a CB_RECALL_ANY; clients are not obligated to return any delegation. The shrinker callback is structurally unable to report progress.

What's more, the single shrinker callback falls back to delegation reaping only when there are no courtesy clients left to reclaim. A single courtesy client is enough to keep a namespace's delegations out of the count it reports.

To begin to resolve these issues, refactor the existing state shrinker into two: one for courtesy clients and one for reaping delegations. Each manages the size of its own population, and the shrinker names become namespace-specific.

---
 fs/nfsd/netns.h     |  6 +++--
 fs/nfsd/nfs4state.c | 77 ++++++++++++++++++++++++++++++++++++++++++-----------
 2 files changed, 65 insertions(+), 18 deletions(-)
fs/nfsd/netns.h +4 -2
index bb62d19430bc..ef01a1cf72ac 100644
@@ -240,8 +240,10 @@ struct nfsd_net {
 	atomic_t		nfsd_courtesy_clients;
 	/* per-namespace; num_delegations in nfs4state.c is host-wide */
 	atomic_long_t		nfsd_delegations;
-	struct shrinker		*nfsd_client_shrinker;
-	struct work_struct	nfsd_shrinker_work;
+	struct shrinker		*nfsd_courtesy_shrinker;
+	struct shrinker		*nfsd_deleg_shrinker;
+	struct work_struct	nfsd_courtesy_work;
+	struct work_struct	nfsd_deleg_work;
 
 	/* last time an admin-revoke happened for NFSv4.0 */
 	time64_t		nfs40_last_revoke;
fs/nfsd/nfs4state.c +61 -16

124 lines not on this page · raw message

Signature · 1 line
2.54.0
author maintainer
#7 lore raw

PATCH v5 7/8 NFSD: Pace the state shrinker's scan requests

In reply to Chuck Lever

Currently, neither of the scan callback functions records anything before returning SHRINK_STOP, so the size of each scan request is discarded. That size is the only real measure NFSD gets of reclaim pressure. Both count callbacks report their population whether or not the reaper is already queued to reclaim it, so reclaim asks again for work that is pending.

Accumulate each courtesy scan request in nfsd_shrink_backlog and subtract the backlog from what that count callback reports. The worker retires the backlog once courtesy_client_reaper() has run. That reaper expires the clients synchronously, so the discount covers exactly the interval the work is pending.

Delegations need a different bound. This is because deleg_reaper() only sends CB_RECALL_ANY and does not track how many delegations were actually returned by the targeted client.

Report the delegations only once NFSD_RECALL_ANY_COOLDOWN_SECS have passed since the last sweep. deleg_reaper() skips any client it recalled from within that window, so an earlier scan request cannot produce another recall.

---
 fs/nfsd/netns.h     |  6 ++++
 fs/nfsd/nfs4state.c | 92 +++++++++++++++++++++++++++++++++++++++++++++--------
 2 files changed, 85 insertions(+), 13 deletions(-)
fs/nfsd/netns.h +6 -0
index ef01a1cf72ac..23923cc4aa47 100644
@@ -245,6 +245,12 @@ struct nfsd_net {
 	struct work_struct	nfsd_courtesy_work;
 	struct work_struct	nfsd_deleg_work;
 
+	/* courtesy scan requests the reaper has not retired yet */
+	atomic_long_t		nfsd_shrink_backlog;
+
+	/* when deleg_reaper() last swept the client list */
+	time64_t		nfsd_last_recall_any;
+
 	/* last time an admin-revoke happened for NFSv4.0 */
 	time64_t		nfs40_last_revoke;
 
fs/nfsd/nfs4state.c +79 -13

165 lines not on this page · raw message

Signature · 1 line
2.54.0
author maintainer
#8 lore raw

PATCH v5 8/8 NFSD: Apportion CB_RECALL_ANY recalls among clients

In reply to Chuck Lever

deleg_reaper() asks each eligible client to return one delegation whenever it runs, whether or not anything needs the memory. A delegation returned before it is needed costs the client an OPEN when it next touches the file. Nothing sizes the request either. The delegation scan callback discards nr_to_scan, which is reclaim's statement of how many objects it wants back.

Record each delegation scan request in nfsd_deleg_backlog and pass the accumulated total to deleg_reaper(). Handing that total to every client would ask for it once per client, so scale it by each client's share of the delegations this sweep can reach. The count callback reports what is left after the outstanding requests, so concurrent reclaimers do not each ask for the same delegations. cl_ra_time keeps the next sweep from returning to the clients this one reached.

Nothing is recalled until a scan arrives. nfs4_laundromat() is the exception. It has no scan request to pass, so it computes what must go for num_delegations to fall below max_delegations, and passes only this namespace's share.

---
 fs/nfsd/netns.h     |   3 ++
 fs/nfsd/nfs4state.c | 131 ++++++++++++++++++++++++++++++++++++++++++----------
 2 files changed, 109 insertions(+), 25 deletions(-)
fs/nfsd/netns.h +3 -0
index 23923cc4aa47..0ce7da20aba3 100644
@@ -248,6 +248,9 @@ struct nfsd_net {
 	/* courtesy scan requests the reaper has not retired yet */
 	atomic_long_t		nfsd_shrink_backlog;
 
+	/* delegation scan requests the reaper has not retired yet */
+	atomic_long_t		nfsd_deleg_backlog;
+
 	/* when deleg_reaper() last swept the client list */
 	time64_t		nfsd_last_recall_any;
 
fs/nfsd/nfs4state.c +106 -25

220 lines not on this page · raw message

Signature · 1 line
2.54.0