NFSD: CB_RECALL_ANY fixes and a meaningful keep count
Subsystems (2)
Chuck Lever
maintainerv5 cover NFSD: CB_RECALL_ANY fixes and a meaningful keep count
NFSD currently sends every CB_RECALL_ANY with craa_objects_to_keep set to zero, as RFC 8881 Section 20.6.3 does not mandate any particular way for a client to choose which delegations to return, if any.
NFSD intends CB_RECALL_ANY only as a signal to return currently unused delegations. To date, only the Linux NFS client has been deeply tested against it. That implementation ignores the craa_objects_to_keep value and returns only one unused delegation of the named types. Therefore the fixed zero craa_objects_to_keep value never produced visible misbehavior during our testing.
However, a zero value can result in non-Linux clients giving back more delegations than is necessary to relieve temporary memory pressure on the server, which needlessly punctures the clients' delegation working set.
Change NFSD so that each CB_RECALL_ANY asks a client to give up a sensible number of delegations instead of all of them at once. To handle the accounting correctly, it is necessary for this series to split the current state shrinker mechanism. The patch descriptions have those details.
---
Changes in v5:
- Exclude clients the sweep skips from the apportionment's divisor.
- Skip clients whose CB_RECALL_ANY callback is still running.
- Discount the delegation shrinker's count by its pending requests.
- Link to v4: https://patch.msgid.link/[email protected]
Changes in v4:
- Avoid a divide by zero when max_delegations is zero.
- Collect Jeff's Reviewed-by on 5-8, given on the v3 posting.
- Link to v3: https://patch.msgid.link/[email protected]
Changes in v3:
- Count delegations per namespace so each shrinker reports its own.
- Give courtesy clients and delegations separate state shrinkers.
- Record the shrinker's scan requests and discount the count by them.
- Scale each client's CB_RECALL_ANY by its share of the scan request.
- Reword the keep-count rationale in the commit message and comment.
- Link to v2: https://patch.msgid.link/[email protected]
Changes in v2:
- Drop the gate that skipped clients holding a single delegation.
- Cover letter rewritten to give performance rationale.
- Link to v1: https://patch.msgid.link/[email protected]
---
Chuck Lever (8):
NFSD: Do not send CB_RECALL_ANY to NFSv4.0 clients
NFSD: Count the delegations held by each client
NFSD: Name directory delegations in the CB_RECALL_ANY type mask
NFSD: Send a meaningful CB_RECALL_ANY keep count
NFSD: Count delegations per network namespace
NFSD: Give delegations their own state shrinker
NFSD: Pace the state shrinker's scan requests
NFSD: Apportion CB_RECALL_ANY recalls among clients
fs/nfsd/netns.h | 17 +++- fs/nfsd/nfs4state.c | 285 +++++++++++++++++++++++++++++++++++++++++++++------- fs/nfsd/state.h | 2 + 3 files changed, 267 insertions(+), 37 deletions(-)
---
base-commit: de7051270554f99559f9b6bcf3c4fb695c7c223d
change-id: 20260810-recall-any-keep-count-f50c2ae1b792
Best regards,
Signature · 1 line
Chuck Lever <[email protected]>
Chuck Lever
maintainerPATCH v5 1/8 NFSD: Do not send CB_RECALL_ANY to NFSv4.0 clients
deleg_reaper() sends CB_RECALL_ANY to every ACTIVE client holding delegations, but CB_RECALL_ANY is an NFSv4.1 operation. An NFSv4.0 client's callback service accepts only CB_GETATTR and CB_RECALL, so it replies OP_ILLEGAL. The decoder maps the unexpected opnum to -EIO, and nfsd4_cb_done() marks the client's callback channel down.
Nothing brings the channel back. nfsd4_run_cb_work() sets NFSD4_CB_UP only for a minor version above zero, and the only nfsd4_probe_callback() call site an NFSv4.0 client reaches is nfsd4_setclientid_confirm(). One visit from the reaper therefore leaves the channel marked down until the client re-establishes its clientid. RENEW then returns NFS4ERR_CB_PATH_DOWN for as long as the client holds delegations. nfsd4_cb_channel_good() stops returning true, so the client is granted no further delegations.
Skip clients at minor version zero.
--- fs/nfsd/nfs4state.c | 2 ++ 1 file changed, 2 insertions(+)
fs/nfsd/nfs4state.c +2 -0
@@ -7945,6 +7945,8 @@ deleg_reaper(struct nfsd_net *nn)list_for_each_safe (pos, next, &nn->client_lru) { clp = list_entry(pos, struct nfs4_client, cl_lru); +if (clp->cl_minorversion ==0 ) +continue ;if (clp->cl_state !=NFSD4_ACTIVE )continue ;if (list_empty (&clp->cl_delegations))
Signature · 1 line
2.54.0
Chuck Lever
maintainerPATCH v5 2/8 NFSD: Count the delegations held by each client
In reply to Chuck Leverstruct nfs4_client records the delegations it holds on cl_delegations but keeps no count of them. deleg_reaper() walks nn->client_lru under nn->client_lock, but cl_delegations is serialized by nn->deleg_lock, which nests outside nn->client_lock. A caller there cannot take nn->deleg_lock to count the list. The cost tells against the walk as
well: an O(n) count per client, on a pass that already visits every client.
Add cl_deleg_count, maintained at the two sites that mutate cl_delegations. Both hold nn->deleg_lock, so the counter is already serialized against itself and needs no atomic of its own. The decrement sits below the delegation_hashed() test, next to the list_del_init it pairs with, so it runs only when the delegation really leaves the list.
A reader that holds only nn->client_lock is not synchronized against either update site, so it can see a count that does not match the list. Such a reader marks the access with data_race() and may not depend on the value for correctness.
No functional change.
--- fs/nfsd/nfs4state.c | 2 ++ fs/nfsd/state.h | 2 ++ 2 files changed, 4 insertions(+)
fs/nfsd/nfs4state.c +2 -0
@@ -1527,6 +1527,7 @@ hash_delegation_locked(struct nfs4_delegation *dp, struct nfs4_file *fp) dp->dl_stid.sc_type =SC_TYPE_DELEG ;list_add (&dp->dl_perfile, &fp->fi_delegations);list_add (&dp->dl_perclnt, &clp->cl_delegations); + clp->cl_deleg_count++;return 0 ; }
@@ -1558,6 +1559,7 @@ unhash_delegation_locked(struct nfs4_delegation *dp, unsigned short statusmask) ++dp->dl_time;spin_lock (&fp->fi_lock);list_del_init (&dp->dl_perclnt); + dp->dl_stid.sc_client->cl_deleg_count--;list_del_init (&dp->dl_recall_lru);list_del_init (&dp->dl_perfile);spin_unlock (&fp->fi_lock);
fs/nfsd/state.h +2 -0
@@ -633,6 +633,8 @@ struct nfs4_client {unsigned int cl_state;atomic_t cl_delegs_in_recall; +/* Length of cl_delegations, updated under nn->deleg_lock */ +unsigned int cl_deleg_count;struct nfsd4_cb_recall_any *cl_ra;time64_t cl_ra_time;
Signature · 1 line
2.54.0
Chuck Lever
maintainerPATCH v5 3/8 NFSD: Name directory delegations in the CB_RECALL_ANY type mask
In reply to Chuck LeverRFC 8881 Section 20.6.3 distinguishes an NFSv4.1 server implementation that shares one pool among all classes of recallable objects from one that keeps separate pools per class. NFSD falls in the former category.
The CB_RECALL_ANY operation's craa_type_mask argument names the types of objects in the recallable resource pool, but NFSD's implementation does not name directory delegations, even though they are allocated through __alloc_init_deleg(), they are counted against the max_delegations budget, and the state shrinker reclaims them.
Add RCA4_TYPE_MASK_DIR_DLG to craa_type_mask so clients that implement directory delegations consider them when choosing which delegations to return.
--- fs/nfsd/nfs4state.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
fs/nfsd/nfs4state.c +2 -1
@@ -7967,7 +7967,8 @@ deleg_reaper(struct nfsd_net *nn) clp->cl_ra_time =ktime_get_boottime_seconds (); clp->cl_ra->ra_keep =0 ; clp->cl_ra->ra_bmval[0 ] =BIT (RCA4_TYPE_MASK_RDATA_DLG ) | -BIT (RCA4_TYPE_MASK_WDATA_DLG ); +BIT (RCA4_TYPE_MASK_WDATA_DLG ) | +BIT (RCA4_TYPE_MASK_DIR_DLG );trace_nfsd_cb_recall_any (clp->cl_ra);nfsd4_run_cb (&clp->cl_ra->ra_cb); }
Signature · 1 line
2.54.0
Chuck Lever
maintainerPATCH v5 4/8 NFSD: Send a meaningful CB_RECALL_ANY keep count
In reply to Chuck Leverdeleg_reaper() sets craa_objects_to_keep to zero on every CB_RECALL_ANY. RFC 8881 Section 20.6.3 defines that field as the number of objects the client may keep, leaving the client to choose which of the excess to return, because the server cannot read lack of recent use as lack of usefulness. Zero asks for every delegation the client holds, including the ones backing files an application still has open.
There is also no reason NFSD has to reclaim the entire delegation working set on the first sign of memory pressure.
Derive the keep count from cl_deleg_count so that each callback asks for one delegation. Both the shrinker and the laundromat re-arm while their condition lasts, so a client with more to give is asked again on the next pass.
The Linux client ignores craa_objects_to_keep and returns unused delegations selected from the type mask alone, so the count changes nothing for it.
--- fs/nfsd/nfs4state.c | 19 ++++++++++++++++--- 1 file changed, 16 insertions(+), 3 deletions(-)
fs/nfsd/nfs4state.c +16 -3
@@ -7942,6 +7942,7 @@ deleg_reaper(struct nfsd_net *nn) {struct list_head *pos, *next;struct nfs4_client *clp; +unsigned int count;spin_lock (&nn->client_lock);list_for_each_safe (pos, next, &nn->client_lru) {
@@ -7951,21 +7952,33 @@ deleg_reaper(struct nfsd_net *nn)continue ;if (clp->cl_state !=NFSD4_ACTIVE )continue ; -if (list_empty (&clp->cl_delegations)) -continue ;if (atomic_read (&clp->cl_delegs_in_recall))continue ;if (ktime_get_boottime_seconds () - clp->cl_ra_time <5 )continue ;if (clp->cl_cb_state !=NFSD4_CB_UP )continue ; +/* +* This read races with hash_delegation_locked() and +* unhash_delegation_locked() on other CPUs. A stale +* count only skews the keep value; the next +* laundromat pass sees a more current one. +*/ + count =data_race (READ_ONCE (clp->cl_deleg_count)); +if (!count) +continue ;if (test_and_set_bit (NFSD4_CALLBACK_RUNNING , &clp->cl_ra->ra_cb.cb_flags))continue ;/* release in nfsd4_cb_recall_any_release */ kref_get (&clp->cl_nfsdfs.cl_ref); clp->cl_ra_time =ktime_get_boottime_seconds (); - clp->cl_ra->ra_keep =0 ; +/* +* Ask for a single delegation. Recalling one before it +* is needed costs the client an OPEN when it next +* touches the file. +*/ + clp->cl_ra->ra_keep = count -1 ; clp->cl_ra->ra_bmval[0 ] =BIT (RCA4_TYPE_MASK_RDATA_DLG ) |BIT (RCA4_TYPE_MASK_WDATA_DLG ) |BIT (RCA4_TYPE_MASK_DIR_DLG );
Signature · 1 line
2.54.0
Chuck Lever
maintainerPATCH v5 5/8 NFSD: Count delegations per network namespace
In reply to Chuck LeverThe state shrinker is allocated per network namespace, but nfsd4_state_shrinker_count() reports num_delegations, which counts the delegations held by the whole host. Every namespace therefore reports every delegation on the server. Reclaim sees the population multiplied by the number of namespaces running NFSD. A namespace holding no delegations of its own still reports a nonzero count and queues its reaper, which then finds nothing to recall.
Count the delegations in each namespace and report that instead. num_delegations stays for the admission check in __alloc_init_deleg() and the ceiling check in nfs4_laundromat(). Both compare against max_delegations, which is sized from host memory and so remains a host-wide limit.
--- fs/nfsd/netns.h | 2 ++ fs/nfsd/nfs4state.c | 7 ++++++- 2 files changed, 8 insertions(+), 1 deletion(-)
fs/nfsd/netns.h +2 -0
@@ -238,6 +238,8 @@ struct nfsd_net {int nfs4_max_clients;atomic_t nfsd_courtesy_clients; +/* per-namespace; num_delegations in nfs4state.c is host-wide */ +atomic_long_t nfsd_delegations;struct shrinker *nfsd_client_shrinker;struct work_struct nfsd_shrinker_work;
fs/nfsd/nfs4state.c +6 -1
@@ -1161,6 +1161,7 @@ static struct nfs4_ol_stateid * nfs4_alloc_open_stateid(struct nfs4_client *clp)*/ static void nfs4_free_deleg (struct nfs4_stid *stid) { +struct nfsd_net *nn =net_generic (stid->sc_client->net, nfsd_net_id);struct nfs4_delegation *dp =delegstateid (stid);WARN_ON_ONCE (!list_empty (&stid->sc_cp_list));
@@ -1171,6 +1172,7 @@ static void nfs4_free_deleg(struct nfs4_stid *stid)nfsd41_cb_destroy_referring_call_list (&dp->dl_recall);kmem_cache_free (deleg_slab, stid);atomic_long_dec (&num_delegations); +atomic_long_dec (&nn->nfsd_delegations); }/*
@@ -1255,6 +1257,7 @@ __alloc_init_deleg(struct nfs4_client *clp, struct nfs4_file *fp, struct nfs4_clnt_odstate *odstate, u32 dl_type, void (*sc_free)(struct nfs4_stid *)) { + struct nfsd_net *nn = net_generic(clp->net, nfsd_net_id);struct nfs4_delegation *dp;struct nfs4_stid *stid;long n;
@@ -1263,6 +1266,7 @@ __alloc_init_deleg(struct nfs4_client *clp, struct nfs4_file *fp,return NULL ; n =atomic_long_inc_return (&num_delegations); +atomic_long_inc (&nn->nfsd_delegations);if (n <0 || n > max_delegations) goto out_dec;
@@ -1295,6 +1299,7 @@ __alloc_init_deleg(struct nfs4_client *clp, struct nfs4_file *fp,return dp; out_dec:atomic_long_dec (&num_delegations); +atomic_long_dec (&nn->nfsd_delegations);return NULL ; }
@@ -5564,7 +5569,7 @@ nfsd4_state_shrinker_count(struct shrinker *shrink, struct shrink_control *sc) count =atomic_read (&nn->nfsd_courtesy_clients);if (!count) - count =atomic_long_read (&num_delegations); + count =atomic_long_read (&nn->nfsd_delegations);if (count)queue_work (laundry_wq, &nn->nfsd_shrinker_work);return (unsigned long )count;
Signature · 1 line
2.54.0
Chuck Lever
maintainerPATCH v5 6/8 NFSD: Give delegations their own state shrinker
In reply to Chuck LeverSince commit 44df6f439a17 ("NFSD: add delegation reaper to react to low memory condition"), nfsd_client_shrinker has managed two unrelated populations of objects.
One population is courtesy clients. Shrinking that population can be done synchronously and without risk of deadlock. The shrinker callback could return a precise count of the number of objects that were released.
The other population is delegations. Shrinking that population requires sending a CB_RECALL_ANY; clients are not obligated to return any delegation. The shrinker callback is structurally unable to report progress.
What's more, the single shrinker callback falls back to delegation reaping only when there are no courtesy clients left to reclaim. A single courtesy client is enough to keep a namespace's delegations out of the count it reports.
To begin to resolve these issues, refactor the existing state shrinker into two: one for courtesy clients and one for reaping delegations. Each manages the size of its own population, and the shrinker names become namespace-specific.
--- fs/nfsd/netns.h | 6 +++-- fs/nfsd/nfs4state.c | 77 ++++++++++++++++++++++++++++++++++++++++++----------- 2 files changed, 65 insertions(+), 18 deletions(-)
fs/nfsd/netns.h +4 -2
@@ -240,8 +240,10 @@ struct nfsd_net {atomic_t nfsd_courtesy_clients;/* per-namespace; num_delegations in nfs4state.c is host-wide */ atomic_long_t nfsd_delegations; -struct shrinker *nfsd_client_shrinker; -struct work_struct nfsd_shrinker_work; +struct shrinker *nfsd_courtesy_shrinker; +struct shrinker *nfsd_deleg_shrinker; +struct work_struct nfsd_courtesy_work; +struct work_struct nfsd_deleg_work;/* last time an admin-revoke happened for NFSv4.0 */ time64_t nfs40_last_revoke;
fs/nfsd/nfs4state.c +61 -16
124 lines not on this page · raw message
Signature · 1 line
2.54.0
Chuck Lever
maintainerPATCH v5 7/8 NFSD: Pace the state shrinker's scan requests
In reply to Chuck LeverCurrently, neither of the scan callback functions records anything before returning SHRINK_STOP, so the size of each scan request is discarded. That size is the only real measure NFSD gets of reclaim pressure. Both count callbacks report their population whether or not the reaper is already queued to reclaim it, so reclaim asks again for work that is pending.
Accumulate each courtesy scan request in nfsd_shrink_backlog and subtract the backlog from what that count callback reports. The worker retires the backlog once courtesy_client_reaper() has run. That reaper expires the clients synchronously, so the discount covers exactly the interval the work is pending.
Delegations need a different bound. This is because deleg_reaper() only sends CB_RECALL_ANY and does not track how many delegations were actually returned by the targeted client.
Report the delegations only once NFSD_RECALL_ANY_COOLDOWN_SECS have passed since the last sweep. deleg_reaper() skips any client it recalled from within that window, so an earlier scan request cannot produce another recall.
--- fs/nfsd/netns.h | 6 ++++ fs/nfsd/nfs4state.c | 92 +++++++++++++++++++++++++++++++++++++++++++++-------- 2 files changed, 85 insertions(+), 13 deletions(-)
fs/nfsd/netns.h +6 -0
@@ -245,6 +245,12 @@ struct nfsd_net {struct work_struct nfsd_courtesy_work;struct work_struct nfsd_deleg_work; +/* courtesy scan requests the reaper has not retired yet */ +atomic_long_t nfsd_shrink_backlog; + +/* when deleg_reaper() last swept the client list */ +time64_t nfsd_last_recall_any; +/* last time an admin-revoke happened for NFSv4.0 */ time64_t nfs40_last_revoke;
fs/nfsd/nfs4state.c +79 -13
165 lines not on this page · raw message
Signature · 1 line
2.54.0
Chuck Lever
maintainerPATCH v5 8/8 NFSD: Apportion CB_RECALL_ANY recalls among clients
In reply to Chuck Leverdeleg_reaper() asks each eligible client to return one delegation whenever it runs, whether or not anything needs the memory. A delegation returned before it is needed costs the client an OPEN when it next touches the file. Nothing sizes the request either. The delegation scan callback discards nr_to_scan, which is reclaim's statement of how many objects it wants back.
Record each delegation scan request in nfsd_deleg_backlog and pass the accumulated total to deleg_reaper(). Handing that total to every client would ask for it once per client, so scale it by each client's share of the delegations this sweep can reach. The count callback reports what is left after the outstanding requests, so concurrent reclaimers do not each ask for the same delegations. cl_ra_time keeps the next sweep from returning to the clients this one reached.
Nothing is recalled until a scan arrives. nfs4_laundromat() is the exception. It has no scan request to pass, so it computes what must go for num_delegations to fall below max_delegations, and passes only this namespace's share.
--- fs/nfsd/netns.h | 3 ++ fs/nfsd/nfs4state.c | 131 ++++++++++++++++++++++++++++++++++++++++++---------- 2 files changed, 109 insertions(+), 25 deletions(-)
fs/nfsd/netns.h +3 -0
@@ -248,6 +248,9 @@ struct nfsd_net {/* courtesy scan requests the reaper has not retired yet */ atomic_long_t nfsd_shrink_backlog; +/* delegation scan requests the reaper has not retired yet */ +atomic_long_t nfsd_deleg_backlog; +/* when deleg_reaper() last swept the client list */ time64_t nfsd_last_recall_any;
fs/nfsd/nfs4state.c +106 -25
220 lines not on this page · raw message
Signature · 1 line
2.54.0