NFS: flexfiles device notifications and caching for wide striped layouts
Subsystems (2)
cover NFS: flexfiles device notifications and caching for wide striped layouts
A flexfiles layout striped across many data servers puts hundreds to ~1000 distinct deviceIDs in play for a single mount. This series makes the client's CB_NOTIFY_DEVICEID handling actually work under live layouts, implements the deviceID-deletion race recovery RFC 8881 asks for, and scales the device caches to that number of devices.
Patches 1-4 are standalone fixes (Cc: stable) for latent bugs in the striped-layout paths. The CB_NOTIFY_DEVICEID decoder gated consuming ndc_immediate on the layout type instead of the notification type, so the flag was silently lost and any multi-item cnda_changes<> array misaligned after the first entry. Read data-server selection truncated the file offset to 32 bits, picking the wrong stripe's DS for availability probing and coalescing above 4 GiB. And page coalescing was bounded on a segment-relative offset in both the flexfiles and files layouts, so an unaligned segment let a coalesced I/O straddle a stripe boundary and send the bytes past it to the wrong data server.
Patches 5-13 make CHANGE notifications work under live layouts. RFC 8881 Section 12.2.10 has CB_NOTIFY_DEVICEID CHANGE exist precisely so a server can re-point a deviceID without recalling the layouts that reference it, but the client's only response today is to unhash the cached device -- which never reaches the references pinned inside the layout driver's segments, so I/O keeps going to the old mapping until the layouts are freed. Each in-flight I/O is given its own device-node reference, the mirror's pinned pointer becomes RCU-managed, and a new layout-driver hook re-resolves the device in place, honoring ndc_immediate. A GETDEVICEINFO reply that raced a CHANGE is discarded rather than installed stale.
Patches 14-17 implement the deviceID-deletion race of RFC 8881 Section 18.40.4: layouts still referencing a deleted deviceID are tested with TEST_STATEID, revoked state is recovered, and the deletion is confirmed with GETDEVICEINFO before escalating to lease recovery. A DELETE that no live layout references keeps today's cheap path.
Patches 18-21 scale the device caches. The global deviceid hash grows from 32 to 256 buckets (a load factor of ~31 at 1000 devices today), and the per-net data-server cache moves from a single list to hash buckets keyed by the DS address set -- which also fixes a latent aliasing bug, since the old comparator matched on subset rather than equality and merged two data servers whenever one's address set contained the other's. Last, the flexfiles driver gains a dataserver_nconnect module parameter alongside its existing dataserver_timeo and dataserver_retrans knobs:
data-server clients otherwise inherit the MDS nconnect, which at ~1000 DSes and nconnect=16 means ~16k sockets and their slot tables for little gain on a workload striping across the DSes anyway. It defaults to 0, preserving today's inherit-from-MDS behavior.
Tested end-to-end against reffs, a notification-capable reference server, at 256 and 1000 devices, including KASAN and lockdep runs under concurrent notification storms.
A related series, "NFS: size the LAYOUTGET reply buffer for wide flexfiles layouts", lifts the single-page LAYOUTGET reply buffer that today caps a segment at roughly 28 stripes. The two are independent and apply cleanly in either order.
Benjamin Coddington (21):
pNFS: Fix CB_NOTIFY_DEVICEID CHANGE to consume ndc_immediate NFSv4/flexfiles: Use the full 64-bit offset for read DS selection NFSv4/flexfiles: Bound page coalescing on the absolute stripe offset NFSv4/filelayout: Anchor page coalescing on pattern_offset NFSv4/flexfiles: Reference the device node across DS setup NFSv4/flexfiles: Carry the device node reference across each I/O NFSv4/flexfiles: Hold a device node reference for layoutstats encoding NFSv4/flexfiles: Make the pinned device node pointer RCU-managed
pNFS: Add a reresolve_deviceid layout driver hook NFSv4/flexfiles: Implement in-place device re-resolve on CHANGE
NFSv4: Dispatch CB_NOTIFY_DEVICEID CHANGE to an in-place refresh NFSv4/flexfiles: Honor ndc_immediate on CB_NOTIFY_DEVICEID CHANGE
pNFS: Discard a GETDEVICEINFO reply that raced a CHANGE notification
pNFS: Add deviceid reference query and collection walkers NFSv4/pnfs: Recover revoked layouts on a deleted deviceID NFSv4/pnfs: Confirm a deviceID delete via GETDEVICEINFO NFSv4/pnfs: Dispatch CB_NOTIFY_DEVICEID DELETE to race recovery NFSv4/pnfs: Grow the deviceid cache hash table
NFSv4/pnfs: Re-home the data-server cache onto hash buckets NFSv4/pnfs: Key the data-server cache by its address set NFSv4/flexfiles: Add a dataserver_nconnect cap
fs/nfs/callback_proc.c | 30 +- fs/nfs/callback_xdr.c | 2 +- fs/nfs/client.c | 6 +- fs/nfs/filelayout/filelayout.c | 8 +- fs/nfs/filelayout/filelayoutdev.c | 2 +- fs/nfs/flexfilelayout/flexfilelayout.c | 383 ++++++++++++++-------- fs/nfs/flexfilelayout/flexfilelayout.h | 44 +-- fs/nfs/flexfilelayout/flexfilelayoutdev.c | 193 +++++++---- fs/nfs/internal.h | 3 +- fs/nfs/netns.h | 5 +- fs/nfs/nfs3client.c | 9 +- fs/nfs/nfs4_fs.h | 2 + fs/nfs/nfs4client.c | 7 +- fs/nfs/nfs4proc.c | 113 +++++++ fs/nfs/nfs4state.c | 3 + fs/nfs/pnfs.c | 292 +++++++++++++++++ fs/nfs/pnfs.h | 81 ++++- fs/nfs/pnfs_dev.c | 35 +- fs/nfs/pnfs_nfs.c | 93 +++++- include/linux/nfs_fs_sb.h | 2 + include/linux/nfs_xdr.h | 2 + 21 files changed, 1069 insertions(+), 246 deletions(-)
base-commit: db2ddb87143519e20a95aa36c60b36107b736a58
Signature · 1 line
2.53.0
PATCH 1/21 pNFS: Fix CB_NOTIFY_DEVICEID CHANGE to consume ndc_immediate
decode_devicenotify_args() gated consuming the trailing ndc_immediate boolean on cbd_layout_type instead of cbd_notify_type. cbd_layout_type holds the pNFS layout type (e.g. 4 for flexfiles) and is never equal to NOTIFY_DEVICEID4_CHANGE (2), so ndc_immediate was never consumed: it was silently lost for a single CHANGE, and a multi-item cnda_changes<> array misaligned after the first item and failed with NFS4ERR_INVAL/BADXDR.
Key the decode off the notification type.
--- fs/nfs/callback_xdr.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
fs/nfs/callback_xdr.c +1 -1
@@ -312,7 +312,7 @@ __be32 decode_devicenotify_args(struct svc_rqst *rqstp,memcpy (dev->cbd_dev_id.data, p,NFS4_DEVICEID4_SIZE ); p +=XDR_QUADLEN (NFS4_DEVICEID4_SIZE ); -if (dev->cbd_layout_type ==NOTIFY_DEVICEID4_CHANGE ) { +if (dev->cbd_notify_type ==NOTIFY_DEVICEID4_CHANGE ) { p =xdr_inline_decode (xdr,sizeof (uint32_t ));if (unlikely (p ==NULL )) { status =htonl (NFS4ERR_BADXDR );
Signature · 1 line
2.53.0
PATCH 2/21 NFSv4/flexfiles: Use the full 64-bit offset for read DS selection
In reply to Benjamin CoddingtonThe read data-server selection helpers declared their offset parameter as u32, truncating the file offset before nfs4_ff_layout_calc_dss_id() computes the stripe index. For striped reads at offsets >= 4 GiB this probes DS availability and sizes coalescing against the wrong stripe's data server; the read itself recomputes dss_id from the full offset, so data is not corrupted. The write path already uses the full offset.
Widen the offset parameter to u64 in the five read-selection helpers.
--- fs/nfs/flexfilelayout/flexfilelayout.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
fs/nfs/flexfilelayout/flexfilelayout.c +5 -5
@@ -878,7 +878,7 @@ ff_layout_mark_ds_reachable(struct pnfs_layout_segment *lseg, u32 idx, u32 dss_istatic struct nfs4_pnfs_ds *ff_layout_choose_ds_for_read (struct pnfs_layout_segment *lseg,u32 start_idx,u32 *best_idx, -u32 offset,u32 *dss_id, +u64 offset,u32 *dss_id,bool check_device) {struct nfs4_ff_layout_segment *fls =FF_LAYOUT_LSEG (lseg);
@@ -914,7 +914,7 @@ ff_layout_choose_ds_for_read(struct pnfs_layout_segment *lseg,static struct nfs4_pnfs_ds *ff_layout_choose_any_ds_for_read (struct pnfs_layout_segment *lseg,u32 start_idx,u32 *best_idx, -u32 offset,u32 *dss_id) +u64 offset,u32 *dss_id) {return ff_layout_choose_ds_for_read (lseg, start_idx, best_idx, offset, dss_id, false);
@@ -923,7 +923,7 @@ ff_layout_choose_any_ds_for_read(struct pnfs_layout_segment *lseg,static struct nfs4_pnfs_ds *ff_layout_choose_valid_ds_for_read (struct pnfs_layout_segment *lseg,u32 start_idx,u32 *best_idx, -u32 offset,u32 *dss_id) +u64 offset,u32 *dss_id) {return ff_layout_choose_ds_for_read (lseg, start_idx, best_idx, offset, dss_id, true);
@@ -932,7 +932,7 @@ ff_layout_choose_valid_ds_for_read(struct pnfs_layout_segment *lseg,static struct nfs4_pnfs_ds *ff_layout_choose_best_ds_for_read (struct pnfs_layout_segment *lseg,u32 start_idx,u32 *best_idx, -u32 offset,u32 *dss_id) +u64 offset,u32 *dss_id) {struct nfs4_pnfs_ds *ds;
@@ -947,7 +947,7 @@ ff_layout_choose_best_ds_for_read(struct pnfs_layout_segment *lseg,static struct nfs4_pnfs_ds *ff_layout_get_ds_for_read (struct nfs_pageio_descriptor *pgio,u32 *best_idx, -u32 offset, +u64 offset,u32 *dss_id) {struct pnfs_layout_segment *lseg = pgio->pg_lseg;
Signature · 1 line
2.53.0
PATCH 3/21 NFSv4/flexfiles: Bound page coalescing on the absolute stripe offset
In reply to Benjamin Coddingtonff_layout_pg_test() was copied from the files layout, which partitions data servers relative to pattern_offset and bounds coalescing with a segment-relative offset. Flexfiles has no pattern_offset:nfs4_ff_layout_calc_dss_id() partitions on the absolute file offset (RFC 8435 Section 6). So when a layout segment's offset is not stripe-unit-aligned, the segment-relative coalescing window is shifted off the absolute stripe grid and a coalesced I/O can straddle a stripe boundary, sending the bytes past it to the wrong data server.
Bound coalescing on the absolute offset to match calc_dss_id(). Aligned segments, including all whole-file layouts, are unchanged.
--- fs/nfs/flexfilelayout/flexfilelayout.c | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-)
fs/nfs/flexfilelayout/flexfilelayout.c +3 -4
@@ -996,7 +996,6 @@ ff_layout_pg_test(struct nfs_pageio_descriptor *pgio, struct nfs_page *prev,unsigned int size;u64 p_stripe, r_stripe;u32 stripe_offset; -u64 segment_offset = pgio->pg_lseg->pls_range.offset;u32 stripe_unit =FF_LAYOUT_LSEG (pgio->pg_lseg)->stripe_unit;/* calls nfs_generic_pg_test */
@@ -1008,8 +1007,8 @@ ff_layout_pg_test(struct nfs_pageio_descriptor *pgio, struct nfs_page *prev,/* see if req and prev are in the same stripe */ if (prev) { - p_stripe = (u64 )req_offset (prev) - segment_offset; - r_stripe = (u64 )req_offset (req) - segment_offset; + p_stripe = (u64 )req_offset (prev); + r_stripe = (u64 )req_offset (req);do_div (p_stripe, stripe_unit);do_div (r_stripe, stripe_unit);
@@ -1018,7 +1017,7 @@ ff_layout_pg_test(struct nfs_pageio_descriptor *pgio, struct nfs_page *prev, }/* calculate remaining bytes in the current stripe */ -div_u64_rem ((u64 )req_offset (req) - segment_offset, +div_u64_rem ((u64 )req_offset (req), stripe_unit, &stripe_offset);WARN_ON_ONCE (stripe_offset > stripe_unit);
Signature · 1 line
2.53.0
PATCH 4/21 NFSv4/filelayout: Anchor page coalescing on pattern_offset
In reply to Benjamin Coddingtonfilelayout_pg_test() bounds page coalescing to a single stripe unit using an offset relative to pls_range.offset, but the data server is selected by nfs4_fl_calc_j_index() using an offset relative to pattern_offset. When a segment's pattern_offset and range offset are not congruent modulo the stripe unit, the coalescing window is shifted off the DS-selection grid, so a coalesced I/O can straddle a stripe boundary and send the bytes past it to the wrong data server.
Anchor coalescing on pattern_offset to match nfs4_fl_calc_j_index(). Stripe-congruent segments, including the common whole-file pattern_offset 0 case, are unchanged.
--- fs/nfs/filelayout/filelayout.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
fs/nfs/filelayout/filelayout.c +4 -4
@@ -796,7 +796,7 @@ filelayout_pg_test(struct nfs_pageio_descriptor *pgio, struct nfs_page *prev,unsigned int size;u64 p_stripe, r_stripe;u32 stripe_offset; -u64 segment_offset = pgio->pg_lseg->pls_range.offset; +u64 pattern_offset =FILELAYOUT_LSEG (pgio->pg_lseg)->pattern_offset;u32 stripe_unit =FILELAYOUT_LSEG (pgio->pg_lseg)->stripe_unit;/* calls nfs_generic_pg_test */
@@ -808,8 +808,8 @@ filelayout_pg_test(struct nfs_pageio_descriptor *pgio, struct nfs_page *prev,/* see if req and prev are in the same stripe */ if (prev) { - p_stripe = (u64 )req_offset (prev) - segment_offset; - r_stripe = (u64 )req_offset (req) - segment_offset; + p_stripe = (u64 )req_offset (prev) - pattern_offset; + r_stripe = (u64 )req_offset (req) - pattern_offset;do_div (p_stripe, stripe_unit);do_div (r_stripe, stripe_unit);
@@ -818,7 +818,7 @@ filelayout_pg_test(struct nfs_pageio_descriptor *pgio, struct nfs_page *prev, }/* calculate remaining bytes in the current stripe */ -div_u64_rem ((u64 )req_offset (req) - segment_offset, +div_u64_rem ((u64 )req_offset (req) - pattern_offset, stripe_unit, &stripe_offset);WARN_ON_ONCE (stripe_offset > stripe_unit);
Signature · 1 line
2.53.0
PATCH 5/21 NFSv4/flexfiles: Reference the device node across DS setup
In reply to Benjamin CoddingtonThe flexfiles I/O setup paths read the mirror's pinned per-stripe device node (mirror->dss[dss_id].mirror_ds) repeatedly and locklessly, relying on the mirror's lifetime pin. To prepare for re-resolving that pointer in place on CB_NOTIFY_DEVICEID CHANGE, readers must hold their own reference on the node they are using rather than trusting the pin.
Replace ff_layout_init_mirror_ds() with ff_layout_get_mirror_ds(), which resolves on first use as before but returns the node with its own reference held. Thread the referenced node through nfs4_ff_layout_prepare_ds() and the DS selection helpers so each setup path snapshots the node once, and drop the reference when setup is done.
No functional change: the pointer is still resolved once and pinned for the life of the mirror.
--- fs/nfs/flexfilelayout/flexfilelayout.c | 140 ++++++++++++++-------- fs/nfs/flexfilelayout/flexfilelayout.h | 16 ++- fs/nfs/flexfilelayout/flexfilelayoutdev.c | 81 ++++++++----- 3 files changed, 149 insertions(+), 88 deletions(-)
fs/nfs/flexfilelayout/flexfilelayout.c +87 -53
@@ -875,7 +875,7 @@ ff_layout_mark_ds_reachable(struct pnfs_layout_segment *lseg, u32 idx, u32 dss_infs4_mark_deviceid_available (devid); } -static struct nfs4_pnfs_ds * +static struct nfs4_ff_layout_ds * ff_layout_choose_ds_for_read(struct pnfs_layout_segment *lseg, u32 start_idx, u32 *best_idx, u64 offset, u32 *dss_id,
@@ -883,7 +883,9 @@ ff_layout_choose_ds_for_read(struct pnfs_layout_segment *lseg, {struct nfs4_ff_layout_segment *fls =FF_LAYOUT_LSEG (lseg);struct nfs4_ff_layout_mirror *mirror; -struct nfs4_pnfs_ds *ds =ERR_PTR (-EAGAIN ); +struct nfs4_ff_layout_ds *mirror_ds; +struct nfs4_ff_layout_ds *ret =ERR_PTR (-EAGAIN ); +struct nfs4_pnfs_ds *ds;u32 idx;/* mirrors are initially sorted by efficiency */
@@ -893,25 +895,32 @@ ff_layout_choose_ds_for_read(struct pnfs_layout_segment *lseg, fls->stripe_unit, fls->mirror_array[idx]->dss_count, offset); - ds =nfs4_ff_layout_prepare_ds (lseg, mirror, *dss_id, false); -if (IS_ERR (ds)) + mirror_ds =ff_layout_get_mirror_ds (lseg->pls_layout, mirror, + *dss_id); + ds =nfs4_ff_layout_prepare_ds (lseg, mirror, mirror_ds, + *dss_id, false); +if (IS_ERR (ds)) { +nfs4_ff_layout_put_deviceid (mirror_ds); + ret =ERR_CAST (ds);continue ; + }if (check_device && -nfs4_test_deviceid_unavailable (&mirror->dss[*dss_id].mirror_ds->id_node)) { +nfs4_test_deviceid_unavailable (&mirror_ds->id_node)) { +nfs4_ff_layout_put_deviceid (mirror_ds);// reinitialize the error state in case if this is the last iteration - ds =ERR_PTR (-EINVAL ); + ret =ERR_PTR (-EINVAL );continue ; } *best_idx = idx; -break ; +return mirror_ds; } -return ds; +return ret; } -static struct nfs4_pnfs_ds * +static struct nfs4_ff_layout_ds *ff_layout_choose_any_ds_for_read (struct pnfs_layout_segment *lseg,u32 start_idx,u32 *best_idx,u64 offset,u32 *dss_id)
@@ -920,7 +929,7 @@ ff_layout_choose_any_ds_for_read(struct pnfs_layout_segment *lseg, offset, dss_id, false); } -static struct nfs4_pnfs_ds * +static struct nfs4_ff_layout_ds *ff_layout_choose_valid_ds_for_read (struct pnfs_layout_segment *lseg,u32 start_idx,u32 *best_idx,u64 offset,u32 *dss_id)
@@ -929,34 +938,36 @@ ff_layout_choose_valid_ds_for_read(struct pnfs_layout_segment *lseg, offset, dss_id, true); } -static struct nfs4_pnfs_ds * +static struct nfs4_ff_layout_ds *ff_layout_choose_best_ds_for_read (struct pnfs_layout_segment *lseg,u32 start_idx,u32 *best_idx,u64 offset,u32 *dss_id) { -struct nfs4_pnfs_ds *ds; +struct nfs4_ff_layout_ds *mirror_ds; - ds =ff_layout_choose_valid_ds_for_read (lseg, start_idx, best_idx, - offset, dss_id); -if (!IS_ERR (ds)) -return ds; + mirror_ds =ff_layout_choose_valid_ds_for_read (lseg, start_idx, + best_idx, offset, + dss_id); +if (!IS_ERR (mirror_ds)) +return mirror_ds;return ff_layout_choose_any_ds_for_read (lseg, start_idx, best_idx, offset, dss_id); } -static struct nfs4_pnfs_ds * +static struct nfs4_ff_layout_ds *ff_layout_get_ds_for_read (struct nfs_pageio_descriptor *pgio,u32 *best_idx,u64 offset,u32 *dss_id) {struct pnfs_layout_segment *lseg = pgio->pg_lseg; -struct nfs4_pnfs_ds *ds; +struct nfs4_ff_layout_ds *mirror_ds; - ds =ff_layout_choose_best_ds_for_read (lseg, pgio->pg_mirror_idx, - best_idx, offset, dss_id); -if (!IS_ERR (ds) || !pgio->pg_mirror_idx) -return ds; + mirror_ds =ff_layout_choose_best_ds_for_read (lseg, + pgio->pg_mirror_idx, + best_idx, offset, dss_id); +if (!IS_ERR (mirror_ds) || !pgio->pg_mirror_idx) +return mirror_ds;return ff_layout_choose_best_ds_for_read (lseg,0 , best_idx, offset, dss_id); }
@@ -1031,8 +1042,7 @@ ff_layout_pg_init_read(struct nfs_pageio_descriptor *pgio, struct nfs_page *req) {struct nfs_pgio_mirror *pgm; -struct nfs4_ff_layout_mirror *mirror; -struct nfs4_pnfs_ds *ds; +struct nfs4_ff_layout_ds *mirror_ds;u32 ds_idx, dss_id; if (NFS_SERVER(pgio->pg_inode)->flags &
@@ -1054,9 +1064,9 @@ ff_layout_pg_init_read(struct nfs_pageio_descriptor *pgio,/* Reset wb_nio, since getting layout segment was successful */ req->wb_nio =0 ; - ds =ff_layout_get_ds_for_read (pgio, &ds_idx, -req_offset (req), &dss_id); -if (IS_ERR (ds)) { + mirror_ds =ff_layout_get_ds_for_read (pgio, &ds_idx, +req_offset (req), &dss_id); +if (IS_ERR (mirror_ds)) {if (!ff_layout_no_fallback_to_mds (pgio->pg_lseg)) goto out_mds;pnfs_generic_pg_cleanup (pgio);
@@ -1065,9 +1075,9 @@ ff_layout_pg_init_read(struct nfs_pageio_descriptor *pgio, goto retry; } - mirror =FF_LAYOUT_COMP (pgio->pg_lseg, ds_idx); pgm = &pgio->pg_mirrors[0 ]; - pgm->pg_bsize = mirror->dss[dss_id].mirror_ds->ds_versions[0 ].rsize; + pgm->pg_bsize = mirror_ds->ds_versions[0 ].rsize; +nfs4_ff_layout_put_deviceid (mirror_ds); pgio->pg_mirror_idx = ds_idx;return ;
@@ -1102,6 +1112,7 @@ ff_layout_pg_init_write(struct nfs_pageio_descriptor *pgio, struct nfs_page *req) {struct nfs4_ff_layout_mirror *mirror; +struct nfs4_ff_layout_ds *mirror_ds;struct nfs_pgio_mirror *pgm;struct nfs4_pnfs_ds *ds;u32 i, dss_id;
@@ -1133,9 +1144,12 @@ ff_layout_pg_init_write(struct nfs_pageio_descriptor *pgio,FF_LAYOUT_LSEG (pgio->pg_lseg)->stripe_unit, mirror->dss_count, req_offset(req)); + mirror_ds =ff_layout_get_mirror_ds (pgio->pg_lseg->pls_layout, + mirror, dss_id); ds =nfs4_ff_layout_prepare_ds (pgio->pg_lseg, mirror, - dss_id, true); + mirror_ds, dss_id, true);if (IS_ERR (ds)) { +nfs4_ff_layout_put_deviceid (mirror_ds);if (!ff_layout_no_fallback_to_mds (pgio->pg_lseg)) goto out_mds;pnfs_generic_pg_cleanup (pgio);
@@ -1144,7 +1158,8 @@ ff_layout_pg_init_write(struct nfs_pageio_descriptor *pgio, goto retry; } pgm = &pgio->pg_mirrors[i]; - pgm->pg_bsize = mirror->dss[dss_id].mirror_ds->ds_versions[0 ].wsize; + pgm->pg_bsize = mirror_ds->ds_versions[0 ].wsize; +nfs4_ff_layout_put_deviceid (mirror_ds); } if (NFS_SERVER(pgio->pg_inode)->flags &
@@ -1276,14 +1291,16 @@ static void ff_layout_resend_pnfs_read(struct nfs_pgio_header *hdr)u32 idx = hdr->pgio_mirror_idx +1 ;u32 new_idx =0 ;u32 dss_id =0 ; -struct nfs4_pnfs_ds *ds; +struct nfs4_ff_layout_ds *mirror_ds; - ds =ff_layout_choose_any_ds_for_read (hdr->lseg, idx, &new_idx, - hdr->args.offset, &dss_id); -if (IS_ERR (ds)) + mirror_ds =ff_layout_choose_any_ds_for_read (hdr->lseg, idx, &new_idx, + hdr->args.offset, &dss_id); +if (IS_ERR (mirror_ds)) {pnfs_error_mark_layout_for_return (hdr->inode, hdr->lseg); -else + }else { +nfs4_ff_layout_put_deviceid (mirror_ds);ff_layout_send_layouterror (hdr->lseg); + }pnfs_read_resend_pnfs (hdr, new_idx); }
@@ -2144,6 +2161,7 @@ ff_layout_read_pagelist(struct nfs_pgio_header *hdr)struct rpc_clnt *ds_clnt;struct nfsd_file *localio;struct nfs4_ff_layout_mirror *mirror; +struct nfs4_ff_layout_ds *mirror_ds;const struct cred *ds_cred;loff_t offset = hdr->args.offset;u32 idx = hdr->pgio_mirror_idx;
@@ -2161,22 +2179,24 @@ ff_layout_read_pagelist(struct nfs_pgio_header *hdr)FF_LAYOUT_LSEG (lseg)->stripe_unit, mirror->dss_count, offset); - ds =nfs4_ff_layout_prepare_ds (lseg, mirror, dss_id, false); + mirror_ds =ff_layout_get_mirror_ds (lseg->pls_layout, mirror, dss_id); + ds =nfs4_ff_layout_prepare_ds (lseg, mirror, mirror_ds, dss_id, false);if (IS_ERR (ds)) { ds_fatal_error =nfs_error_is_fatal (PTR_ERR (ds)); goto out_failed; } - ds_clnt =nfs4_ff_find_or_create_ds_client (mirror, ds->ds_clp, - hdr->inode, dss_id); + ds_clnt =nfs4_ff_find_or_create_ds_client (mirror_ds, ds->ds_clp, + hdr->inode);if (IS_ERR (ds_clnt)) goto out_failed; - ds_cred =ff_layout_get_ds_cred (mirror, &lseg->pls_range, hdr->cred, dss_id); + ds_cred =ff_layout_get_ds_cred (mirror, &lseg->pls_range, hdr->cred, + mirror_ds, dss_id);if (!ds_cred) goto out_failed; - vers =nfs4_ff_layout_ds_version (mirror, dss_id); + vers =nfs4_ff_layout_ds_version (mirror_ds);dprintk ("%s USE DS: %s cl_count %d vers %d\n" , __func__, ds->ds_remotestr,refcount_read (&ds->ds_clp->cl_count), vers);
@@ -2188,7 +2208,8 @@ ff_layout_read_pagelist(struct nfs_pgio_header *hdr)if (fh) hdr->args.fh = fh; -nfs4_ff_layout_select_ds_stateid (mirror, dss_id, &hdr->args.stateid); +nfs4_ff_layout_select_ds_stateid (mirror, mirror_ds, dss_id, + &hdr->args.stateid);/* * Note that if we ever decide to split across DSes,
@@ -2211,9 +2232,11 @@ ff_layout_read_pagelist(struct nfs_pgio_header *hdr) &ff_layout_read_call_ops_v4, 0, RPC_TASK_SOFTCONN, localio);put_cred (ds_cred); +nfs4_ff_layout_put_deviceid (mirror_ds);return PNFS_ATTEMPTED ; out_failed: +nfs4_ff_layout_put_deviceid (mirror_ds);if (ff_layout_avoid_mds_available_ds (lseg) && !ds_fatal_error)return PNFS_TRY_AGAIN ;if (ff_layout_no_fallback_to_mds (lseg)) {
@@ -2239,6 +2262,7 @@ ff_layout_write_pagelist(struct nfs_pgio_header *hdr, int sync)struct rpc_clnt *ds_clnt;struct nfsd_file *localio;struct nfs4_ff_layout_mirror *mirror; +struct nfs4_ff_layout_ds *mirror_ds;const struct cred *ds_cred;loff_t offset = hdr->args.offset;int vers;
@@ -2252,22 +2276,24 @@ ff_layout_write_pagelist(struct nfs_pgio_header *hdr, int sync)FF_LAYOUT_LSEG (lseg)->stripe_unit, mirror->dss_count, offset); - ds =nfs4_ff_layout_prepare_ds (lseg, mirror, dss_id, true); + mirror_ds =ff_layout_get_mirror_ds (lseg->pls_layout, mirror, dss_id); + ds =nfs4_ff_layout_prepare_ds (lseg, mirror, mirror_ds, dss_id, true);if (IS_ERR (ds)) { ds_fatal_error =nfs_error_is_fatal (PTR_ERR (ds)); goto out_failed; } - ds_clnt =nfs4_ff_find_or_create_ds_client (mirror, ds->ds_clp, - hdr->inode, dss_id); + ds_clnt =nfs4_ff_find_or_create_ds_client (mirror_ds, ds->ds_clp, + hdr->inode);if (IS_ERR (ds_clnt)) goto out_failed; - ds_cred =ff_layout_get_ds_cred (mirror, &lseg->pls_range, hdr->cred, dss_id); + ds_cred =ff_layout_get_ds_cred (mirror, &lseg->pls_range, hdr->cred, + mirror_ds, dss_id);if (!ds_cred) goto out_failed; - vers =nfs4_ff_layout_ds_version (mirror, dss_id); + vers =nfs4_ff_layout_ds_version (mirror_ds); dprintk("%s ino %llu sync %d req %zu@%llu DS: %s cl_count %d vers %d\n", __func__, hdr->inode->i_ino, sync, (size_t) hdr->args.count,
@@ -2282,7 +2308,8 @@ ff_layout_write_pagelist(struct nfs_pgio_header *hdr, int sync)if (fh) hdr->args.fh = fh; -nfs4_ff_layout_select_ds_stateid (mirror, dss_id, &hdr->args.stateid); +nfs4_ff_layout_select_ds_stateid (mirror, mirror_ds, dss_id, + &hdr->args.stateid);/* * Note that if we ever decide to split across DSes,
@@ -2304,9 +2331,11 @@ ff_layout_write_pagelist(struct nfs_pgio_header *hdr, int sync) &ff_layout_write_call_ops_v4, sync, RPC_TASK_SOFTCONN, localio);put_cred (ds_cred); +nfs4_ff_layout_put_deviceid (mirror_ds);return PNFS_ATTEMPTED ; out_failed: +nfs4_ff_layout_put_deviceid (mirror_ds);if (ff_layout_avoid_mds_available_ds (lseg) && !ds_fatal_error)return PNFS_TRY_AGAIN ;if (ff_layout_no_fallback_to_mds (lseg)) {
@@ -2341,6 +2370,7 @@ static int ff_layout_initiate_commit(struct nfs_commit_data *data, int how)struct rpc_clnt *ds_clnt;struct nfsd_file *localio;struct nfs4_ff_layout_mirror *mirror; +struct nfs4_ff_layout_ds *mirror_ds =NULL ;const struct cred *ds_cred;u32 idx, dss_id;int vers, ret;
@@ -2353,20 +2383,22 @@ static int ff_layout_initiate_commit(struct nfs_commit_data *data, int how) idx =calc_mirror_idx_from_commit (lseg, data->ds_commit_index); mirror =FF_LAYOUT_COMP (lseg, idx); dss_id =calc_dss_id_from_commit (lseg, data->ds_commit_index); - ds =nfs4_ff_layout_prepare_ds (lseg, mirror, dss_id, true); + mirror_ds =ff_layout_get_mirror_ds (lseg->pls_layout, mirror, dss_id); + ds =nfs4_ff_layout_prepare_ds (lseg, mirror, mirror_ds, dss_id, true);if (IS_ERR (ds)) goto out_err; - ds_clnt =nfs4_ff_find_or_create_ds_client (mirror, ds->ds_clp, - data->inode, dss_id); + ds_clnt =nfs4_ff_find_or_create_ds_client (mirror_ds, ds->ds_clp, + data->inode);if (IS_ERR (ds_clnt)) goto out_err; - ds_cred =ff_layout_get_ds_cred (mirror, &lseg->pls_range, data->cred, dss_id); + ds_cred =ff_layout_get_ds_cred (mirror, &lseg->pls_range, data->cred, + mirror_ds, dss_id);if (!ds_cred) goto out_err; - vers =nfs4_ff_layout_ds_version (mirror, dss_id); + vers =nfs4_ff_layout_ds_version (mirror_ds); dprintk("%s ino %llu, how %d cl_count %d vers %d\n", __func__, data->inode->i_ino, how, refcount_read(&ds->ds_clp->cl_count),
@@ -2392,8 +2424,10 @@ static int ff_layout_initiate_commit(struct nfs_commit_data *data, int how) &ff_layout_commit_call_ops_v4, how, RPC_TASK_SOFTCONN, localio);put_cred (ds_cred); +nfs4_ff_layout_put_deviceid (mirror_ds);return ret; out_err: +nfs4_ff_layout_put_deviceid (mirror_ds);pnfs_generic_prepare_to_resend_writes (data);pnfs_generic_commit_release (data);return -EAGAIN ;
fs/nfs/flexfilelayout/flexfilelayout.h +11 -5
@@ -207,9 +207,9 @@ ff_layout_no_read_on_rw(struct pnfs_layout_segment *lseg) }static inline int -nfs4_ff_layout_ds_version (const struct nfs4_ff_layout_mirror *mirror,u32 dss_id) +nfs4_ff_layout_ds_version (const struct nfs4_ff_layout_ds *mirror_ds) { -return mirror->dss[dss_id].mirror_ds->ds_versions[0 ].version; +return mirror_ds->ds_versions[0 ].version; } static inline u32
@@ -245,23 +245,29 @@ struct nfs_fh * nfs4_ff_layout_select_ds_fh(struct nfs4_ff_layout_mirror *mirror, u32 dss_id);void nfs4_ff_layout_select_ds_stateid (const struct nfs4_ff_layout_mirror *mirror, +const struct nfs4_ff_layout_ds *mirror_ds,u32 dss_id,nfs4_stateid *stateid); +struct nfs4_ff_layout_ds * +ff_layout_get_mirror_ds (struct pnfs_layout_hdr *lo, +struct nfs4_ff_layout_mirror *mirror, +u32 dss_id);struct nfs4_pnfs_ds *nfs4_ff_layout_prepare_ds (struct pnfs_layout_segment *lseg,struct nfs4_ff_layout_mirror *mirror, +struct nfs4_ff_layout_ds *mirror_ds,u32 dss_id,bool fail_return);struct rpc_clnt * -nfs4_ff_find_or_create_ds_client (struct nfs4_ff_layout_mirror *mirror, +nfs4_ff_find_or_create_ds_client (const struct nfs4_ff_layout_ds *mirror_ds,struct nfs_client *ds_clp, -struct inode *inode, -u32 dss_id); +struct inode *inode);const struct cred *ff_layout_get_ds_cred (struct nfs4_ff_layout_mirror *mirror,const struct pnfs_layout_range *range,const struct cred *mdscred, +const struct nfs4_ff_layout_ds *mirror_ds,u32 dss_id);bool ff_layout_avoid_mds_available_ds (struct pnfs_layout_segment *lseg);bool ff_layout_avoid_read_on_rw (struct pnfs_layout_segment *lseg);
fs/nfs/flexfilelayout/flexfilelayoutdev.c +51 -30
162 lines not on this page · raw message
Signature · 1 line
2.53.0
PATCH 6/21 NFSv4/flexfiles: Carry the device node reference across each I/O
In reply to Benjamin CoddingtonRather than dropping the device node reference when DS setup completes, transfer it to the in-flight I/O: carry it on nfs_pgio_header and nfs_commit_data as ds_dev, and release it when the header or commit data is released, alongside the lseg reference.
Convert the completion paths to use the carried node instead of re-reading the mirror's pinned pointer: DS error tracking, marking the deviceid available/unavailable, and deleting the deviceid on connection errors now act on the node the I/O was actually sent to. Once a CHANGE notification can re-point the pinned pointer mid-flight, this keeps error attribution on the old device rather than its replacement (and removes a NULL dereference had the pointer been reset to NULL).
FF_LAYOUT_DEVID_NODE() is now unused; remove it.
--- fs/nfs/flexfilelayout/flexfilelayout.c | 82 ++++++++++------------- fs/nfs/flexfilelayout/flexfilelayout.h | 15 +---- fs/nfs/flexfilelayout/flexfilelayoutdev.c | 11 +-- fs/nfs/pnfs.c | 2 + fs/nfs/pnfs.h | 8 +++ fs/nfs/pnfs_nfs.c | 1 + include/linux/nfs_xdr.h | 2 + 7 files changed, 58 insertions(+), 63 deletions(-)
fs/nfs/flexfilelayout/flexfilelayout.c +37 -45
@@ -857,24 +857,6 @@ nfs4_ff_layout_stat_io_end_write(struct rpc_task *task,spin_unlock (&mirror->lock); } -static void -ff_layout_mark_ds_unreachable (struct pnfs_layout_segment *lseg,u32 idx,u32 dss_id) -{ -struct nfs4_deviceid_node *devid =FF_LAYOUT_DEVID_NODE (lseg, idx, dss_id); - -if (devid) -nfs4_mark_deviceid_unavailable (devid); -} - -static void -ff_layout_mark_ds_reachable (struct pnfs_layout_segment *lseg,u32 idx,u32 dss_id) -{ -struct nfs4_deviceid_node *devid =FF_LAYOUT_DEVID_NODE (lseg, idx, dss_id); - -if (devid) -nfs4_mark_deviceid_available (devid); -} -static struct nfs4_ff_layout_ds * ff_layout_choose_ds_for_read(struct pnfs_layout_segment *lseg, u32 start_idx, u32 *best_idx,
@@ -1333,11 +1315,10 @@ static int ff_layout_async_handle_error_v4(struct rpc_task *task,struct nfs4_state *state, struct nfs_client *clp, struct pnfs_layout_segment *lseg, - u32 idx, u32 dss_id) + struct nfs4_deviceid_node *devid) { struct pnfs_layout_hdr *lo = lseg->pls_layout;struct inode *inode = lo->plh_inode; -struct nfs4_deviceid_node *devid =FF_LAYOUT_DEVID_NODE (lseg, idx, dss_id);struct nfs4_slot_table *tbl = &clp->cl_session->fc_slot_table;switch (op_status) {
@@ -1409,8 +1390,9 @@ static int ff_layout_async_handle_error_v4(struct rpc_task *task,case -ENODEV :dprintk ("%s DS connection error %d\n" , __func__, task->tk_status); -nfs4_delete_deviceid (devid->ld, devid->nfs_client, - &devid->deviceid); +if (devid) +nfs4_delete_deviceid (devid->ld, devid->nfs_client, + &devid->deviceid);rpc_wake_up (&tbl->slot_tbl_waitq);break ;default :
@@ -1434,9 +1416,8 @@ static int ff_layout_async_handle_error_v3(struct rpc_task *task,u32 op_status, struct nfs_client *clp, struct pnfs_layout_segment *lseg, - u32 idx, u32 dss_id) + struct nfs4_deviceid_node *devid) { - struct nfs4_deviceid_node *devid = FF_LAYOUT_DEVID_NODE(lseg, idx, dss_id); switch (op_status) { case NFS_OK:
@@ -1482,8 +1463,9 @@ static int ff_layout_async_handle_error_v3(struct rpc_task *task,default :dprintk ("%s DS connection error %d\n" , __func__, task->tk_status); -nfs4_delete_deviceid (devid->ld, devid->nfs_client, - &devid->deviceid); +if (devid) +nfs4_delete_deviceid (devid->ld, devid->nfs_client, + &devid->deviceid); } out_reset_to_pnfs:/* FIXME: Need to prevent infinite looping here. */
@@ -1500,12 +1482,13 @@ static int ff_layout_async_handle_error(struct rpc_task *task,struct nfs4_state *state, struct nfs_client *clp, struct pnfs_layout_segment *lseg, - u32 idx, u32 dss_id) + struct nfs4_deviceid_node *devid) { int vers = clp->cl_nfs_mod->rpc_vers->number;if (task->tk_status >=0 ) { -ff_layout_mark_ds_reachable (lseg, idx, dss_id); +if (devid) +nfs4_mark_deviceid_available (devid);return 0 ; }
@@ -1516,10 +1499,10 @@ static int ff_layout_async_handle_error(struct rpc_task *task,switch (vers) {case 3 :return ff_layout_async_handle_error_v3 (task, op_status, clp, - lseg, idx, dss_id); + lseg, devid);case 4 :return ff_layout_async_handle_error_v4 (task, op_status, state, - clp, lseg, idx, dss_id); + clp, lseg, devid);default :/* should never happen */ WARN_ON_ONCE (1 );
@@ -1528,6 +1511,7 @@ static int ff_layout_async_handle_error(struct rpc_task *task, }static void ff_layout_io_track_ds_error (struct pnfs_layout_segment *lseg, +struct nfs4_deviceid_node *devid,u32 idx,u32 dss_id,u64 offset,u64 length,u32 *op_status,int opnum,int error) {
@@ -1566,8 +1550,8 @@ static void ff_layout_io_track_ds_error(struct pnfs_layout_segment *lseg, mirror =FF_LAYOUT_COMP (lseg, idx); err =ff_layout_track_ds_error (FF_LAYOUT_FROM_HDR (lseg->pls_layout), - mirror, dss_id, offset, length, status, opnum, -nfs_io_gfp_mask ()); + mirror, devid, dss_id, offset, length, + status, opnum,nfs_io_gfp_mask ());switch (status) {case NFS4ERR_DELAY :
@@ -1575,7 +1559,8 @@ static void ff_layout_io_track_ds_error(struct pnfs_layout_segment *lseg,case NFS4ERR_PERM :break ;case NFS4ERR_NXIO : -ff_layout_mark_ds_unreachable (lseg, idx, dss_id); +if (devid) +nfs4_mark_deviceid_unavailable (devid);/* * Don't return the layout if this is a read and we still * have layouts to try
@@ -1603,7 +1588,7 @@ static int ff_layout_read_done_cb(struct rpc_task *task,int err;if (task->tk_status <0 ) { - ff_layout_io_track_ds_error(hdr->lseg, + ff_layout_io_track_ds_error(hdr->lseg, hdr->ds_dev, hdr->pgio_mirror_idx, dss_id, hdr->args.offset, hdr->args.count, &hdr->res.op_status, OP_READ,
@@ -1614,8 +1599,7 @@ static int ff_layout_read_done_cb(struct rpc_task *task, err =ff_layout_async_handle_error (task, hdr->res.op_status, hdr->args.context->state, hdr->ds_clp, hdr->lseg, - hdr->pgio_mirror_idx, - dss_id); + hdr->ds_dev);trace_nfs4_pnfs_read (hdr, err);clear_bit (NFS_IOHDR_RESEND_PNFS , &hdr->flags);
@@ -1808,7 +1792,7 @@ static int ff_layout_write_done_cb(struct rpc_task *task,int err;if (task->tk_status <0 ) { - ff_layout_io_track_ds_error(hdr->lseg, + ff_layout_io_track_ds_error(hdr->lseg, hdr->ds_dev, hdr->pgio_mirror_idx, dss_id, hdr->args.offset, hdr->args.count, &hdr->res.op_status, OP_WRITE,
@@ -1819,8 +1803,7 @@ static int ff_layout_write_done_cb(struct rpc_task *task, err =ff_layout_async_handle_error (task, hdr->res.op_status, hdr->args.context->state, hdr->ds_clp, hdr->lseg, - hdr->pgio_mirror_idx, - dss_id); + hdr->ds_dev);trace_nfs4_pnfs_write (hdr, err);clear_bit (NFS_IOHDR_RESEND_PNFS , &hdr->flags);
@@ -1862,7 +1845,7 @@ static int ff_layout_commit_done_cb(struct rpc_task *task,u32 dss_id =calc_dss_id_from_commit (data->lseg, data->ds_commit_index);if (task->tk_status <0 ) { -ff_layout_io_track_ds_error (data->lseg, idx, dss_id, +ff_layout_io_track_ds_error (data->lseg, data->ds_dev, idx, dss_id, data->args.offset, data->args.count, &data->res.op_status,OP_COMMIT , task->tk_status);
@@ -1870,8 +1853,8 @@ static int ff_layout_commit_done_cb(struct rpc_task *task, } err =ff_layout_async_handle_error (task, data->res.op_status, -NULL , data->ds_clp, data->lseg, idx, - dss_id); +NULL , data->ds_clp, data->lseg, + data->ds_dev);trace_nfs4_pnfs_commit_ds (data, err);switch (err) {
@@ -2226,13 +2209,16 @@ ff_layout_read_pagelist(struct nfs_pgio_header *hdr)ff_layout_read_record_layoutstats_start (&hdr->task, hdr); } +/* Transfer the device node reference to the I/O; put on release */ +pnfs_put_ds_dev (hdr->ds_dev); + hdr->ds_dev = &mirror_ds->id_node; +/* Perform an asynchronous read to ds */ nfs_initiate_pgio (ds_clnt, hdr, ds_cred, ds->ds_clp->rpc_ops, vers ==3 ? &ff_layout_read_call_ops_v3 : &ff_layout_read_call_ops_v4,0 ,RPC_TASK_SOFTCONN , localio);put_cred (ds_cred); -nfs4_ff_layout_put_deviceid (mirror_ds);return PNFS_ATTEMPTED ; out_failed:
@@ -2325,13 +2311,16 @@ ff_layout_write_pagelist(struct nfs_pgio_header *hdr, int sync)ff_layout_write_record_layoutstats_start (&hdr->task, hdr); } +/* Transfer the device node reference to the I/O; put on release */ +pnfs_put_ds_dev (hdr->ds_dev); + hdr->ds_dev = &mirror_ds->id_node; +/* Perform an asynchronous write */ nfs_initiate_pgio (ds_clnt, hdr, ds_cred, ds->ds_clp->rpc_ops, vers ==3 ? &ff_layout_write_call_ops_v3 : &ff_layout_write_call_ops_v4, sync,RPC_TASK_SOFTCONN , localio);put_cred (ds_cred); -nfs4_ff_layout_put_deviceid (mirror_ds);return PNFS_ATTEMPTED ; out_failed:
@@ -2419,12 +2408,15 @@ static int ff_layout_initiate_commit(struct nfs_commit_data *data, int how)ff_layout_commit_record_layoutstats_start (&data->task, data); } +/* Transfer the device node reference to the commit; put on release */ +pnfs_put_ds_dev (data->ds_dev); + data->ds_dev = &mirror_ds->id_node; + ret =nfs_initiate_commit (ds_clnt, data, ds->ds_clp->rpc_ops, vers ==3 ? &ff_layout_commit_call_ops_v3 : &ff_layout_commit_call_ops_v4, how,RPC_TASK_SOFTCONN , localio);put_cred (ds_cred); -nfs4_ff_layout_put_deviceid (mirror_ds);return ret; out_err:nfs4_ff_layout_put_deviceid (mirror_ds);
fs/nfs/flexfilelayout/flexfilelayout.h +1 -14
@@ -162,20 +162,6 @@ FF_LAYOUT_COMP(struct pnfs_layout_segment *lseg, u32 idx)return NULL ; } -static inline struct nfs4_deviceid_node * -FF_LAYOUT_DEVID_NODE (struct pnfs_layout_segment *lseg,u32 idx,u32 dss_id) -{ -struct nfs4_ff_layout_mirror *mirror =FF_LAYOUT_COMP (lseg, idx); - -if (mirror !=NULL ) { -struct nfs4_ff_layout_ds *mirror_ds = mirror->dss[dss_id].mirror_ds; - -if (!IS_ERR_OR_NULL (mirror_ds)) -return &mirror_ds->id_node; - } -return NULL ; -} -static inline u32 FF_LAYOUT_MIRROR_COUNT (struct pnfs_layout_segment *lseg) {
@@ -232,6 +218,7 @@ void nfs4_ff_layout_put_deviceid(struct nfs4_ff_layout_ds *mirror_ds);void nfs4_ff_layout_free_deviceid (struct nfs4_ff_layout_ds *mirror_ds);int ff_layout_track_ds_error (struct nfs4_flexfile_layout *flo,struct nfs4_ff_layout_mirror *mirror, +const struct nfs4_deviceid_node *devid,u32 dss_id,u64 offset,u64 length,int status,enum nfs_opnum4 opnum,gfp_t gfp_flags);void ff_layout_send_layouterror (struct pnfs_layout_segment *lseg);
fs/nfs/flexfilelayout/flexfilelayoutdev.c +7 -4
@@ -255,6 +255,7 @@ ff_layout_add_ds_error_locked(struct nfs4_flexfile_layout *flo,int ff_layout_track_ds_error (struct nfs4_flexfile_layout *flo,struct nfs4_ff_layout_mirror *mirror, +const struct nfs4_deviceid_node *devid,u32 dss_id,u64 offset,u64 length,int status,enum nfs_opnum4 opnum,gfp_t gfp_flags) {
@@ -263,7 +264,7 @@ int ff_layout_track_ds_error(struct nfs4_flexfile_layout *flo,if (status ==0 )return 0 ; -if (IS_ERR_OR_NULL (mirror->dss[dss_id].mirror_ds)) +if (devid ==NULL )return -EINVAL ; dserr =kmalloc_obj (*dserr, gfp_flags);
@@ -276,8 +277,7 @@ int ff_layout_track_ds_error(struct nfs4_flexfile_layout *flo, dserr->status = status; dserr->opnum = opnum;nfs4_stateid_copy (&dserr->stateid, &mirror->dss[dss_id].stateid); -memcpy (&dserr->deviceid, &mirror->dss[dss_id].mirror_ds->id_node.deviceid, -NFS4_DEVICEID4_SIZE ); +memcpy (&dserr->deviceid, &devid->deviceid,NFS4_DEVICEID4_SIZE );spin_lock (&flo->generic_hdr.plh_inode->i_lock);ff_layout_add_ds_error_locked (flo, dserr);
@@ -442,7 +442,10 @@ nfs4_ff_layout_prepare_ds(struct pnfs_layout_segment *lseg, } noconnect:ff_layout_track_ds_error (FF_LAYOUT_FROM_HDR (lseg->pls_layout), - mirror, dss_id, lseg->pls_range.offset, + mirror, +IS_ERR_OR_NULL (mirror_ds) ? +NULL : &mirror_ds->id_node, + dss_id, lseg->pls_range.offset, lseg->pls_range.length,NFS4ERR_NXIO ,OP_ILLEGAL ,GFP_NOIO );ff_layout_send_layouterror (lseg);
fs/nfs/pnfs.c +2 -0
@@ -3104,6 +3104,7 @@ pnfs_do_write(struct nfs_pageio_descriptor *desc,static void pnfs_writehdr_free (struct nfs_pgio_header *hdr) { +pnfs_put_ds_dev (hdr->ds_dev);pnfs_put_lseg (hdr->lseg);nfs_pgio_header_free (hdr); }
@@ -3249,6 +3250,7 @@ pnfs_do_read(struct nfs_pageio_descriptor *desc, struct nfs_pgio_header *hdr)static void pnfs_readhdr_free (struct nfs_pgio_header *hdr) { +pnfs_put_ds_dev (hdr->ds_dev);pnfs_put_lseg (hdr->lseg);nfs_pgio_header_free (hdr); }
fs/nfs/pnfs.h +8 -0
@@ -384,6 +384,14 @@ void nfs4_init_deviceid_node(struct nfs4_deviceid_node *, struct nfs_server *, const struct nfs4_deviceid *); bool nfs4_put_deviceid_node(struct nfs4_deviceid_node *);void nfs4_mark_deviceid_available (struct nfs4_deviceid_node *node); + +/* Put the device node reference carried by an in-flight I/O, if any */ +static inline void pnfs_put_ds_dev (struct nfs4_deviceid_node *dev) +{ +if (dev) +nfs4_put_deviceid_node (dev); +} +void nfs4_mark_deviceid_unavailable (struct nfs4_deviceid_node *node);bool nfs4_test_deviceid_unavailable (struct nfs4_deviceid_node *node);void nfs4_deviceid_purge_client (const struct nfs_client *);
fs/nfs/pnfs_nfs.c +1 -0
@@ -55,6 +55,7 @@ void pnfs_generic_commit_release(void *calldata)struct nfs_commit_data *data = calldata; data->completion_ops->completion (data); +pnfs_put_ds_dev (data->ds_dev);pnfs_put_lseg (data->lseg);nfs_put_client (data->ds_clp);nfs_commitdata_release (data);
include/linux/nfs_xdr.h +2 -0
@@ -1687,6 +1687,7 @@ struct nfs_pgio_header {struct nfs_client *ds_clp;/* pNFS data server */ u32 ds_commit_idx;/* ds index if ds_clp is set */ u32 pgio_mirror_idx;/* mirror index in pgio layer */ +struct nfs4_deviceid_node *ds_dev;/* device node ref held across the I/O */ };struct nfs_mds_commit_info {
@@ -1725,6 +1726,7 @@ struct nfs_commit_data {struct nfs_open_context *context;struct pnfs_layout_segment *lseg;struct nfs_client *ds_clp;/* pNFS data server */ +struct nfs4_deviceid_node *ds_dev;/* device node ref held across the commit */ int ds_commit_index;loff_t lwb;const struct rpc_call_ops *mds_ops;
Signature · 1 line
2.53.0
PATCH 7/21 NFSv4/flexfiles: Hold a device node reference for layoutstats encoding
In reply to Benjamin Coddingtonff_layout_mirror_prepare_stats() records a stripe's device under i_lock, but the layoutstats/layoutreturn XDR encode runs later and dereferences the mirror's pinned device node (for the DS netaddr) with no reference of its own. Once a CHANGE notification can re-point the pinned node, that read becomes use-after-free.
Take a node reference at prepare time and carry it with the stripe in a per-devinfo nfs4_ff_layoutstat_priv, released in ff_layout_free_layoutstats(). The priv entries live alongside the devinfo array: co-allocated for LAYOUTSTATS, an additional member of nfs4_flexfile_layoutreturn_args for LAYOUTRETURN.
--- fs/nfs/flexfilelayout/flexfilelayout.c | 53 +++++++++++++++++--------- fs/nfs/flexfilelayout/flexfilelayout.h | 11 ++++++ 2 files changed, 47 insertions(+), 17 deletions(-)
fs/nfs/flexfilelayout/flexfilelayout.c +36 -17
@@ -44,10 +44,11 @@ static void ff_layout_read_record_layoutstats_done(struct rpc_task *task,static int ff_layout_mirror_prepare_stats (struct pnfs_layout_hdr *lo,struct nfs42_layoutstat_devinfo *devinfo, +struct nfs4_ff_layoutstat_priv *priv,int dev_limit,enum nfs4_ff_op_type type);static void ff_layout_encode_ff_layoutupdate (struct xdr_stream *xdr,const struct nfs42_layoutstat_devinfo *devinfo, -struct nfs4_ff_layout_ds_stripe *dss_info); +struct nfs4_ff_layoutstat_priv *priv);static struct pnfs_layout_hdr *ff_layout_alloc_layout_hdr (struct inode *inode,gfp_t gfp_flags)
@@ -2703,7 +2704,7 @@ ff_layout_prepare_layoutreturn(struct nfs4_layoutreturn_args *args)spin_lock (&args->inode->i_lock); ff_args->num_dev =ff_layout_mirror_prepare_stats ( - &ff_layout->generic_hdr, &ff_args->devinfo[0 ], + &ff_layout->generic_hdr, &ff_args->devinfo[0 ], &ff_args->priv[0 ],ARRAY_SIZE (ff_args->devinfo),NFS4_FF_OP_LAYOUTRETURN );spin_unlock (&args->inode->i_lock);
@@ -2878,10 +2879,11 @@ ff_layout_encode_io_latency(struct xdr_stream *xdr,static void ff_layout_encode_ff_layoutupdate (struct xdr_stream *xdr,const struct nfs42_layoutstat_devinfo *devinfo, -struct nfs4_ff_layout_ds_stripe *dss_info) +struct nfs4_ff_layoutstat_priv *priv) { +struct nfs4_ff_layout_ds_stripe *dss_info = priv->dss_info;struct nfs4_pnfs_ds_addr *da; -struct nfs4_pnfs_ds *ds = dss_info->mirror_ds->ds; +struct nfs4_pnfs_ds *ds = priv->mirror_ds->ds;struct nfs_fh *fh = &dss_info->fh_versions[0 ];__be32 *p;
@@ -2928,10 +2930,10 @@ ff_layout_encode_layoutstats(struct xdr_stream *xdr, const void *args,static void ff_layout_free_layoutstats (struct nfs4_xdr_opaque_data *opaque) { -struct nfs4_ff_layout_ds_stripe *dss_info = opaque->data; -struct nfs4_ff_layout_mirror *mirror = dss_info->mirror; +struct nfs4_ff_layoutstat_priv *priv = opaque->data; -ff_layout_put_mirror (mirror); +nfs4_ff_layout_put_deviceid (priv->mirror_ds); +ff_layout_put_mirror (priv->dss_info->mirror); }static const struct nfs4_xdr_opaque_ops layoutstat_ops = {
@@ -2942,12 +2944,13 @@ static const struct nfs4_xdr_opaque_ops layoutstat_ops = {static int ff_layout_mirror_prepare_stats (struct pnfs_layout_hdr *lo,struct nfs42_layoutstat_devinfo *devinfo, +struct nfs4_ff_layoutstat_priv *priv,int dev_limit,enum nfs4_ff_op_type type) {struct nfs4_flexfile_layout *ff_layout =FF_LAYOUT_FROM_HDR (lo);struct nfs4_ff_layout_mirror *mirror;struct nfs4_ff_layout_ds_stripe *dss_info; -struct nfs4_deviceid_node *dev; +struct nfs4_ff_layout_ds *mirror_ds;int i =0 , dss_id;list_for_each_entry (mirror, &ff_layout->mirrors, mirrors) {
@@ -2955,7 +2958,8 @@ ff_layout_mirror_prepare_stats(struct pnfs_layout_hdr *lo, dss_info = &mirror->dss[dss_id];if (i >= dev_limit)break ; -if (IS_ERR_OR_NULL (dss_info->mirror_ds)) + mirror_ds = dss_info->mirror_ds; +if (IS_ERR_OR_NULL (mirror_ds))continue ; if (!test_and_clear_bit(NFS4_FF_MIRROR_STAT_AVAIL, &mirror->flags) &&
@@ -2964,9 +2968,14 @@ ff_layout_mirror_prepare_stats(struct pnfs_layout_hdr *lo,/* mirror refcount put in cleanup_layoutstats */ if (!refcount_inc_not_zero (&mirror->ref))continue ; - dev = &dss_info->mirror_ds->id_node; +/* +* The mirror's pin holds the node while we're under +* i_lock; take a reference for the encode, put in +* ff_layout_free_layoutstats(). +*/ +atomic_inc (&mirror_ds->id_node.ref);memcpy (&devinfo->dev_id, - &dev->deviceid, + &mirror_ds->id_node.deviceid,NFS4_DEVICEID4_SIZE ); devinfo->offset =0 ; devinfo->length =NFS4_MAX_UINT64 ;
@@ -2982,9 +2991,12 @@ ff_layout_mirror_prepare_stats(struct pnfs_layout_hdr *lo,spin_unlock (&mirror->lock); devinfo->layout_type =LAYOUT_FLEX_FILES ; devinfo->ld_private.ops = &layoutstat_ops; - devinfo->ld_private.data = &mirror->dss[dss_id]; + priv->dss_info = dss_info; + priv->mirror_ds = mirror_ds; + devinfo->ld_private.data = priv; devinfo++; + priv++; i++; } }
@@ -2995,21 +3007,28 @@ static int ff_layout_prepare_layoutstats(struct nfs42_layoutstat_args *args) {struct pnfs_layout_hdr *lo;struct nfs4_flexfile_layout *ff_layout; +struct nfs4_ff_layoutstat_priv *priv;const int dev_count =PNFS_LAYOUTSTATS_MAXDEV ; -/* For now, send at most PNFS_LAYOUTSTATS_MAXDEV statistics */ - args->devinfo =kmalloc_objs (*args->devinfo, dev_count, -nfs_io_gfp_mask ()); +/* +* For now, send at most PNFS_LAYOUTSTATS_MAXDEV statistics. +* The per-devinfo private entries are co-allocated after the +* devinfo array and freed along with it. +*/ + args->devinfo =kmalloc (dev_count * (sizeof (*args->devinfo) + +sizeof (*priv)), +nfs_io_gfp_mask ());if (!args->devinfo)return -ENOMEM ; + priv = (struct nfs4_ff_layoutstat_priv *)&args->devinfo[dev_count];spin_lock (&args->inode->i_lock); lo =NFS_I (args->inode)->layout;if (lo &&pnfs_layout_is_valid (lo)) { ff_layout =FF_LAYOUT_FROM_HDR (lo); args->num_dev =ff_layout_mirror_prepare_stats ( - &ff_layout->generic_hdr, &args->devinfo[0 ], dev_count, -NFS4_FF_OP_LAYOUTSTATS ); + &ff_layout->generic_hdr, &args->devinfo[0 ], priv, + dev_count,NFS4_FF_OP_LAYOUTSTATS ); }else args->num_dev =0 ;spin_unlock (&args->inode->i_lock);
fs/nfs/flexfilelayout/flexfilelayout.h +11 -0
@@ -124,9 +124,20 @@ struct nfs4_flexfile_layout {unsigned long flags; }; +/* +* Per-devinfo private data for a layoutstats/layoutreturn encode: the +* stripe the stats describe plus a reference on its device node so the +* node (and its DS addresses) stay valid until the XDR encode runs. +*/ +struct nfs4_ff_layoutstat_priv { +struct nfs4_ff_layout_ds_stripe *dss_info; +struct nfs4_ff_layout_ds *mirror_ds; +}; +struct nfs4_flexfile_layoutreturn_args {struct list_head errors;struct nfs42_layoutstat_devinfo devinfo[FF_LAYOUTSTATS_MAXDEV ]; +struct nfs4_ff_layoutstat_priv priv[FF_LAYOUTSTATS_MAXDEV ];unsigned int num_errors;unsigned int num_dev;struct page *pages[1 ];
Signature · 1 line
2.53.0
PATCH 8/21 NFSv4/flexfiles: Make the pinned device node pointer RCU-managed
In reply to Benjamin CoddingtonAnnotate mirror->dss[dss_id].mirror_ds as __rcu and convert the remaining readers, completing the preparation for re-pointing the pinned node while I/O is in flight:
- ff_layout_get_mirror_ds() takes its reference under rcu_read_lock() with atomic_inc_not_zero(), retrying if it races a reset; the resolve path takes the caller's reference before publishing the node so a concurrent reset cannot free it under the caller. This is the shape of get_huge_zero_folio(): try-ref, else create and install with cmpxchg (retrying on a lost race) holding one reference for the pointer and one for the caller, torn down elsewhere by xchg + put.
- The availability scans hold rcu_read_lock() across the walk; they only test flags on the RCU-freed node.
- ff_layout_cancel_io() takes a reference around the cancel/disconnect calls, which may block.
- ff_layout_mirror_prepare_stats() runs under i_lock, which a future re-pointing walk must also hold.
- ff_layout_free_mirror() tears down the last reference; no concurrency.
The pointer is still only ever set once per mirror lifetime, so there is no behavior change; this commit makes the subsequent in-place re-resolve on CB_NOTIFY_DEVICEID CHANGE safe to introduce.
--- fs/nfs/flexfilelayout/flexfilelayout.c | 29 ++++-- fs/nfs/flexfilelayout/flexfilelayout.h | 2 +- fs/nfs/flexfilelayout/flexfilelayoutdev.c | 112 ++++++++++++++-------- 3 files changed, 92 insertions(+), 51 deletions(-)
fs/nfs/flexfilelayout/flexfilelayout.c +20 -9
@@ -313,7 +313,9 @@ static void ff_layout_free_mirror(struct nfs4_ff_layout_mirror *mirror) cred =rcu_access_pointer (mirror->dss[dss_id].rw_cred);put_cred (cred);nfs_close_local_fh (&mirror->dss[dss_id].nfl); -nfs4_ff_layout_put_deviceid (mirror->dss[dss_id].mirror_ds); +/* the last reference to the mirror is gone; no concurrency */ +nfs4_ff_layout_put_deviceid (rcu_dereference_protected ( + mirror->dss[dss_id].mirror_ds,1 )); }kfree (mirror->dss);
@@ -2476,22 +2478,29 @@ static void ff_layout_cancel_io(struct pnfs_layout_segment *lseg)for (idx =0 ; idx < flseg->mirror_array_cnt; idx++) { mirror = flseg->mirror_array[idx];for (dss_id =0 ; dss_id < mirror->dss_count; dss_id++) { - mirror_ds = mirror->dss[dss_id].mirror_ds; -if (IS_ERR_OR_NULL (mirror_ds)) +rcu_read_lock (); + mirror_ds =rcu_dereference (mirror->dss[dss_id].mirror_ds); +if (IS_ERR_OR_NULL (mirror_ds) || + !atomic_inc_not_zero (&mirror_ds->id_node.ref)) { +rcu_read_unlock ();continue ; - ds = mirror->dss[dss_id].mirror_ds->ds; + } +rcu_read_unlock (); + ds = mirror_ds->ds;if (!ds) -continue ; + goto next; ds_clp = ds->ds_clp;if (!ds_clp) -continue ; + goto next; clnt = ds_clp->cl_rpcclient;if (!clnt) -continue ; + goto next;if (!rpc_cancel_tasks (clnt, -EAGAIN , ff_layout_match_io, lseg)) -continue ; + goto next;rpc_clnt_disconnect (clnt); +next: +nfs4_ff_layout_put_deviceid (mirror_ds); } } }
@@ -2958,7 +2967,9 @@ ff_layout_mirror_prepare_stats(struct pnfs_layout_hdr *lo, dss_info = &mirror->dss[dss_id];if (i >= dev_limit)break ; - mirror_ds = dss_info->mirror_ds; + mirror_ds =rcu_dereference_protected ( + dss_info->mirror_ds, +lockdep_is_held (&lo->plh_inode->i_lock));if (IS_ERR_OR_NULL (mirror_ds))continue ; if (!test_and_clear_bit(NFS4_FF_MIRROR_STAT_AVAIL,
fs/nfs/flexfilelayout/flexfilelayout.h +1 -1
@@ -79,7 +79,7 @@ struct nfs4_ff_layout_ds_stripe {struct nfs4_ff_layout_mirror *mirror;struct nfs4_deviceid devid;u32 efficiency; -struct nfs4_ff_layout_ds *mirror_ds; +struct nfs4_ff_layout_ds __rcu *mirror_ds;u32 fh_versions_cnt;struct nfs_fh *fh_versions;nfs4_stateid stateid;
fs/nfs/flexfilelayout/flexfilelayoutdev.c +71 -41
155 lines not on this page · raw message
Signature · 1 line
2.53.0
PATCH 9/21 pNFS: Add a reresolve_deviceid layout driver hook
In reply to Benjamin CoddingtonRFC 8881 Section 12.2.10 lets a server change a deviceid's mapping under live layouts by sending CB_NOTIFY_DEVICEID CHANGE instead of recalling the layouts, but the client's only response today is to unhash the cached device, which never reaches references pinned inside a layout driver's segments.
Add a reresolve_deviceid hook to pnfs_layoutdriver_type and a generic driver, pnfs_layout_reresolve_deviceid_byclid(), that walks every layout on every server of the client and invokes the hook under the layout inode's i_lock. Because the final put of a device node can sleep (it may tear down the DS nfs_client), the hook must not drop references
itself: it collects the nodes it un-pins on a list through the new nfs4_deviceid_node.put_list member, and the generic driver puts them once all locks are dropped.
No driver implements the hook yet, and nothing calls the walker: no behavior change.
--- fs/nfs/pnfs.c | 68 +++++++++++++++++++++++++++++++++++++++++++++++++++ fs/nfs/pnfs.h | 19 ++++++++++++++ 2 files changed, 87 insertions(+)
fs/nfs/pnfs.c +68 -0
@@ -2876,6 +2876,74 @@ pnfs_layout_return_unused_byclid(struct nfs_client *clp, &range); } +struct pnfs_reresolve_deviceid_args { +const struct pnfs_layoutdriver_type *ld; +const struct nfs4_deviceid *id; +bool immediate; +struct list_head put_list; +}; + +static int pnfs_layout_reresolve_deviceid_byserver (struct nfs_server *server, +void *data) +{ +struct pnfs_reresolve_deviceid_args *args = data; +struct pnfs_layout_hdr *lo; +struct inode *inode; + +if (server->pnfs_curr_ld != args->ld) +return 0 ; + +rcu_read_lock (); +list_for_each_entry_rcu (lo, &server->layouts, plh_layouts) { + inode = lo->plh_inode; +if (!inode) +continue ; +spin_lock (&inode->i_lock); +if (lo->plh_inode == inode &&pnfs_layout_is_valid (lo)) + args->ld->reresolve_deviceid (lo, args->id, + args->immediate, + &args->put_list); +spin_unlock (&inode->i_lock); + } +rcu_read_unlock (); +return 0 ; +} + +/* +* pnfs_layout_reresolve_deviceid_byclid - re-point live layouts at a +* changed deviceid mapping (CB_NOTIFY_DEVICEID CHANGE) +* +* Walk every layout of @clp's servers using @ld and invoke the driver's +* reresolve_deviceid hook for @id. The hook runs under each layout +* inode's i_lock and defers its device node puts to the put_list, which +* is drained here once all locks are dropped (the final put can sleep). +*/ +void +pnfs_layout_reresolve_deviceid_byclid (struct nfs_client *clp, +const struct pnfs_layoutdriver_type *ld, +const struct nfs4_deviceid *id, +bool immediate) +{ +struct pnfs_reresolve_deviceid_args args = { + .ld = ld, + .id = id, + .immediate = immediate, + .put_list =LIST_HEAD_INIT (args.put_list), + }; +struct nfs4_deviceid_node *node, *tmp; + +if (!ld->reresolve_deviceid) +return ; + +nfs_client_for_each_server (clp, + pnfs_layout_reresolve_deviceid_byserver, &args); + +list_for_each_entry_safe (node, tmp, &args.put_list, put_list) { +list_del (&node->put_list); +nfs4_put_deviceid_node (node); + } +} +/* Check if we have we have a valid layout but if there isn't an intersection * between the request and the pgio->pg_lseg, put this pgio->pg_lseg away. */
fs/nfs/pnfs.h +19 -0
@@ -170,6 +170,17 @@ struct pnfs_layoutdriver_type {struct nfs4_deviceid_node * (*alloc_deviceid_node) (struct nfs_server *server,struct pnfs_device *pdev,gfp_t gfp_flags); +/* +* Re-resolve any of @lo's references to the changed deviceid @id +* (CB_NOTIFY_DEVICEID CHANGE). Called under @lo's inode i_lock; +* must not sleep. Device node references given up here must be +* collected on @put_list (via node->put_list) for the caller to +* put once all locks are dropped. +*/ +void (*reresolve_deviceid)(struct pnfs_layout_hdr *lo, +const struct nfs4_deviceid *id, +bool immediate, +struct list_head *put_list);int (*prepare_layoutreturn) (struct nfs4_layoutreturn_args *);
@@ -353,6 +364,10 @@ void pnfs_error_mark_layout_for_return(struct inode *inode, struct pnfs_layout_segment *lseg);void pnfs_layout_return_unused_byclid (struct nfs_client *clp,enum pnfs_iomode iomode); +void pnfs_layout_reresolve_deviceid_byclid (struct nfs_client *clp, +const struct pnfs_layoutdriver_type *ld, +const struct nfs4_deviceid *id, +bool immediate);int pnfs_layout_handle_reboot (struct nfs_client *clp);/* nfs4_deviceid_flags */
@@ -373,6 +388,10 @@ struct nfs4_deviceid_node {struct nfs4_deviceid deviceid;struct rcu_head rcu;atomic_t ref; +/* deferred-put linkage for in-place re-resolve; owned by whoever +* exchanged this node out of its pinning pointer +*/ +struct list_head put_list; }; struct nfs4_deviceid_node *
Signature · 1 line
2.53.0
PATCH 10/21 NFSv4/flexfiles: Implement in-place device re-resolve on CHANGE
In reply to Benjamin CoddingtonImplement the reresolve_deviceid hook: walk the layout's mirrors and, for every stripe whose raw deviceid matches the changed one, exchange the pinned device node out for NULL. In-flight I/O completes on the old node through its own reference; the next I/O to the stripe re-resolves via ff_layout_get_mirror_ds() and picks up the server's new mapping with a fresh GETDEVICEINFO. The un-pinned references are handed back on the walker's list to be put outside the locks.
Matching uses the raw deviceid decoded from the layout (dss[].devid), so stripes whose node was never resolved, or already exchanged, are untouched. Nothing dispatches CHANGE to the walker yet: no behavior change.
--- fs/nfs/flexfilelayout/flexfilelayout.c | 36 ++++++++++++++++++++++++++ 1 file changed, 36 insertions(+)
fs/nfs/flexfilelayout/flexfilelayout.c +36 -0
@@ -2505,6 +2505,41 @@ static void ff_layout_cancel_io(struct pnfs_layout_segment *lseg) } } +/* +* The server changed the mapping for deviceid @id (CB_NOTIFY_DEVICEID +* CHANGE). Un-pin every stripe device node resolved from @id in @lo's +* mirrors: in-flight I/O completes on the old node via its own +* reference, and the next I/O to the stripe re-resolves, picking up the +* new mapping with a fresh GETDEVICEINFO (the stale cache entry has +* already been unhashed). +* +* Called under @lo's inode i_lock; the un-pinned references are handed +* to the caller on @head to put outside the lock. +*/ +static void ff_layout_reresolve_deviceid (struct pnfs_layout_hdr *lo, +const struct nfs4_deviceid *id, +bool immediate, +struct list_head *head) +{ +struct nfs4_flexfile_layout *flo =FF_LAYOUT_FROM_HDR (lo); +struct nfs4_ff_layout_mirror *mirror; +struct nfs4_ff_layout_ds *old; +u32 dss_id; + +list_for_each_entry (mirror, &flo->mirrors, mirrors) { +for (dss_id =0 ; dss_id < mirror->dss_count; dss_id++) { +if (memcmp (&mirror->dss[dss_id].devid, id, +sizeof (*id)) !=0 ) +continue ; + old =unrcu_pointer ( +xchg (&mirror->dss[dss_id].mirror_ds,NULL )); +if (IS_ERR_OR_NULL (old)) +continue ; +list_add (&old->id_node.put_list, head); + } + } +} +static struct pnfs_ds_commit_info *ff_layout_get_ds_info (struct inode *inode) {
@@ -3088,6 +3123,7 @@ static struct pnfs_layoutdriver_type flexfilelayout_type = { .pg_write_ops = &ff_layout_pg_write_ops, .get_ds_info = ff_layout_get_ds_info, .free_deviceid_node = ff_layout_free_deviceid_node, + .reresolve_deviceid = ff_layout_reresolve_deviceid, .read_pagelist = ff_layout_read_pagelist, .write_pagelist = ff_layout_write_pagelist, .alloc_deviceid_node = ff_layout_alloc_deviceid_node,
Signature · 1 line
2.53.0
PATCH 11/21 NFSv4: Dispatch CB_NOTIFY_DEVICEID CHANGE to an in-place refresh
In reply to Benjamin Coddingtonnfs4_callback_devicenotify() treated CHANGE identically to DELETE: both only unhashed the cached device, so references pinned under live layouts kept sending I/O to the old mapping until the layouts were freed. Per RFC 8881 Section 12.2.10, CHANGE exists precisely so a server can change a mapping without recalling the layouts.
For CHANGE, after unhashing the stale cache entry (so re-resolution cannot re-pin it), invoke the layout driver's re-resolve walker to re-point pinned references under live layouts. DELETE is unchanged, and drivers without a reresolve_deviceid hook see no change.
--- fs/nfs/callback_proc.c | 9 +++++++++ 1 file changed, 9 insertions(+)
fs/nfs/callback_proc.c +9 -0
@@ -390,7 +390,16 @@ __be32 nfs4_callback_devicenotify(void *argp, void *resp,if (!ld)continue ; } +/* +* Unhash the cached device first so re-resolution cannot +* re-pin the stale node, then re-point any references +* pinned under live layouts (RFC 8881 Section 12.2.10). +*/ nfs4_delete_deviceid (ld, cps->clp, &dev->cbd_dev_id); +if (dev->cbd_notify_type ==NOTIFY_DEVICEID4_CHANGE ) +pnfs_layout_reresolve_deviceid_byclid (cps->clp, ld, + &dev->cbd_dev_id, + dev->cbd_immediate); }pnfs_put_layoutdriver (ld); out:
Signature · 1 line
2.53.0
PATCH 12/21 NFSv4/flexfiles: Honor ndc_immediate on CB_NOTIFY_DEVICEID CHANGE
In reply to Benjamin CoddingtonWhen a CHANGE notification carries ndc_immediate, RFC 8881 Section 20.12 says the change is enforced immediately and the client might not be able to complete pending I/O. In addition to un-pinning the stripe's device node, mark the old node unavailable so paths still holding a reference stop selecting it; I/O the server rejects is re-driven onto the freshly resolved mapping by the existing DS error handling.
The walk can also exchange out a node that already carries the new
mapping: a re-resolve that completed between the unhash and the walk reaching the stripe, raced by this walk or by the walk of a later CHANGE for the same deviceid. Ripping such a node out is harmless (it is still hashed, so the next I/O re-pins it from the cache), but it must not be marked unavailable. Stale vintages are distinguishable by
hashed-ness: every superseded node was unhashed before its walk began and is never re-inserted, while a fresh node is inserted before it is installed. Only mark nodes that are no longer hashed.
Without ndc_immediate, pending I/O drains on the old mapping and only new I/O re-resolves, as before.
--- fs/nfs/flexfilelayout/flexfilelayout.c | 17 +++++++++++++++++ 1 file changed, 17 insertions(+)
fs/nfs/flexfilelayout/flexfilelayout.c +17 -0
@@ -2535,6 +2535,23 @@ static void ff_layout_reresolve_deviceid(struct pnfs_layout_hdr *lo, xchg(&mirror->dss[dss_id].mirror_ds, NULL));if (IS_ERR_OR_NULL (old))continue ; +/* +* ndc_immediate: the server is enforcing the change +* now and pending I/O may not complete. Mark the +* old node unavailable so anyone still holding a +* reference stops selecting it; I/O the server +* rejects is re-driven onto the new mapping by the +* existing DS error handling. +* +* Only stale vintages are unhashed: a node still +* hashed was fetched after the stale cache entry +* was removed and already carries the new mapping +* (it can be exchanged out here by a walk that +* lost a race with the re-resolve, or with the +* walk of a later CHANGE); don't mark it. +*/ +if (immediate &&hlist_unhashed (&old->id_node.node)) +nfs4_mark_deviceid_unavailable (&old->id_node);list_add (&old->id_node.put_list, head); } }
Signature · 1 line
2.53.0
PATCH 13/21 pNFS: Discard a GETDEVICEINFO reply that raced a CHANGE notification
In reply to Benjamin CoddingtonRFC 8881 Section 18.40.4: a GETDEVICEINFO reply in flight while the server changes the device mapping may carry the pre-change mapping; if it is inserted into the cache after the CHANGE notification unhashed the stale entry, the client re-caches stale data.
Track a change epoch, bumped when a CHANGE notification is processed before the stale entry is unhashed. nfs4_find_get_deviceid() snapshots the epoch before issuing GETDEVICEINFO and, serialized against the unhash by nfs4_deviceid_lock at insert time, discards the reply and refetches if the epoch moved. A stale insert that instead precedes the unhash is removed by the unhash itself, so both orderings end with the post-change mapping.
--- fs/nfs/callback_proc.c | 4 ++++ fs/nfs/pnfs.h | 1 + fs/nfs/pnfs_dev.c | 25 +++++++++++++++++++++++++ 3 files changed, 30 insertions(+)
fs/nfs/callback_proc.c +4 -0
@@ -394,7 +394,11 @@ __be32 nfs4_callback_devicenotify(void *argp, void *resp,* Unhash the cached device first so re-resolution cannot * re-pin the stale node, then re-point any references * pinned under live layouts (RFC 8881 Section 12.2.10). +* The epoch bump lets an in-flight GETDEVICEINFO detect +* that its reply may predate the change. */ +if (dev->cbd_notify_type ==NOTIFY_DEVICEID4_CHANGE ) +nfs4_deviceid_bump_change_epoch ();nfs4_delete_deviceid (ld, cps->clp, &dev->cbd_dev_id);if (dev->cbd_notify_type ==NOTIFY_DEVICEID4_CHANGE ) pnfs_layout_reresolve_deviceid_byclid(cps->clp, ld,
fs/nfs/pnfs.h +1 -0
@@ -399,6 +399,7 @@ nfs4_find_get_deviceid(struct nfs_server *server, const struct nfs4_deviceid *id, const struct cred *cred, gfp_t gfp_mask);void nfs4_delete_deviceid (const struct pnfs_layoutdriver_type *,const struct nfs_client *,const struct nfs4_deviceid *); +void nfs4_deviceid_bump_change_epoch (void );void nfs4_init_deviceid_node (struct nfs4_deviceid_node *,struct nfs_server *,const struct nfs4_deviceid *);bool nfs4_put_deviceid_node (struct nfs4_deviceid_node *);
fs/nfs/pnfs_dev.c +25 -0
@@ -181,6 +181,22 @@ __nfs4_find_get_deviceid(struct nfs_server *server,return d; } +/* +* Bumped when a CB_NOTIFY_DEVICEID CHANGE is processed. A GETDEVICEINFO +* reply fetched across a change may carry the pre-change mapping; the +* epoch lets nfs4_find_get_deviceid() detect that and refetch. The +* counter is ordered by nfs4_deviceid_lock: the notification handler +* bumps it before unhashing the stale entry under the lock, so an +* insert serialized after the unhash observes the new epoch. +*/ +static atomic_t nfs4_deviceid_change_epoch =ATOMIC_INIT (0 ); + +void +nfs4_deviceid_bump_change_epoch (void ) +{ +atomic_inc (&nfs4_deviceid_change_epoch); +} +struct nfs4_deviceid_node * nfs4_find_get_deviceid(struct nfs_server *server, const struct nfs4_deviceid *id, const struct cred *cred,
@@ -188,11 +204,14 @@ nfs4_find_get_deviceid(struct nfs_server *server, {long hash =nfs4_deviceid_hash (id);struct nfs4_deviceid_node *d, *new; +int epoch; +retry: d =__nfs4_find_get_deviceid (server, id, hash);if (d) goto found; + epoch =atomic_read (&nfs4_deviceid_change_epoch); new =nfs4_get_device_info (server, id, cred, gfp_mask);if (!new) {trace_nfs4_find_deviceid (server, id, -ENOENT );
@@ -200,6 +219,12 @@ nfs4_find_get_deviceid(struct nfs_server *server, }spin_lock (&nfs4_deviceid_lock); +if (atomic_read (&nfs4_deviceid_change_epoch) != epoch) { +/* a mapping changed while we fetched; ours may be stale */ +spin_unlock (&nfs4_deviceid_lock); + server->pnfs_curr_ld->free_deviceid_node (new); + goto retry; + } d =__nfs4_find_get_deviceid (server, id, hash);if (d) {spin_unlock (&nfs4_deviceid_lock);
Signature · 1 line
2.53.0
PATCH 14/21 pNFS: Add deviceid reference query and collection walkers
In reply to Benjamin CoddingtonThe CB_NOTIFY_DEVICEID DELETE race recovery (RFC 8881 Section 18.40.4) needs to ask whether any live layout still references a deviceID, and to enumerate those layouts for TEST_STATEID. Add a layout_references_deviceid hook (sibling of reresolve_deviceid; the flexfiles implementation memcmps each mirror stripe's decoded devid, valid independent of the pinned device node) and two walkers over the byserver pattern:
- pnfs_layout_deviceid_referenced_byclid(): boolean existence query, early-stopping, entirely under i_lock.
- pnfs_layout_collect_deviceid_refs(): collects each matching layout with the hdr pinned, the inode grabbed with its superblock active (a pinned hdr does not hold its inode -- same discipline as the bulk-destroy walker), and the layout stateid and cred snapshotted under i_lock, so the caller can issue sleeping RPCs against the collection.
No callers yet; no behavior change.
--- fs/nfs/flexfilelayout/flexfilelayout.c | 17 +++ fs/nfs/pnfs.c | 155 +++++++++++++++++++++++++ fs/nfs/pnfs.h | 29 +++++ 3 files changed, 201 insertions(+)
fs/nfs/flexfilelayout/flexfilelayout.c +17 -0
@@ -2505,6 +2505,22 @@ static void ff_layout_cancel_io(struct pnfs_layout_segment *lseg) } } +/* Called under @lo's inode i_lock. */ +static bool ff_layout_references_deviceid (struct pnfs_layout_hdr *lo, +const struct nfs4_deviceid *id) +{ +struct nfs4_flexfile_layout *flo =FF_LAYOUT_FROM_HDR (lo); +struct nfs4_ff_layout_mirror *mirror; +u32 dss_id; + +list_for_each_entry (mirror, &flo->mirrors, mirrors) +for (dss_id =0 ; dss_id < mirror->dss_count; dss_id++) +if (memcmp (&mirror->dss[dss_id].devid, id, +sizeof (*id)) ==0 ) +return true; +return false; +} +/* * The server changed the mapping for deviceid @id (CB_NOTIFY_DEVICEID * CHANGE). Un-pin every stripe device node resolved from @id in @lo's
@@ -3141,6 +3157,7 @@ static struct pnfs_layoutdriver_type flexfilelayout_type = { .get_ds_info = ff_layout_get_ds_info, .free_deviceid_node = ff_layout_free_deviceid_node, .reresolve_deviceid = ff_layout_reresolve_deviceid, + .layout_references_deviceid = ff_layout_references_deviceid, .read_pagelist = ff_layout_read_pagelist, .write_pagelist = ff_layout_write_pagelist, .alloc_deviceid_node = ff_layout_alloc_deviceid_node,
fs/nfs/pnfs.c +155 -0
161 lines not on this page · raw message
fs/nfs/pnfs.h +29 -0
@@ -181,6 +181,12 @@ struct pnfs_layoutdriver_type {const struct nfs4_deviceid *id, bool immediate, struct list_head *put_list); +/* +* Does @lo hold any reference to deviceid @id? Called under +* @lo's inode i_lock; must not sleep. +*/ +bool (*layout_references_deviceid)(struct pnfs_layout_hdr *lo, +const struct nfs4_deviceid *id);int (*prepare_layoutreturn) (struct nfs4_layoutreturn_args *);
@@ -368,6 +374,29 @@ void pnfs_layout_reresolve_deviceid_byclid(struct nfs_client *clp,const struct pnfs_layoutdriver_type *ld, const struct nfs4_deviceid *id, bool immediate); +bool pnfs_layout_deviceid_referenced_byclid (struct nfs_client *clp, +const struct pnfs_layoutdriver_type *ld, +const struct nfs4_deviceid *id); + +/* +* One live layout referencing a deviceID, collected for the +* CB_NOTIFY_DEVICEID DELETE recovery: the hdr is pinned, the inode +* igrab'd with its superblock active, and the layout stateid and +* cred snapshotted for TEST_STATEID. +*/ +struct nfs4_deviceid_ref { +struct list_head node; +struct pnfs_layout_hdr *lo; +struct inode *inode; +nfs4_stateid stateid; +const struct cred *cred; +}; + +void pnfs_layout_collect_deviceid_refs (struct nfs_client *clp, +const struct pnfs_layoutdriver_type *ld, +const struct nfs4_deviceid *id, +struct list_head *result); +void pnfs_layout_put_deviceid_refs (struct list_head *result);int pnfs_layout_handle_reboot (struct nfs_client *clp);/* nfs4_deviceid_flags */
Signature · 1 line
2.53.0
PATCH 15/21 NFSv4/pnfs: Recover revoked layouts on a deleted deviceID
In reply to Benjamin CoddingtonRFC 8881 Section 18.40.4: when CB_NOTIFY_DEVICEID deletes a deviceID the client believes live layouts still reference, layouts referring to it may have been revoked -- TEST_STATEID each referring layout's stateid and recover the revoked ones (mark the layout stateid invalid, free the lsegs, FREE_STATEID to acknowledge).
The callback thread cannot issue fore-channel RPCs, so suspects are queued on the nfs_client (dedup'd, holding a layoutdriver reference) and resolved by a new state-manager step keyed on NFS4CLNT_DEVICEID_DELETE. The worker re-collects the referring layouts via the deviceid-ref walker rather than trusting a carried snapshot, so layouts legitimately returned or recalled in between are skipped. If every referring layout turned out revoked, the delete is confirmed by the revocations themselves and the cached device is dropped; a layout the server still considers valid leaves the device cached (verifying the delete with GETDEVICEINFO comes next).
Nothing enqueues suspects yet, so no behavior change.
--- fs/nfs/nfs4_fs.h | 2 ++ fs/nfs/nfs4client.c | 2 ++ fs/nfs/nfs4proc.c | 70 +++++++++++++++++++++++++++++++++++++++ fs/nfs/nfs4state.c | 3 ++ fs/nfs/pnfs.c | 67 +++++++++++++++++++++++++++++++++++++ fs/nfs/pnfs.h | 19 +++++++++++ include/linux/nfs_fs_sb.h | 2 ++ 7 files changed, 165 insertions(+)
fs/nfs/nfs4_fs.h +2 -0
@@ -52,6 +52,7 @@ enum nfs4_client_state {NFS4CLNT_RECALL_ANY_LAYOUT_READ ,NFS4CLNT_RECALL_ANY_LAYOUT_RW ,NFS4CLNT_DELEGRETURN_DELAYED , + NFS4CLNT_DEVICEID_DELETE, };#define NFS4_RENEW_TIMEOUT 0x01
@@ -493,6 +494,7 @@ int nfs41_discover_server_trunking(struct nfs_client *clp, struct nfs_client **, const struct cred *);extern void nfs4_schedule_session_recovery (struct nfs4_session *,int );extern void nfs41_notify_server (struct nfs_client *); +extern void nfs4_deviceid_delete_recover_run (struct nfs_client *clp);bool nfs4_check_serverowner_major_id (struct nfs41_server_owner *o1,struct nfs41_server_owner *o2);
fs/nfs/nfs4client.c +2 -0
@@ -217,6 +217,7 @@ struct nfs_client *nfs4_alloc_client(const struct nfs_client_initdata *cl_init) clp->cl_last_renewal = jiffies;init_waitqueue_head (&clp->cl_lock_waitq);INIT_LIST_HEAD (&clp->pending_cb_stateids); +INIT_LIST_HEAD (&clp->cl_deviceid_deletes);if (cl_init->minorversion !=0 )__set_bit (NFS_CS_INFINITE_SLOTS , &clp->cl_flags);
@@ -285,6 +286,7 @@ static void nfs4_shutdown_client(struct nfs_client *clp)nfs4_kill_renewd (clp); clp->cl_mvops->shutdown_client (clp);nfs4_destroy_callback (clp); +pnfs_deviceid_delete_queue_free (clp);if (__test_and_clear_bit (NFS_CS_IDMAP , &clp->cl_res_state))nfs_idmap_delete (clp);
fs/nfs/nfs4proc.c +70 -0
76 lines not on this page · raw message
fs/nfs/nfs4state.c +3 -0
@@ -2643,6 +2643,9 @@ static void nfs4_state_manager(struct nfs_client *clp)set_bit (NFS4CLNT_RUN_MANAGER , &clp->cl_state); }nfs4_layoutreturn_any_run (clp); +if (test_and_clear_bit (NFS4CLNT_DEVICEID_DELETE , + &clp->cl_state)) +nfs4_deviceid_delete_recover_run (clp);clear_bit (NFS4CLNT_RECALL_RUNNING , &clp->cl_state); }
fs/nfs/pnfs.c +67 -0
73 lines not on this page · raw message
fs/nfs/pnfs.h +19 -0
@@ -397,6 +397,25 @@ void pnfs_layout_collect_deviceid_refs(struct nfs_client *clp,const struct nfs4_deviceid *id, struct list_head *result);void pnfs_layout_put_deviceid_refs (struct list_head *result); + +/* +* A CB_NOTIFY_DEVICEID DELETE naming a deviceID that live layouts +* still reference (RFC 8881 Section 18.40.4). Queued on +* nfs_client.cl_deviceid_deletes under cl_lock for the state manager +* to resolve; holds a layoutdriver reference. +*/ +struct nfs4_deviceid_delete { +struct list_head list; +const struct pnfs_layoutdriver_type *ld; +struct nfs4_deviceid id; +}; + +void pnfs_deviceid_delete_mark (struct nfs_client *clp, +const struct pnfs_layoutdriver_type *ld, +const struct nfs4_deviceid *id); +struct nfs4_deviceid_delete *pnfs_deviceid_delete_dequeue ( +struct nfs_client *clp); +void pnfs_deviceid_delete_queue_free (struct nfs_client *clp);int pnfs_layout_handle_reboot (struct nfs_client *clp);/* nfs4_deviceid_flags */
include/linux/nfs_fs_sb.h +2 -0
@@ -101,6 +101,8 @@ struct nfs_client {/* The flags used for obtaining the clientid during EXCHANGE_ID */ u32 cl_exchange_flags;struct nfs4_session *cl_session;/* shared session */ +/* CB_NOTIFY_DEVICEID DELETE suspects, protected by cl_lock */ +struct list_head cl_deviceid_deletes;bool cl_preserve_clid;struct nfs41_server_owner *cl_serverowner;struct nfs41_server_scope *cl_serverscope;
Signature · 1 line
2.53.0
PATCH 16/21 NFSv4/pnfs: Confirm a deviceID delete via GETDEVICEINFO
In reply to Benjamin CoddingtonRFC 8881 Section 18.40.4: if TEST_STATEID says at least one layout referring to the deleted deviceID is still valid, the delete cannot be trusted -- verify it with GETDEVICEINFO. The device really being gone while the server also considers a referring layout valid means the server is faulty; recover by re-establishing the client ID (nfs4_schedule_lease_recovery drives the prescribed EXCHANGE_ID) and drop the cached device. Any other answer -- including the device existing, i.e. an erroneous DELETE -- keeps the cached device and the layout intact.
The raw-status probe calls nfs4_proc_getdeviceinfo() directly because nfs4_get_device_info() swallows the RPC status and cannot distinguish NFS4ERR_NOENT from a transient failure. A one-page reply buffer is enough: a device too large for it fails with something other than -ENOENT, which still proves existence.
Still nothing enqueues suspects; no behavior change.
--- fs/nfs/nfs4proc.c | 57 +++++++++++++++++++++++++++++++++++++++++------ 1 file changed, 50 insertions(+), 7 deletions(-)
fs/nfs/nfs4proc.c +50 -7
@@ -10441,6 +10441,36 @@ static int nfs41_free_stateid(struct nfs_server *server,return ret; } +/* +* Ask the server whether a deviceID exists, surfacing the raw status +* (nfs4_get_device_info() swallows it). A one-page reply buffer is +* enough for the answer: a device too large for it fails with +* something other than -ENOENT, which still proves existence. +*/ +static int nfs4_deviceid_validate (struct nfs_server *server, +const struct pnfs_layoutdriver_type *ld, +const struct nfs4_deviceid *id,const struct cred *cred) +{ +struct pnfs_device pdev; +struct page *page; +int status; + + page =alloc_page (GFP_KERNEL ); +if (!page) +return -ENOMEM ; + +memset (&pdev,0 ,sizeof (pdev)); +memcpy (&pdev.dev_id, id,sizeof (pdev.dev_id)); + pdev.layout_type = ld->id; + pdev.pages = &page; + pdev.pglen =PAGE_SIZE ; + pdev.maxcount =PAGE_SIZE - nfs41_maxgetdevinfo_overhead; + + status =nfs4_proc_getdeviceinfo (server, &pdev, cred); +__free_page (page); +return status; +} +/* * A CB_NOTIFY_DEVICEID DELETE named a deviceID that live layouts still * referenced -- a conformant server never does this (RFC 8881 Section
@@ -10449,16 +10479,21 @@ static int nfs41_free_stateid(struct nfs_server *server,* invalid, free the lsegs, FREE_STATEID). If every referring layout * turned out revoked, the delete is confirmed by the revocations and * the cached device is dropped. A layout the server still considers -* valid means the delete cannot be trusted; leave the device cached. +* valid means the delete cannot be trusted: verify it with +* GETDEVICEINFO. The device really being gone under a valid layout +* means the server is faulty -- recover by re-establishing the client +* ID (Section 18.40.4 prescribes EXCHANGE_ID). Any other answer +* (including the device existing: an erroneous DELETE) keeps the +* cached device. */ static void nfs4_deviceid_delete_recover (struct nfs_client *clp,const struct pnfs_layoutdriver_type *ld,const struct nfs4_deviceid *id) {LIST_HEAD (layouts); -struct nfs4_deviceid_ref *ref; +struct nfs4_deviceid_ref *ref, *confirm =NULL ;bool revoked = false; -bool referenced = false; +int status;pnfs_layout_collect_deviceid_refs (clp, ld, id, &layouts);
@@ -10466,13 +10501,13 @@ static void nfs4_deviceid_delete_recover(struct nfs_client *clp,struct pnfs_layout_hdr *lo = ref->lo;struct inode *inode = ref->inode;LIST_HEAD (head); -int status; status =nfs41_test_stateid (NFS_SERVER (inode), &ref->stateid, ref->cred);switch (status) {case NFS_OK : - referenced = true; +if (!confirm) + confirm = ref;break ;case -NFS4ERR_ADMIN_REVOKED :case -NFS4ERR_DELEG_REVOKED :
@@ -10494,10 +10529,18 @@ static void nfs4_deviceid_delete_recover(struct nfs_client *clp,break ; } } -pnfs_layout_put_deviceid_refs (&layouts); -if (revoked && !referenced) +if (confirm) { + status =nfs4_deviceid_validate (NFS_SERVER (confirm->inode), + ld, id, confirm->cred); +if (status == -ENOENT ) { +nfs4_schedule_lease_recovery (clp); +nfs4_delete_deviceid (ld, clp, id); + } + }else if (revoked) {nfs4_delete_deviceid (ld, clp, id); + } +pnfs_layout_put_deviceid_refs (&layouts); }void nfs4_deviceid_delete_recover_run (struct nfs_client *clp)
Signature · 1 line
2.53.0
PATCH 17/21 NFSv4/pnfs: Dispatch CB_NOTIFY_DEVICEID DELETE to race recovery
In reply to Benjamin CoddingtonTurn on the RFC 8881 Section 18.40.4 DELETE handling: a DELETE for a deviceID that no live layout references keeps today's cheap behavior (drop the cached device, no state-manager wake). A DELETE for a deviceID that live layouts still reference -- which a conformant server never sends (Section 20.12) -- is deferred to the state manager, which TEST_STATEIDs the referring layouts, recovers revoked ones, and confirms or refutes the delete with GETDEVICEINFO.
The deferred case no longer unhashes the device immediately: if the recovery concludes the DELETE was erroneous (the deviceID still exists and a referring layout is still valid), the client keeps using the cached device.
--- fs/nfs/callback_proc.c | 31 +++++++++++++++++++++++-------- 1 file changed, 23 insertions(+), 8 deletions(-)
fs/nfs/callback_proc.c +23 -8
@@ -391,19 +391,34 @@ __be32 nfs4_callback_devicenotify(void *argp, void *resp,continue ; }/* -* Unhash the cached device first so re-resolution cannot -* re-pin the stale node, then re-point any references -* pinned under live layouts (RFC 8881 Section 12.2.10). -* The epoch bump lets an in-flight GETDEVICEINFO detect -* that its reply may predate the change. +* CHANGE: unhash the cached device first so re-resolution +* cannot re-pin the stale node, then re-point any +* references pinned under live layouts (RFC 8881 Section +* 12.2.10). The epoch bump lets an in-flight +* GETDEVICEINFO detect that its reply may predate the +* change. +* +* DELETE: a deviceID still referenced by a live layout +* must never be deleted (Section 20.12), so a referenced +* one may be racing layout revocation -- defer to the +* state manager for the Section 18.40.4 recovery (this +* thread cannot issue the fore-channel RPCs it needs). +* Unreferenced: "no harm is done" -- drop the cached +* device as before. */ -if (dev->cbd_notify_type ==NOTIFY_DEVICEID4_CHANGE ) +if (dev->cbd_notify_type ==NOTIFY_DEVICEID4_CHANGE ) {nfs4_deviceid_bump_change_epoch (); -nfs4_delete_deviceid (ld, cps->clp, &dev->cbd_dev_id); -if (dev->cbd_notify_type ==NOTIFY_DEVICEID4_CHANGE ) +nfs4_delete_deviceid (ld, cps->clp, &dev->cbd_dev_id);pnfs_layout_reresolve_deviceid_byclid (cps->clp, ld, &dev->cbd_dev_id, dev->cbd_immediate); + }else if (pnfs_layout_deviceid_referenced_byclid (cps->clp, + ld, &dev->cbd_dev_id)) { +pnfs_deviceid_delete_mark (cps->clp, ld, + &dev->cbd_dev_id); + }else { +nfs4_delete_deviceid (ld, cps->clp, &dev->cbd_dev_id); + } }pnfs_put_layoutdriver (ld); out:
Signature · 1 line
2.53.0
PATCH 18/21 NFSv4/pnfs: Grow the deviceid cache hash table
In reply to Benjamin CoddingtonThe global deviceid cache has 32 buckets shared by every server and layout type. pNFS striping deployments put hundreds to ~1000 distinct deviceIDs in play per mount (mirrors x stripes), a load factor of ~31: resolution and re-resolution bursts walk long chains.
Grow to 256 buckets (2KB fixed): load factor ~4 at 1000 devices. The existing byte-folding hash distributes fine at any power-of-two size (the multiplier is odd), so bucket count is the only lever. Alternatives considered and set aside: per-client tables do not help the headline single-mount striping case, and a resizing rhashtable is unwarranted for a burst-only (never per-I/O) lookup path.
--- fs/nfs/pnfs_dev.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-)
fs/nfs/pnfs_dev.c +9 -1
@@ -40,8 +40,16 @@/* * Device ID RCU cache. A device ID is unique per server and layout type. +* +* 256 buckets keep chains short for pNFS striping deployments with +* hundreds to ~1000 distinct deviceIDs (load factor ~4 at 1000) at a +* fixed cost of 2KB. The hash folds every deviceID byte through +* x = x * 37 + byte; 37 is odd, so the low-bit mask distributes +* deviceIDs differing in any single byte perfectly at any +* power-of-two table size -- bucket count, not the hash, is the +* scaling lever. */ -#define NFS4_DEVICE_ID_HASH_BITS 5 +#define NFS4_DEVICE_ID_HASH_BITS 8#define NFS4_DEVICE_ID_HASH_SIZE (1 << NFS4_DEVICE_ID_HASH_BITS)#define NFS4_DEVICE_ID_HASH_MASK (NFS4_DEVICE_ID_HASH_SIZE - 1)
Signature · 1 line
2.53.0
PATCH 19/21 NFSv4/pnfs: Re-home the data-server cache onto hash buckets
In reply to Benjamin CoddingtonMechanical conversion of the per-net data-server cache from a single list_head to an array of hlist buckets: nfs4_pnfs_ds.ds_node becomes an hlist_node, netns init/teardown cover every bucket, and removal uses hlist_del_init (which needs no bucket reference).
Insertion still targets bucket 0 and lookup still scans every entry, so behavior and cost are unchanged -- keying the buckets by the DS address set comes next. Splitting the structural churn from the keying keeps a bisect able to tell a list-conversion bug from a hash-key bug.
--- fs/nfs/client.c | 6 ++++-- fs/nfs/netns.h | 5 ++++- fs/nfs/pnfs.h | 2 +- fs/nfs/pnfs_nfs.c | 17 ++++++++++------- 4 files changed, 19 insertions(+), 11 deletions(-)
fs/nfs/client.c +4 -2
@@ -1283,7 +1283,8 @@ void nfs_clients_init(struct net *net)INIT_LIST_HEAD (&nn->nfs_volume_list);#if IS_ENABLED (CONFIG_NFS_V4 )idr_init (&nn->cb_ident_idr); -INIT_LIST_HEAD (&nn->nfs4_data_server_cache); +for (int i =0 ; i <NFS4_DS_CACHE_HASH_SIZE ; i++) +INIT_HLIST_HEAD (&nn->nfs4_data_server_cache[i]);spin_lock_init (&nn->nfs4_data_server_lock);#endif /* CONFIG_NFS_V4 */ spin_lock_init (&nn->nfs_client_lock);
@@ -1303,7 +1304,8 @@ void nfs_clients_exit(struct net *net)WARN_ON_ONCE (!list_empty (&nn->nfs_client_list));WARN_ON_ONCE (!list_empty (&nn->nfs_volume_list));#if IS_ENABLED (CONFIG_NFS_V4 ) -WARN_ON_ONCE (!list_empty (&nn->nfs4_data_server_cache)); +for (int i =0 ; i <NFS4_DS_CACHE_HASH_SIZE ; i++) +WARN_ON_ONCE (!hlist_empty (&nn->nfs4_data_server_cache[i]));#endif /* CONFIG_NFS_V4 */ }
fs/nfs/netns.h +4 -1
@@ -31,7 +31,10 @@ struct nfs_net {unsigned short nfs_callback_tcpport;unsigned short nfs_callback_tcpport6;int cb_users[NFS4_MAX_MINOR_VERSION +1 ]; -struct list_head nfs4_data_server_cache; +#define NFS4_DS_CACHE_HASH_BITS 8 +#define NFS4_DS_CACHE_HASH_SIZE (1 << NFS4_DS_CACHE_HASH_BITS) +/* hashed by nfs4_ds_addrs_hash() over the DS's address set */ +struct hlist_head nfs4_data_server_cache[NFS4_DS_CACHE_HASH_SIZE ];spinlock_t nfs4_data_server_lock;#endif /* CONFIG_NFS_V4 */ struct nfs_netns_client *nfs_client;
fs/nfs/pnfs.h +1 -1
@@ -57,7 +57,7 @@ struct nfs4_pnfs_ds_addr { };struct nfs4_pnfs_ds { -struct list_head ds_node;/* nfs4_pnfs_dev_hlist dev_dslist */ +struct hlist_node ds_node;/* nfs_net nfs4_data_server_cache */ char *ds_remotestr;/* comma sep list of addrs */ struct list_head ds_addrs;const struct net *ds_net;
fs/nfs/pnfs_nfs.c +10 -7
@@ -604,16 +604,19 @@ _same_data_server_addrs_locked(const struct list_head *dsaddrs1, }/* -* Lookup DS by addresses. nfs4_ds_cache_lock is held +* Lookup DS by addresses. nfs4_data_server_lock is held */ static struct nfs4_pnfs_ds *_data_server_lookup_locked (const struct nfs_net *nn,const struct list_head *dsaddrs) {struct nfs4_pnfs_ds *ds; -list_for_each_entry (ds, &nn->nfs4_data_server_cache, ds_node) -if (_same_data_server_addrs_locked (&ds->ds_addrs, dsaddrs)) -return ds; +for (int i =0 ; i <NFS4_DS_CACHE_HASH_SIZE ; i++) +hlist_for_each_entry (ds, &nn->nfs4_data_server_cache[i], + ds_node) +if (_same_data_server_addrs_locked (&ds->ds_addrs, + dsaddrs)) +return ds;return NULL ; }
@@ -659,7 +662,7 @@ void nfs4_pnfs_ds_put(struct nfs4_pnfs_ds *ds)struct nfs_net *nn =net_generic (ds->ds_net, nfs_net_id);if (refcount_dec_and_lock (&ds->ds_count, &nn->nfs4_data_server_lock)) { -list_del_init (&ds->ds_node); +hlist_del_init (&ds->ds_node);spin_unlock (&nn->nfs4_data_server_lock);destroy_ds (ds); }
@@ -745,10 +748,10 @@ nfs4_pnfs_ds_add(const struct net *net, struct list_head *dsaddrs, gfp_t gfp_flalist_splice_init (dsaddrs, &ds->ds_addrs); ds->ds_remotestr = remotestr;refcount_set (&ds->ds_count,1 ); -INIT_LIST_HEAD (&ds->ds_node); +INIT_HLIST_NODE (&ds->ds_node); ds->ds_net = net; ds->ds_clp =NULL ; -list_add (&ds->ds_node, &nn->nfs4_data_server_cache); +hlist_add_head (&ds->ds_node, &nn->nfs4_data_server_cache[0 ]);dprintk ("%s add new data server %s\n" , __func__, ds->ds_remotestr); }else {
Signature · 1 line
2.53.0
PATCH 20/21 NFSv4/pnfs: Key the data-server cache by its address set
In reply to Benjamin CoddingtonPopulating the data-server cache was quadratic: every GETDEVICEINFO decode scanned the whole per-net cache under one spinlock, with an O(n*m) address-set comparison per entry. A striping mount with hundreds to ~1000 distinct data servers pays that in a burst at first access and again on notification-driven re-resolution.
Hash each DS to a bucket keyed on its address set: per-address jhash over exactly the fields same_sockaddr() compares, combined by addition so multipath ordering cannot change the bucket, and the existing comparator as the in-bucket tiebreaker. Lookup and insert now touch one bucket; teardown is unchanged (hlist_del_init needs no bucket).
The comparator also tightens from subset to set-equality to agree with the order-independent key. The subset test was itself a latent
bug: two data servers where one's address set contained the other's were wrongly merged into whichever was cached first. A flexfiles or files-layout device decodes a deterministic multipath list per GETDEVICEINFO, so equal sets are what servers actually hand out.
--- fs/nfs/pnfs_nfs.c | 73 +++++++++++++++++++++++++++++++++++++++++------ 1 file changed, 64 insertions(+), 9 deletions(-)
fs/nfs/pnfs_nfs.c +64 -9
@@ -15,6 +15,8 @@#include "nfs4session.h" #include "internal.h" +#include <linux/hash.h> +#include <linux/jhash.h> #include "pnfs.h" #include "netns.h" #include "nfs4trace.h"
@@ -577,8 +579,10 @@ same_sockaddr(struct sockaddr *addr1, struct sockaddr *addr2) }/* -* Checks if 'dsaddrs1' contains a subset of 'dsaddrs2'. If it does, -* declare a match. +* Checks if 'dsaddrs1' and 'dsaddrs2' contain the same set of +* addresses. If they do, declare a match. Equal element counts +* plus a subset test give set equality (neither side can hold +* duplicates: each list decodes one device's multipath addresses). */ static bool _same_data_server_addrs_locked(const struct list_head *dsaddrs1,
@@ -588,6 +592,10 @@ _same_data_server_addrs_locked(const struct list_head *dsaddrs1,struct sockaddr *sa1, *sa2;bool match = false; +if (list_count_nodes ((struct list_head *)dsaddrs1) != +list_count_nodes ((struct list_head *)dsaddrs2)) +return false; +list_for_each_entry (da1, dsaddrs1, da_node) { sa1 = (struct sockaddr *)&da1->da_addr; match = false;
@@ -603,6 +611,53 @@ _same_data_server_addrs_locked(const struct list_head *dsaddrs1,return match; } +/* +* Hash exactly the fields same_sockaddr() compares: family, address +* bytes and port. The v6 scope id is deliberately left out -- it +* only discriminates link-local addresses, and hashing it would be +* harmless but pointless: unequal link-local addresses that share a +* bucket are still separated by the comparator. +*/ +static u32 +nfs4_ds_addr_hash (const struct sockaddr *sa) +{ +u32 h = sa->sa_family; + +switch (sa->sa_family) { +case AF_INET : { +const struct sockaddr_in *a = (const struct sockaddr_in *)sa; + + h =jhash (&a->sin_addr.s_addr,sizeof (a->sin_addr.s_addr), h); + h =jhash (&a->sin_port,sizeof (a->sin_port), h); +break ; + } +case AF_INET6 : { +const struct sockaddr_in6 *a = (const struct sockaddr_in6 *)sa; + + h =jhash (&a->sin6_addr,sizeof (a->sin6_addr), h); + h =jhash (&a->sin6_port,sizeof (a->sin6_port), h); +break ; + } + } +return h; +} + +/* +* Bucket index for a DS's address set. Per-address hashes combine +* by addition so the multipath list order cannot change the bucket, +* matching the order-independent set comparison above. +*/ +static u32 +nfs4_ds_addrs_hash (const struct list_head *dsaddrs) +{ +const struct nfs4_pnfs_ds_addr *da; +u32 h =0 ; + +list_for_each_entry (da, dsaddrs, da_node) + h +=nfs4_ds_addr_hash ((const struct sockaddr *)&da->da_addr); +return hash_32 (h,NFS4_DS_CACHE_HASH_BITS ); +} +/* * Lookup DS by addresses. nfs4_data_server_lock is held */
@@ -610,13 +665,11 @@ static struct nfs4_pnfs_ds * _data_server_lookup_locked(const struct nfs_net *nn, const struct list_head *dsaddrs) { struct nfs4_pnfs_ds *ds; +u32 bucket =nfs4_ds_addrs_hash (dsaddrs); -for (int i =0 ; i <NFS4_DS_CACHE_HASH_SIZE ; i++) -hlist_for_each_entry (ds, &nn->nfs4_data_server_cache[i], - ds_node) -if (_same_data_server_addrs_locked (&ds->ds_addrs, - dsaddrs)) -return ds; +hlist_for_each_entry (ds, &nn->nfs4_data_server_cache[bucket], ds_node) +if (_same_data_server_addrs_locked (&ds->ds_addrs, dsaddrs)) +return ds;return NULL ; }
@@ -751,7 +804,9 @@ nfs4_pnfs_ds_add(const struct net *net, struct list_head *dsaddrs, gfp_t gfp_flaINIT_HLIST_NODE (&ds->ds_node); ds->ds_net = net; ds->ds_clp =NULL ; -hlist_add_head (&ds->ds_node, &nn->nfs4_data_server_cache[0 ]); +hlist_add_head (&ds->ds_node, + &nn->nfs4_data_server_cache[ +nfs4_ds_addrs_hash (&ds->ds_addrs)]);dprintk ("%s add new data server %s\n" , __func__, ds->ds_remotestr); }else {
Signature · 1 line
2.53.0
PATCH 21/21 NFSv4/flexfiles: Add a dataserver_nconnect cap
In reply to Benjamin CoddingtonData-server clients inherit the MDS nconnect setting. A striping mount multiplies that by every distinct data server: at ~1000 DSes and nconnect=16 the client opens ~16k sockets plus their sunrpc slot tables, and striping already spreads I/O across the data servers, so a high per-DS transport count buys little for that workload.
Add a dataserver_nconnect module parameter to the flexfiles layout driver alongside its existing dataserver_timeo and dataserver_retrans knobs, and thread the value through nfs4_pnfs_ds_connect() to both the v3 and v4 data-server client setup paths. The default of 0 preserves today's inherit-from-MDS behavior, so the cap is opt-in and non-regressing. The files layout passes 0, unchanged.
--- fs/nfs/filelayout/filelayoutdev.c | 2 +- fs/nfs/flexfilelayout/flexfilelayoutdev.c | 7 +++++++ fs/nfs/internal.h | 3 ++- fs/nfs/nfs3client.c | 9 +++++++-- fs/nfs/nfs4client.c | 5 ++++- fs/nfs/pnfs.h | 3 ++- fs/nfs/pnfs_nfs.c | 20 +++++++++++++------- 7 files changed, 36 insertions(+), 13 deletions(-)
fs/nfs/filelayout/filelayoutdev.c +1 -1
@@ -279,7 +279,7 @@ nfs4_fl_prepare_ds(struct pnfs_layout_segment *lseg, u32 ds_idx) goto out_test_devid; status =nfs4_pnfs_ds_connect (s, ds, devid, dataserver_timeo, - dataserver_retrans,4 , + dataserver_retrans,0 ,4 , s->nfs_client->cl_minorversion);if (status) {nfs4_mark_deviceid_unavailable (devid);
fs/nfs/flexfilelayout/flexfilelayoutdev.c +7 -0
@@ -20,6 +20,7 @@static unsigned int dataserver_timeo =NFS_DEF_TCP_TIMEO ;static unsigned int dataserver_retrans; +static unsigned int dataserver_nconnect;static bool ff_layout_has_available_ds (struct pnfs_layout_segment *lseg);
@@ -439,6 +440,7 @@ nfs4_ff_layout_prepare_ds(struct pnfs_layout_segment *lseg,*/ status =nfs4_pnfs_ds_connect (s, ds, &mirror_ds->id_node, dataserver_timeo, dataserver_retrans, + dataserver_nconnect, mirror_ds->ds_versions[0 ].version, mirror_ds->ds_versions[0 ].minor_version);
@@ -696,3 +698,8 @@ module_param(dataserver_timeo, uint, 0644);MODULE_PARM_DESC (dataserver_timeo,"The time (in tenths of a second) the " "NFSv4.1 client waits for a response from a " " data server before it retries an NFS request." ); +module_param (dataserver_nconnect, uint,0644 ); +MODULE_PARM_DESC (dataserver_nconnect,"The maximum number of connections " +"the NFSv4.1 client opens to each data server, " +"capping the value inherited from the MDS nconnect " +"mount option. 0 (default) applies no cap." );
fs/nfs/internal.h +2 -1
@@ -250,6 +250,7 @@ extern struct nfs_client *nfs4_set_ds_client(struct nfs_server *mds_srv, int ds_addrlen, int ds_proto, unsigned int ds_timeo, unsigned int ds_retrans, + unsigned int ds_nconnect, u32 minor_version);extern struct rpc_clnt *nfs4_find_or_create_ds_client (struct nfs_client *,struct inode *);
@@ -258,7 +259,7 @@ extern void nfs4_session_limit_xasize(struct nfs_server *server);extern struct nfs_client *nfs3_set_ds_client (struct nfs_server *mds_srv,const struct sockaddr_storage *ds_addr,int ds_addrlen,int ds_proto,unsigned int ds_timeo, -unsigned int ds_retrans); +unsigned int ds_retrans,unsigned int ds_nconnect);#ifdef CONFIG_PROC_FS extern int __init nfs_fs_proc_init (void );extern void nfs_fs_proc_exit (void );
fs/nfs/nfs3client.c +7 -2
@@ -84,7 +84,8 @@ struct nfs_server *nfs3_clone_server(struct nfs_server *source,*/ struct nfs_client *nfs3_set_ds_client (struct nfs_server *mds_srv,const struct sockaddr_storage *ds_addr,int ds_addrlen, -int ds_proto,unsigned int ds_timeo,unsigned int ds_retrans) +int ds_proto,unsigned int ds_timeo,unsigned int ds_retrans, +unsigned int ds_nconnect) {struct rpc_timeout ds_timeout;unsigned long connect_timeout = ds_timeo * (ds_retrans +1 ) *HZ /10 ;
@@ -124,8 +125,12 @@ struct nfs_client *nfs3_set_ds_client(struct nfs_server *mds_srv, fallthrough;case XPRT_TRANSPORT_RDMA :case XPRT_TRANSPORT_TCP : -if (mds_clp->cl_nconnect >1 ) +if (mds_clp->cl_nconnect >1 ) { cl_init.nconnect = mds_clp->cl_nconnect; +if (ds_nconnect) + cl_init.nconnect =min (cl_init.nconnect, + ds_nconnect); + } }if (mds_srv->flags &NFS_MOUNT_NORESVPORT )
fs/nfs/nfs4client.c +4 -1
@@ -793,7 +793,7 @@ static int nfs4_set_client(struct nfs_server *server,struct nfs_client *nfs4_set_ds_client (struct nfs_server *mds_srv,const struct sockaddr_storage *ds_addr,int ds_addrlen,int ds_proto,unsigned int ds_timeo,unsigned int ds_retrans, -u32 minor_version) +unsigned int ds_nconnect,u32 minor_version) {struct rpc_timeout ds_timeout;struct nfs_client *mds_clp = mds_srv->nfs_client;
@@ -831,6 +831,9 @@ struct nfs_client *nfs4_set_ds_client(struct nfs_server *mds_srv,case XPRT_TRANSPORT_TCP :if (mds_clp->cl_nconnect >1 ) { cl_init.nconnect = mds_clp->cl_nconnect; +if (ds_nconnect) + cl_init.nconnect =min (cl_init.nconnect, + ds_nconnect); cl_init.max_connect =NFS_MAX_TRANSPORTS ; } }
fs/nfs/pnfs.h +2 -1
@@ -497,7 +497,8 @@ struct nfs4_pnfs_ds *nfs4_pnfs_ds_add(const struct net *net,void nfs4_pnfs_v3_ds_connect_unload (void );int nfs4_pnfs_ds_connect (struct nfs_server *mds_srv,struct nfs4_pnfs_ds *ds,struct nfs4_deviceid_node *devid,unsigned int timeo, -unsigned int retrans,u32 version,u32 minor_version); +unsigned int retrans,unsigned int nconnect, +u32 version,u32 minor_version);struct nfs4_pnfs_ds_addr *nfs4_decode_mp_ds_addr (struct net *net,struct xdr_stream *xdr,gfp_t gfp_flags);
fs/nfs/pnfs_nfs.c +13 -7
@@ -842,7 +842,8 @@ static struct nfs_client *(*get_v3_ds_connect)(int ds_addrlen, int ds_proto, unsigned int ds_timeo, - unsigned int ds_retrans); + unsigned int ds_retrans, + unsigned int ds_nconnect);static bool load_v3_ds_connect (void ) {
@@ -865,7 +866,8 @@ void nfs4_pnfs_v3_ds_connect_unload(void)static int _nfs4_pnfs_v3_ds_connect (struct nfs_server *mds_srv,struct nfs4_pnfs_ds *ds,unsigned int timeo, -unsigned int retrans) +unsigned int retrans, +unsigned int nconnect) {struct nfs_client *clp =ERR_PTR (-EIO );struct nfs_client *mds_clp = mds_srv->nfs_client;
@@ -917,7 +919,7 @@ static int _nfs4_pnfs_v3_ds_connect(struct nfs_server *mds_srv, ds_proto =XPRT_TRANSPORT_TCP_TLS ; clp =get_v3_ds_connect (mds_srv, &da->da_addr, da->da_addrlen, - ds_proto, timeo, retrans); + ds_proto, timeo, retrans, nconnect);if (IS_ERR (clp))continue ; clp->cl_rpcclient->cl_softerr =0 ;
@@ -940,6 +942,7 @@ static int _nfs4_pnfs_v4_ds_connect(struct nfs_server *mds_srv,struct nfs4_pnfs_ds *ds, unsigned int timeo, unsigned int retrans, + unsigned int nconnect, u32 minor_version) { struct nfs_client *clp = ERR_PTR(-EIO);
@@ -1030,7 +1033,8 @@ static int _nfs4_pnfs_v4_ds_connect(struct nfs_server *mds_srv, clp =nfs4_set_ds_client (mds_srv, &da->da_addr, da->da_addrlen, ds_proto, - timeo, retrans, minor_version); + timeo, retrans, nconnect, + minor_version);if (IS_ERR (clp))continue ;
@@ -1063,7 +1067,8 @@ static int _nfs4_pnfs_v4_ds_connect(struct nfs_server *mds_srv,*/ int nfs4_pnfs_ds_connect (struct nfs_server *mds_srv,struct nfs4_pnfs_ds *ds,struct nfs4_deviceid_node *devid,unsigned int timeo, -unsigned int retrans,u32 version,u32 minor_version) +unsigned int retrans,unsigned int nconnect, +u32 version,u32 minor_version) {int err;
@@ -1082,11 +1087,12 @@ int nfs4_pnfs_ds_connect(struct nfs_server *mds_srv, struct nfs4_pnfs_ds *ds,switch (version) {case 3 : - err =_nfs4_pnfs_v3_ds_connect (mds_srv, ds, timeo, retrans); + err =_nfs4_pnfs_v3_ds_connect (mds_srv, ds, timeo, retrans, + nconnect);break ;case 4 : err =_nfs4_pnfs_v4_ds_connect (mds_srv, ds, timeo, retrans, - minor_version); + nconnect, minor_version);break ;default :dprintk ("%s: unsupported DS version %d\n" , __func__, version);
Signature · 1 line
2.53.0
Note to reviewers - this one had numerous problems identified by sashiko:
https://sashiko.dev/#/patchset/cover.1786653063.git.bcodding%40hammerspace.com
I'm working on a v2..
Ben
Quoted 106 lines · Benjamin Coddington
A flexfiles layout striped across many data servers puts hundreds to ~1000 distinct deviceIDs in play for a single mount. This series makes the client's
CB_NOTIFY_DEVICEIDhandling actually work under live layouts, implements the deviceID-deletion race recovery RFC 8881 asks for, and scales the device caches to that number of devices.Patches 1-4 are standalone fixes (Cc: stable) for latent bugs in the striped-layout paths. The
CB_NOTIFY_DEVICEIDdecoder gated consumingndc_immediateon the layout type instead of the notification type, so the flag was silently lost and any multi-item cnda_changes<> array misaligned after the first entry. Read data-server selection truncated the file offset to 32 bits, picking the wrong stripe's DS for availability probing and coalescing above 4 GiB. And page coalescing was bounded on a segment-relative offset in both the flexfiles and files layouts, so an unaligned segment let a coalesced I/O straddle a stripe boundary and send the bytes past it to the wrong data server.Patches 5-13 make CHANGE notifications work under live layouts. RFC 8881 Section 12.2.10 has
CB_NOTIFY_DEVICEIDCHANGE exist precisely so a server can re-point a deviceID without recalling the layouts that reference it, but the client's only response today is to unhash the cached device -- which never reaches the references pinned inside the layout driver's segments, so I/O keeps going to the old mapping until the layouts are freed. Each in-flight I/O is given its own device-node reference, the mirror's pinned pointer becomes RCU-managed, and a new layout-driver hook re-resolves the device in place, honoringndc_immediate. A GETDEVICEINFO reply that raced a CHANGE is discarded rather than installed stale.Patches 14-17 implement the deviceID-deletion race of RFC 8881 Section 18.40.4: layouts still referencing a deleted deviceID are tested with
TEST_STATEID, revoked state is recovered, and the deletion is confirmed with GETDEVICEINFO before escalating to lease recovery. A DELETE that no live layout references keeps today's cheap path.Patches 18-21 scale the device caches. The global deviceid hash grows from 32 to 256 buckets (a load factor of ~31 at 1000 devices today), and the per-net data-server cache moves from a single list to hash buckets keyed by the DS address set -- which also fixes a latent aliasing bug, since the old comparator matched on subset rather than equality and merged two data servers whenever one's address set contained the other's. Last, the flexfiles driver gains a
dataserver_nconnectmodule parameter alongside its existingdataserver_timeoanddataserver_retransknobs:
data-server clients otherwise inherit the MDS nconnect, which at ~1000 DSes and nconnect=16 means ~16k sockets and their slot tables for little gain on a workload striping across the DSes anyway. It defaults to 0, preserving today's inherit-from-MDS behavior.Tested end-to-end against reffs, a notification-capable reference server, at 256 and 1000 devices, including KASAN and lockdep runs under concurrent notification storms.
A related series, "NFS: size the LAYOUTGET reply buffer for wide flexfiles layouts", lifts the single-page LAYOUTGET reply buffer that today caps a segment at roughly 28 stripes. The two are independent and apply cleanly in either order.
Benjamin Coddington (21):
pNFS: FixCB_NOTIFY_DEVICEIDCHANGE to consumendc_immediateNFSv4/flexfiles: Use the full 64-bit offset for read DS selection NFSv4/flexfiles: Bound page coalescing on the absolute stripe offset NFSv4/filelayout: Anchor page coalescing onpattern_offsetNFSv4/flexfiles: Reference the device node across DS setup NFSv4/flexfiles: Carry the device node reference across each I/O NFSv4/flexfiles: Hold a device node reference for layoutstats encoding NFSv4/flexfiles: Make the pinned device node pointer RCU-managed
pNFS: Add areresolve_deviceidlayout driver hook NFSv4/flexfiles: Implement in-place device re-resolve on CHANGE
NFSv4: DispatchCB_NOTIFY_DEVICEIDCHANGE to an in-place refresh NFSv4/flexfiles: Honorndc_immediateonCB_NOTIFY_DEVICEIDCHANGE
pNFS: Discard a GETDEVICEINFO reply that raced a CHANGE notification
pNFS: Add deviceid reference query and collection walkers NFSv4/pnfs: Recover revoked layouts on a deleted deviceID NFSv4/pnfs: Confirm a deviceID delete via GETDEVICEINFO NFSv4/pnfs: DispatchCB_NOTIFY_DEVICEIDDELETE to race recovery NFSv4/pnfs: Grow the deviceid cache hash table
NFSv4/pnfs: Re-home the data-server cache onto hash buckets NFSv4/pnfs: Key the data-server cache by its address set NFSv4/flexfiles: Add adataserver_nconnectcapfs/nfs/callback_proc.c | 30 +- fs/nfs/callback_xdr.c | 2 +- fs/nfs/client.c | 6 +- fs/nfs/filelayout/filelayout.c | 8 +- fs/nfs/filelayout/filelayoutdev.c | 2 +- fs/nfs/flexfilelayout/flexfilelayout.c | 383 ++++++++++++++-------- fs/nfs/flexfilelayout/flexfilelayout.h | 44 +-- fs/nfs/flexfilelayout/flexfilelayoutdev.c | 193 +++++++---- fs/nfs/internal.h | 3 +- fs/nfs/netns.h | 5 +- fs/nfs/nfs3client.c | 9 +- fs/nfs/nfs4_fs.h | 2 + fs/nfs/nfs4client.c | 7 +- fs/nfs/nfs4proc.c | 113 +++++++ fs/nfs/nfs4state.c | 3 + fs/nfs/pnfs.c | 292 +++++++++++++++++ fs/nfs/pnfs.h | 81 ++++- fs/nfs/pnfs_dev.c | 35 +- fs/nfs/pnfs_nfs.c | 93 +++++- include/linux/nfs_fs_sb.h | 2 + include/linux/nfs_xdr.h | 2 + 21 files changed, 1069 insertions(+), 246 deletions(-)base-commit:
db2ddb87143519e20a95aa36c60b36107b736a58Signature · 1 line
2.53.0