Skip to content

pNFS/flexfiles: mirror instance error handling

Benjamin Coddington linux-nfs 2 versions – New

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v2 Latest3 patches 0 repliesNew
  2. v13 patches 1 replySuperseded

Changes in v2 from the cover letter

- Rebase onto [1]. No functional change from v1; every conflict was in context that series rewrites:

- nfs4_ff_layout_prepare_ds() now takes the device node the caller has already resolved and referenced, so patch 1 replaces the trailing fail_return argument of a five-argument signature, and three of the five call sites wrap to stay inside 80 columns.

- ff_layout_cancel_io() now walks mirror->dss[].mirror_ds under RCU and takes a reference on the node, so patch 2 applies its device-ID filter ahead of the RCU dereference rather than in place of a plain pointer read. The filter still compares the raw device ID from the layout, which is immutable across the in-place device re-resolve that series adds.

- ff_layout_resend_pnfs_read() now yields a device node rather than a data server and puts it on the success path, so patch 2's NULL device-ID argument moves into the braced error arm.

[1] https://lore.kernel.org/linux-nfs/[email protected]/

Patches in v2 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/3NFSv4/flexfiles: report the intended opnum when DS connection setup failsfs/nfs/flexfilelayout/flexfilelayout.c fs/nfs/flexfilelayout/flexfilelayout.h fs/nfs/flexfilelayout/flexfilelayoutdev.c+17 -120 replies
2/3pNFS: allow layout drivers to cancel I/O to a single devicefs/nfs/blocklayout/blocklayout.c fs/nfs/callback_proc.c fs/nfs/filelayout/filelayout.c +4 more+46 -300 replies
3/3NFSv4/flexfiles: only cancel I/O to a failed mirror instancefs/nfs/flexfilelayout/flexfilelayout.c fs/nfs/flexfilelayout/flexfilelayoutdev.c+8 -40 replies

Reviews trailers per patch

All 3 patches have a Reviewed-by or Acked-by.

PatchTigran Mkrtchyan
1/3R-b
2/3R-b
3/3R-b

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v2 thread

Cover letter [PATCH v2 0/3] pNFS/flexfiles: mirror instance error handling

When a flexfiles data server fails, the client marks the layout for return and cancels the layout segment's in-flight I/O. ff_layout_cancel_io() cancels I/O to every mirror instance rather than just the failed one, and cancelling a request that has already been transmitted does not stop the data server from executing it. Those writes may complete after the client has sent its LAYOUTRETURN, and the metadata server will see writes to a file for which no write layout is outstanding.

Patch 1 reports the operation the client was preparing to send when a data server connection fails, instead of OP_ILLEGAL.

Patches 2 and 3 let a layout driver cancel I/O to a single device, and pass the failed device down from the flexfiles error paths, so that I/O to the remaining mirrors drains before the LAYOUTRETURN is sent.

Layout recalls that set clora_changed, bulk returns and revocations are
unchanged: they continue to cancel I/O to every device.

This version applies on top of the flexfiles device-notification and caching series [1], which rewrites the data-server setup and I/O-cancellation paths these patches touch.

v1 posting:
https://lore.kernel.org/linux-nfs/[email protected]/

Changes since v1:

- Rebase onto [1]. No functional change from v1; every conflict was in context that series rewrites:

- nfs4_ff_layout_prepare_ds() now takes the device node the caller has already resolved and referenced, so patch 1 replaces the trailing fail_return argument of a five-argument signature, and three of the five call sites wrap to stay inside 80 columns.

- ff_layout_cancel_io() now walks mirror->dss[].mirror_ds under RCU and takes a reference on the node, so patch 2 applies its device-ID filter ahead of the RCU dereference rather than in place of a plain pointer read. The filter still compares the raw device ID from the layout, which is immutable across the in-place device re-resolve that series adds.

- ff_layout_resend_pnfs_read() now yields a device node rather than a data server and puts it on the success path, so patch 2's NULL device-ID argument moves into the braced error arm.

[1] https://lore.kernel.org/linux-nfs/[email protected]/

Benjamin Coddington (3):
NFSv4/flexfiles: report the intended opnum when DS connection setup
fails
pNFS: allow layout drivers to cancel I/O to a single device NFSv4/flexfiles: only cancel I/O to a failed mirror instance

 fs/nfs/blocklayout/blocklayout.c          |  6 +++--
 fs/nfs/callback_proc.c                    |  2 +-
 fs/nfs/filelayout/filelayout.c            |  4 +--
 fs/nfs/flexfilelayout/flexfilelayout.c    | 32 +++++++++++++++--------
 fs/nfs/flexfilelayout/flexfilelayout.h    |  2 +-
 fs/nfs/flexfilelayout/flexfilelayoutdev.c | 17 +++++++-----
 fs/nfs/pnfs.c                             | 32 ++++++++++++++---------
 fs/nfs/pnfs.h                             | 14 ++++++----
 8 files changed, 67 insertions(+), 42 deletions(-)
Signature · 1 line
2.53.0