Skip to content

Fix NFS commit to DS

[email protected] linux-nfs 2 versions – New

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v2 Latest22 patches 0 repliesNew
  2. v122 patches 0 repliesSuperseded

Changes in v2 from the cover letter

- Fix compile issues when !defined(CONFIG_NFS_V4_1)
- Code clean ups

Patches in v2 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/22pNFS/flexfiles: Simplify allocation of the mirror arrayfs/nfs/flexfilelayout/flexfilelayout.c fs/nfs/flexfilelayout/flexfilelayout.h+6 -170 replies
2/22NFS/pNFS: Refactor pnfs_generic_commit_pagelist()fs/nfs/pnfs_nfs.c fs/nfs/write.c+76 -1010 replies
3/22pNFS: Add a helper to allocate the array of bucketsfs/nfs/pnfs.h fs/nfs/pnfs_nfs.c include/linux/nfs_xdr.h+46 -30 replies
4/22NFSv4/pnfs: Support a list of commit arrays in struct pnfs_ds_commit_infofs/nfs/direct.c fs/nfs/filelayout/filelayout.c fs/nfs/flexfilelayout/flexfilelayout.c +2 more+18 -10 replies
5/22NFSv4/pNFS: Scan the full list of commit arrays when committingfs/nfs/pnfs_nfs.c+40 -120 replies
6/22pNFS: Support per-layout segment commits in pnfs_generic_recover_commit_reqs()fs/nfs/pnfs_nfs.c+33 -80 replies
7/22pNFS: Support per-layout segment commits in pnfs_generic_commit_pagelist()fs/nfs/pnfs_nfs.c+16 -00 replies
8/22NFS/pNFS: Allow O_DIRECT to release the DS commitinfofs/nfs/direct.c fs/nfs/pnfs.h+17 -00 replies
9/22NFS: commit errors should be fatalfs/nfs/direct.c+30 -20 replies
10/22NFS: Fix O_DIRECT commit verifier handlingfs/nfs/direct.c fs/nfs/internal.h fs/nfs/write.c+22 -1240 replies
11/22NFS/pNFS: Support commit arrays in nfs_clear_pnfs_ds_commit_verifiers()fs/nfs/internal.h+16 -30 replies
12/22pNFS: Add infrastructure for cleaning up per-layout commit structuresfs/nfs/filelayout/filelayout.c fs/nfs/flexfilelayout/flexfilelayout.c fs/nfs/internal.h +4 more+121 -40 replies
13/22pNFS: Enable per-layout segment commit structuresfs/nfs/filelayout/filelayout.c fs/nfs/flexfilelayout/flexfilelayout.c fs/nfs/pnfs.h +1 more+117 -60 replies
14/22NFS/pNFS: Add a helper pnfs_generic_search_commit_reqs()fs/nfs/filelayout/filelayout.c fs/nfs/pnfs.h fs/nfs/pnfs_nfs.c+54 -310 replies
15/22NFS: Remove bucket array from struct pnfs_ds_commit_infofs/nfs/direct.c fs/nfs/filelayout/filelayout.c fs/nfs/flexfilelayout/flexfilelayout.c +3 more+1 -1850 replies
16/22NFS/pNFS: Clean up pNFS commit operationsfs/nfs/direct.c fs/nfs/filelayout/filelayout.c fs/nfs/flexfilelayout/flexfilelayout.c +3 more+98 -710 replies
17/22NFS/pNFS: Simplify bucket layout segment reference countingfs/nfs/pnfs_nfs.c include/linux/nfs_xdr.h+21 -210 replies
18/22NFS/pNFS: Fix pnfs_layout_mark_request_commit() invalid layout segment handlingfs/nfs/pnfs_nfs.c+12 -160 replies
19/22pNFS/flexfile: Don't merge layout segments if the mirrors don't matchfs/nfs/flexfilelayout/flexfilelayout.c+19 -00 replies
20/22pNFS/flexfiles: Check the layout segment range before doing I/Ofs/nfs/flexfilelayout/flexfilelayout.c fs/nfs/pnfs.c fs/nfs/pnfs.h+13 -30 replies
21/22pNFS/flexfiles: remove requirement for whole file layoutsfs/nfs/flexfilelayout/flexfilelayout.c+0 -210 replies
22/22pNFS/flexfiles: Specify the layout segment range in LAYOUTGETfs/nfs/flexfilelayout/flexfilelayout.c+4 -40 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v2 yet.

Read the v2 thread

Cover letter [PATCH v2 00/22] Fix NFS commit to DS

From: Trond Myklebust <[email protected]>

The commit to DS code is currently broken in the face of layout segments that do not cover the full file case. In particular, if the layout is a flexfile mirrored layout, we see that if two layout segments have differing numbers of mirrors, or just have different ordering of mirrors, then the current code can lead to data corruption and even kernel Oopses on the client.

The root cause of these problems is the fact that we have one array of commit lists per struct pnfs_ds_commit_info, and that once that array is set up, it cannot change. This despite the fact that layout segments can be recalled and changed on the fly.
This patch set attempts to fix that problem by changing the array of commit lists into a list of lists. It does so in a way that ensures 1-1 correspondence between a layout segment and a set of commits for the case of buffered writes, and a similar setup for a collection of O_DIRECT writes.

Since most of this work is acting on code that is currently common to both the files layout and the flexfiles layout, I've fixed up both layout types in the same way.

---
v2:
- Fix compile issues when !defined(CONFIG_NFS_V4_1)
- Code clean ups

Trond Myklebust (22):
pNFS/flexfiles: Simplify allocation of the mirror array NFS/pNFS: Refactor pnfs_generic_commit_pagelist()
pNFS: Add a helper to allocate the array of buckets NFSv4/pnfs: Support a list of commit arrays in struct
pnfs_ds_commit_info
NFSv4/pNFS: Scan the full list of commit arrays when committing
pNFS: Support per-layout segment commits in
pnfs_generic_recover_commit_reqs()
pNFS: Support per-layout segment commits in
pnfs_generic_commit_pagelist()
NFS/pNFS: Allow O_DIRECT to release the DS commitinfo
NFS: commit errors should be fatal
NFS: Fix O_DIRECT commit verifier handling
NFS/pNFS: Support commit arrays in
nfs_clear_pnfs_ds_commit_verifiers()
pNFS: Add infrastructure for cleaning up per-layout commit structures
pNFS: Enable per-layout segment commit structures NFS/pNFS: Add a helper pnfs_generic_search_commit_reqs()
NFS: Remove bucket array from struct pnfs_ds_commit_info NFS/pNFS: Clean up pNFS commit operations
NFS/pNFS: Simplify bucket layout segment reference counting NFS/pNFS: Fix pnfs_layout_mark_request_commit() invalid layout segment
handling
pNFS/flexfile: Don't merge layout segments if the mirrors don't match pNFS/flexfiles: Check the layout segment range before doing I/O pNFS/flexfiles: remove requirement for whole file layouts pNFS/flexfiles: Specify the layout segment range in LAYOUTGET

 fs/nfs/direct.c                        | 174 +++------
 fs/nfs/filelayout/filelayout.c         | 163 +++-----
 fs/nfs/flexfilelayout/flexfilelayout.c | 203 ++++------
 fs/nfs/flexfilelayout/flexfilelayout.h |   2 +-
 fs/nfs/internal.h                      |  26 +-
 fs/nfs/pnfs.c                          |   4 +-
 fs/nfs/pnfs.h                          | 137 +++++--
 fs/nfs/pnfs_nfs.c                      | 514 ++++++++++++++++++-------
 fs/nfs/write.c                         |  16 +-
 include/linux/nfs_xdr.h                |  32 +-
 10 files changed, 709 insertions(+), 562 deletions(-)
Signature · 1 line
2.25.1