Skip to content

Add Flexfile Layout Module

Tom Haynes linux-nfs 6 versions – In review

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v6 Latest53 patches 7 repliesIn review
  2. v551 patches 3 repliesSuperseded
  3. v450 patches 0 repliesSuperseded
Earlier versions · 3
VersionPostedPatchesRepliesStatus
v349 patches24 repliesSuperseded
v249 patches27 repliesSuperseded
v150 patches18 repliesSuperseded

Patches in v6 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/53pnfs: Prepare for flexfiles by pulling out common codefs/nfs/Makefile fs/nfs/filelayout/filelayout.c fs/nfs/filelayout/filelayout.h +3 more+330 -2900 replies
2/53pnfs: Do not grab the commit_info lock twice when rescheduling writesfs/nfs/direct.c fs/nfs/pnfs.h fs/nfs/pnfs_nfs.c+23 -260 replies
3/53nfs41: pull data server cache from file layout to generic pnfsfs/nfs/filelayout/filelayout.h fs/nfs/filelayout/filelayoutdev.c fs/nfs/pnfs.h +1 more+265 -2520 replies
4/53nfs41: pull decode_ds_addr from file layout to generic pnfsfs/nfs/filelayout/filelayoutdev.c fs/nfs/pnfs.h fs/nfs/pnfs_nfs.c+154 -1500 replies
5/53nfs41: pull nfs4_ds_connect from file layout to generic pnfsfs/nfs/filelayout/filelayoutdev.c fs/nfs/pnfs.h fs/nfs/pnfs_nfs.c+89 -730 replies
6/53nfs41: allow LD to choose DS connection auth flavorfs/nfs/filelayout/filelayoutdev.c fs/nfs/internal.h fs/nfs/nfs4client.c +2 more+14 -90 replies
7/53nfs41: move file layout macros to generic pnfsfs/nfs/filelayout/filelayout.h fs/nfs/pnfs.h+11 -100 replies
8/53nfsv3: introduce nfs3_set_ds_clientfs/nfs/internal.h fs/nfs/nfs3_fs.h fs/nfs/nfs3client.c +2 more+46 -50 replies
9/53nfs41: allow LD to choose DS connection version/minor_versionfs/nfs/filelayout/filelayoutdev.c fs/nfs/internal.h fs/nfs/nfs4client.c +2 more+14 -80 replies
10/53nfs41: create NFSv3 DS connection if specifiedfs/nfs/nfs4super.c fs/nfs/pnfs.h fs/nfs/pnfs_nfs.c+89 -51 reply
11/53pnfs: Add nfs_rpc_ops in calls to nfs_initiate_pgiofs/nfs/filelayout/filelayout.c fs/nfs/internal.h fs/nfs/pagelist.c +3 more+13 -80 replies
12/53nfs: allow different protocol in nfs_initiate_commitfs/nfs/filelayout/filelayout.c fs/nfs/internal.h fs/nfs/pnfs_nfs.c +1 more+7 -40 replies
13/53nfs4: pass slot table to nfs40_setup_sequencefs/nfs/nfs4_fs.h fs/nfs/nfs4proc.c+17 -110 replies
14/53nfs4: export nfs4_sequence_donefs/nfs/nfs4_fs.h fs/nfs/nfs4proc.c+7 -40 replies
15/53nfs: allow to specify cred in nfs_initiate_pgiofs/nfs/filelayout/filelayout.c fs/nfs/internal.h fs/nfs/pagelist.c+14 -110 replies
16/53NFSv4.1/NFSv3: Add pNFS callbacks for nfs3_(read|write|commit)_done()fs/nfs/nfs3proc.c+9 -00 replies
17/53sunrpc: add rpc_count_iostats_idxinclude/linux/sunrpc/metrics.h net/sunrpc/stats.c+21 -70 replies
18/53nfs: set hostname when creating nfsv3 ds connectionfs/nfs/nfs3client.c+7 -00 replies
19/53nfs/flexclient: export pnfs_layoutcommit_inodefs/nfs/pnfs.c+1 -00 replies
20/53nfs41: close a small race window when adding new layout to global listfs/nfs/pnfs.c+3 -50 replies
21/53nfs41: serialize first layoutget of a filefs/nfs/pnfs.c fs/nfs/pnfs.h+32 -40 replies
22/53nfs: save server READ/WRITE/COMMIT statusfs/nfs/nfs2xdr.c fs/nfs/nfs3xdr.c fs/nfs/nfs4xdr.c +1 more+15 -30 replies
23/53nfs41: pass iomode through layoutreturn argsfs/nfs/nfs4xdr.c fs/nfs/pnfs.c include/linux/nfs_xdr.h+3 -10 replies
24/53nfs41: make a helper function to send layoutreturnfs/nfs/pnfs.c+33 -200 replies
25/53nfs41: add a helper to mark layout for returnfs/nfs/pnfs.c fs/nfs/pnfs.h+59 -00 replies
26/53nfs41: don't use a layout if it is marked for returningfs/nfs/nfs4proc.c fs/nfs/pnfs.c fs/nfs/pnfs.h+20 -50 replies
27/53nfs41: send layoutreturn in last put_lsegfs/nfs/pnfs.c+37 -10 replies
28/53nfs41: clear NFS_LAYOUT_RETURN if layoutreturn is sent or failed to sendfs/nfs/nfs4proc.c fs/nfs/pnfs.c+6 -00 replies
29/53nfs/filelayout: use pnfs_error_mark_layout_for_returnfs/nfs/filelayout/filelayout.c fs/nfs/pnfs_nfs.c+1 -110 replies
30/53nfs: introduce pg_cleanup op for pgio descriptorsfs/nfs/pagelist.c include/linux/nfs_page.h+5 -10 replies
31/53pnfs: release lseg in pnfs_generic_pg_cleanupfs/nfs/blocklayout/blocklayout.c fs/nfs/filelayout/filelayout.c fs/nfs/objlayout/objio_osd.c +2 more+21 -180 replies
32/53nfs: handle overlapping reqs in lock_and_joinfs/nfs/write.c+11 -60 replies
33/53nfs: rename pgio header ds_idx to ds_commit_idxfs/nfs/direct.c fs/nfs/filelayout/filelayout.c include/linux/nfs_xdr.h+9 -110 replies
34/53pnfs: pass ds_commit_idx through the commit pathfs/nfs/direct.c fs/nfs/filelayout/filelayout.c fs/nfs/internal.h +3 more+24 -170 replies
35/53nfs: add mirroring support to pgio layerfs/nfs/direct.c fs/nfs/internal.h fs/nfs/objlayout/objio_osd.c +6 more+311 -670 replies
36/53nfs: mirroring support for direct iofs/nfs/direct.c+57 -140 replies
37/53pnfs: fail comparison when bucket verifier not setfs/nfs/direct.c+5 -10 replies
38/53nfs41: add a debug warning if we destroy an unempty layoutfs/nfs/pnfs.c+2 -00 replies
39/53nfs: only reset desc->pg_mirror_idx when mirroring is supportedfs/nfs/internal.h fs/nfs/pagelist.c+11 -40 replies
40/53nfs: add nfs_pgio_current_mirror helperfs/nfs/internal.h fs/nfs/objlayout/objio_osd.c fs/nfs/pagelist.c +1 more+24 -140 replies
41/53pnfs: allow LD to ask to resend read through pnfsfs/nfs/pnfs.c fs/nfs/pnfs.h+16 -10 replies
42/53nfs41: add range to layoutreturn argsfs/nfs/nfs4xdr.c fs/nfs/pnfs.c include/linux/nfs_xdr.h+7 -50 replies
43/53nfs41: allow async version layoutreturnfs/nfs/nfs4proc.c fs/nfs/pnfs.c fs/nfs/pnfs.h+16 -80 replies
44/53nfs41: introduce NFS_LAYOUT_RETURN_BEFORE_CLOSEfs/nfs/nfs4proc.c fs/nfs/pnfs.c fs/nfs/pnfs.h+36 -72 replies
45/53nfs/flexfiles: send layoutreturn before freeing lsegfs/nfs/pnfs.c+56 -250 replies
46/53nfs41: add NFS_LAYOUT_RETRY_LAYOUTGET to layout header flagsfs/nfs/pnfs.c fs/nfs/pnfs.h+21 -00 replies
47/53nfs: add a helper to set NFS_ODIRECT_RESCHED_WRITES to direct writesfs/nfs/direct.c fs/nfs/internal.h+7 -00 replies
48/53nfs41: wait for LAYOUTRETURN before retrying LAYOUTGETfs/nfs/nfs4proc.c fs/nfs/pnfs.c fs/nfs/pnfs.h+45 -30 replies
49/53nfs: count DIO good bytes correctly with mirroringfs/nfs/direct.c+8 -40 replies
50/53pnfs/flexfiles: Add the FlexFile Layout Driverfs/nfs/Kconfig fs/nfs/Makefile fs/nfs/flexfilelayout/Makefile +10 more+2325 -120 replies
51/53pnfs: Update documentation on the Layout DriversDocumentation/filesystems/nfs/pnfs.txt+7 -60 replies
52/53nfs41: .init_read and .init_write can be called with valid pg_lsegfs/nfs/pnfs.c+20 -210 replies
53/53pnfs: lookup new lseg at lseg boundaryfs/nfs/pnfs.c+8 -20 replies

Reviews trailers per patch

6 of 53 patches have a Reviewed-by or Acked-by.

PatchJeff Layton
1/53
2/53A-b
3/53R-b
4/53R-b
5/53R-b
6/53R-b
7/53R-b
8/53
9/53
10/53
11/53
12/53
13/53
14/53
15/53
16/53
17/53
18/53
19/53
20/53
21/53
22/53
23/53
24/53
25/53
26/53
27/53
28/53
29/53
30/53
31/53
32/53
33/53
34/53
35/53
36/53
37/53
38/53
39/53
40/53
41/53
42/53
43/53
44/53
45/53
46/53
47/53
48/53
49/53
50/53
51/53
52/53
53/53

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v6 thread

Cover letter [PATCH v6 00/53] Add Flexfile Layout Module

Hi Trond,

This patchset introduces the Flexfile Layout Module for the client.

It will correspond to draft 5
(http://tools.ietf.org/id/draft-ietf-nfsv4-flex-files-05.txt)
of the Parallel NFS (pNFS) Flexible File Layout
(https://datatracker.ietf.org/doc/draft-ietf-nfsv4-flex-files/).

This version has been rebased to v3.19-rc7 and is ready for inclusion in 3.20.

The difference from v5 is that:
- we fixed 3 nits found by the kbuild test robot
- we fixed the generic helper functions for .init_read and .init_write
- we fixed handling multiple layout segments with the blocklayout

The [flexfiles] branch of

git://git.linux-nfs.org/projects/loghyr/linux-nfs.git

also has this code.

Thanks,
Tom

Peng Tao (37):
nfs41: pull data server cache from file layout to generic pnfs
nfs41: pull decode_ds_addr from file layout to generic pnfs
nfs41: pull nfs4_ds_connect from file layout to generic pnfs
nfs41: allow LD to choose DS connection auth flavor
nfs41: move file layout macros to generic pnfs
nfsv3: introduce nfs3_set_ds_client
nfs41: allow LD to choose DS connection version/minor_version
nfs41: create NFSv3 DS connection if specified
nfs: allow different protocol in nfs_initiate_commit
nfs4: pass slot table to nfs40_setup_sequence
nfs4: export nfs4_sequence_done
nfs: allow to specify cred in nfs_initiate_pgio
nfs: set hostname when creating nfsv3 ds connection nfs/flexclient: export pnfs_layoutcommit_inode
nfs41: close a small race window when adding new layout to global list
nfs41: serialize first layoutget of a file
nfs: save server READ/WRITE/COMMIT status
nfs41: pass iomode through layoutreturn args
nfs41: make a helper function to send layoutreturn
nfs41: add a helper to mark layout for return
nfs41: don't use a layout if it is marked for returning
nfs41: send layoutreturn in last put_lseg
nfs41: clear NFS_LAYOUT_RETURN if layoutreturn is sent or failed to
send
nfs/filelayout: use pnfs_error_mark_layout_for_return
nfs41: add a debug warning if we destroy an unempty layout
nfs: only reset desc->pg_mirror_idx when mirroring is supported
nfs: add nfs_pgio_current_mirror helper
pnfs: allow LD to ask to resend read through pnfs
nfs41: add range to layoutreturn args
nfs41: allow async version layoutreturn
nfs41: introduce NFS_LAYOUT_RETURN_BEFORE_CLOSE nfs/flexfiles: send layoutreturn before freeing lseg
nfs41: add NFS_LAYOUT_RETRY_LAYOUTGET to layout header flags
nfs: add a helper to set NFS_ODIRECT_RESCHED_WRITES to direct writes
nfs41: wait for LAYOUTRETURN before retrying LAYOUTGET
nfs: count DIO good bytes correctly with mirroring
nfs41: .init_read and .init_write can be called with valid pg_lseg

Tom Haynes (5):
pnfs: Prepare for flexfiles by pulling out common code
pnfs: Do not grab the commit_info lock twice when rescheduling writes
pnfs: Add nfs_rpc_ops in calls to nfs_initiate_pgio pnfs/flexfiles: Add the FlexFile Layout Driver
pnfs: Update documentation on the Layout Drivers

Trond Myklebust (1):
NFSv4.1/NFSv3: Add pNFS callbacks for nfs3_(read|write|commit)_done()

Weston Andros Adamson (10):
sunrpc: add rpc_count_iostats_idx
nfs: introduce pg_cleanup op for pgio descriptors
pnfs: release lseg in pnfs_generic_pg_cleanup
nfs: handle overlapping reqs in lock_and_join
nfs: rename pgio header ds_idx to ds_commit_idx
pnfs: pass ds_commit_idx through the commit path
nfs: add mirroring support to pgio layer
nfs: mirroring support for direct io
pnfs: fail comparison when bucket verifier not set
pnfs: lookup new lseg at lseg boundary

 Documentation/filesystems/nfs/pnfs.txt    |   13 +-
 fs/nfs/Kconfig                            |    5 +
 fs/nfs/Makefile                           |    3 +-
 fs/nfs/blocklayout/blocklayout.c          |    2 +
 fs/nfs/direct.c                           |  112 +-
 fs/nfs/filelayout/filelayout.c            |  315 +-----
 fs/nfs/filelayout/filelayout.h            |   40 -
 fs/nfs/filelayout/filelayoutdev.c         |  469 +--------
 fs/nfs/flexfilelayout/Makefile            |    5 +
 fs/nfs/flexfilelayout/flexfilelayout.c    | 1574 +++++++++++++++++++++++++++++
 fs/nfs/flexfilelayout/flexfilelayout.h    |  155 +++
 fs/nfs/flexfilelayout/flexfilelayoutdev.c |  552 ++++++++++
 fs/nfs/idmap.c                            |    3 +-
 fs/nfs/internal.h                         |   31 +-
 fs/nfs/nfs2xdr.c                          |   10 +-
 fs/nfs/nfs3_fs.h                          |    2 +
 fs/nfs/nfs3client.c                       |   41 +
 fs/nfs/nfs3proc.c                         |    9 +
 fs/nfs/nfs3super.c                        |    2 +-
 fs/nfs/nfs3xdr.c                          |    3 +
 fs/nfs/nfs4_fs.h                          |    6 +
 fs/nfs/nfs4client.c                       |    7 +-
 fs/nfs/nfs4proc.c                         |   48 +-
 fs/nfs/nfs4super.c                        |    3 +
 fs/nfs/nfs4xdr.c                          |    9 +-
 fs/nfs/objlayout/objio_osd.c              |    5 +-
 fs/nfs/pagelist.c                         |  294 +++++-
 fs/nfs/pnfs.c                             |  470 +++++++--
 fs/nfs/pnfs.h                             |  121 ++-
 fs/nfs/pnfs_nfs.c                         |  840 +++++++++++++++
 fs/nfs/read.c                             |   33 +-
 fs/nfs/write.c                            |   52 +-
 include/linux/nfs4.h                      |    1 +
 include/linux/nfs_fs_sb.h                 |    9 +-
 include/linux/nfs_idmap.h                 |    2 +
 include/linux/nfs_page.h                  |   22 +-
 include/linux/nfs_xdr.h                   |    6 +-
 include/linux/sunrpc/metrics.h            |    4 +
 net/sunrpc/stats.c                        |   26 +-
 39 files changed, 4263 insertions(+), 1041 deletions(-)
 create mode 100644 fs/nfs/flexfilelayout/Makefile
 create mode 100644 fs/nfs/flexfilelayout/flexfilelayout.c
 create mode 100644 fs/nfs/flexfilelayout/flexfilelayout.h
 create mode 100644 fs/nfs/flexfilelayout/flexfilelayoutdev.c
 create mode 100644 fs/nfs/pnfs_nfs.c
Signature · 1 line
1.9.3