NFS Direct: align misaligned DIO for LOCALIO
Get series with b4
b4 am [email protected]Versions newest first · linked by author and title
- v11 Latest7 patches 3 repliesIn review
- v107 patches 14 repliesSuperseded
- v97 patches 9 repliesSuperseded
Earlier versions · 5
| Version | Posted | Patches | Replies | Status |
|---|---|---|---|---|
| v8 | 9 patches | 0 replies | Superseded | |
| v7 | 11 patches | 0 replies | Superseded | |
| v6 | 7 patches | 0 replies | Superseded | |
| v2 | 7 patches | 6 replies | Superseded | |
| v1 RFC | 6 patches | 2 replies | Superseded |
Changes in v11 from the cover letter
- Fixed patch 6's nfstrace.h compiler errors due to #include order in nfs2xdr.c
- Rebased patches 2 and 5 to reduce churn/indentation for EINVAL error logging.
- Cc Chuck and Jeff for review of patch 2's fs/nfsd/localio.c change to add .nfsd_file_dio_alignment callback to nfsd_localio_operations
Patches in v11 No applies recorded yet
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/7 | nfs/localio: make trace_nfs_local_open_fh more usefulfs/ fs/ | +6 | 0 replies | |
| 2/7 | nfs/localio: avoid issuing misaligned IO using O_DIRECTfs/ fs/ include/ | +68 | 3 replies | |
| 2/7 | nfs/localio: avoid issuing misaligned IO using O_DIRECTfs/ fs/ include/ | +70 | 0 replies | |
| 3/7 | nfs/localio: refactor iocb and iov_iter_bvec initializationfs/ | +33 | 0 replies | |
| 4/7 | nfs/localio: refactor iocb initializationfs/ | +56 | 0 replies | |
| 5/7 | nfs/localio: add proper O_DIRECT support for READ and WRITEfs/ | +201 | 0 replies | |
| 6/7 | nfs/localio: add tracepoints for misaligned DIO READ and WRITE supportfs/ fs/ fs/ +2 more | +90 | 0 replies | |
| 7/7 | NFS: add basic STATX_DIOALIGN and STATX_DIO_READ_ALIGN supportfs/ | +15 | 0 replies |
Reviews trailers per patch
2 of 8 patches have a Reviewed-by or Acked-by.
| Patch | Jeff Layton |
|---|---|
| 1/7 | |
| 2/7 | R-b |
| 2/7 | R-b |
| 3/7 | |
| 4/7 | |
| 5/7 | |
| 6/7 | |
| 7/7 |
Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.
Cover letter [PATCH v11 0/7] NFS Direct: align misaligned DIO for LOCALIO
Hi Anna,
NFS and LOCALIO in particular benefit from avoiding the page cache for workloads that have a working set that is significantly larger than available system memory. NFS DIRECT makes it possible to always enable LOCALIO to use O_DIRECT even if the IO is not DIO-aligned.
This patchset's changes are focused on NFS LOCALIO (fs/nfs/localio.c); as such they will not impact NFS at all unless CONFIG_NFS_LOCALIO=y
These changes have been tested quite a lot, using workloads from load generators fio and dt. But also using MLperf benchmark suite's npz loader library (written in python) that creates a particularly nasty 1MB IO pattern which is DIO-aligned on disk but _not_ in memory (so it triggers the need to fallback from DIO to buffered IO). I'm happy to share more specifics on test tools and commandlines if needed.
Changes since v10 (09.17.2025):
- Fixed patch 6's nfstrace.h compiler errors due to #include order in nfs2xdr.c
- Rebased patches 2 and 5 to reduce churn/indentation for EINVAL error logging.
- Cc Chuck and Jeff for review of patch 2's fs/nfsd/localio.c change to add .nfsd_file_dio_alignment callback to nfsd_localio_operations
Changes since v9 (09.15.2025):
- Updated patch 3 to make bvec init while loop similar to nfsd_iter_read()
- To support both sync and async completion of DIO (sync completion will happen if underlying driver is ramdisk, etc): push DIO's nfs_local_pgio_done() down from nfs_local_{read,write}_done to nfs_local_{read,write}_aio_complete
Changes since v8 (08.15.2025)
- Removed all fs/nfs/direct.c changes and pushed this misaligned DIO handling down to LOCALIO where it belongs.
- Updated all commit headers to reflect changes to code.
- Because misaligned DIO is now handled properly in LOCALIO, removed the nfs modparam 'localio_O_DIRECT_semantics' that was added during v6.14 to require users opt-in to the requirement that all O_DIRECT be properly DIO-aligned.
- Enhanced LOCALIO's DIO WRITE code to handle potential for VFS to return -ENOTBLK if it fails to invalidate page cache on WRITE.
- Verified various test IO workloads function as expected; including a misaligned DIO test that failed with the previous v8 direct.c implementation.
- Verified performance remains high with LOCALIO on fast NVMe.
- Verified sparse and checkpatch.pl are clean.
Earlier changelog was provided in v8's 0th patch header, see:
https://lore.kernel.org/linux-nfs/[email protected]/
All review appreciated, thanks.
Mike
Mike Snitzer (7):
nfs/localio: make trace_nfs_local_open_fh more useful nfs/localio: avoid issuing misaligned IO using O_DIRECT nfs/localio: refactor iocb and iov_iter_bvec initialization nfs/localio: refactor iocb initialization
nfs/localio: add proper O_DIRECT support for READ and WRITE nfs/localio: add tracepoints for misaligned DIO READ and WRITE support
NFS: add basic STATX_DIOALIGN and STATX_DIO_READ_ALIGN support
fs/nfs/inode.c | 15 ++ fs/nfs/internal.h | 10 + fs/nfs/localio.c | 405 ++++++++++++++++++++++++++++--------- fs/nfs/nfs2xdr.c | 2 +- fs/nfs/nfs3xdr.c | 2 +- fs/nfs/nfstrace.h | 76 ++++++- fs/nfsd/localio.c | 11 + include/linux/nfslocalio.h | 2 + 8 files changed, 421 insertions(+), 102 deletions(-)
Signature · 1 line
2.44.0