Skip to content

nfs: NFSv4.2 client support for UNCACHEABLE_FILE_DATA and UNCACHEABLE_DIRENT_METADATA

Mike Snitzer linux-nfs 3 versions – New

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v4 Latest6 patches 0 repliesNew
  2. v26 patches 0 repliesSuperseded
  3. v14 patches 5 repliesSuperseded

Changes in v4 from the cover letter

- Fix a regression that broke regular-file OPEN against any server that advertises UNCACHEABLE_DIRENT_METADATA (attr 88). v3 placed attr 88 in the shared request bitmaps (nfs4_fattr_bitmap, nfs4_pnfs_open_bitmap), which are used verbatim as the OPEN "open_bitmap" -- a GETATTR that is masked only by the server's supported set and is not type-gated -- so every regular-file OPEN queried a directory-only attribute and the server answered NFS4ERR_INVAL, failing opens broadly. Patch 4 no longer adds attr 88 to those bitmaps; patch 5 instead has nfs4_bitmap_copy_adjust() add it for directory targets (gated on server support). attr 87 continues to ride the shared bitmaps: it is legal, and wanted, on a regular-file OPEN (to drive O_DIRECT).
- Patch 2: close the same class of hole in the other direction -- UNCACHEABLE_FILE_DATA (attr 87) requested on a directory via DELEGRETURN. nfs4_update_changeattr_locked() (which only ever runs on directory inodes) was setting the file-only NFS_INO_INVALID_UNCACHEABLE_FILE_DATA bit, and nfs4_bitmask_set() translated it into a request for attr 87 with no object-type check; with directory delegation support, DELEGRETURN of a directory delegation would then query attr 87 on a directory and fail with NFS4ERR_INVAL. Drop the bit from the directory-only aggregation and request attr 87 in nfs4_bitmask_set() only for regular files.
- Patch 4: document at the inode recording site why the directory attribute needs no NFS_INO_INVALID_* cache-validity bit -- it is requested unconditionally for directories via the type gate, so it is refetched on every directory GETATTR and cannot go stale.
- Commit-message tidy-ups (Link: tags, line wrapping); no other changes.

Patches in v4 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/6nfs4.2: add UNCACHEABLE_FILE_DATA attribute supportfs/nfs/inode.c fs/nfs/nfs4proc.c fs/nfs/nfs4trace.h +5 more+88 -100 replies
2/6nfs4.2: request UNCACHEABLE_FILE_DATA only for regular filesfs/nfs/inode.c fs/nfs/nfs4proc.c+45 -70 replies
3/6nfs4.2: open UNCACHEABLE_FILE_DATA files with O_DIRECTfs/nfs/dir.c fs/nfs/nfs4file.c fs/nfs/nfs4proc.c +1 more+17 -00 replies
4/6nfs4.2: add UNCACHEABLE_DIRENT_METADATA attribute supportfs/nfs/inode.c fs/nfs/nfs4proc.c fs/nfs/nfs4trace.h +4 more+64 -40 replies
5/6nfs4.2: request UNCACHEABLE_DIRENT_METADATA only for directoriesfs/nfs/nfs4proc.c+17 -40 replies
6/6nfs4.2: honor UNCACHEABLE_DIRENT_METADATA by refetching readdirfs/nfs/dir.c fs/nfs/nfstrace.h+17 -20 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v4 yet.

Read the v4 thread

Cover letter [PATCH v4 0/6] nfs: NFSv4.2 client support for UNCACHEABLE_FILE_DATA and UNCACHEABLE_DIRENT_METADATA

This series adds Linux NFSv4.2 client support for two companion, per-object "uncacheable" attributes that let a server advise the client to stop caching state that changes faster than client caches can track:

- UNCACHEABLE_FILE_DATA (FATTR4 87, draft-ietf-nfsv4-uncacheable-files [1]) -- a per-regular-file boolean: suppress caching of the file's data, both write-behind and read caching.

- UNCACHEABLE_DIRENT_METADATA (FATTR4 88, draft-ietf-nfsv4-uncacheable-directories [2]) -- a per-directory
boolean: retrieve directory-entry metadata (names and per-entry size and timestamps) from the server on each READDIR rather than serving it from the client's readdir cache.

Both are OPTIONAL, read-write booleans; the two are independent and apply to disjoint object types (a regular file may carry 87, a directory 88). This client honors a server-set attribute; it does not set either (that is left to server/administrator policy). The motivating deployments expose a single namespace concurrently through NFSv4.2, NFSv3, and SMB, plus server-side policy engines, so file data and directory contents can change faster than a typical client cache lifetime -- producing stale reads and read-modify-write "write holes" for file data, and stale size/timestamp listings for directories.

For UNCACHEABLE_FILE_DATA, a marked regular file is opened O_DIRECT, which suppresses read and write-behind caching and satisfies the spec's durability invariant via the existing direct-I/O path.

For UNCACHEABLE_DIRENT_METADATA, readdir on a marked directory bypasses the readdir cache and refetches from the server, forcing READDIRPLUS so the per-entry attributes the attribute governs (size and timestamps) are refreshed rather than served stale from the inode attribute caches.

Each attribute is requested only for the object type it applies to, since a server must reject a query on any other type with NFS4ERR_INVAL.

The series is organized as:

1/6 decode UNCACHEABLE_FILE_DATA, track per-exported-filesystem
support, and record it on the inode.

  2/6  request UNCACHEABLE_FILE_DATA only for regular files.
  3/6  open uncacheable regular files O_DIRECT.
  4/6  decode UNCACHEABLE_DIRENT_METADATA, track per-exported-filesystem

support, and record it on the inode.
5/6 request UNCACHEABLE_DIRENT_METADATA only for directories. 6/6 honor UNCACHEABLE_DIRENT_METADATA: refetch readdir (forcing
READDIRPLUS) on a marked directory.

[1] https://datatracker.ietf.org/doc/draft-ietf-nfsv4-uncacheable-files/
[2] https://datatracker.ietf.org/doc/draft-ietf-nfsv4-uncacheable-directories/

Changes since v3:
- Fix a regression that broke regular-file OPEN against any server that advertises UNCACHEABLE_DIRENT_METADATA (attr 88). v3 placed attr 88 in the shared request bitmaps (nfs4_fattr_bitmap, nfs4_pnfs_open_bitmap), which are used verbatim as the OPEN "open_bitmap" -- a GETATTR that is masked only by the server's supported set and is not type-gated -- so every regular-file OPEN queried a directory-only attribute and the server answered NFS4ERR_INVAL, failing opens broadly. Patch 4 no longer adds attr 88 to those bitmaps; patch 5 instead has nfs4_bitmap_copy_adjust() add it for directory targets (gated on server support). attr 87 continues to ride the shared bitmaps: it is legal, and wanted, on a regular-file OPEN (to drive O_DIRECT).
- Patch 2: close the same class of hole in the other direction -- UNCACHEABLE_FILE_DATA (attr 87) requested on a directory via DELEGRETURN. nfs4_update_changeattr_locked() (which only ever runs on directory inodes) was setting the file-only NFS_INO_INVALID_UNCACHEABLE_FILE_DATA bit, and nfs4_bitmask_set() translated it into a request for attr 87 with no object-type check; with directory delegation support, DELEGRETURN of a directory delegation would then query attr 87 on a directory and fail with NFS4ERR_INVAL. Drop the bit from the directory-only aggregation and request attr 87 in nfs4_bitmask_set() only for regular files.
- Patch 4: document at the inode recording site why the directory attribute needs no NFS_INO_INVALID_* cache-validity bit -- it is requested unconditionally for directories via the type gate, so it is refetched on every directory GETATTR and cannot go stale.
- Commit-message tidy-ups (Link: tags, line wrapping); no other changes.

Changes since v2:
- Patch 1 (nfs4_fattr_bitmap): place FATTR4_WORD2_UNCACHEABLE_FILE_DATA as the first, unconditional word2 entry and OR in FATTR4_WORD2_SECURITY_LABEL under CONFIG_NFS_V4_SECURITY_LABEL, so the word2 initializer no longer begins with a stray '|' (which fails to build) when CONFIG_NFS_V4_SECURITY_LABEL is disabled.

Changes since v1:
- Drop the v1 1/4 xdrgen patch that added Documentation/sunrpc/xdr/ nfs4_2.x and a generated <linux/sunrpc/xdrgen/nfs4_2.h>. Instead open-code FATTR4_UNCACHEABLE_FILE_DATA in <linux/nfs4.h> alongside the other hand-defined FATTR4 protocol-extension constants, and drop the generated NFS4_fattr4_uncacheable_file_data_sz macro (its single XDR word is folded into nfs4_fattr_value_maxsz).
- Store the per-inode flag as a bool bitfield (bool uncacheable_file_data : 1) and simplify the sites that record it.
- Remove stray blank lines introduced in nfs4_atomic_open() and the nfs4trace.h attribute-flags list.
- Add client support for the companion UNCACHEABLE_DIRENT_METADATA attribute, attr 88 (patches 4-6).

Mike Snitzer (5):
nfs4.2: request UNCACHEABLE_FILE_DATA only for regular files nfs4.2: open UNCACHEABLE_FILE_DATA files with O_DIRECT nfs4.2: add UNCACHEABLE_DIRENT_METADATA attribute support nfs4.2: request UNCACHEABLE_DIRENT_METADATA only for directories nfs4.2: honor UNCACHEABLE_DIRENT_METADATA by refetching readdir

Tom Haynes (1):
nfs4.2: add UNCACHEABLE_FILE_DATA attribute support

 fs/nfs/dir.c            | 22 ++++++++++--
 fs/nfs/inode.c          | 39 ++++++++++++++++++--
 fs/nfs/nfs4file.c       |  2 ++
 fs/nfs/nfs4proc.c       | 80 ++++++++++++++++++++++++++++++++++++++---
 fs/nfs/nfs4trace.h      |  4 ++-
 fs/nfs/nfs4xdr.c        | 64 ++++++++++++++++++++++++++++++++-
 fs/nfs/nfstrace.h       |  4 ++-
 include/linux/nfs4.h    | 18 ++++++++++
 include/linux/nfs_fs.h  |  5 +++
 include/linux/nfs_xdr.h | 11 +++++-
 10 files changed, 235 insertions(+), 14 deletions(-)
Signature · 1 line
2.47.3