nfs: NFSv4.2 client support for UNCACHEABLE_FILE_DATA and UNCACHEABLE_DIRENT_METADATA
Subsystems (2)
v3 cover nfs: NFSv4.2 client support for UNCACHEABLE_FILE_DATA and UNCACHEABLE_DIRENT_METADATA
This series adds Linux NFSv4.2 client support for two companion, per-object "uncacheable" attributes that let a server advise the client to stop caching state that changes faster than client caches can track:
- UNCACHEABLE_FILE_DATA (FATTR4 87, draft-ietf-nfsv4-uncacheable-files [1]) -- a per-regular-file boolean: suppress caching of the file's data, both write-behind and read caching.
- UNCACHEABLE_DIRENT_METADATA (FATTR4 88, draft-ietf-nfsv4-uncacheable-directories [2]) -- a per-directory
boolean: retrieve directory-entry metadata (names and per-entry size and timestamps) from the server on each READDIR rather than serving it from the client's readdir cache.
Both are OPTIONAL, read-write booleans; the two are independent and apply to disjoint object types (a regular file may carry 87, a directory 88). This client honors a server-set attribute; it does not set either (that is left to server/administrator policy). The motivating deployments expose a single namespace concurrently through NFSv4.2, NFSv3, and SMB, plus server-side policy engines, so file data and directory contents can change faster than a typical client cache lifetime -- producing stale reads and read-modify-write "write holes" for file data, and stale size/timestamp listings for directories.
For UNCACHEABLE_FILE_DATA, a marked regular file is opened O_DIRECT, which suppresses read and write-behind caching and satisfies the spec's durability invariant via the existing direct-I/O path.
For UNCACHEABLE_DIRENT_METADATA, readdir on a marked directory bypasses the readdir cache and refetches from the server, forcing READDIRPLUS so the per-entry attributes the attribute governs (size and timestamps) are refreshed rather than served stale from the inode attribute caches.
Each attribute is requested only for the object type it applies to, since a server must reject a query on any other type with NFS4ERR_INVAL.
The series is organized as:
1/6 decode UNCACHEABLE_FILE_DATA, track per-exported-filesystem
support, and record it on the inode.
2/6 request UNCACHEABLE_FILE_DATA only for regular files. 3/6 open uncacheable regular files O_DIRECT. 4/6 decode UNCACHEABLE_DIRENT_METADATA, track per-exported-filesystem
support, and record it on the inode.
5/6 request UNCACHEABLE_DIRENT_METADATA only for directories. 6/6 honor UNCACHEABLE_DIRENT_METADATA: refetch readdir (forcing
READDIRPLUS) on a marked directory.
[1] https://datatracker.ietf.org/doc/draft-ietf-nfsv4-uncacheable-files/
[2] https://datatracker.ietf.org/doc/draft-ietf-nfsv4-uncacheable-directories/
Changes since v2:
- Patch 1 (nfs4_fattr_bitmap): place FATTR4_WORD2_UNCACHEABLE_FILE_DATA as the first, unconditional word2 entry and OR in FATTR4_WORD2_SECURITY_LABEL under CONFIG_NFS_V4_SECURITY_LABEL, so the word2 initializer no longer begins with a stray '|' (which fails to build) when CONFIG_NFS_V4_SECURITY_LABEL is disabled.
Changes since v1:
- Drop the v1 1/4 xdrgen patch that added Documentation/sunrpc/xdr/ nfs4_2.x and a generated <linux/sunrpc/xdrgen/nfs4_2.h>. Instead open-code FATTR4_UNCACHEABLE_FILE_DATA in <linux/nfs4.h> alongside the other hand-defined FATTR4 protocol-extension constants, and drop the generated NFS4_fattr4_uncacheable_file_data_sz macro (its single XDR word is folded into nfs4_fattr_value_maxsz).
- Store the per-inode flag as a bool bitfield (bool uncacheable_file_data : 1) and simplify the sites that record it.
- Remove stray blank lines introduced in nfs4_atomic_open() and the nfs4trace.h attribute-flags list.
- Add client support for the companion UNCACHEABLE_DIRENT_METADATA attribute, attr 88 (patches 4-6).
Mike Snitzer (5):
nfs4.2: request UNCACHEABLE_FILE_DATA only for regular files nfs4.2: open UNCACHEABLE_FILE_DATA files with O_DIRECT nfs4.2: add UNCACHEABLE_DIRENT_METADATA attribute support nfs4.2: request UNCACHEABLE_DIRENT_METADATA only for directories nfs4.2: honor UNCACHEABLE_DIRENT_METADATA by refetching readdir
Tom Haynes (1):
nfs4.2: add UNCACHEABLE_FILE_DATA attribute support
fs/nfs/dir.c | 22 +++++++++++-- fs/nfs/inode.c | 32 +++++++++++++++++-- fs/nfs/nfs4file.c | 2 ++ fs/nfs/nfs4proc.c | 69 +++++++++++++++++++++++++++++++++++++---- fs/nfs/nfs4trace.h | 4 ++- fs/nfs/nfs4xdr.c | 64 +++++++++++++++++++++++++++++++++++++- fs/nfs/nfstrace.h | 4 ++- include/linux/nfs4.h | 18 +++++++++++ include/linux/nfs_fs.h | 5 +++ include/linux/nfs_xdr.h | 11 ++++++- 10 files changed, 216 insertions(+), 15 deletions(-)
Signature · 1 line
2.47.3
PATCH v3 1/6 nfs4.2: add UNCACHEABLE_FILE_DATA attribute support
From: Tom Haynes <[email protected]>
Recognize the NFSv4.2 per-file UNCACHEABLE_FILE_DATA attribute (attr 87, draft-ietf-nfsv4-uncacheable-files): decode it via GETATTR, track per- exported-filesystem support, and record on the inode whether a regular file's data must not be cached. Acting on the attribute (opening such files O_DIRECT) is done by a subsequent change.
If the NFSv4 server reports a regular file's UNCACHEABLE_FILE_DATA as true, it indicates the file's data must not be cached; the client records this in NFS_I(inode)->uncacheable_file_data for use by the I/O paths.
The UNCACHEABLE_FILE_DATA attribute applies only to regular files (NF4REG); per the draft a server MUST reject a query of it on any other object type with NFS4ERR_INVAL. A subsequent commit gates the client accordingly.
See: https://datatracker.ietf.org/doc/draft-ietf-nfsv4-uncacheable-files/
--- fs/nfs/inode.c | 22 +++++++++++++++++++--- fs/nfs/nfs4proc.c | 15 ++++++++++++--- fs/nfs/nfs4trace.h | 3 ++- fs/nfs/nfs4xdr.c | 35 ++++++++++++++++++++++++++++++++++- fs/nfs/nfstrace.h | 3 ++- include/linux/nfs4.h | 9 +++++++++ include/linux/nfs_fs.h | 3 +++ include/linux/nfs_xdr.h | 8 +++++++- 8 files changed, 88 insertions(+), 10 deletions(-)
fs/nfs/inode.c +19 -3
@@ -507,6 +507,7 @@ nfs_fhget(struct super_block *sb, struct nfs_fh *fh, struct nfs_fattr *fattr) inode->i_blocks =0 ; nfsi->write_io =0 ; nfsi->read_io =0 ; + nfsi->uncacheable_file_data = false; nfsi->read_cache_jiffies = fattr->time_start; nfsi->attr_gencount = fattr->gencount;
@@ -561,6 +562,11 @@ nfs_fhget(struct super_block *sb, struct nfs_fh *fh, struct nfs_fattr *fattr) }else if (fattr_supported &NFS_ATTR_FATTR_SPACE_USED && fattr->size !=0 )nfs_set_cache_invalid (inode,NFS_INO_INVALID_BLOCKS ); +if (fattr->valid &NFS_ATTR_FATTR_UNCACHEABLE_FILE_DATA ) + nfsi->uncacheable_file_data = + fattr->aux_flags &NFS_AUX_UNCACHEABLE_FILE_DATA ; +else if (fattr_supported &NFS_ATTR_FATTR_UNCACHEABLE_FILE_DATA ) +nfs_set_cache_invalid (inode,NFS_INO_INVALID_UNCACHEABLE_FILE_DATA );nfs_setsecurity (inode, fattr);
@@ -1975,7 +1981,8 @@ static int nfs_inode_finish_partial_attr_update(const struct nfs_fattr *fattr,NFS_INO_INVALID_ATIME |NFS_INO_INVALID_CTIME |NFS_INO_INVALID_MTIME |NFS_INO_INVALID_SIZE |NFS_INO_INVALID_BLOCKS |NFS_INO_INVALID_OTHER | -NFS_INO_INVALID_NLINK |NFS_INO_INVALID_BTIME ; +NFS_INO_INVALID_NLINK |NFS_INO_INVALID_BTIME | +NFS_INO_INVALID_UNCACHEABLE_FILE_DATA ;unsigned long cache_validity =NFS_I (inode)->cache_validity;enum nfs4_change_attr_type ctype =NFS_SERVER (inode)->change_attr_type;
@@ -2297,7 +2304,8 @@ static int nfs_update_inode(struct inode *inode, struct nfs_fattr *fattr) nfsi->cache_validity &= ~(NFS_INO_INVALID_ATTR |NFS_INO_INVALID_ATIME |NFS_INO_REVAL_FORCED - |NFS_INO_INVALID_BLOCKS ); + |NFS_INO_INVALID_BLOCKS + |NFS_INO_INVALID_UNCACHEABLE_FILE_DATA );/* Do atomic weak cache consistency updates */ nfs_wcc_update_inode (inode, fattr);
@@ -2337,7 +2345,8 @@ static int nfs_update_inode(struct inode *inode, struct nfs_fattr *fattr) | NFS_INO_INVALID_NLINK |NFS_INO_INVALID_MODE |NFS_INO_INVALID_OTHER - |NFS_INO_INVALID_BTIME ; + |NFS_INO_INVALID_BTIME + |NFS_INO_INVALID_UNCACHEABLE_FILE_DATA ;if (S_ISDIR (inode->i_mode))nfs_force_lookup_revalidate (inode); attr_changed = true;
@@ -2461,6 +2470,13 @@ static int nfs_update_inode(struct inode *inode, struct nfs_fattr *fattr) nfsi->cache_validity |= save_cache_validity &NFS_INO_INVALID_BLOCKS ; +if (fattr->valid &NFS_ATTR_FATTR_UNCACHEABLE_FILE_DATA ) + nfsi->uncacheable_file_data = + fattr->aux_flags &NFS_AUX_UNCACHEABLE_FILE_DATA ; +else if (fattr_supported &NFS_ATTR_FATTR_UNCACHEABLE_FILE_DATA ) + nfsi->cache_validity |= + save_cache_validity &NFS_INO_INVALID_UNCACHEABLE_FILE_DATA ; +/* Update attrtimeo value if we're out of the unstable period */ if (attr_changed) {nfs_inc_stats (inode,NFSIOS_ATTRINVALIDATE );
fs/nfs/nfs4proc.c +12 -3
@@ -225,8 +225,9 @@ const u32 nfs4_fattr_bitmap[3] = { | FATTR4_WORD1_TIME_METADATA |FATTR4_WORD1_TIME_MODIFY |FATTR4_WORD1_MOUNTED_ON_FILEID , + FATTR4_WORD2_UNCACHEABLE_FILE_DATA #ifdef CONFIG_NFS_V4_SECURITY_LABEL -FATTR4_WORD2_SECURITY_LABEL + |FATTR4_WORD2_SECURITY_LABEL #endif };
@@ -250,6 +251,7 @@ static const u32 nfs4_pnfs_open_bitmap[3] = { #ifdef CONFIG_NFS_V4_SECURITY_LABEL | FATTR4_WORD2_SECURITY_LABEL #endif + | FATTR4_WORD2_UNCACHEABLE_FILE_DATA }; static const u32 nfs4_open_noattr_bitmap[3] = {
@@ -327,6 +329,9 @@ static void nfs4_bitmap_copy_adjust(__u32 *dst, const __u32 *src,if (!(cache_validity &NFS_INO_INVALID_BTIME )) dst[1 ] &= ~FATTR4_WORD1_TIME_CREATE ; +if (!(cache_validity &NFS_INO_INVALID_UNCACHEABLE_FILE_DATA )) + dst[2 ] &= ~FATTR4_WORD2_UNCACHEABLE_FILE_DATA ; +if (nfs_have_delegated_mtime (inode)) {if (!(cache_validity &NFS_INO_INVALID_ATIME )) dst[1 ] &= ~(FATTR4_WORD1_TIME_ACCESS |FATTR4_WORD1_TIME_ACCESS_SET );
@@ -1238,7 +1243,7 @@ nfs4_update_changeattr_locked(struct inode *inode,NFS_INO_INVALID_SIZE |NFS_INO_INVALID_OTHER |NFS_INO_INVALID_BLOCKS |NFS_INO_INVALID_NLINK |NFS_INO_INVALID_MODE |NFS_INO_INVALID_BTIME | -NFS_INO_INVALID_XATTR ; +NFS_INO_INVALID_XATTR |NFS_INO_INVALID_UNCACHEABLE_FILE_DATA ; nfsi->attrtimeo =NFS_MINATTRTIMEO (inode); } nfsi->attrtimeo_timestamp = jiffies;
@@ -3857,7 +3862,7 @@ static void nfs4_close_context(struct nfs_open_context *ctx, int is_sync)#define FATTR4_WORD1_NFS40_MASK (2*FATTR4_WORD1_MOUNTED_ON_FILEID - 1UL)#define FATTR4_WORD2_NFS41_MASK (2*FATTR4_WORD2_SUPPATTR_EXCLCREAT - 1UL) -#define FATTR4_WORD2_NFS42_MASK (2*FATTR4_WORD2_OPEN_ARGUMENTS - 1UL) +#define FATTR4_WORD2_NFS42_MASK (2*FATTR4_WORD2_UNCACHEABLE_FILE_DATA - 1UL)#define FATTR4_WORD2_NFS42_TIME_DELEG_MASK \ (FATTR4_WORD2_TIME_DELEG_MODIFY|FATTR4_WORD2_TIME_DELEG_ACCESS)
@@ -3981,6 +3986,8 @@ static int _nfs4_server_capabilities(struct nfs_server *server, struct nfs_fh *fmemcpy (server->attr_bitmask_nl, res.attr_bitmask,sizeof (server->attr_bitmask)); server->attr_bitmask_nl[2 ] &= ~FATTR4_WORD2_SECURITY_LABEL ; +if (!(res.attr_bitmask[2 ] &FATTR4_WORD2_UNCACHEABLE_FILE_DATA )) + server->fattr_valid &= ~NFS_ATTR_FATTR_UNCACHEABLE_FILE_DATA ;if (res.open_caps.oa_share_access_want[0 ] &NFS4_SHARE_WANT_OPEN_XOR_DELEGATION )
@@ -5809,6 +5816,8 @@ void nfs4_bitmask_set(__u32 bitmask[], const __u32 src[], bitmask[1 ] |=FATTR4_WORD1_SPACE_USED ;if (cache_validity &NFS_INO_INVALID_BTIME ) bitmask[1 ] |=FATTR4_WORD1_TIME_CREATE ; +if (cache_validity &NFS_INO_INVALID_UNCACHEABLE_FILE_DATA ) + bitmask[2 ] |=FATTR4_WORD2_UNCACHEABLE_FILE_DATA ;if (cache_validity &NFS_INO_INVALID_SIZE ) bitmask[0 ] |=FATTR4_WORD0_SIZE ;
fs/nfs/nfs4trace.h +2 -1
@@ -33,7 +33,8 @@ { NFS_ATTR_FATTR_CHANGE, "CHANGE" }, \ { NFS_ATTR_FATTR_OWNER_NAME, "OWNER_NAME" }, \ { NFS_ATTR_FATTR_GROUP_NAME, "GROUP_NAME" }, \ - { NFS_ATTR_FATTR_BTIME, "BTIME" }) + { NFS_ATTR_FATTR_BTIME, "BTIME" }, \ + { NFS_ATTR_FATTR_UNCACHEABLE_FILE_DATA, "UNCACHEABLE_FILE_DATA" }) DECLARE_EVENT_CLASS(nfs4_clientid_event, TP_PROTO(
fs/nfs/nfs4xdr.c +34 -1
@@ -120,7 +120,8 @@ static int decode_layoutget(struct xdr_stream *xdr, struct rpc_rqst *req, 3*nfstime4_maxsz + \ nfs4_owner_maxsz + \ nfs4_group_maxsz + nfs4_label_maxsz + \ - decode_mdsthreshold_maxsz)) + decode_mdsthreshold_maxsz + \ + 1)) /* uncacheable_file_data */#define nfs4_fattr_maxsz (nfs4_fattr_bitmap_maxsz + \ nfs4_fattr_value_maxsz)#define decode_getattr_maxsz (op_decode_hdr_maxsz + nfs4_fattr_maxsz)
@@ -4380,6 +4381,30 @@ static int decode_attr_open_arguments(struct xdr_stream *xdr, uint32_t *bitmap,return 0 ; } +static int decode_attr_uncacheable_file_data (struct xdr_stream *xdr,uint32_t *bitmap, +uint32_t *res,uint64_t *flags) +{ +int status =0 ; +__be32 *p; + +if (unlikely (bitmap[2 ] & (FATTR4_WORD2_UNCACHEABLE_FILE_DATA -1U ))) +return -EIO ; +if (likely (bitmap[2 ] &FATTR4_WORD2_UNCACHEABLE_FILE_DATA )) { + p =xdr_inline_decode (xdr,4 ); +if (unlikely (!p)) +return -EIO ; +if (be32_to_cpup (p)) + *res |=NFS_AUX_UNCACHEABLE_FILE_DATA ; +else + *res &= ~NFS_AUX_UNCACHEABLE_FILE_DATA ; + bitmap[2 ] &= ~FATTR4_WORD2_UNCACHEABLE_FILE_DATA ; + *flags |=NFS_ATTR_FATTR_UNCACHEABLE_FILE_DATA ; + } +dprintk ("%s: uncacheable_file_data: =%s\n" , __func__, + (*res &NFS_AUX_UNCACHEABLE_FILE_DATA ) ==0 ?"false" :"true" ); +return status; +} +static int verify_attr_len (struct xdr_stream *xdr,unsigned int savep,uint32_t attrlen) {unsigned int attrwords =XDR_QUADLEN (attrlen);
@@ -4725,6 +4750,8 @@ static int decode_getfattr_attrs(struct xdr_stream *xdr, uint32_t *bitmap,uint32_t type;int32_t err; + fattr->aux_flags =0 ; + status =decode_attr_type (xdr, bitmap, &type);if (status <0 ) goto xdr_error;
@@ -4843,6 +4870,12 @@ static int decode_getfattr_attrs(struct xdr_stream *xdr, uint32_t *bitmap, goto xdr_error; fattr->valid |= status; + status =decode_attr_uncacheable_file_data (xdr, bitmap, &fattr->aux_flags, + &fattr->valid); +if (status <0 ) + goto xdr_error; + + status =0 ; xdr_error:dprintk ("%s: xdr returned %d\n" , __func__, -status);return status;
fs/nfs/nfstrace.h +2 -1
@@ -33,7 +33,8 @@ { NFS_INO_INVALID_XATTR, "INVALID_XATTR" }, \ { NFS_INO_INVALID_NLINK, "INVALID_NLINK" }, \ { NFS_INO_INVALID_MODE, "INVALID_MODE" }, \ - { NFS_INO_INVALID_BTIME, "INVALID_BTIME" }) + { NFS_INO_INVALID_BTIME, "INVALID_BTIME" }, \ + { NFS_INO_INVALID_UNCACHEABLE_FILE_DATA, "INVALID_UNCACHEABLE_FILE_DATA" }) #define nfs_show_nfsi_flags(v) \ __print_flags(v, "|", \
include/linux/nfs4.h +9 -0
@@ -389,6 +389,14 @@ enum { FATTR4_XATTR_SUPPORT = 82, }; +/* +* Symbol name and value are from draft-ietf-nfsv4-uncacheable-files +* Section 7. "XDR for Uncacheable Attribute" +*/ +enum { +FATTR4_UNCACHEABLE_FILE_DATA =87 , +}; +/* * The following internal definitions enable processing the above * attribute bits within 32-bit word boundaries.
@@ -475,6 +483,7 @@ enum {#define FATTR4_WORD2_ACL_TRUEFORM_SCOPE BIT(FATTR4_ACL_TRUEFORM_SCOPE - 64)#define FATTR4_WORD2_POSIX_DEFAULT_ACL BIT(FATTR4_POSIX_DEFAULT_ACL - 64)#define FATTR4_WORD2_POSIX_ACCESS_ACL BIT(FATTR4_POSIX_ACCESS_ACL - 64) +#define FATTR4_WORD2_UNCACHEABLE_FILE_DATA BIT(FATTR4_UNCACHEABLE_FILE_DATA - 64)/* MDS threshold bitmap bits */ #define THRESHOLD_RD (1UL << 0)
include/linux/nfs_fs.h +3 -0
@@ -162,6 +162,8 @@ struct nfs_inode {struct timespec64 btime; + bool uncacheable_file_data : 1; +/* * read_cache_jiffies is when we started read-caching this inode. * attrtimeo is for how long the cached information is assumed
@@ -319,6 +321,7 @@ struct nfs4_copy_state {#define NFS_INO_INVALID_NLINK BIT(16)/* cached nlinks is invalid */ #define NFS_INO_INVALID_MODE BIT(17)/* cached mode is invalid */ #define NFS_INO_INVALID_BTIME BIT(18)/* cached btime is invalid */ +#define NFS_INO_INVALID_UNCACHEABLE_FILE_DATA BIT(19)/* cached uncacheable_file_data is invalid */ #define NFS_INO_INVALID_ATTR (NFS_INO_INVALID_CHANGE \ | NFS_INO_INVALID_CTIME \
include/linux/nfs_xdr.h +7 -1
@@ -17,6 +17,9 @@#define NFS_BITMASK_SZ 3 +/* aux_flags in nfs_fattr */ +#define NFS_AUX_UNCACHEABLE_FILE_DATA BIT(0) +struct nfs4_string {unsigned int len;char *data;
@@ -68,6 +71,7 @@ struct nfs_fattr {struct timespec64 mtime;struct timespec64 ctime;struct timespec64 btime; +__u32 aux_flags;/* NFSv4 auxiliary flags bitfield */ __u64 change_attr;/* NFSv4 change attribute */ __u64 pre_change_attr;/* pre-op NFSv4 change attribute */ __u64 pre_size;/* pre_op_attr.size */
@@ -108,6 +112,7 @@ struct nfs_fattr {#define NFS_ATTR_FATTR_GROUP_NAME BIT_ULL(24)#define NFS_ATTR_FATTR_V4_SECURITY_LABEL BIT_ULL(25)#define NFS_ATTR_FATTR_BTIME BIT_ULL(26) +#define NFS_ATTR_FATTR_UNCACHEABLE_FILE_DATA BIT_ULL(27)#define NFS_ATTR_FATTR (NFS_ATTR_FATTR_TYPE \ | NFS_ATTR_FATTR_MODE \
@@ -129,7 +134,8 @@ struct nfs_fattr {#define NFS_ATTR_FATTR_V4 (NFS_ATTR_FATTR \ | NFS_ATTR_FATTR_SPACE_USED \ | NFS_ATTR_FATTR_BTIME \ - | NFS_ATTR_FATTR_V4_SECURITY_LABEL) + | NFS_ATTR_FATTR_V4_SECURITY_LABEL \ + | NFS_ATTR_FATTR_UNCACHEABLE_FILE_DATA)/* * Maximal number of supported layout drivers.
Signature · 1 line
2.47.3
PATCH v3 2/6 nfs4.2: request UNCACHEABLE_FILE_DATA only for regular files
In reply to Mike SnitzerThe UNCACHEABLE_FILE_DATA attribute applies only to regular files (NF4REG); per draft-ietf-nfsv4-uncacheable-files a server MUST reject a query of it on any other object type with NFS4ERR_INVAL. The previous commit decodes and tracks the attribute but does not gate it: the bit rides in the per-server attribute bitmask (server->attr_bitmask) and in the generic getattr request bitmap (nfs4_fattr_bitmap), so it would be requested for non-regular objects too -- e.g. a plain directory GETATTR, a LOOKUP that resolves to a directory, or a CREATE (which only ever makes non-regular objects). A strict server would fail those compounds.
Gate the client accordingly:
- Only set NFS_INO_INVALID_UNCACHEABLE_FILE_DATA on regular-file inodes, so the attribute is never (re)requested for directories or other non-regular objects via the delegation GETATTR or nfs4_bitmask_set() refresh paths.
- Gate the request by object type at the single choke point nfs4_bitmap_copy_adjust(), which clears FATTR4_WORD2_UNCACHEABLE_FILE_DATA unless the target inode is a regular file (a NULL inode -- unknown object type -- clears it too). This already covers GETATTR, SETATTR and LINK; route LOOKUP, LOOKUPP and CREATE through it as well.
The bit is kept in server->attr_bitmask (it is server-supported, and OPEN still requests it via its regular-file-only open_bitmap), so no bespoke per-data-file bitmask plumbing is needed. The remaining getattr-bearing compounds are already safe: ACCESS, DELEGRETURN, WRITE, CLOSE and LAYOUTCOMMIT use server->cache_consistency_bitmask (no word2 attributes) or operate on regular files; READDIR does not encode the bit; and LOOKUP_ROOT, FSINFO, STATFS and PATHCONF use fixed bitmaps without it.
--- fs/nfs/inode.c | 6 ++++-- fs/nfs/nfs4proc.c | 37 ++++++++++++++++++++++++++++++++++--- 2 files changed, 38 insertions(+), 5 deletions(-)
fs/nfs/inode.c +4 -2
@@ -565,7 +565,8 @@ nfs_fhget(struct super_block *sb, struct nfs_fh *fh, struct nfs_fattr *fattr)if (fattr->valid &NFS_ATTR_FATTR_UNCACHEABLE_FILE_DATA ) nfsi->uncacheable_file_data = fattr->aux_flags &NFS_AUX_UNCACHEABLE_FILE_DATA ; -else if (fattr_supported &NFS_ATTR_FATTR_UNCACHEABLE_FILE_DATA ) +else if (S_ISREG (inode->i_mode) && + (fattr_supported &NFS_ATTR_FATTR_UNCACHEABLE_FILE_DATA ))nfs_set_cache_invalid (inode,NFS_INO_INVALID_UNCACHEABLE_FILE_DATA );nfs_setsecurity (inode, fattr);
@@ -2473,7 +2474,8 @@ static int nfs_update_inode(struct inode *inode, struct nfs_fattr *fattr)if (fattr->valid &NFS_ATTR_FATTR_UNCACHEABLE_FILE_DATA ) nfsi->uncacheable_file_data = fattr->aux_flags &NFS_AUX_UNCACHEABLE_FILE_DATA ; -else if (fattr_supported &NFS_ATTR_FATTR_UNCACHEABLE_FILE_DATA ) +else if (S_ISREG (inode->i_mode) && + (fattr_supported &NFS_ATTR_FATTR_UNCACHEABLE_FILE_DATA )) nfsi->cache_validity |= save_cache_validity &NFS_INO_INVALID_UNCACHEABLE_FILE_DATA ;
fs/nfs/nfs4proc.c +34 -3
@@ -308,6 +308,15 @@ static void nfs4_bitmap_copy_adjust(__u32 *dst, const __u32 *src,unsigned long cache_validity;memcpy (dst, src,NFS4_BITMASK_SZ *sizeof (*dst)); +/* +* The uncacheable_file_data attribute applies only to regular files +* (NF4REG); a server must reject a query of it on any other object +* type with NFS4ERR_INVAL. Never request it unless the target is +* known to be a regular file (callers with an unknown object type, +* e.g. LOOKUP, pass a NULL inode). +*/ +if (!inode || !S_ISREG (inode->i_mode)) + dst[2 ] &= ~FATTR4_WORD2_UNCACHEABLE_FILE_DATA ;if (!inode || !nfs_have_read_or_write_delegation (inode))return ;
@@ -4598,6 +4607,7 @@ static int _nfs4_proc_lookup(struct rpc_clnt *clnt, struct inode *dir, .rpc_resp = &res, };unsigned short task_flags =0 ; +__u32 bitmask[NFS4_BITMASK_SZ ];if (nfs_server_capable (dir,NFS_CAP_MOVEABLE )) task_flags =RPC_TASK_MOVEABLE ;
@@ -4606,7 +4616,13 @@ static int _nfs4_proc_lookup(struct rpc_clnt *clnt, struct inode *dir,if (nfs_lookup_is_soft_revalidate (dentry)) task_flags |=RPC_TASK_TIMEOUT ; - args.bitmask =nfs4_bitmask (server, fattr->label); +/* +* The looked-up object's type is unknown here, so gate out the +* regular-file-only uncacheable_file_data attribute (NULL inode). +*/ +nfs4_bitmap_copy_adjust (bitmask,nfs4_bitmask (server, fattr->label), +NULL ,0 ); + args.bitmask = bitmask;nfs_fattr_init (fattr);
@@ -4720,13 +4736,20 @@ static int _nfs4_proc_lookupp(struct inode *inode, .rpc_resp = &res, };unsigned short task_flags =0 ; +__u32 bitmask[NFS4_BITMASK_SZ ];if (server->flags &NFS_MOUNT_SOFTREVAL ) task_flags |=RPC_TASK_TIMEOUT ;if (server->caps &NFS_CAP_MOVEABLE ) task_flags |=RPC_TASK_MOVEABLE ; - args.bitmask =nfs4_bitmask (server, fattr->label); +/* +* The looked-up object's type is unknown here, so gate out the +* regular-file-only uncacheable_file_data attribute (NULL inode). +*/ +nfs4_bitmap_copy_adjust (bitmask,nfs4_bitmask (server, fattr->label), +NULL ,0 ); + args.bitmask = bitmask;nfs_fattr_init (fattr);nfs4_init_sequence (server->nfs_client, &args.seq_args, &res.seq_res,0 ,0 );
@@ -5141,6 +5164,7 @@ struct nfs4_createdata {struct nfs4_create_res res;struct nfs_fh fh;struct nfs_fattr fattr; +u32 bitmask[NFS4_BITMASK_SZ ]; }; static struct nfs4_createdata *nfs4_alloc_createdata(struct inode *dir,
@@ -5164,7 +5188,14 @@ static struct nfs4_createdata *nfs4_alloc_createdata(struct inode *dir, data->arg.name = name; data->arg.attrs = sattr; data->arg.ftype = ftype; - data->arg.bitmask =nfs4_bitmask (server, data->fattr.label); +/* +* CREATE only makes non-regular objects, so gate out the +* regular-file-only uncacheable_file_data attribute (NULL inode). +*/ +nfs4_bitmap_copy_adjust (data->bitmask, +nfs4_bitmask (server, data->fattr.label), +NULL ,0 ); + data->arg.bitmask = data->bitmask; data->arg.umask =current_umask (); data->res.server = server; data->res.fh = &data->fh;
Signature · 1 line
2.47.3
PATCH v3 3/6 nfs4.2: open UNCACHEABLE_FILE_DATA files with O_DIRECT
In reply to Mike SnitzerHonor the per-file UNCACHEABLE_FILE_DATA attribute by transparently opening such regular files with O_DIRECT, so reads and writes bypass the page cache as the attribute requires, without the application having to request O_DIRECT itself.
This follows the model the specification describes: the attribute is "similar in intent to O_DIRECT" and clients "retain flexibility in how they satisfy the requirements" (draft-ietf-nfsv4-uncacheable-files Section 4.4, "Relationship to Direct I/O"), and its Implementation Status (Section 6) describes a prototype Linux client that "treats the attribute as an indication to use O_DIRECT-like behavior for file access".
Introduce an NFS_CONTEXT_O_DIRECT open-context flag: nfs4_atomic_open() sets it when the resolved inode has uncacheable_file_data set (and the open is not O_APPEND), and the open paths nfs_atomic_open() and nfs4_file_open() apply O_DIRECT to the file when the flag is set.
The I/O mode is thus selected at open time and is not changed for an already-open file: a later change to the attribute takes effect on the next open. The specification permits this -- a client that has already opened a file MAY continue with its existing caching behavior and apply the updated attribute to subsequent operations (Section 5).
The delegation interaction in Section 4.3 was considered: it permits read caching to remain when another NFSv4.2 mechanism, such as a delegation, already ensures a consistent view of the file. That relaxation is optional ("may remain appropriate") and read-only -- it does not relax write-behind suppression (Section 4.1) or the WRITE durability invariant (Section 4.2). This implementation deliberately does not take it: an uncacheable file is opened O_DIRECT regardless of any delegation held, which is compliant (read caching is simply suppressed more aggressively than the Section 4.3 minimum) and avoids decoupling read vs write caching behind a single open flag. Relaxing reads under a delegation is left as a possible future optimization.
Section 6 observes the benefit holds "for applications that issue well-formed I/O requests". That alignment caveat does not constrain the Linux NFS client's over-the-wire path: the client readily issues misaligned I/O using O_DIRECT over SunRPC to the remote NFS server. The only place a fallback from O_DIRECT to buffered I/O for misaligned I/O applies is NFS LOCALIO (fs/nfs/localio.c), which detects non-DIO-aligned I/O and falls back internally; that path is unaffected by this change.
See: https://datatracker.ietf.org/doc/draft-ietf-nfsv4-uncacheable-files/
--- fs/nfs/dir.c | 4 ++++ fs/nfs/nfs4file.c | 2 ++ fs/nfs/nfs4proc.c | 10 ++++++++++ include/linux/nfs_fs.h | 1 + 4 files changed, 17 insertions(+)
fs/nfs/dir.c +4 -0
@@ -2208,6 +2208,10 @@ int nfs_atomic_open(struct inode *dir, struct dentry *dentry, goto out; } file->f_mode |=FMODE_CAN_ODIRECT ; +if (test_bit (NFS_CONTEXT_O_DIRECT , &ctx->flags)) { + file->f_flags |=O_DIRECT ; + open_flags |=O_DIRECT ; + } err =nfs_finish_open (ctx, ctx->dentry, file, open_flags);trace_nfs_atomic_open_exit (dir, ctx, open_flags, err);
fs/nfs/nfs4file.c +2 -0
@@ -91,6 +91,8 @@ nfs4_file_open(struct inode *inode, struct file *filp)nfs_fscache_open_file (inode, filp); err =0 ; filp->f_mode |=FMODE_CAN_ODIRECT ; +if (test_bit (NFS_CONTEXT_O_DIRECT , &ctx->flags)) + filp->f_flags |=O_DIRECT ; out_put_ctx:put_nfs_open_context (ctx);
fs/nfs/nfs4proc.c +10 -0
@@ -3853,6 +3853,16 @@ nfs4_atomic_open(struct inode *dir, struct nfs_open_context *ctx,if (IS_ERR (state))return ERR_CAST (state); + +/* +* Use O_DIRECT if file was marked as Uncacheable, see: +* https://datatracker.ietf.org/doc/draft-ietf-nfsv4-uncacheable-files/ +*/ +if (!(open_flags &O_DIRECT ) &&NFS_I (state->inode)->uncacheable_file_data) { +if (!(open_flags &O_APPEND )) +set_bit (NFS_CONTEXT_O_DIRECT , &ctx->flags); + } +return state->inode; }
include/linux/nfs_fs.h +1 -0
@@ -110,6 +110,7 @@ struct nfs_open_context {#define NFS_CONTEXT_UNLOCK (3)#define NFS_CONTEXT_FILE_OPEN (4)#define NFS_CONTEXT_WRITE_SYNC (5) +#define NFS_CONTEXT_O_DIRECT (6)struct nfs4_threshold *mdsthreshold;struct list_head list;
Signature · 1 line
2.47.3
PATCH v3 4/6 nfs4.2: add UNCACHEABLE_DIRENT_METADATA attribute support
In reply to Mike SnitzerRecognize the NFSv4.2 per-directory UNCACHEABLE_DIRENT_METADATA attribute (attr 88, draft-ietf-nfsv4-uncacheable-directories): decode it via GETATTR, track per-exported-filesystem support, and record on the inode whether a directory's directory-entry metadata must not be cached. Honoring the attribute (refetching directory-entry metadata from the server on each READDIR) is done by a subsequent change.
If the NFSv4 server reports a directory's UNCACHEABLE_DIRENT_METADATA as true, it indicates the directory's directory-entry metadata must not be cached; the client records this in NFS_I(inode)->uncacheable_dirent_metadata for use by the readdir path.
The UNCACHEABLE_DIRENT_METADATA attribute applies only to directory objects (NF4DIR) and is independent of the companion UNCACHEABLE_FILE_DATA attribute (attr 87); the two govern different aspects of client caching and may be used separately. A subsequent commit gates the client accordingly so the attribute is requested only for directories.
Unlike the per-file UNCACHEABLE_FILE_DATA attribute, this directory attribute is deliberately not tied to the cache_validity / file-delegation machinery (nfs4_bitmap_copy_adjust()'s delegation block, nfs4_bitmask_set()):
a directory cannot hold an NFSv4 read/write (file) delegation, and per the draft a server must recall or withhold a directory delegation while the attribute is set, so there is never an authoritative cached state to optimize against. The attribute is simply requested on every directory GETATTR and recorded on receipt.
See: https://datatracker.ietf.org/doc/draft-ietf-nfsv4-uncacheable-directories/
--- fs/nfs/inode.c | 8 ++++++++ fs/nfs/nfs4proc.c | 6 +++++- fs/nfs/nfs4trace.h | 3 ++- fs/nfs/nfs4xdr.c | 31 ++++++++++++++++++++++++++++++- include/linux/nfs4.h | 9 +++++++++ include/linux/nfs_fs.h | 1 + include/linux/nfs_xdr.h | 5 ++++- 7 files changed, 59 insertions(+), 4 deletions(-)
fs/nfs/inode.c +8 -0
@@ -508,6 +508,7 @@ nfs_fhget(struct super_block *sb, struct nfs_fh *fh, struct nfs_fattr *fattr) nfsi->write_io =0 ; nfsi->read_io =0 ; nfsi->uncacheable_file_data = false; + nfsi->uncacheable_dirent_metadata = false; nfsi->read_cache_jiffies = fattr->time_start; nfsi->attr_gencount = fattr->gencount;
@@ -568,6 +569,9 @@ nfs_fhget(struct super_block *sb, struct nfs_fh *fh, struct nfs_fattr *fattr)else if (S_ISREG (inode->i_mode) && (fattr_supported &NFS_ATTR_FATTR_UNCACHEABLE_FILE_DATA ))nfs_set_cache_invalid (inode,NFS_INO_INVALID_UNCACHEABLE_FILE_DATA ); +if (fattr->valid &NFS_ATTR_FATTR_UNCACHEABLE_DIRENT_METADATA ) + nfsi->uncacheable_dirent_metadata = + fattr->aux_flags &NFS_AUX_UNCACHEABLE_DIRENT_METADATA ;nfs_setsecurity (inode, fattr);
@@ -2479,6 +2483,10 @@ static int nfs_update_inode(struct inode *inode, struct nfs_fattr *fattr) nfsi->cache_validity |= save_cache_validity &NFS_INO_INVALID_UNCACHEABLE_FILE_DATA ; +if (fattr->valid &NFS_ATTR_FATTR_UNCACHEABLE_DIRENT_METADATA ) + nfsi->uncacheable_dirent_metadata = + fattr->aux_flags &NFS_AUX_UNCACHEABLE_DIRENT_METADATA ; +/* Update attrtimeo value if we're out of the unstable period */ if (attr_changed) {nfs_inc_stats (inode,NFSIOS_ATTRINVALIDATE );
fs/nfs/nfs4proc.c +5 -1
@@ -226,6 +226,7 @@ const u32 nfs4_fattr_bitmap[3] = { | FATTR4_WORD1_TIME_MODIFY |FATTR4_WORD1_MOUNTED_ON_FILEID ,FATTR4_WORD2_UNCACHEABLE_FILE_DATA + | FATTR4_WORD2_UNCACHEABLE_DIRENT_METADATA #ifdef CONFIG_NFS_V4_SECURITY_LABEL |FATTR4_WORD2_SECURITY_LABEL #endif
@@ -252,6 +253,7 @@ static const u32 nfs4_pnfs_open_bitmap[3] = { |FATTR4_WORD2_SECURITY_LABEL #endif | FATTR4_WORD2_UNCACHEABLE_FILE_DATA + |FATTR4_WORD2_UNCACHEABLE_DIRENT_METADATA }; static const u32 nfs4_open_noattr_bitmap[3] = {
@@ -3881,7 +3883,7 @@ static void nfs4_close_context(struct nfs_open_context *ctx, int is_sync)#define FATTR4_WORD1_NFS40_MASK (2*FATTR4_WORD1_MOUNTED_ON_FILEID - 1UL)#define FATTR4_WORD2_NFS41_MASK (2*FATTR4_WORD2_SUPPATTR_EXCLCREAT - 1UL) -#define FATTR4_WORD2_NFS42_MASK (2*FATTR4_WORD2_UNCACHEABLE_FILE_DATA - 1UL) +#define FATTR4_WORD2_NFS42_MASK (2*FATTR4_WORD2_UNCACHEABLE_DIRENT_METADATA - 1UL)#define FATTR4_WORD2_NFS42_TIME_DELEG_MASK \ (FATTR4_WORD2_TIME_DELEG_MODIFY|FATTR4_WORD2_TIME_DELEG_ACCESS)
@@ -4007,6 +4009,8 @@ static int _nfs4_server_capabilities(struct nfs_server *server, struct nfs_fh *f server->attr_bitmask_nl[2 ] &= ~FATTR4_WORD2_SECURITY_LABEL ;if (!(res.attr_bitmask[2 ] &FATTR4_WORD2_UNCACHEABLE_FILE_DATA )) server->fattr_valid &= ~NFS_ATTR_FATTR_UNCACHEABLE_FILE_DATA ; +if (!(res.attr_bitmask[2 ] &FATTR4_WORD2_UNCACHEABLE_DIRENT_METADATA )) + server->fattr_valid &= ~NFS_ATTR_FATTR_UNCACHEABLE_DIRENT_METADATA ;if (res.open_caps.oa_share_access_want[0 ] &NFS4_SHARE_WANT_OPEN_XOR_DELEGATION )
fs/nfs/nfs4trace.h +2 -1
@@ -34,7 +34,8 @@ { NFS_ATTR_FATTR_OWNER_NAME, "OWNER_NAME" }, \ { NFS_ATTR_FATTR_GROUP_NAME, "GROUP_NAME" }, \ { NFS_ATTR_FATTR_BTIME, "BTIME" }, \ - { NFS_ATTR_FATTR_UNCACHEABLE_FILE_DATA, "UNCACHEABLE_FILE_DATA" }) + { NFS_ATTR_FATTR_UNCACHEABLE_FILE_DATA, "UNCACHEABLE_FILE_DATA" }, \ + { NFS_ATTR_FATTR_UNCACHEABLE_DIRENT_METADATA, "UNCACHEABLE_DIRENT_METADATA" }) DECLARE_EVENT_CLASS(nfs4_clientid_event, TP_PROTO(
fs/nfs/nfs4xdr.c +30 -1
@@ -121,7 +121,7 @@ static int decode_layoutget(struct xdr_stream *xdr, struct rpc_rqst *req, nfs4_owner_maxsz + \ nfs4_group_maxsz + nfs4_label_maxsz + \ decode_mdsthreshold_maxsz + \ - 1)) /* uncacheable_file_data */ + 1)) /* uncacheable_file_data / dirent_metadata */#define nfs4_fattr_maxsz (nfs4_fattr_bitmap_maxsz + \ nfs4_fattr_value_maxsz)#define decode_getattr_maxsz (op_decode_hdr_maxsz + nfs4_fattr_maxsz)
@@ -4405,6 +4405,30 @@ static int decode_attr_uncacheable_file_data(struct xdr_stream *xdr, uint32_t *breturn status; } +static int decode_attr_uncacheable_dirent_metadata (struct xdr_stream *xdr,uint32_t *bitmap, +uint32_t *res,uint64_t *flags) +{ +int status =0 ; +__be32 *p; + +if (unlikely (bitmap[2 ] & (FATTR4_WORD2_UNCACHEABLE_DIRENT_METADATA -1U ))) +return -EIO ; +if (likely (bitmap[2 ] &FATTR4_WORD2_UNCACHEABLE_DIRENT_METADATA )) { + p =xdr_inline_decode (xdr,4 ); +if (unlikely (!p)) +return -EIO ; +if (be32_to_cpup (p)) + *res |=NFS_AUX_UNCACHEABLE_DIRENT_METADATA ; +else + *res &= ~NFS_AUX_UNCACHEABLE_DIRENT_METADATA ; + bitmap[2 ] &= ~FATTR4_WORD2_UNCACHEABLE_DIRENT_METADATA ; + *flags |=NFS_ATTR_FATTR_UNCACHEABLE_DIRENT_METADATA ; + } +dprintk ("%s: uncacheable_dirent_metadata: =%s\n" , __func__, + (*res &NFS_AUX_UNCACHEABLE_DIRENT_METADATA ) ==0 ?"false" :"true" ); +return status; +} +static int verify_attr_len (struct xdr_stream *xdr,unsigned int savep,uint32_t attrlen) {unsigned int attrwords =XDR_QUADLEN (attrlen);
@@ -4875,6 +4899,11 @@ static int decode_getfattr_attrs(struct xdr_stream *xdr, uint32_t *bitmap,if (status <0 ) goto xdr_error; + status =decode_attr_uncacheable_dirent_metadata (xdr, bitmap, &fattr->aux_flags, + &fattr->valid); +if (status <0 ) + goto xdr_error; + status =0 ; xdr_error:dprintk ("%s: xdr returned %d\n" , __func__, -status);
include/linux/nfs4.h +9 -0
@@ -397,6 +397,14 @@ enum { FATTR4_UNCACHEABLE_FILE_DATA = 87, }; +/* +* Symbol name and value are from draft-ietf-nfsv4-uncacheable-directories +* Section 8. "XDR for Uncacheable Dirents Attribute" +*/ +enum { +FATTR4_UNCACHEABLE_DIRENT_METADATA =88 , +}; +/* * The following internal definitions enable processing the above * attribute bits within 32-bit word boundaries.
@@ -484,6 +492,7 @@ enum {#define FATTR4_WORD2_POSIX_DEFAULT_ACL BIT(FATTR4_POSIX_DEFAULT_ACL - 64)#define FATTR4_WORD2_POSIX_ACCESS_ACL BIT(FATTR4_POSIX_ACCESS_ACL - 64)#define FATTR4_WORD2_UNCACHEABLE_FILE_DATA BIT(FATTR4_UNCACHEABLE_FILE_DATA - 64) +#define FATTR4_WORD2_UNCACHEABLE_DIRENT_METADATA BIT(FATTR4_UNCACHEABLE_DIRENT_METADATA - 64)/* MDS threshold bitmap bits */ #define THRESHOLD_RD (1UL << 0)
include/linux/nfs_fs.h +1 -0
@@ -164,6 +164,7 @@ struct nfs_inode {struct timespec64 btime; bool uncacheable_file_data : 1; + bool uncacheable_dirent_metadata : 1;/* * read_cache_jiffies is when we started read-caching this inode.
include/linux/nfs_xdr.h +4 -1
@@ -19,6 +19,7 @@/* aux_flags in nfs_fattr */ #define NFS_AUX_UNCACHEABLE_FILE_DATA BIT(0) +#define NFS_AUX_UNCACHEABLE_DIRENT_METADATA BIT(1)struct nfs4_string {unsigned int len;
@@ -113,6 +114,7 @@ struct nfs_fattr {#define NFS_ATTR_FATTR_V4_SECURITY_LABEL BIT_ULL(25)#define NFS_ATTR_FATTR_BTIME BIT_ULL(26)#define NFS_ATTR_FATTR_UNCACHEABLE_FILE_DATA BIT_ULL(27) +#define NFS_ATTR_FATTR_UNCACHEABLE_DIRENT_METADATA BIT_ULL(28)#define NFS_ATTR_FATTR (NFS_ATTR_FATTR_TYPE \ | NFS_ATTR_FATTR_MODE \
@@ -135,7 +137,8 @@ struct nfs_fattr { | NFS_ATTR_FATTR_SPACE_USED \ |NFS_ATTR_FATTR_BTIME \ |NFS_ATTR_FATTR_V4_SECURITY_LABEL \ - | NFS_ATTR_FATTR_UNCACHEABLE_FILE_DATA) + |NFS_ATTR_FATTR_UNCACHEABLE_FILE_DATA \ + | NFS_ATTR_FATTR_UNCACHEABLE_DIRENT_METADATA)/* * Maximal number of supported layout drivers.
Signature · 1 line
2.47.3
PATCH v3 5/6 nfs4.2: request UNCACHEABLE_DIRENT_METADATA only for directories
In reply to Mike SnitzerThe UNCACHEABLE_DIRENT_METADATA attribute (attr 88) applies only to directory objects (NF4DIR); per draft-ietf-nfsv4-uncacheable-directories a server must reject a query of it on any other object type with NFS4ERR_INVAL. Gate the request by object type at the single choke point nfs4_bitmap_copy_adjust(), stripping FATTR4_WORD2_UNCACHEABLE_DIRENT_METADATA unless the target is known to be a directory (callers with an unknown object type, e.g. LOOKUP/LOOKUPP/CREATE, pass a NULL inode and are already routed through this helper). This mirrors the NF4REG gating of the companion UNCACHEABLE_FILE_DATA attribute: a regular file requests file_data and never dirent_metadata, a directory requests dirent_metadata and never file_data, and an unknown/other object requests neither.
The bit stays in server->attr_bitmask, so OPEN (which requests the regular-file open bitmap) is unaffected. This type gate runs before the helper's read/write (file) delegation handling and is the only adjustment the directory attribute needs: a directory cannot hold a file delegation, so the delegation-based suppression below it never applies.
See: https://datatracker.ietf.org/doc/draft-ietf-nfsv4-uncacheable-directories/
--- fs/nfs/nfs4proc.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-)
fs/nfs/nfs4proc.c +7 -4
@@ -312,13 +312,16 @@ static void nfs4_bitmap_copy_adjust(__u32 *dst, const __u32 *src,memcpy (dst, src,NFS4_BITMASK_SZ *sizeof (*dst));/* * The uncacheable_file_data attribute applies only to regular files -* (NF4REG); a server must reject a query of it on any other object -* type with NFS4ERR_INVAL. Never request it unless the target is -* known to be a regular file (callers with an unknown object type, -* e.g. LOOKUP, pass a NULL inode). +* (NF4REG) and the uncacheable_dirent_metadata attribute only to +* directories (NF4DIR); a server must reject a query of either on any +* other object type with NFS4ERR_INVAL. Never request either unless +* the target is known to be of the matching type (callers with an +* unknown object type, e.g. LOOKUP, pass a NULL inode). */ if (!inode || !S_ISREG (inode->i_mode)) dst[2 ] &= ~FATTR4_WORD2_UNCACHEABLE_FILE_DATA ; +if (!inode || !S_ISDIR (inode->i_mode)) + dst[2 ] &= ~FATTR4_WORD2_UNCACHEABLE_DIRENT_METADATA ;if (!inode || !nfs_have_read_or_write_delegation (inode))return ;
Signature · 1 line
2.47.3
PATCH v3 6/6 nfs4.2: honor UNCACHEABLE_DIRENT_METADATA by refetching readdir
In reply to Mike SnitzerHonor the per-directory UNCACHEABLE_DIRENT_METADATA attribute: when a directory is marked uncacheable, nfs_readdir() bypasses the readdir page cache and refetches directory-entry metadata from the server on every readdir, satisfying the always-refetch semantics the attribute requires (draft-ietf-nfsv4-uncacheable-directories Section 5.1).
Rather than searching the cached readdir folios, nfs_readdir() forces the -EBADCOOKIE path so the request is served by uncached_readdir(); the dir_cookie == 0 (start-of-directory) case is included so the very first readdir of an uncacheable directory also goes to the server. A tracepoint records when an uncacheable directory bypasses the cache.
The metadata the attribute governs is the per-entry size and timestamps, which are carried only by READDIRPLUS (a plain READDIR refreshes names but leaves the entries' attributes to the inode attribute caches). So also force READDIRPLUS for an uncacheable directory (when the server is capable): nfs_use_readdirplus() otherwise enables it only at the start of the directory or once cache usage crosses a threshold, and the cache- bypassing path above never accrues that usage -- which would leave continuation READDIRs of a large directory refreshing names but serving stale per-entry attributes. Forcing READDIRPLUS makes each READDIR refresh the entries' attribute caches (via nfs_prime_dcache() -> nfs_refresh_inode()), so a subsequent stat() observes current values.
The attribute does not change NFSv4.2 change-attribute semantics: the client continues to use the directory change attribute for validation; this only suppresses serving READDIR responses from the local cache.
See: https://datatracker.ietf.org/doc/draft-ietf-nfsv4-uncacheable-directories/
--- fs/nfs/dir.c | 18 ++++++++++++++++-- fs/nfs/nfstrace.h | 1 + 2 files changed, 17 insertions(+), 2 deletions(-)
fs/nfs/dir.c +16 -2
@@ -667,6 +667,14 @@ static bool nfs_use_readdirplus(struct inode *dir, struct dir_context *ctx,return false;if (NFS_SERVER (dir)->flags &NFS_MOUNT_FORCE_RDIRPLUS )return true; +/* +* An uncacheable directory must refetch directory-entry metadata +* (including per-entry size and timestamps) from the server on each +* READDIR; force READDIRPLUS so those attributes are refreshed on +* every call rather than left stale in the inode attribute caches. +*/ +if (NFS_I (dir)->uncacheable_dirent_metadata) +return true;if (ctx->pos ==0 || cache_hits + cache_misses >NFS_READDIR_CACHE_USAGE_THRESHOLD )return true;
@@ -1274,12 +1282,18 @@ static int nfs_readdir(struct file *file, struct dir_context *ctx) desc->clear_cache = force_clear; do { - res = readdir_search_pagecache(desc); + if (nfsi->uncacheable_dirent_metadata) { + res = -EBADCOOKIE; + trace_nfs_readdir_uncacheable_directory(inode); + } else { + res = readdir_search_pagecache(desc); + } if (res == -EBADCOOKIE) { res = 0; /* This means either end of directory */ - if (desc->dir_cookie && !desc->eof) { + if ((desc->dir_cookie || nfsi->uncacheable_dirent_metadata) && + !desc->eof) { /* Or that the server has 'lost' a cookie */ res = uncached_readdir(desc); if (res == 0)
fs/nfs/nfstrace.h +1 -0
@@ -181,6 +181,7 @@ DEFINE_NFS_INODE_EVENT_DONE(nfs_fsync_exit);DEFINE_NFS_INODE_EVENT (nfs_access_enter);DEFINE_NFS_INODE_EVENT_DONE (nfs_set_cache_invalid);DEFINE_NFS_INODE_EVENT (nfs_readdir_force_readdirplus); +DEFINE_NFS_INODE_EVENT (nfs_readdir_uncacheable_directory);DEFINE_NFS_INODE_EVENT_DONE (nfs_readdir_cache_fill_done);DEFINE_NFS_INODE_EVENT_DONE (nfs_readdir_uncached_done);
Signature · 1 line
2.47.3