NFSv4 READDIR d_type fixup
Subsystems (2)
v3 cover NFSv4 READDIR d_type fixup
This is a combined posting of two previously posted patches. The first unconditionally adds the type attribute to the list of requested attributes for v4 READDIR. The second patch enables a per-mount modification via sysfs of any of the attributes the client will currently decode for v4 READDIR.
The first patch solves a real problem but may cause a performance regression for some servers that require extra processing to return inode information along with the namespace. We performed an informal survey of most of the major NFSv4 server vendors and although we did not learn of any that may be impacted, the potential remains.
The second patch gives us a way to disable this new READDIR behavior should we find a serious impact in an existing setup. I would appreciate serious consideration of this patch in light of the number of claimed performance regressions that have been reported almost every time we touch this sensitive operation on the client.
For example:
echo 0x800 0x800000 0x0 > /sys/fs/nfs/0\:57/v4_readdir_attrs
.. will revert the behavior change from patch 1.
Changes on v2:
- fix patch 2/2 to compile without CONFIG_NFS_4
Changes on v3:
- fix a lkp warning about line spacing
- hide the v4_readdir_attrs behind CONFIG_NFS_EXPERT_SYSFS
Benjamin Coddington (2):
NFSv4: Always ask for type with READDIR
NFSv4: Allow per-mount tuning of READDIR attrs
fs/nfs/Kconfig | 7 +++ fs/nfs/client.c | 4 ++ fs/nfs/nfs4client.c | 4 ++ fs/nfs/nfs4proc.c | 1 + fs/nfs/nfs4xdr.c | 9 ++-- fs/nfs/sysfs.c | 91 +++++++++++++++++++++++++++++++++++++++ include/linux/nfs_fs_sb.h | 1 + include/linux/nfs_xdr.h | 1 + 8 files changed, 113 insertions(+), 5 deletions(-)
Signature · 1 line
2.41.0
PATCH v3 1/2 NFSv4: Always ask for type with READDIR
Again we have claimed regressions for walking a directory tree, this time with the "find" utility which always tries to optimize away asking for any attributes until it has a complete list of entries. This behavior makes the readdir plus heuristic do the wrong thing, which causes a storm of GETATTRs to determine each entry's type in order to continue the walk.
For v4 add the type attribute to each READDIR request to include it no matter the heuristic. This allows a simple find command to proceed quickly through a directory tree.
--- fs/nfs/nfs4xdr.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
fs/nfs/nfs4xdr.c +2 -2
@@ -1602,7 +1602,7 @@ static void encode_read(struct xdr_stream *xdr, const struct nfs_pgio_args *argsstatic void encode_readdir (struct xdr_stream *xdr,const struct nfs4_readdir_arg *readdir,struct rpc_rqst *req,struct compound_hdr *hdr) {uint32_t attrs[3 ] = { -FATTR4_WORD0_RDATTR_ERROR , +FATTR4_WORD0_TYPE |FATTR4_WORD0_RDATTR_ERROR ,FATTR4_WORD1_MOUNTED_ON_FILEID , };uint32_t dircount = readdir->count;
@@ -1612,7 +1612,7 @@ static void encode_readdir(struct xdr_stream *xdr, const struct nfs4_readdir_argunsigned int i;if (readdir->plus) { - attrs[0 ] |=FATTR4_WORD0_TYPE |FATTR4_WORD0_CHANGE |FATTR4_WORD0_SIZE | + attrs[0 ] |=FATTR4_WORD0_CHANGE |FATTR4_WORD0_SIZE |FATTR4_WORD0_FSID |FATTR4_WORD0_FILEHANDLE |FATTR4_WORD0_FILEID ; attrs[1 ] |=FATTR4_WORD1_MODE |FATTR4_WORD1_NUMLINKS |FATTR4_WORD1_OWNER | FATTR4_WORD1_OWNER_GROUP|FATTR4_WORD1_RAWDEV|
Signature · 1 line
2.41.0
PATCH v3 2/2 NFSv4: Allow per-mount tuning of READDIR attrs
In reply to Benjamin CoddingtonExpose a per-mount knob in sysfs to set the READDIR requested attributes for a non-plus READDIR request.
For example:
echo 0x800 0x800000 0x0 > /sys/fs/nfs/0\:57/v4_readdir_attrs
.. will revert the client to only request rdattr_error and mounted_on_fileid for any non "plus" READDIR, as before the patch preceeding this one in this series. This provides existing installations an option to fix a potential performance regression that may occur after NFS clients update to request additional default READDIR attributes.
Hide the functionality behind CONFIG_NFS_EXPERT_SYSFS.
--- fs/nfs/Kconfig | 7 +++ fs/nfs/client.c | 4 ++ fs/nfs/nfs4client.c | 4 ++ fs/nfs/nfs4proc.c | 1 + fs/nfs/nfs4xdr.c | 7 ++- fs/nfs/sysfs.c | 91 +++++++++++++++++++++++++++++++++++++++ include/linux/nfs_fs_sb.h | 1 + include/linux/nfs_xdr.h | 1 + 8 files changed, 112 insertions(+), 4 deletions(-)
fs/nfs/Kconfig +7 -0
@@ -212,3 +212,10 @@ config NFS_V4_2_READ_PLUS default y help Choose Y here to enable use of the NFS v4.2 READ_PLUS operation. + +config NFS_EXPERT_SYSFS + bool "NFS: Enable the support for expert tuning of nfs behaviors" + default n + help + This option allows potentially dangerous modifications to the default + behaviors of the NFS client via files within sysfs.
fs/nfs/client.c +4 -0
@@ -922,6 +922,10 @@ void nfs_server_copy_userdata(struct nfs_server *target, struct nfs_server *sour target->options = source->options; target->auth_info = source->auth_info; target->port = source->port; +#if IS_ENABLED (CONFIG_NFS_V4 ) +memcpy (target->readdir_attrs, source->readdir_attrs, +sizeof (target->readdir_attrs)); +#endif }EXPORT_SYMBOL_GPL (nfs_server_copy_userdata);
fs/nfs/nfs4client.c +4 -0
@@ -1115,6 +1115,10 @@ static int nfs4_server_common_setup(struct nfs_server *server,nfs4_server_set_init_caps (server); +/* Default (non-plus) v4 readdir attributes */ + server->readdir_attrs[0 ] =FATTR4_WORD0_TYPE |FATTR4_WORD0_RDATTR_ERROR ; + server->readdir_attrs[1 ] =FATTR4_WORD1_MOUNTED_ON_FILEID ; +/* Probe the root fh to retrieve its FSID and filehandle */ error =nfs4_get_rootfh (server, mntfh, auth_probe);if (error <0 )
fs/nfs/nfs4proc.c +1 -0
@@ -5113,6 +5113,7 @@ static int _nfs4_proc_readdir(struct nfs_readdir_arg *nr_arg, .pgbase = 0, .count = nr_arg->page_len, .plus = nr_arg->plus, + .server = server, };struct nfs4_readdir_res res;struct rpc_message msg = {
fs/nfs/nfs4xdr.c +3 -4
@@ -1601,16 +1601,15 @@ static void encode_read(struct xdr_stream *xdr, const struct nfs_pgio_args *argsstatic void encode_readdir (struct xdr_stream *xdr,const struct nfs4_readdir_arg *readdir,struct rpc_rqst *req,struct compound_hdr *hdr) { -uint32_t attrs[3 ] = { -FATTR4_WORD0_TYPE |FATTR4_WORD0_RDATTR_ERROR , -FATTR4_WORD1_MOUNTED_ON_FILEID , - }; +uint32_t attrs[3 ];uint32_t dircount = readdir->count;uint32_t maxcount = readdir->count;__be32 *p, verf[2 ];uint32_t attrlen =0 ;unsigned int i; +memcpy (attrs, readdir->server->readdir_attrs,sizeof (attrs)); +if (readdir->plus) { attrs[0 ] |=FATTR4_WORD0_CHANGE |FATTR4_WORD0_SIZE |FATTR4_WORD0_FSID |FATTR4_WORD0_FILEHANDLE |FATTR4_WORD0_FILEID ;
fs/nfs/sysfs.c +91 -0
103 lines not on this page · raw message
include/linux/nfs_fs_sb.h +1 -0
@@ -219,6 +219,7 @@ struct nfs_server {of change attribute, size, ctime and mtime attributes supported by the server */ +u32 readdir_attrs[3 ];/* V4 tuneable default readdir attrs */ u32 acl_bitmask;/* V4 bitmask representing the ACEs that are supported on this filesystem */
include/linux/nfs_xdr.h +1 -0
@@ -1142,6 +1142,7 @@ struct nfs4_readdir_arg {struct page ** pages;/* zero-copy data */ unsigned int pgbase;/* zero-copy data */ const u32 * bitmask; +const struct nfs_server *server;bool plus; };
Signature · 1 line
2.41.0