Cifs: cache entries for the cached directory
Subsystems (2)
Steve, List
Updated version of the patch to add caching of directory entries for the cached dir.
I have updated with all the suggestions from Enzo and Paulo.
In this patch, once you have done the initial 'ls /mnt' and future listings of the root of the share will be served straight out of cache until the lease is broken.
The first three patches are small and isolated for easy review
and the fourth patch is what actually adds the caching of entries.
These functions are now at a stage that we should be able to start looking at replacing the "single cached direcotry" that is linked to the tcon to be a list of <=n most recent directories we have accessed. Thus expanding it to cache a lot more directories than just the root. But that will come later once this is stabilized and road-tested. At least I think with these all changes it should be fairly easy to expand the use to more directories.
PATCH 1/4 cifs: move definition of cifs_fattr earlier in cifsglob.h
This only moves these definitions to come earlier in the file but not change the definition itself.
This is done to reduce the amount of changes in future patches.
--- fs/cifs/cifsglob.h | 62 +++++++++++++++++++++++----------------------- 1 file changed, 31 insertions(+), 31 deletions(-)
fs/cifs/cifsglob.h +31 -31
@@ -1009,6 +1009,37 @@ cap_unix(struct cifs_ses *ses)return ses->server->vals->cap_unix & ses->capabilities; } +/* +* common struct for holding inode info when searching for or updating an +* inode with new info +*/ + +#define CIFS_FATTR_DFS_REFERRAL 0x1 +#define CIFS_FATTR_DELETE_PENDING 0x2 +#define CIFS_FATTR_NEED_REVAL 0x4 +#define CIFS_FATTR_INO_COLLISION 0x8 +#define CIFS_FATTR_UNKNOWN_NLINK 0x10 +#define CIFS_FATTR_FAKE_ROOT_INO 0x20 + +struct cifs_fattr { +u32 cf_flags; +u32 cf_cifsattrs; +u64 cf_uniqueid; +u64 cf_eof; +u64 cf_bytes; +u64 cf_createtime; +kuid_t cf_uid; +kgid_t cf_gid; +umode_t cf_mode; +dev_t cf_rdev; +unsigned int cf_nlink; +unsigned int cf_dtype; +struct timespec64 cf_atime; +struct timespec64 cf_mtime; +struct timespec64 cf_ctime; +u32 cf_cifstag; +}; +struct cached_fid {bool is_valid:1 ;/* Do we have a useable root fid */ bool file_all_info_is_valid:1 ;
@@ -1641,37 +1672,6 @@ struct file_list {struct cifsFileInfo *cfile; }; -/* -* common struct for holding inode info when searching for or updating an -* inode with new info -*/ - -#define CIFS_FATTR_DFS_REFERRAL 0x1 -#define CIFS_FATTR_DELETE_PENDING 0x2 -#define CIFS_FATTR_NEED_REVAL 0x4 -#define CIFS_FATTR_INO_COLLISION 0x8 -#define CIFS_FATTR_UNKNOWN_NLINK 0x10 -#define CIFS_FATTR_FAKE_ROOT_INO 0x20 - -struct cifs_fattr { -u32 cf_flags; -u32 cf_cifsattrs; -u64 cf_uniqueid; -u64 cf_eof; -u64 cf_bytes; -u64 cf_createtime; -kuid_t cf_uid; -kgid_t cf_gid; -umode_t cf_mode; -dev_t cf_rdev; -unsigned int cf_nlink; -unsigned int cf_dtype; -struct timespec64 cf_atime; -struct timespec64 cf_mtime; -struct timespec64 cf_ctime; -u32 cf_cifstag; -}; -static inline void free_dfs_info_param (struct dfs_info3_param *param) {if (param) {
Signature · 1 line
2.30.2
PATCH 2/4 cifs: check for smb1 in open_cached_dir()
In reply to Ronnie SahlbergCheck protocol version in open_cached_dir() and return not supported for SMB1. This allows us to call open_cached_dir() from code that is common to both smb1 and smb2/3 in future patches without having to do this check in the call-site.
At the same time, add a check if tcon is valid or not for the same reason.
--- fs/cifs/smb2ops.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
fs/cifs/smb2ops.c +2 -1
@@ -776,7 +776,8 @@ int open_cached_dir(unsigned int xid, struct cifs_tcon *tcon,struct cifs_fid *pfid;struct dentry *dentry; -if (tcon->nohandlecache) +if (tcon ==NULL || tcon->nohandlecache || +is_smb1_server (tcon->ses->server))return -ENOTSUPP ;if (cifs_sb->root ==NULL )
Signature · 1 line
2.30.2
PATCH 3/4 cifs: set the CREATE_NOT_FILE when opening the directory in use_cached_dir()
In reply to Ronnie SahlbergThis enforces that we can only do this for directories and not normal files or else the server will return an error.
This means that we will have conditionally check IF the path refers to a directory or not in all the call-sites where we are unsure. Right now this check is for "" i.e. root.
--- fs/cifs/smb2inode.c | 5 ++++- fs/cifs/smb2ops.c | 5 +++-- 2 files changed, 7 insertions(+), 3 deletions(-)
fs/cifs/smb2inode.c +4 -1
@@ -514,8 +514,11 @@ smb2_query_path_info(const unsigned int xid, struct cifs_tcon *tcon,if (smb2_data ==NULL )return -ENOMEM ; +if (strcmp (full_path,"" )) + rc = -ENOENT ; +else + rc =open_cached_dir (xid, tcon, full_path, cifs_sb, &cfid);/* If it is a root and its handle is cached then use it */ - rc =open_cached_dir (xid, tcon, full_path, cifs_sb, &cfid);if (!rc) {if (tcon->crfid.file_all_info_is_valid) { move_smb2_info_to_cifs(data,
fs/cifs/smb2ops.c +3 -2
@@ -825,7 +825,7 @@ int open_cached_dir(unsigned int xid, struct cifs_tcon *tcon, rqst[0 ].rq_nvec =SMB2_CREATE_IOV_SIZE ; oparms.tcon = tcon; - oparms.create_options =cifs_create_options (cifs_sb,0 ); + oparms.create_options =cifs_create_options (cifs_sb,CREATE_NOT_FILE ); oparms.desired_access =FILE_READ_ATTRIBUTES ; oparms.disposition =FILE_OPEN ; oparms.fid = pfid;
@@ -2696,7 +2696,8 @@ smb2_query_info_compound(const unsigned int xid, struct cifs_tcon *tcon, resp_buftype[0 ] = resp_buftype[1 ] = resp_buftype[2 ] =CIFS_NO_BUFFER ;memset (rsp_iov,0 ,sizeof (rsp_iov)); - rc =open_cached_dir (xid, tcon, path, cifs_sb, &cfid); +if (!strcmp (path,"" )) + rc =open_cached_dir (xid, tcon, path, cifs_sb, &cfid);memset (&open_iov,0 ,sizeof (open_iov)); rqst[0 ].rq_iov = open_iov;
Signature · 1 line
2.30.2
Re: [PATCH 3/4] cifs: set the CREATE_NOT_FILE when opening the directory in use_cached_dir()
tentatively merged the first 3 of the series (still looking at patch
4) pending testing
Quoted 58 lines · Ronnie Sahlberg
This enforces that we can only do this for directories and not normal files or else the server will return an error.
This means that we will have conditionally check IF the path refers to a directory or not in all the call-sites where we are unsure. Right now this check is for "" i.e. root.Reviewed-by Paulo Alcantara (SUSE) <[email protected]>Reviewed-by Enzo Matsumiya <[email protected]>Signed-off-by Ronnie Sahlberg <[email protected]>--- fs/cifs/smb2inode.c | 5 ++++- fs/cifs/smb2ops.c | 5 +++-- 2 files changed, 7 insertions(+), 3 deletions(-)fs/cifs/smb2inode.c +4
-1@@ -514,8 +514,11 @@ smb2_query_path_info(const unsigned int xid, struct cifs_tcon *tcon,if (smb2_data ==NULL )return -ENOMEM ; +if (strcmp (full_path,"" )) + rc = -ENOENT ; +else + rc =open_cached_dir (xid, tcon, full_path, cifs_sb, &cfid);/* If it is a root and its handle is cached then use it */ - rc =open_cached_dir (xid, tcon, full_path, cifs_sb, &cfid);if (!rc) {if (tcon->crfid.file_all_info_is_valid) { move_smb2_info_to_cifs(data,fs/cifs/smb2ops.c +3
-3@@ -825,7 +825,7 @@ int open_cached_dir(unsigned int xid, struct cifs_tcon *tcon, rqst[0 ].rq_nvec =SMB2_CREATE_IOV_SIZE ; oparms.tcon = tcon; - oparms.create_options =cifs_create_options (cifs_sb,0 ); + oparms.create_options =cifs_create_options (cifs_sb,CREATE_NOT_FILE ); oparms.desired_access =FILE_READ_ATTRIBUTES ; oparms.disposition =FILE_OPEN ; oparms.fid = pfid;@@ -2696,7 +2696,8 @@ smb2_query_info_compound(const unsigned int xid, struct cifs_tcon *tcon, resp_buftype[0 ] = resp_buftype[1 ] = resp_buftype[2 ] =CIFS_NO_BUFFER ;memset (rsp_iov,0 ,sizeof (rsp_iov)); - rc =open_cached_dir (xid, tcon, path, cifs_sb, &cfid); +if (!strcmp (path,"" )) + rc =open_cached_dir (xid, tcon, path, cifs_sb, &cfid);memset (&open_iov,0 ,sizeof (open_iov)); rqst[0 ].rq_iov = open_iov; --2.30.2
Signature · 3 lines
Thanks, Steve
PATCH 4/4 cifs: cache the dirents for entries in a cached directory
In reply to Ronnie SahlbergThis adds caching of the directory entries for a cached directory while we keep a lease on the directory.
--- fs/cifs/cifsglob.h | 22 ++++++ fs/cifs/misc.c | 2 + fs/cifs/readdir.c | 181 +++++++++++++++++++++++++++++++++++++++++++-- fs/cifs/smb2ops.c | 16 ++++ 4 files changed, 213 insertions(+), 8 deletions(-)
fs/cifs/cifsglob.h +22 -0
@@ -1040,6 +1040,27 @@ struct cifs_fattr {u32 cf_cifstag; }; +struct cached_dirent { +struct list_head entry; +char *name; +int namelen; +loff_t pos; + +struct cifs_fattr fattr; +}; + +struct cached_dirents { +bool is_valid:1 ; +bool is_failed:1 ; +struct dir_context *ctx;/* +* Only used to make sure we only take entries +* from a single context. Never dereferenced. +*/ +struct mutex de_mutex; +int pos;/* Expected ctx->pos */ +struct list_head entries; +}; +struct cached_fid {bool is_valid:1 ;/* Do we have a useable root fid */ bool file_all_info_is_valid:1 ;
@@ -1052,6 +1073,7 @@ struct cached_fid {struct dentry *dentry;struct work_struct lease_break;struct smb2_file_all_info file_all_info; +struct cached_dirents dirents; };/*
fs/cifs/misc.c +2 -0
@@ -114,6 +114,8 @@ tconInfoAlloc(void)kfree (ret_buf);return NULL ; } +INIT_LIST_HEAD (&ret_buf->crfid.dirents.entries); +mutex_init (&ret_buf->crfid.dirents.de_mutex);atomic_inc (&tconInfoAllocCount); ret_buf->status =TID_NEW ;
fs/cifs/readdir.c +173 -8
255 lines not on this page · raw message
fs/cifs/smb2ops.c +16 -0
@@ -699,6 +699,7 @@ smb2_close_cached_fid(struct kref *ref) { struct cached_fid *cfid = container_of(ref, struct cached_fid, refcount); +struct cached_dirent *dirent, *q;if (cfid->is_valid) {cifs_dbg (FYI ,"clear cached root file handle\n" );
@@ -718,6 +719,21 @@ smb2_close_cached_fid(struct kref *ref)dput (cfid->dentry); cfid->dentry =NULL ; } +/* +* Delete all cached dirent names +*/ +mutex_lock (&cfid->dirents.de_mutex); +list_for_each_entry_safe (dirent, q, &cfid->dirents.entries, entry) { +list_del (&dirent->entry); +kfree (dirent->name); +kfree (dirent); + } + cfid->dirents.is_valid =0 ; + cfid->dirents.is_failed =0 ; + cfid->dirents.ctx =NULL ; + cfid->dirents.pos =0 ; +mutex_unlock (&cfid->dirents.de_mutex); + }void close_cached_dir (struct cached_fid *cfid)
Signature · 1 line
2.30.2
Re: [PATCH 4/4] cifs: cache the dirents for entries in a cached directory
Hi Ronnie,
fs/cifs/cifsglob.h struct cifs_fattr { · line 1054
Quoted 3 lines · Ronnie Sahlberg
fs/cifs/cifsglob.h +3
-0+struct cached_dirents { +bool is_valid:1 ; +bool is_failed:1 ;
Just as I commented in the other thread, having both fields above confuses me. Shouldn't using is_valid/!is_valid be enough?
Cheers,
Enzo
Re: [PATCH 4/4] cifs: cache the dirents for entries in a cached directory
Quoted 10 lines · Enzo Matsumiya
Hi Ronnie,
fs/cifs/cifsglob.h struct cifs_fattr { · line 1054
Quoted 3 lines · Ronnie Sahlberg
fs/cifs/cifsglob.h +3
-0+struct cached_dirents { +bool is_valid:1 ; +bool is_failed:1 ;Just as I commented in the other thread, having both fields above confuses me. Shouldn't using is_valid/!is_valid be enough?
We need two because semantically there are three states we need to describe:
1, we have successfully cached all entries and cache is valid
2, we are in the process of building the cache, and the cache might eventually become complete and valid
3, we are in the process of building the cache, but there has been a failure and the cache will never be complete or valid
Quoted 6 lines
Cheers,
Enzo
Re: [PATCH 4/4] cifs: cache the dirents for entries in a cached directory
tentatively merged this patch (and the 3 earlier ones are already in for-next) into cifs-2.6.git for-next pending testing.
Did minor whitespace cleanup
Quoted 27 lines · Leif Sahlberg
Quoted 10 lines · Enzo Matsumiya
Hi Ronnie,
fs/cifs/cifsglob.h struct cifs_fattr { · line 1054
Quoted 3 lines · Ronnie Sahlberg
fs/cifs/cifsglob.h +3
-0+struct cached_dirents { +bool is_valid:1 ; +bool is_failed:1 ;Just as I commented in the other thread, having both fields above confuses me. Shouldn't using is_valid/!is_valid be enough?
We need two because semantically there are three states we need to describe:
1, we have successfully cached all entries and cache is valid
2, we are in the process of building the cache, and the cache might eventually become complete and valid
3, we are in the process of building the cache, but there has been a failure and the cache will never be complete or validQuoted 6 lines
Cheers,
Enzo
Signature · 3 lines
Thanks, Steve