Skip to contentSkip to conversation

NFS: Fix fscache super_cookie index_key from changing after umount

PATCH In review Dave Wysochanski 2 participants 5 messages Apr 15, 2020 – Apr 16, 2020

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/3 NFS: Fix fscache super_cookie index_key from changing after umount

Commit 402cb8dda949 ("fscache: Attach the index key and aux data to the cookie") added the index_key and index_key_len parameters to fscache_acquire_cookie(), and updated the callers in the NFS client. One of the callers was inside nfs_fscache_get_super_cookie() and was changed to use the full struct nfs_fscache_key as the index_key. However, a couple members of this structure contain pointers and thus will change each time the same NFS share is remounted. Since index_key is used for fscache_cookie->key_hash and this subsequently is used to compare cookies, the effectiveness of fscache with NFS is reduced to the point at which a umount
occurs. Any subsequent remount of the same share will cause a
unique NFS super_block index_key and key_hash to be generated for the same data, rendering any prior fscache data unable to be found. A simple reproducer demonstrates the problem.

1. Mount share with 'fsc', create a file, drop page cache systemctl start cachefilesd
mount -o vers=3,fsc 127.0.0.1:/export /mnt
dd if=/dev/zero of=/mnt/file1.bin bs=4096 count=1 echo 3 > /proc/sys/vm/drop_caches

2. Read file into page cache and fscache, then unmount dd if=/mnt/file1.bin of=/dev/null bs=4096 count=1 umount /mnt

3. Remount and re-read which should come from fscache mount -o vers=3,fsc 127.0.0.1:/export /mnt echo 3 > /proc/sys/vm/drop_caches
dd if=/mnt/file1.bin of=/dev/null bs=4096 count=1

4. Check for READ ops in mountstats - there should be none grep READ: /proc/self/mountstats

Looking at the history and the removed function, nfs_super_get_key(), we should only use nfs_fscache_key.key plus any uniquifier, for the fscache index_key.

Fixes 402cb8dda949 ("fscache: Attach the index key and aux data to the cookie")
---
 fs/nfs/fscache.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
fs/nfs/fscache.c +2 -1
index 1abf126c2df4..8eff1fd806b1 100644
@@ -188,7 +188,8 @@ void nfs_fscache_get_super_cookie(struct super_block *sb, const char *uniq, int
 	/* create a cache index for looking up filehandles */
 	nfss->fscache = fscache_acquire_cookie(nfss->nfs_client->fscache,
 					       &nfs_fscache_super_index_def,
-					       key, sizeof(*key) + ulen,
+					       &key->key,
+					       sizeof(key->key) + ulen,
 					       NULL, 0,
 					       nfss, 0, true);
 	dfprintk(FSCACHE, "NFS: get superblock cookie (0x%p/0x%p)\n",
Signature · 1 line
1.8.3.1
#1 lore raw

PATCH 2/3 NFS: Fix fscache super_cookie allocation

Commit f2aedb713c28 ("NFS: Add fs_context support.") reworked NFS mount code paths for fs_context support which included super_block initialization. In the process there was an extra return left in the code and so we never call nfs_fscache_get_super_cookie even if 'fsc' is given on as mount option. In addition, there is an extra check inside nfs_fscache_get_super_cookie for the NFS_OPTION_FSCACHE which is unnecessary since the only caller nfs_get_cache_cookie checks this flag.

Fixes f2aedb713c28 ("NFS: Add fs_context support.")
---
 fs/nfs/fscache.c | 2 --
 fs/nfs/super.c   | 1 -
 2 files changed, 3 deletions(-)
fs/nfs/fscache.c +0 -2
index 8eff1fd806b1..f51718415606 100644
@@ -118,8 +118,6 @@ void nfs_fscache_get_super_cookie(struct super_block *sb, const char *uniq, int
 
 	nfss->fscache_key = NULL;
 	nfss->fscache = NULL;
-	if (!(nfss->options & NFS_OPTION_FSCACHE))
-		return;
 	if (!uniq) {
 		uniq = "";
 		ulen = 1;
fs/nfs/super.c +0 -1
index 59ef3b13ccca..cc34aa3a8ba4 100644
@@ -1189,7 +1189,6 @@ static void nfs_get_cache_cookie(struct super_block *sb,
 			uniq = ctx->fscache_uniq;
 			ulen = strlen(ctx->fscache_uniq);
 		}
-		return;
 	}
 
 	nfs_fscache_get_super_cookie(sb, uniq, ulen);
Signature · 1 line
1.8.3.1
#2 lore raw

PATCH 3/3 NFSv4: Fix fscache cookie aux_data to ensure change_attr is included

In reply to Dave Wysochanski

Commit 402cb8dda949 ("fscache: Attach the index key and aux data to the cookie") added the aux_data and aux_data_len to parameters to fscache_acquire_cookie(), and updated the callers in the NFS client. In the process it modified the aux_data to include the change_attr, but missed adding change_attr to a couple places where aux_data was used. Specifically, when opening a file and the change_attr is not added, the following attempt to lookup an object will fail inside cachefiles_check_object_xattr() = -116 due to nfs_fscache_inode_check_aux() failing memcmp on auxdata and returning FSCACHE_CHECKAUX_OBSOLETE.

Fix this by adding nfs_fscache_update_auxdata() to set the auxdata from all relevant fields in the inode, including the change_attr.

Fixes 402cb8dda949 ("fscache: Attach the index key and aux data to the cookie")
---
 fs/nfs/fscache.c | 34 ++++++++++++++++------------------
 1 file changed, 16 insertions(+), 18 deletions(-)
fs/nfs/fscache.c +16 -18
index f51718415606..e3240f9f9570 100644
@@ -225,6 +225,19 @@ void nfs_fscache_release_super_cookie(struct super_block *sb)
 	}
 }
 
+static nfs_fscache_update_auxdata(struct nfs_fscache_inode_auxdata *auxdata,
+				  struct nfs_inode *nfsi)
+{
+	memset(auxdata, 0, sizeof(*auxdata));
+	auxdata->mtime_sec  = nfsi->vfs_inode.i_mtime.tv_sec;
+	auxdata->mtime_nsec = nfsi->vfs_inode.i_mtime.tv_nsec;
+	auxdata->ctime_sec  = nfsi->vfs_inode.i_ctime.tv_sec;
+	auxdata->ctime_nsec = nfsi->vfs_inode.i_ctime.tv_nsec;
+
+	if (NFS_SERVER(&nfsi->vfs_inode)->nfs_client->rpc_ops->version == 4)
+		auxdata->change_attr = inode_peek_iversion_raw(&nfsi->vfs_inode);
+}
+
 /*
  * Initialise the per-inode cache cookie pointer for an NFS inode.
  */
@@ -238,14 +251,7 @@ void nfs_fscache_init_inode(struct inode *inode)
 	if (!(nfss->fscache && S_ISREG(inode->i_mode)))
 		return;
 
-	memset(&auxdata, 0, sizeof(auxdata));
-	auxdata.mtime_sec  = nfsi->vfs_inode.i_mtime.tv_sec;
-	auxdata.mtime_nsec = nfsi->vfs_inode.i_mtime.tv_nsec;
-	auxdata.ctime_sec  = nfsi->vfs_inode.i_ctime.tv_sec;
-	auxdata.ctime_nsec = nfsi->vfs_inode.i_ctime.tv_nsec;
-
-	if (NFS_SERVER(&nfsi->vfs_inode)->nfs_client->rpc_ops->version == 4)
-		auxdata.change_attr = inode_peek_iversion_raw(&nfsi->vfs_inode);
+	nfs_fscache_update_auxdata(&auxdata, nfsi);
 
 	nfsi->fscache = fscache_acquire_cookie(NFS_SB(inode->i_sb)->fscache,
 					       &nfs_fscache_inode_object_def,
@@ -265,11 +271,7 @@ void nfs_fscache_clear_inode(struct inode *inode)
 
 	dfprintk(FSCACHE, "NFS: clear cookie (0x%p/0x%p)\n", nfsi, cookie);
 
-	memset(&auxdata, 0, sizeof(auxdata));
-	auxdata.mtime_sec  = nfsi->vfs_inode.i_mtime.tv_sec;
-	auxdata.mtime_nsec = nfsi->vfs_inode.i_mtime.tv_nsec;
-	auxdata.ctime_sec  = nfsi->vfs_inode.i_ctime.tv_sec;
-	auxdata.ctime_nsec = nfsi->vfs_inode.i_ctime.tv_nsec;
+	nfs_fscache_update_auxdata(&auxdata, nfsi);
 	fscache_relinquish_cookie(cookie, &auxdata, false);
 	nfsi->fscache = NULL;
 }
@@ -309,11 +311,7 @@ void nfs_fscache_open_file(struct inode *inode, struct file *filp)
 	if (!fscache_cookie_valid(cookie))
 		return;
 
-	memset(&auxdata, 0, sizeof(auxdata));
-	auxdata.mtime_sec  = nfsi->vfs_inode.i_mtime.tv_sec;
-	auxdata.mtime_nsec = nfsi->vfs_inode.i_mtime.tv_nsec;
-	auxdata.ctime_sec  = nfsi->vfs_inode.i_ctime.tv_sec;
-	auxdata.ctime_nsec = nfsi->vfs_inode.i_ctime.tv_nsec;
+	nfs_fscache_update_auxdata(&auxdata, nfsi);
 
 	if (inode_is_open_for_write(inode)) {
 		dfprintk(FSCACHE, "NFS: nfsi 0x%p disabling cache\n", nfsi);
Signature · 1 line
1.8.3.1
#3 lore raw

Re: [PATCH 3/3] NFSv4: Fix fscache cookie aux_data to ensure change_attr is included

Thank you for the patch! Perhaps something to improve:

Hi Dave,

Thank you for the patch! Perhaps something to improve:

[auto build test WARNING on nfs/linux-next]
[also build test WARNING on v5.7-rc1 next-20200415]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system. BTW, we also suggest to use '--base' option to specify the base tree in git format-patch, please see https://stackoverflow.com/a/37406982]

url:    https://github.com/0day-ci/linux/commits/Dave-Wysochanski/NFS-Fix-fscache-super_cookie-index_key-from-changing-after-umount/20200416-070856
base:   git://git.linux-nfs.org/projects/trondmy/linux-nfs.git linux-next
config: c6x-allyesconfig (attached as .config)
compiler: c6x-elf-gcc (GCC) 9.3.0

reproduce:
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day GCC_VERSION=9.3.0 make.cross ARCH=c6x

If you fix the issue, kindly add following tag as appropriate

All warnings (new ones prefixed by >>):

>> fs/nfs/fscache.c:228:8: warning: return type defaults to 'int' [-Wreturn-type]
     228 | static nfs_fscache_update_auxdata(struct nfs_fscache_inode_auxdata *auxdata,
         |        ^~~~~~~~~~~~~~~~~~~~~~~~~~
   fs/nfs/fscache.c: In function 'nfs_fscache_update_auxdata':
>> fs/nfs/fscache.c:239:1: warning: control reaches end of non-void function [-Wreturn-type]
     239 | }
         | ^

vim +/int +228 fs/nfs/fscache.c

227

 > 228	static nfs_fscache_update_auxdata(struct nfs_fscache_inode_auxdata *auxdata,
   229					  struct nfs_inode *nfsi)
   230	{
   231		memset(auxdata, 0, sizeof(*auxdata));
   232		auxdata->mtime_sec  = nfsi->vfs_inode.i_mtime.tv_sec;
   233		auxdata->mtime_nsec = nfsi->vfs_inode.i_mtime.tv_nsec;
   234		auxdata->ctime_sec  = nfsi->vfs_inode.i_ctime.tv_sec;
   235		auxdata->ctime_nsec = nfsi->vfs_inode.i_ctime.tv_nsec;
   236	
   237		if (NFS_SERVER(&nfsi->vfs_inode)->nfs_client->rpc_ops->version == 4)
   238			auxdata->change_attr = inode_peek_iversion_raw(&nfsi->vfs_inode);
 > 239	}

240

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org

#4 lore raw

Re: [PATCH 3/3] NFSv4: Fix fscache cookie aux_data to ensure change_attr is included

Apologies for somehow missing this, resent v2 of 3/3 with 'void' added to prototype.

Should be
static void nfs_fscache_update_auxdata(struct
nfs_fscache_inode_auxdata *auxdata,

Quoted 54 lines · kbuild test robot

Hi Dave,

Thank you for the patch! Perhaps something to improve:

[auto build test WARNING on nfs/linux-next]
[also build test WARNING on v5.7-rc1 next-20200415]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system. BTW, we also suggest to use '--base' option to specify the base tree in git format-patch, please see https://stackoverflow.com/a/37406982]

url:    https://github.com/0day-ci/linux/commits/Dave-Wysochanski/NFS-Fix-fscache-super_cookie-index_key-from-changing-after-umount/20200416-070856
base:   git://git.linux-nfs.org/projects/trondmy/linux-nfs.git linux-next
config: c6x-allyesconfig (attached as .config)
compiler: c6x-elf-gcc (GCC) 9.3.0

reproduce:
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day GCC_VERSION=9.3.0 make.cross ARCH=c6x

If you fix the issue, kindly add following tag as appropriate

All warnings (new ones prefixed by >>):

Quoted 1 line
Quoted 1 line

fs/nfs/fscache.c:228:8: warning: return type defaults to 'int' [-Wreturn-type]

228 | static nfs_fscache_update_auxdata(struct nfs_fscache_inode_auxdata *auxdata,
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
fs/nfs/fscache.c: In function 'nfs_fscache_update_auxdata':

Quoted 1 line
Quoted 1 line

fs/nfs/fscache.c:239:1: warning: control reaches end of non-void function [-Wreturn-type]

     239 | }
         | ^

vim +/int +228 fs/nfs/fscache.c

   227
Quoted 1 line

228 static nfs_fscache_update_auxdata(struct nfs_fscache_inode_auxdata *auxdata,

229 struct nfs_inode *nfsi)

   230  {
   231          memset(auxdata, 0, sizeof(*auxdata));
   232          auxdata->mtime_sec  = nfsi->vfs_inode.i_mtime.tv_sec;
   233          auxdata->mtime_nsec = nfsi->vfs_inode.i_mtime.tv_nsec;
   234          auxdata->ctime_sec  = nfsi->vfs_inode.i_ctime.tv_sec;
   235          auxdata->ctime_nsec = nfsi->vfs_inode.i_ctime.tv_nsec;
   236
   237          if (NFS_SERVER(&nfsi->vfs_inode)->nfs_client->rpc_ops->version == 4)
   238                  auxdata->change_attr = inode_peek_iversion_raw(&nfsi->vfs_inode);
Quoted 1 line

239 }

240

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org