cifs: ignore cached share root handle closing errors
Subsystems (2)
PATCH 1/1 cifs: ignore cached share root handle closing errors
Fix tcon use-after-free and NULL ptr deref.
Customer system crashes with the following kernel log:
[462233.169868] CIFS VFS: Cancelling wait for mid 4894753 cmd: 14 => a QUERY DIR
[462233.228045] CIFS VFS: cifs_put_smb_ses: Session Logoff failure rc=-4
[462233.305922] CIFS VFS: cifs_put_smb_ses: Session Logoff failure rc=-4
[462233.306205] CIFS VFS: cifs_put_smb_ses: Session Logoff failure rc=-4
[462233.347060] CIFS VFS: cifs_put_smb_ses: Session Logoff failure rc=-4
[462233.347107] CIFS VFS: Close unmatched open
[462233.347113] BUG: unable to handle kernel NULL pointer dereference at 0000000000000038
...
[exception RIP: cifs_put_tcon+0xa0] (this is doing tcon->ses->server) #6 [...] smb2_cancelled_close_fid at ... [cifs]
#7 [...] process_one_work at ...
#8 [...] worker_thread at ...
#9 [...] kthread at ...
The most likely explanation we have is:
* When we put the last reference of a tcon (refcount=0), we close the cached share root handle.
* If closing a handle is interupted, SMB2_close() will queue a SMB2_close() in a work thread.
* The queued object keeps a tcon ref so we bump the tcon refcount, jumping from 0 to 1.
* We reach the end of cifs_put_tcon(), we free the tcon object despite it now having a refcount of 1.
* The queued work now runs, but the tcon, ses & server was freed in the meantime resulting in a crash.
THREAD 1
========cifs_put_tcon => tcon refcount reach 0
SMB2_tdis
close_shroot_lease
close_shroot_lease_locked => if cached root has lease && refcount reach 0
smb2_close_cached_fid => if cached root valid
SMB2_close => retry close in a worker thread if interrupted
smb2_handle_cancelled_close
__smb2_handle_cancelled_close => !! tcon refcount bump 0 => 1 !!
INIT_WORK(&cancelled->work, smb2_cancelled_close_fid); queue_work(cifsiod_wq, &cancelled->work) => queue work
tconInfoFree(tcon); ==> freed!
cifs_put_smb_ses(ses); ==> freed!
THREAD 2 (workqueue)
========smb2_cancelled_close_fid
SMB2_close(0, cancelled->tcon, ...); => use-after-free of tcon
cifs_put_tcon(cancelled->tcon); => tcon refcount reach 0 second time
*CRASH*
--- fs/cifs/smb2ops.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
fs/cifs/smb2ops.c +3 -2
@@ -607,8 +607,9 @@ smb2_close_cached_fid(struct kref *ref)if (cfid->is_valid) {cifs_dbg (FYI ,"clear cached root file handle\n" ); -SMB2_close (0 , cfid->tcon, cfid->fid->persistent_fid, - cfid->fid->volatile_fid); +/* ignore errors here & do not retry in worker thread */ +SMB2_close_flags (0 , cfid->tcon, cfid->fid->persistent_fid, + cfid->fid->volatile_fid,0 ); cfid->is_valid = false; cfid->file_all_info_is_valid = false; cfid->has_lease = false;
Signature · 1 line
2.16.4
kbuild test robot
build botThank you for the patch! Yet something to improve:
Hi Aurelien,
Thank you for the patch! Yet something to improve:
[auto build test ERROR on cifs/for-next]
[also build test ERROR on v5.6 next-20200331]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system. BTW, we also suggest to use '--base' option to specify the base tree in git format-patch, please see https://stackoverflow.com/a/37406982]
url: https://github.com/0day-ci/linux/commits/Aurelien-Aptel/cifs-ignore-cached-share-root-handle-closing-errors/20200401-025615 base: git://git.samba.org/sfrench/cifs-2.6.git for-next config: x86_64-randconfig-s1-20200401 (attached as .config) compiler: gcc-6 (Debian 6.3.0-18+deb9u1) 6.3.0 20170516 reproduce: # save the attached .config to linux build tree make ARCH=x86_64
If you fix the issue, kindly add following tag
All errors (new ones prefixed by >>):
fs/cifs/smb2ops.c: In function 'smb2_close_cached_fid':
>> fs/cifs/smb2ops.c:611:3: error: implicit declaration of function 'SMB2_close_flags' [-Werror=implicit-function-declaration]
SMB2_close_flags(0, cfid->tcon, cfid->fid->persistent_fid,
^~~~~~~~~~~~~~~~cc1: some warnings being treated as errors
vim +/SMB2_close_flags +611 fs/cifs/smb2ops.c
601
602 static void
603 smb2_close_cached_fid(struct kref *ref)
604 {
605 struct cached_fid *cfid = container_of(ref, struct cached_fid,
606 refcount);
607
608 if (cfid->is_valid) {
609 cifs_dbg(FYI, "clear cached root file handle\n");
610 /* ignore errors here & do not retry in worker thread */
> 611 SMB2_close_flags(0, cfid->tcon, cfid->fid->persistent_fid,
612 cfid->fid->volatile_fid, 0);
613 cfid->is_valid = false;
614 cfid->file_all_info_is_valid = false;
615 cfid->has_lease = false;
616 }
617 }
618
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org
kbuild test robot
build botThank you for the patch! Yet something to improve:
Hi Aurelien,
Thank you for the patch! Yet something to improve:
[auto build test ERROR on cifs/for-next]
[also build test ERROR on v5.6 next-20200331]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system. BTW, we also suggest to use '--base' option to specify the base tree in git format-patch, please see https://stackoverflow.com/a/37406982]
url: https://github.com/0day-ci/linux/commits/Aurelien-Aptel/cifs-ignore-cached-share-root-handle-closing-errors/20200401-025615 base: git://git.samba.org/sfrench/cifs-2.6.git for-next config: s390-randconfig-a001-20200331 (attached as .config) compiler: s390-linux-gcc (GCC) 9.3.0
reproduce:
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# save the attached .config to linux build tree
GCC_VERSION=9.3.0 make.cross ARCH=s390
If you fix the issue, kindly add following tag
All errors (new ones prefixed by >>):
fs/cifs/smb2ops.c: In function 'smb2_close_cached_fid':
>> fs/cifs/smb2ops.c:611:3: error: implicit declaration of function 'SMB2_close_flags'; did you mean 'SMB2_close_free'? [-Werror=implicit-function-declaration]
611 | SMB2_close_flags(0, cfid->tcon, cfid->fid->persistent_fid,
| ^~~~~~~~~~~~~~~~
| SMB2_close_freecc1: some warnings being treated as errors
vim +611 fs/cifs/smb2ops.c
601
602 static void
603 smb2_close_cached_fid(struct kref *ref)
604 {
605 struct cached_fid *cfid = container_of(ref, struct cached_fid,
606 refcount);
607
608 if (cfid->is_valid) {
609 cifs_dbg(FYI, "clear cached root file handle\n");
610 /* ignore errors here & do not retry in worker thread */
> 611 SMB2_close_flags(0, cfid->tcon, cfid->fid->persistent_fid,
612 cfid->fid->volatile_fid, 0);
613 cfid->is_valid = false;
614 cfid->file_all_info_is_valid = false;
615 cfid->has_lease = false;
616 }
617 }
618
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org