Special files broken against Samba master
Subsystems (2)
Hi!
Against Samba master special files are broken with posix extensions.
On the Samba server:
root@smb3-posix:/home/samba# ls -li /srv/samba/test/posix/ insgesamt 20 131958 brw-r--r-- 1 root root 0, 0 15. Nov 12:04 blockdev 131965 crw-r--r-- 1 root root 1, 1 15. Nov 12:04 chardev 131966 prw-r--r-- 1 samba samba 0 15. Nov 12:05 fif 131953 -rw-rwxrw-+ 2 samba samba 4 18. Nov 11:37 file 131953 -rw-rwxrw-+ 2 samba samba 4 18. Nov 11:37 hardlink 131957 lrwxrwxrwx 1 samba samba 4 15. Nov 12:03 symlink -> file 131954 -rwxrwxr-x+ 1 samba samba 0 18. Nov 15:28 symlinkoversmb
# mount.cifs //127.0.0.1/posix /mnt/smb3unix/ -o username=samba,password=x,posix
# ls -li /mnt/smb3unix/posix/
ls: cannot access '/mnt/smb3unix/posix/blockdev': No data available
ls: cannot access '/mnt/smb3unix/posix/chardev': No data available
ls: cannot access '/mnt/smb3unix/posix/symlinkoversmb': No data available
ls: cannot access '/mnt/smb3unix/posix/fifo': No data available
ls: cannot access '/mnt/smb3unix/posix/symlink': No data available
total 16
? -????????? ? ? ? ? ? blockdev
? -????????? ? ? ? ? ? chardev
? -????????? ? ? ? ? ? fifo
131953 -rw-rwxrw- 2 root samba 4 Nov 18 11:37 file
131953 -rw-rwxrw- 2 root samba 4 Nov 18 11:37 hardlink
? -????????? ? ? ? ? ? symlink
? -????????? ? ? ? ? ? symlinkoversmbI have two WIP patches that fix this as a side effect of adding support to using the new POSIX type as discussed at SDC, patches attached:
root@smb3-posix:/home/samba# ls -li /mnt/smb3unix/posix/ insgesamt 21 131958 brw-r--r-- 1 root root 0, 0 15. Nov 12:04 blockdev 131965 crw-r--r-- 1 root root 1, 1 15. Nov 12:04 chardev 131966 prw-r--r-- 1 root samba 0 15. Nov 12:05 fifo 131953 -rw-rwxrw- 2 root samba 4 18. Nov 11:37 file 131953 -rw-rwxrw- 2 root samba 4 18. Nov 11:37 hardlink 131957 lrwxrwxrwx 1 root samba 4 15. Nov 12:03 symlink -> file 131954 lrwxrwxr-x 1 root samba 23 18. Nov 15:28 symlinkoversmb -> mnt/smb3unix/posix/file
I'm also optimizing away unneeded reparse point queries (create+fsctl+close) for nfs reparse points where we only need the file type which we now get and use via the smb3 posix type.
I'm sure I'm doing things wrong. Can you help me getting this across the line?
-slow
[1] <https://www.samba.org/~slow/SMB3_POSIX/fscc_posix_extensions.html#posix-file-type-definition>
From a57c32da874f285af266b7fbbaefb7940991d049 Mon Sep 17 00:00:00 2001
From: Ralph Boehme <[email protected]>
Date: Fri, 15 Nov 2024 13:15:50 +0100
Subject: [PATCH 1/3] fs/smb/client: avoid querying SMB2_OP_QUERY_WSL_EA for
SMB3 POSIX
--- fs/smb/client/smb2inode.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
fs/smb/client/smb2inode.c +2 -1
@@ -941,7 +941,8 @@ int smb2_query_path_info(const unsigned int xid,if (rc || !data->reparse_point) goto out; - cmds[num_cmds++] =SMB2_OP_QUERY_WSL_EA ; +if (!tcon->posix_extensions) + cmds[num_cmds++] =SMB2_OP_QUERY_WSL_EA ;/* * Skip SMB2_OP_GET_REPARSE if symlink already parsed in create * response.
Signature · 53 lines
2.47.0 From afd2dce84b1ae970b2ec4421948194746f542cf6 Mon Sep 17 00:00:00 2001 From: Ralph Boehme <[email protected]> Date: Fri, 15 Nov 2024 19:21:04 +0100 Subject: [PATCH 2/3] fs/smb/client: Implement new SMB3 POSIX type Fixes special files against current Samba. On the Samba server: insgesamt 20 131958 brw-r--r-- 1 root root 0, 0 15. Nov 12:04 blockdev 131965 crw-r--r-- 1 root root 1, 1 15. Nov 12:04 chardev 131966 prw-r--r-- 1 samba samba 0 15. Nov 12:05 fifo 131953 -rw-rwxrw-+ 2 samba samba 4 18. Nov 11:37 file 131953 -rw-rwxrw-+ 2 samba samba 4 18. Nov 11:37 hardlink 131957 lrwxrwxrwx 1 samba samba 4 15. Nov 12:03 symlink -> file 131954 -rwxrwxr-x+ 1 samba samba 0 18. Nov 15:28 symlinkoversmb Before: ls: cannot access '/mnt/smb3unix/posix/blockdev': No data available ls: cannot access '/mnt/smb3unix/posix/chardev': No data available ls: cannot access '/mnt/smb3unix/posix/symlinkoversmb': No data available ls: cannot access '/mnt/smb3unix/posix/fifo': No data available ls: cannot access '/mnt/smb3unix/posix/symlink': No data available total 16 ? -????????? ? ? ? ? ? blockdev ? -????????? ? ? ? ? ? chardev ? -????????? ? ? ? ? ? fifo 131953 -rw-rwxrw- 2 root samba 4 Nov 18 11:37 file 131953 -rw-rwxrw- 2 root samba 4 Nov 18 11:37 hardlink ? -????????? ? ? ? ? ? symlink ? -????????? ? ? ? ? ? symlinkoversmb After: insgesamt 21 131958 brw-r--r-- 1 root root 0, 0 15. Nov 12:04 blockdev 131965 crw-r--r-- 1 root root 1, 1 15. Nov 12:04 chardev 131966 prw-r--r-- 1 root samba 0 15. Nov 12:05 fifo 131953 -rw-rwxrw- 2 root samba 4 18. Nov 11:37 file 131953 -rw-rwxrw- 2 root samba 4 18. Nov 11:37 hardlink 131957 lrwxrwxrwx 1 root samba 4 15. Nov 12:03 symlink -> file 131954 lrwxrwxr-x 1 root samba 23 18. Nov 15:28 symlinkoversmb -> mnt/smb3unix/posix/file --- fs/smb/client/cifsproto.h | 1 + fs/smb/client/inode.c | 89 +++++++++++++++++++++++++++++++++++---- fs/smb/client/readdir.c | 35 +++++++-------- fs/smb/client/reparse.c | 84 ++++++++++++++++++++++-------------- 4 files changed, 149 insertions(+), 60 deletions(-)
fs/smb/client/cifsproto.h +1 -0
@@ -676,6 +676,7 @@ int __cifs_sfu_make_node(unsigned int xid, struct inode *inode,int cifs_sfu_make_node (unsigned int xid,struct inode *inode,struct dentry *dentry,struct cifs_tcon *tcon,const char *full_path,umode_t mode,dev_t dev); +umode_t wire_mode_to_posix (u32 wire);#ifdef CONFIG_CIFS_DFS_UPCALL static inline int get_dfs_path(const unsigned int xid, struct cifs_ses *ses,
fs/smb/client/inode.c +80 -9
106 lines not on this page · raw message
fs/smb/client/readdir.c +16 -19
@@ -241,31 +241,28 @@ cifs_posix_to_fattr(struct cifs_fattr *fattr, struct smb2_posix_info *info, fattr->cf_nlink =le32_to_cpu (info->HardLinks); fattr->cf_cifsattrs =le32_to_cpu (info->DosAttributes); -/* -* Since we set the inode type below we need to mask off -* to avoid strange results if bits set above. -* XXX: why not make server&client use the type bits? -*/ - fattr->cf_mode =le32_to_cpu (info->Mode) & ~S_IFMT ; +if (fattr->cf_cifsattrs &ATTR_REPARSE ) { + fattr->cf_cifstag =le32_to_cpu (info->ReparseTag); + } + + fattr->cf_mode =wire_mode_to_posix (le32_to_cpu (info->Mode)); + fattr->cf_dtype =S_DT (le32_to_cpu (info->Mode)); + +switch (fattr->cf_mode &S_IFMT ) { +case S_IFLNK : +case S_IFBLK : +case S_IFCHR : + fattr->cf_flags |=CIFS_FATTR_NEED_REVAL ; +break ; +default : +break ; + }cifs_dbg (FYI ,"posix fattr: dev %d, reparse %d, mode %o\n" ,le32_to_cpu (info->DeviceId),le32_to_cpu (info->ReparseTag),le32_to_cpu (info->Mode)); -if (fattr->cf_cifsattrs &ATTR_DIRECTORY ) { - fattr->cf_mode |=S_IFDIR ; - fattr->cf_dtype =DT_DIR ; - }else { -/* -* mark anything that is not a dir as regular -* file. special files should have the REPARSE -* attribute and will be marked as needing revaluation -*/ - fattr->cf_mode |=S_IFREG ; - fattr->cf_dtype =DT_REG ; - } -sid_to_id (cifs_sb, &parsed.owner, fattr,SIDOWNER );sid_to_id (cifs_sb, &parsed.group, fattr,SIDGROUP ); }
fs/smb/client/reparse.c +52 -32
111 lines not on this page · raw message
Signature · 12 lines
2.47.0 From 436e2dd352980f0be7f43e02c8c24d558a7d54bc Mon Sep 17 00:00:00 2001 From: Ralph Boehme <[email protected]> Date: Mon, 25 Nov 2024 16:19:56 +0100 Subject: [PATCH 3/3] fs/smb/client: cifs_prime_dcache() for SMB3 POSIX reparse points --- fs/smb/client/readdir.c | 18 +++++++++++++++++- 1 file changed, 17 insertions(+), 1 deletion(-)
fs/smb/client/readdir.c +17 -1
@@ -71,6 +71,8 @@ cifs_prime_dcache(struct dentry *parent, struct qstr *name,struct inode *inode;struct super_block *sb = parent->d_sb;struct cifs_sb_info *cifs_sb =CIFS_SB (sb); +bool posix =cifs_sb_master_tcon (cifs_sb)->posix_extensions; +bool reparse_need_reval = false;DECLARE_WAIT_QUEUE_HEAD_ONSTACK (wq);int rc;
@@ -85,7 +87,21 @@ cifs_prime_dcache(struct dentry *parent, struct qstr *name,* this spares us an invalidation. */ retry: -if ((fattr->cf_cifsattrs &ATTR_REPARSE ) || +if (posix) { +switch (fattr->cf_mode &S_IFMT ) { +case S_IFLNK : +case S_IFBLK : +case S_IFCHR : + reparse_need_reval = true; +break ; +default : +break ; + } + }else if (fattr->cf_cifsattrs &ATTR_REPARSE ) { + reparse_need_reval = true; + } + +if (reparse_need_reval || (fattr->cf_flags &CIFS_FATTR_NEED_REVAL ))return ;
Signature · 1 line
2.47.0
Quoted 3 lines · Ralph Boehme
I'm also optimizing away unneeded reparse point queries (create+fsctl+close) for nfs reparse points where we only need the file type which we now get and use via the smb3 posix type.
Awesome! Yeah, we don't need to revalidate the special files with SMB3.1.1 POSIX as we can already parse file type directly from query dir response. We could that in separate patches, so no worries. This will definitely help with getdents(2).
Quoted 2 lines
I'm sure I'm doing things wrong. Can you help me getting this across the line?
Patches look good, thanks. It would be great making sure it didn't regress against NFS server on Windows, but Steve can do that.
Thanks Ralph!
Quoted 6 lines · Paulo Alcantara
Quoted 2 lines · Ralph Boehme
I'm sure I'm doing things wrong. Can you help me getting this across the line?
Patches look good, thanks. It would be great making sure it didn't regress against NFS server on Windows, but Steve can do that.
great, thanks for taking a look! I was expecting I was likely doing some stupid beginner mistakes, but if you think the changes look good, here's a v2 with just my +1 added, without code changes.
Thanks!
-slow
From d73e1672f001b532af18d92ce361f535b5c3e91d Mon Sep 17 00:00:00 2001
From: Ralph Boehme <[email protected]>
Date: Fri, 15 Nov 2024 13:15:50 +0100
Subject: [PATCH 1/3] fs/smb/client: avoid querying SMB2_OP_QUERY_WSL_EA for
SMB3 POSIX
--- fs/smb/client/smb2inode.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
fs/smb/client/smb2inode.c +2 -1
@@ -941,7 +941,8 @@ int smb2_query_path_info(const unsigned int xid,if (rc || !data->reparse_point) goto out; - cmds[num_cmds++] =SMB2_OP_QUERY_WSL_EA ; +if (!tcon->posix_extensions) + cmds[num_cmds++] =SMB2_OP_QUERY_WSL_EA ;/* * Skip SMB2_OP_GET_REPARSE if symlink already parsed in create * response.
Signature · 55 lines
2.47.0 From 371b76b710b14898890253dfbcf405a1f9fdecd8 Mon Sep 17 00:00:00 2001 From: Ralph Boehme <[email protected]> Date: Fri, 15 Nov 2024 19:21:04 +0100 Subject: [PATCH 2/3] fs/smb/client: Implement new SMB3 POSIX type Fixes special files against current Samba. On the Samba server: insgesamt 20 131958 brw-r--r-- 1 root root 0, 0 15. Nov 12:04 blockdev 131965 crw-r--r-- 1 root root 1, 1 15. Nov 12:04 chardev 131966 prw-r--r-- 1 samba samba 0 15. Nov 12:05 fifo 131953 -rw-rwxrw-+ 2 samba samba 4 18. Nov 11:37 file 131953 -rw-rwxrw-+ 2 samba samba 4 18. Nov 11:37 hardlink 131957 lrwxrwxrwx 1 samba samba 4 15. Nov 12:03 symlink -> file 131954 -rwxrwxr-x+ 1 samba samba 0 18. Nov 15:28 symlinkoversmb Before: ls: cannot access '/mnt/smb3unix/posix/blockdev': No data available ls: cannot access '/mnt/smb3unix/posix/chardev': No data available ls: cannot access '/mnt/smb3unix/posix/symlinkoversmb': No data available ls: cannot access '/mnt/smb3unix/posix/fifo': No data available ls: cannot access '/mnt/smb3unix/posix/symlink': No data available total 16 ? -????????? ? ? ? ? ? blockdev ? -????????? ? ? ? ? ? chardev ? -????????? ? ? ? ? ? fifo 131953 -rw-rwxrw- 2 root samba 4 Nov 18 11:37 file 131953 -rw-rwxrw- 2 root samba 4 Nov 18 11:37 hardlink ? -????????? ? ? ? ? ? symlink ? -????????? ? ? ? ? ? symlinkoversmb After: insgesamt 21 131958 brw-r--r-- 1 root root 0, 0 15. Nov 12:04 blockdev 131965 crw-r--r-- 1 root root 1, 1 15. Nov 12:04 chardev 131966 prw-r--r-- 1 root samba 0 15. Nov 12:05 fifo 131953 -rw-rwxrw- 2 root samba 4 18. Nov 11:37 file 131953 -rw-rwxrw- 2 root samba 4 18. Nov 11:37 hardlink 131957 lrwxrwxrwx 1 root samba 4 15. Nov 12:03 symlink -> file 131954 lrwxrwxr-x 1 root samba 23 18. Nov 15:28 symlinkoversmb -> mnt/smb3unix/posix/file Signed-off-by: Ralph Boehme <[email protected]> --- fs/smb/client/cifsproto.h | 1 + fs/smb/client/inode.c | 89 +++++++++++++++++++++++++++++++++++---- fs/smb/client/readdir.c | 35 +++++++-------- fs/smb/client/reparse.c | 84 ++++++++++++++++++++++-------------- 4 files changed, 149 insertions(+), 60 deletions(-)
fs/smb/client/cifsproto.h +1 -0
@@ -676,6 +676,7 @@ int __cifs_sfu_make_node(unsigned int xid, struct inode *inode,int cifs_sfu_make_node (unsigned int xid,struct inode *inode,struct dentry *dentry,struct cifs_tcon *tcon,const char *full_path,umode_t mode,dev_t dev); +umode_t wire_mode_to_posix (u32 wire);#ifdef CONFIG_CIFS_DFS_UPCALL static inline int get_dfs_path(const unsigned int xid, struct cifs_ses *ses,
fs/smb/client/inode.c +80 -9
106 lines not on this page · raw message
fs/smb/client/readdir.c +16 -19
@@ -241,31 +241,28 @@ cifs_posix_to_fattr(struct cifs_fattr *fattr, struct smb2_posix_info *info, fattr->cf_nlink =le32_to_cpu (info->HardLinks); fattr->cf_cifsattrs =le32_to_cpu (info->DosAttributes); -/* -* Since we set the inode type below we need to mask off -* to avoid strange results if bits set above. -* XXX: why not make server&client use the type bits? -*/ - fattr->cf_mode =le32_to_cpu (info->Mode) & ~S_IFMT ; +if (fattr->cf_cifsattrs &ATTR_REPARSE ) { + fattr->cf_cifstag =le32_to_cpu (info->ReparseTag); + } + + fattr->cf_mode =wire_mode_to_posix (le32_to_cpu (info->Mode)); + fattr->cf_dtype =S_DT (le32_to_cpu (info->Mode)); + +switch (fattr->cf_mode &S_IFMT ) { +case S_IFLNK : +case S_IFBLK : +case S_IFCHR : + fattr->cf_flags |=CIFS_FATTR_NEED_REVAL ; +break ; +default : +break ; + }cifs_dbg (FYI ,"posix fattr: dev %d, reparse %d, mode %o\n" ,le32_to_cpu (info->DeviceId),le32_to_cpu (info->ReparseTag),le32_to_cpu (info->Mode)); -if (fattr->cf_cifsattrs &ATTR_DIRECTORY ) { - fattr->cf_mode |=S_IFDIR ; - fattr->cf_dtype =DT_DIR ; - }else { -/* -* mark anything that is not a dir as regular -* file. special files should have the REPARSE -* attribute and will be marked as needing revaluation -*/ - fattr->cf_mode |=S_IFREG ; - fattr->cf_dtype =DT_REG ; - } -sid_to_id (cifs_sb, &parsed.owner, fattr,SIDOWNER );sid_to_id (cifs_sb, &parsed.group, fattr,SIDGROUP ); }
fs/smb/client/reparse.c +52 -32
111 lines not on this page · raw message
Signature · 13 lines
2.47.0 From fedaee4c83cb794c7c417a9687c2b1773fb50a65 Mon Sep 17 00:00:00 2001 From: Ralph Boehme <[email protected]> Date: Mon, 25 Nov 2024 16:19:56 +0100 Subject: [PATCH 3/3] fs/smb/client: cifs_prime_dcache() for SMB3 POSIX reparse points Signed-off-by: Ralph Boehme <[email protected]> --- fs/smb/client/readdir.c | 18 +++++++++++++++++- 1 file changed, 17 insertions(+), 1 deletion(-)
fs/smb/client/readdir.c +17 -1
@@ -71,6 +71,8 @@ cifs_prime_dcache(struct dentry *parent, struct qstr *name,struct inode *inode;struct super_block *sb = parent->d_sb;struct cifs_sb_info *cifs_sb =CIFS_SB (sb); +bool posix =cifs_sb_master_tcon (cifs_sb)->posix_extensions; +bool reparse_need_reval = false;DECLARE_WAIT_QUEUE_HEAD_ONSTACK (wq);int rc;
@@ -85,7 +87,21 @@ cifs_prime_dcache(struct dentry *parent, struct qstr *name,* this spares us an invalidation. */ retry: -if ((fattr->cf_cifsattrs &ATTR_REPARSE ) || +if (posix) { +switch (fattr->cf_mode &S_IFMT ) { +case S_IFLNK : +case S_IFBLK : +case S_IFCHR : + reparse_need_reval = true; +break ; +default : +break ; + } + }else if (fattr->cf_cifsattrs &ATTR_REPARSE ) { + reparse_need_reval = true; + } + +if (reparse_need_reval || (fattr->cf_flags &CIFS_FATTR_NEED_REVAL ))return ;
Signature · 1 line
2.47.0
Looks like a big improvement - fewer roundtrips to Samba. Haven't tried to Windows yet, but it does look like it breaks mounts to ksmbd so I may need a minor change to cifs.ko or ksmbd.
Quoted 15 lines · Ralph Boehme
Quoted 6 lines · Paulo Alcantara
Quoted 2 lines · Ralph Boehme
I'm sure I'm doing things wrong. Can you help me getting this across the line?
Patches look good, thanks. It would be great making sure it didn't regress against NFS server on Windows, but Steve can do that.
great, thanks for taking a look! I was expecting I was likely doing some stupid beginner mistakes, but if you think the changes look good, here's a v2 with just my +1 added, without code changes.
Thanks!
-slow
Signature · 3 lines
Thanks, Steve
Hi Steve!
Thanks for testing! ksmbd needs to be updated to implement the current spec. The next major Samba version 4.22 will ship this stuff, so I'd be good if cifs.ko follows along asap.
Thanks!
-slow
Quoted 23 lines · Steve French
Looks like a big improvement - fewer roundtrips to Samba. Haven't tried to Windows yet, but it does look like it breaks mounts to ksmbd so I may need a minor change to cifs.ko or ksmbd.
Quoted 15 lines · Ralph Boehme
Quoted 6 lines · Paulo Alcantara
Quoted 2 lines · Ralph Boehme
I'm sure I'm doing things wrong. Can you help me getting this across the line?
Patches look good, thanks. It would be great making sure it didn't regress against NFS server on Windows, but Steve can do that.
great, thanks for taking a look! I was expecting I was likely doing some stupid beginner mistakes, but if you think the changes look good, here's a v2 with just my +1 added, without code changes.
Thanks!
-slow
Steve French
Acked-by Paulo Alcantara (Red Hat) Acked-by Paulo Alcantara (Red Hat) Acked-by Paulo Alcantara (Red Hat)I updated patch 2 (0002-fs-smb-client-Implement-new-SMB3-POSIX-type.patch) to address various checkpatch warnings, and added one small changeset to fix mounts to older servers which don't fill in the file type in the level 100 Mode field (0004-smb3.1.1-fix-posix-mounts-to-older-servers.patch), and updated them in cifs-2.6.git for-next. Let me know if any objections. It looks like it does make a big improvement in reporting special files when mounted to Samba, but want to figure out why the dentry cache (revalidate call) still requires the extra roundtrip per file when doing "ls -l" (there may be a way that this enhanced query dir output could be recognized as still valid, and reduce need for extra queries).
See attached.
Quoted 15 lines · Ralph Boehme
Quoted 6 lines · Paulo Alcantara
Quoted 2 lines · Ralph Boehme
I'm sure I'm doing things wrong. Can you help me getting this across the line?
Patches look good, thanks. It would be great making sure it didn't regress against NFS server on Windows, but Steve can do that.
great, thanks for taking a look! I was expecting I was likely doing some stupid beginner mistakes, but if you think the changes look good, here's a v2 with just my +1 added, without code changes.
Thanks!
-slow
Signature · 3 lines
Thanks, Steve
From 28ab8ec89b756453194c654e3a18cfe1728b0309 Mon Sep 17 00:00:00 2001
From: Steve French <[email protected]>
Date: Wed, 4 Dec 2024 17:46:00 -0600
Subject: [PATCH 4/4] smb3.1.1: fix posix mounts to older servers
Some servers which implement the SMB3.1.1 POSIX extensions did not set the file type in the mode in the infolevel 100 response. With recent changes for checking the file type via the mode field, this can cause the root directory to be reported incorrectly and mounts (e.g. to ksmbd) to fail.
--- fs/smb/client/cifsproto.h | 2 +- fs/smb/client/inode.c | 11 ++++++++--- fs/smb/client/readdir.c | 3 ++- 3 files changed, 11 insertions(+), 5 deletions(-)
fs/smb/client/cifsproto.h +1 -1
@@ -669,7 +669,7 @@ int __cifs_sfu_make_node(unsigned int xid, struct inode *inode,int cifs_sfu_make_node (unsigned int xid,struct inode *inode,struct dentry *dentry,struct cifs_tcon *tcon,const char *full_path,umode_t mode,dev_t dev); -umode_t wire_mode_to_posix (u32 wire); +umode_t wire_mode_to_posix (u32 wire,bool is_dir);#ifdef CONFIG_CIFS_DFS_UPCALL static inline int get_dfs_path(const unsigned int xid, struct cifs_ses *ses,
fs/smb/client/inode.c +8 -3
@@ -814,13 +814,17 @@ static u32 wire_filetype_to_posix(u32 wire_type)return posix_filetypes[wire_type]; } -umode_t wire_mode_to_posix (u32 wire) +umode_t wire_mode_to_posix (u32 wire,bool is_dir) {u32 wire_type;u32 mode; wire_type = (wire &POSIX_FILETYPE_MASK ) >>POSIX_FILETYPE_SHIFT ; - mode = (wire_perms_to_posix (wire) |wire_filetype_to_posix (wire_type)); +/* older servers do not set POSIX file type in the mode field in the response */ +if ((wire_type ==0 ) && is_dir) + mode =wire_perms_to_posix (wire) |S_IFDIR ; +else + mode = (wire_perms_to_posix (wire) |wire_filetype_to_posix (wire_type));return (umode_t )mode; }
@@ -860,7 +864,8 @@ static void smb311_posix_info_to_fattr(struct cifs_fattr *fattr, fattr->cf_bytes =le64_to_cpu (info->AllocationSize); fattr->cf_createtime =le64_to_cpu (info->CreationTime); fattr->cf_nlink =le32_to_cpu (info->HardLinks); - fattr->cf_mode =wire_mode_to_posix (le32_to_cpu (info->Mode)); + fattr->cf_mode =wire_mode_to_posix (le32_to_cpu (info->Mode), + fattr->cf_cifsattrs &ATTR_DIRECTORY );if (cifs_open_data_reparse (data) &&cifs_reparse_point_to_fattr (cifs_sb, fattr, data))
fs/smb/client/readdir.c +2 -1
@@ -261,7 +261,8 @@ cifs_posix_to_fattr(struct cifs_fattr *fattr, struct smb2_posix_info *info, fattr->cf_cifstag =le32_to_cpu (info->ReparseTag);/* The Mode field in the response can now include the file type as well */ - fattr->cf_mode =wire_mode_to_posix (le32_to_cpu (info->Mode)); + fattr->cf_mode =wire_mode_to_posix (le32_to_cpu (info->Mode), + fattr->cf_cifsattrs &ATTR_DIRECTORY ); fattr->cf_dtype =S_DT (le32_to_cpu (info->Mode));switch (fattr->cf_mode &S_IFMT ) {
Signature · 1 line
2.43.0
From ca4b2c4607433033e9c4f4659f809af4261d8992 Mon Sep 17 00:00:00 2001
From: Ralph Boehme <[email protected]>
Date: Fri, 15 Nov 2024 13:15:50 +0100
Subject: [PATCH 1/4] fs/smb/client: avoid querying SMB2_OP_QUERY_WSL_EA for
SMB3 POSIX
Avoid extra roundtrip
--- fs/smb/client/smb2inode.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
fs/smb/client/smb2inode.c +2 -1
@@ -943,7 +943,8 @@ int smb2_query_path_info(const unsigned int xid,if (rc || !data->reparse_point) goto out; - cmds[num_cmds++] =SMB2_OP_QUERY_WSL_EA ; +if (!tcon->posix_extensions) + cmds[num_cmds++] =SMB2_OP_QUERY_WSL_EA ;/* * Skip SMB2_OP_GET_REPARSE if symlink already parsed in create * response.
Signature · 1 line
2.43.0
From 6a832bc8bbb22350f7ffe6ecb2d36f261bb96023 Mon Sep 17 00:00:00 2001
From: Ralph Boehme <[email protected]>
Date: Fri, 15 Nov 2024 19:21:04 +0100
Subject: [PATCH 2/4] fs/smb/client: Implement new SMB3 POSIX type
Fixes special files against current Samba.
On the Samba server:
insgesamt 20
131958 brw-r--r-- 1 root root 0, 0 15. Nov 12:04 blockdev 131965 crw-r--r-- 1 root root 1, 1 15. Nov 12:04 chardev 131966 prw-r--r-- 1 samba samba 0 15. Nov 12:05 fifo 131953 -rw-rwxrw-+ 2 samba samba 4 18. Nov 11:37 file 131953 -rw-rwxrw-+ 2 samba samba 4 18. Nov 11:37 hardlink 131957 lrwxrwxrwx 1 samba samba 4 15. Nov 12:03 symlink -> file 131954 -rwxrwxr-x+ 1 samba samba 0 18. Nov 15:28 symlinkoversmb
Before:
ls: cannot access '/mnt/smb3unix/posix/blockdev': No data available
ls: cannot access '/mnt/smb3unix/posix/chardev': No data available
ls: cannot access '/mnt/smb3unix/posix/symlinkoversmb': No data available
ls: cannot access '/mnt/smb3unix/posix/fifo': No data available
ls: cannot access '/mnt/smb3unix/posix/symlink': No data available total 16
? -????????? ? ? ? ? ? blockdev
? -????????? ? ? ? ? ? chardev
? -????????? ? ? ? ? ? fifo
131953 -rw-rwxrw- 2 root samba 4 Nov 18 11:37 file
131953 -rw-rwxrw- 2 root samba 4 Nov 18 11:37 hardlink
? -????????? ? ? ? ? ? symlink
? -????????? ? ? ? ? ? symlinkoversmbAfter:
insgesamt 21
131958 brw-r--r-- 1 root root 0, 0 15. Nov 12:04 blockdev 131965 crw-r--r-- 1 root root 1, 1 15. Nov 12:04 chardev
131966 prw-r--r-- 1 root samba 0 15. Nov 12:05 fifo 131953 -rw-rwxrw- 2 root samba 4 18. Nov 11:37 file 131953 -rw-rwxrw- 2 root samba 4 18. Nov 11:37 hardlink 131957 lrwxrwxrwx 1 root samba 4 15. Nov 12:03 symlink -> file
131954 lrwxrwxr-x 1 root samba 23 18. Nov 15:28 symlinkoversmb -> mnt/smb3unix/posix/file
--- fs/smb/client/cifsproto.h | 1 + fs/smb/client/inode.c | 89 +++++++++++++++++++++++++++++++++++---- fs/smb/client/readdir.c | 35 +++++++-------- fs/smb/client/reparse.c | 84 ++++++++++++++++++++++-------------- 4 files changed, 149 insertions(+), 60 deletions(-)
fs/smb/client/cifsproto.h +1 -0
@@ -669,6 +669,7 @@ int __cifs_sfu_make_node(unsigned int xid, struct inode *inode,int cifs_sfu_make_node (unsigned int xid,struct inode *inode,struct dentry *dentry,struct cifs_tcon *tcon,const char *full_path,umode_t mode,dev_t dev); +umode_t wire_mode_to_posix (u32 wire);#ifdef CONFIG_CIFS_DFS_UPCALL static inline int get_dfs_path(const unsigned int xid, struct cifs_ses *ses,
fs/smb/client/inode.c +80 -9
106 lines not on this page · raw message
fs/smb/client/readdir.c +16 -19
@@ -241,31 +241,28 @@ cifs_posix_to_fattr(struct cifs_fattr *fattr, struct smb2_posix_info *info, fattr->cf_nlink =le32_to_cpu (info->HardLinks); fattr->cf_cifsattrs =le32_to_cpu (info->DosAttributes); -/* -* Since we set the inode type below we need to mask off -* to avoid strange results if bits set above. -* XXX: why not make server&client use the type bits? -*/ - fattr->cf_mode =le32_to_cpu (info->Mode) & ~S_IFMT ; +if (fattr->cf_cifsattrs &ATTR_REPARSE ) + fattr->cf_cifstag =le32_to_cpu (info->ReparseTag); + +/* The Mode field in the response can now include the file type as well */ + fattr->cf_mode =wire_mode_to_posix (le32_to_cpu (info->Mode)); + fattr->cf_dtype =S_DT (le32_to_cpu (info->Mode)); + +switch (fattr->cf_mode &S_IFMT ) { +case S_IFLNK : +case S_IFBLK : +case S_IFCHR : + fattr->cf_flags |=CIFS_FATTR_NEED_REVAL ; +break ; +default : +break ; + }cifs_dbg (FYI ,"posix fattr: dev %d, reparse %d, mode %o\n" ,le32_to_cpu (info->DeviceId),le32_to_cpu (info->ReparseTag),le32_to_cpu (info->Mode)); -if (fattr->cf_cifsattrs &ATTR_DIRECTORY ) { - fattr->cf_mode |=S_IFDIR ; - fattr->cf_dtype =DT_DIR ; - }else { -/* -* mark anything that is not a dir as regular -* file. special files should have the REPARSE -* attribute and will be marked as needing revaluation -*/ - fattr->cf_mode |=S_IFREG ; - fattr->cf_dtype =DT_REG ; - } -sid_to_id (cifs_sb, &parsed.owner, fattr,SIDOWNER );sid_to_id (cifs_sb, &parsed.group, fattr,SIDGROUP ); }
fs/smb/client/reparse.c +52 -32
111 lines not on this page · raw message
Signature · 1 line
2.43.0
From 8cb0bc5436351de8a11eef13b7367d64cc0d6c68 Mon Sep 17 00:00:00 2001
From: Ralph Boehme <[email protected]>
Date: Mon, 25 Nov 2024 16:19:56 +0100
Subject: [PATCH 3/4] fs/smb/client: cifs_prime_dcache() for SMB3 POSIX reparse
points
Spares an extra revalidation request
--- fs/smb/client/readdir.c | 18 +++++++++++++++++- 1 file changed, 17 insertions(+), 1 deletion(-)
fs/smb/client/readdir.c +17 -1
@@ -71,6 +71,8 @@ cifs_prime_dcache(struct dentry *parent, struct qstr *name,struct inode *inode;struct super_block *sb = parent->d_sb;struct cifs_sb_info *cifs_sb =CIFS_SB (sb); +bool posix =cifs_sb_master_tcon (cifs_sb)->posix_extensions; +bool reparse_need_reval = false;DECLARE_WAIT_QUEUE_HEAD_ONSTACK (wq);int rc;
@@ -85,7 +87,21 @@ cifs_prime_dcache(struct dentry *parent, struct qstr *name,* this spares us an invalidation. */ retry: -if ((fattr->cf_cifsattrs &ATTR_REPARSE ) || +if (posix) { +switch (fattr->cf_mode &S_IFMT ) { +case S_IFLNK : +case S_IFBLK : +case S_IFCHR : + reparse_need_reval = true; +break ; +default : +break ; + } + }else if (fattr->cf_cifsattrs &ATTR_REPARSE ) { + reparse_need_reval = true; + } + +if (reparse_need_reval || (fattr->cf_flags &CIFS_FATTR_NEED_REVAL ))return ;
Signature · 1 line
2.43.0
Hi Steve
thanks for digging into this!
Quoted 3 lines · Steve French
I updated patch 2 (0002-fs-smb-client-Implement-new-SMB3-POSIX-type.patch) to address various checkpatch warnings,
oh, sorry for those! I'll try to remember to run it myself next time.
Quoted 5 lines
and added one small changeset to fix
mounts to older servers which don't fill in the file type in the level 100 Mode field (0004-smb3.1.1-fix-posix-mounts-to-older-servers.patch), and updated them in cifs-2.6.git for-next. Let me know if any objections.
hm, not sure I like adding code to support non compliant SMB3 POSIX server implementations at this early stage of implementing them across various projects, clients and servers. Can't Namjae just fix it in ksmbd?
Quoted 6 lines
It looks like it does make a big improvement in reporting special files when mounted to Samba, but want to figure out why the dentry cache (revalidate call) still requires the extra roundtrip per file when doing "ls -l" (there may be a way that this enhanced query dir output could be recognized as still valid, and reduce need for extra queries).
it should only do an extra roundtrip when needed for symlinks, character and block devices as for those we wee need additional info. But for fifos and sockets the SMB2-FIND POSIX-infolevel is sufficient and in my testing I didn't see extra roundtrips for those.
Thanks!
-slow
Quoted 18 lines · Ralph Boehme
Hi Steve
thanks for digging into this!
Quoted 3 lines · Steve French
I updated patch 2 (0002-fs-smb-client-Implement-new-SMB3-POSIX-type.patch) to address various checkpatch warnings,
oh, sorry for those! I'll try to remember to run it myself next time.
Quoted 5 lines
and added one small changeset to fix
mounts to older servers which don't fill in the file type in the level 100 Mode field (0004-smb3.1.1-fix-posix-mounts-to-older-servers.patch), and updated them in cifs-2.6.git for-next. Let me know if any objections.
Quoted 3 lines
hm, not sure I like adding code to support non compliant SMB3 POSIX server implementations at this early stage of implementing them across various projects, clients and servers. Can't Namjae just fix it in ksmbd?
We will make sure ksmbd is changed to fill in the file type, but we should reduce risks (where fix is easy) of breaking mounts to existing servers, especially since this has been in for more than three years and since the sanity check is safe and easy (ATTRIBUTE_DIRECTORY
will always be set correctly by a server). There is also at least
one other server that implemented earlier version of the SMB3.1.1 POSIX Extensions so better to be safe when we can do a minor check like this on the client so we don't risk any regressions, and it also will reduce risk of a future server bug breaking mounts.
Signature · 3 lines
Thanks, Steve
Hello, this patch is incomplete and still does not fix the main problem that SMB2_OP_QUERY_WSL_EA command does not work with any Windows SMB server except the last Windows Server version. On non-recent Windows versions it is not possible to set both EAs and reparse point at the same time and Windows SMB server is returning error when trying to query EAs on file with reparse point.
Which basically means that it is not possible to query data about the special files from Windows SMB server (except 2022 version).
More details are in the email which I wrote in September:
https://lore.kernel.org/linux-cifs/20240928140939.vjndryndfngzq7x4@pali/
I proposed similar but extended patch which skips asking for EAs based on reparse point:
https://lore.kernel.org/linux-cifs/[email protected]/
But it was somehow rejected as the proper solution should be different:
https://lore.kernel.org/linux-cifs/20240917210707.4lt4obty7wlmm42j@pali/
And that is why I'm surprised that the batch below was accepted...
Are you planning to implement a proper solution? If not that I would propose to reconsider my original idea, which will workaround also Windows SMB server, and not only Samba server.
As I wrote in previous emails, I think that this is a serious issue as it disallow to use any reparse points against non-recent Windows Server systems.
Quoted 26 lines · On Wednesday 27 November 2024 20:23:29 Ralph Boehme wrote:
From
a57c32da874f285af266b7fbbaefb7940991d049Mon Sep 17 00:00:00 2001
From: Ralph Boehme <[email protected]>
Date: Fri, 15 Nov 2024 13:15:50 +0100
Subject: [PATCH 1/3]fs/smb/client: avoid queryingSMB2_OP_QUERY_WSL_EAfor
SMB3 POSIX--- fs/smb/client/smb2inode.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)fs/smb/client/smb2inode.c +2
-1@@ -941,7 +941,8 @@ int smb2_query_path_info(const unsigned int xid,if (rc || !data->reparse_point) goto out; - cmds[num_cmds++] =SMB2_OP_QUERY_WSL_EA ; +if (!tcon->posix_extensions) + cmds[num_cmds++] =SMB2_OP_QUERY_WSL_EA ;/* * Skip SMB2_OP_GET_REPARSE if symlink already parsed in create * response. Signature · 1 line
2.47.0
Quoted 6 lines · Pali Rohár
Hello, this patch is incomplete and still does not fix the main problem that
SMB2_OP_QUERY_WSL_EAcommand does not work with any Windows SMB server except the last Windows Server version. On non-recent Windows versions it is not possible to set both EAs and reparse point at the same time and Windows SMB server is returning error when trying to query EAs on file with reparse point.
No, Ralph's patch has nothing to do with your problem. SMB3.1.1 POSIX will support NFS reparse points with no EAs, so makes sense skipping SMB2_OP_QUERY_WSL_EA altogether for posix case.
Quoted 9 lines
Which basically means that it is not possible to query data about the special files from Windows SMB server (except 2022 version).
More details are in the email which I wrote in September:
https://lore.kernel.org/linux-cifs/20240928140939.vjndryndfngzq7x4@pali/I proposed similar but extended patch which skips asking for EAs based on reparse point:
https://lore.kernel.org/linux-cifs/[email protected]/
Yes, that patch looks incorrect and untested. Can you tell me how is @data->reparse.tag supposed to be set, for non-readdir case, if the compound request wasn't sent yet? Have you tried to stat(2) those files with your patch?
Quoted 2 lines
But it was somehow rejected as the proper solution should be different:
https://lore.kernel.org/linux-cifs/20240917210707.4lt4obty7wlmm42j@pali/
Yes. Have you sent a patch with the proposed solution yet?
Quoted 9 lines · On Monday 09 December 2024 16:28:08 Paulo Alcantara wrote:
Quoted 6 lines · Pali Rohár
Hello, this patch is incomplete and still does not fix the main problem that
SMB2_OP_QUERY_WSL_EAcommand does not work with any Windows SMB server except the last Windows Server version. On non-recent Windows versions it is not possible to set both EAs and reparse point at the same time and Windows SMB server is returning error when trying to query EAs on file with reparse point.
Quoted 3 lines
No, Ralph's patch has nothing to do with your problem. SMB3.1.1 POSIX will support NFS reparse points with no EAs, so makes sense skipping
SMB2_OP_QUERY_WSL_EAaltogether for posix case.
Ok, so this is just a coincidence that skip was added at the same place. And I though that it is the same thing (which now I see that is not).
Quoted 6 lines
Quoted 6 lines
Which basically means that it is not possible to query data about the special files from Windows SMB server (except 2022 version).
More details are in the email which I wrote in September:
https://lore.kernel.org/linux-cifs/20240928140939.vjndryndfngzq7x4@pali/
Quoted 8 lines
Quoted 3 lines
I proposed similar but extended patch which skips asking for EAs based on reparse point:
https://lore.kernel.org/linux-cifs/[email protected]/Yes, that patch looks incorrect and untested. Can you tell me how is @data->reparse.tag supposed to be set, for non-readdir case, if the compound request wasn't sent yet? Have you tried to stat(2) those files with your patch?
I need to rethink about it. Basically I already dropped this patch as I was expecting the proper solution.
Quoted 4 lines
Quoted 2 lines
But it was somehow rejected as the proper solution should be different:
https://lore.kernel.org/linux-cifs/20240917210707.4lt4obty7wlmm42j@pali/Yes. Have you sent a patch with the proposed solution yet?
I have not sent, I was not working on it. I was in impression that you are going to implement the proper solution.
Quoted 6 lines · Pali Rohár
Quoted 2 lines · On Monday 09 December 2024 16:28:08 Paulo Alcantara wrote:
Yes. Have you sent a patch with the proposed solution yet?
I have not sent, I was not working on it. I was in impression that you are going to implement the proper solution.
Yes, but priorities have changed, unfortunately.
Quoted 10 lines · On Monday 09 December 2024 16:42:54 Paulo Alcantara wrote:
Quoted 6 lines · Pali Rohár
Quoted 2 lines · On Monday 09 December 2024 16:28:08 Paulo Alcantara wrote:
Yes. Have you sent a patch with the proposed solution yet?
I have not sent, I was not working on it. I was in impression that you are going to implement the proper solution.
Yes, but priorities have changed, unfortunately.
Ok. Would you have a time to prepare a patch? I have still feeling that the smb2_compound_op() code and EAs code is rather complicated for me. I would like if somebody else could look at it, as I have feeling if I try to do it, it can end up with something more broken...
Quoted 4 lines · Pali Rohár
Ok. Would you have a time to prepare a patch? I have still feeling that the
smb2_compound_op()code and EAs code is rather complicated for me. I would like if somebody else could look at it, as I have feeling if I try to do it, it can end up with something more broken...
Yeah, it's a bit tricky. I'll let you know when I have patch so I can try it against your old servers.
Quoted 9 lines · On Tuesday 10 December 2024 10:19:26 Paulo Alcantara wrote:
Quoted 4 lines · Pali Rohár
Ok. Would you have a time to prepare a patch? I have still feeling that the
smb2_compound_op()code and EAs code is rather complicated for me. I would like if somebody else could look at it, as I have feeling if I try to do it, it can end up with something more broken...Yeah, it's a bit tricky. I'll let you know when I have patch so I can try it against your old servers.
I looked at that code and it looks to be really more complicated to implement partial parsing of the reply, as compound code is bound to many places and code paths.
Meanwhile I have tried to implement a new version to retry the call without querying EAs when server returns that EAs on specific paths are unsupported:
https://patchwork.kernel.org/project/cifs-client/patch/[email protected]/
I think that this approach should now work correctly.