cifs: ls of mount point gives input/output error (probably related to CIFS: getdents() broken for large dirs)
Subsystems (2)
I mount via cifs a windows XP share, df gives me correct sizes, but when I ls the mount point i get input/output error.
strace: http://pastebin.com/WXf8M1nu
mount --verbose -t cifs -o username=administrator,password=blahblah //192.168.0.11/jobs /mnt/backups/montaz/jobs mount.cifs kernel mount options: ip=192.168.0.11,unc=\\192.168.0.11\jobs,,ver=1,user=administrator,pass=********
df
//192.168.0.11/jobs 114464 105196 9268 92% /mnt/backups/montaz/jobs
ls /mnt/backups/montaz/jobs/
ls: reading directory /mnt/backups/montaz/jobs/: Input/output error total 0
the fun thing is that i can cd to a lower level directory, and ls works fine there! only the mount point has the problem
ls /mnt/backups/montaz/jobs/test
total 44K
drwxr-xr-x 1 root root 0 Apr 30 2010 blah blah/
......
kernel version 3.2rc7
this seems to be related to :
https://lkml.org/lkml/2011/8/1/427
Re: [3.0.0+][Regression][Bisected] CIFS: getdents() broken for large dirs
Please CC because i am not subscribed
Quoted 25 lines · On Thu, 29 Dec 2011 02:08:57 +0200 Konstantinos Skarlatos…
I mount via cifs a windows XP share, df gives me correct sizes, but when I ls the mount point i get input/output error.
strace: http://pastebin.com/WXf8M1numount --verbose -t cifs -o username=administrator,password=blahblah //192.168.0.11/jobs
/mnt/backups/montaz/jobs
mount.cifs kernel mount options:
ip=192.168.0.11,unc=\\192.168.0.11\jobs,,ver=1,user=administrator,pass=********df
//192.168.0.11/jobs 114464
105196 9268 92%/mnt/backups/montaz/jobsls
/mnt/backups/montaz/jobs/
ls: reading directory/mnt/backups/montaz/jobs/: Input/output error total 0the fun thing is that i can cd to a lower level directory, and ls works fine there! only the mount point has the problem
ls
/mnt/backups/montaz/jobs/test
total 44K
drwxr-xr-x 1 root root 0 Apr 30 2010 blah blah/
......
Quoted 6 lines
kernel version 3.2rc7
this seems to be related to :
https://lkml.org/lkml/2011/8/1/427
Re: [3.0.0+][Regression][Bisected] CIFS:getdents()broken for large dirs
Hmmm, maybe. What makes you think that it's related? What sort of server are you seeing this against?
What would be most helpful is a capture and/or some cifsFYI info. Instructions on how to capture that are here:
http://wiki.samba.org/index.php/LinuxCIFS_troubleshooting
Would you be able to provide those?
Thanks,
Signature · 1 line
Jeff Layton <[email protected]>
Quoted 30 lines · Konstantinos Skarlatos
I mount via cifs a windows XP share, df gives me correct sizes, but when I ls the mount point i get input/output error.
strace: http://pastebin.com/WXf8M1numount --verbose -t cifs -o username=administrator,password=blahblah //192.168.0.11/jobs
/mnt/backups/montaz/jobs
mount.cifs kernel mount options:
ip=192.168.0.11,unc=\\192.168.0.11\jobs,,ver=1,user=administrator,pass=********df
//192.168.0.11/jobs 114464 105196 9268 92%/mnt/backups/montaz/jobsls
/mnt/backups/montaz/jobs/
ls: reading directory/mnt/backups/montaz/jobs/: Input/output error total 0the fun thing is that i can cd to a lower level directory, and ls works fine there! only the mount point has the problem
ls
/mnt/backups/montaz/jobs/test
total 44K
drwxr-xr-x 1 root root 0 Apr 30 2010 blah blah/ekernel version 3.2rc7
this seems to be related to :
https://lkml.org/lkml/2011/8/1/427
Re: [3.0.0+][Regression][Bisected] CIFS:getdents()broken for large dirs
Quoted 5 lines
Please CC because i am not subscribed
--
To unsubscribe from this list: send the line "unsubscribe linux-cifs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
I was working on phrasing a question for MS.
Noticed this problem too.
MS-CIFS - v20101001, in section 2.2.2.3.1, does not list info level codes higher than 0x104 for Trans2 subcommand Trans2_find_first2.
Yet servers such as Windows 2008 and Windows 7, respond to info levels such as 0x105 correctly. But Windows XP responds to this info level (0x105) incorrectly. The Find_First2 data has fewer entries than the ones listed in search Count field of Find First2 parameters.
Servers such as Windows XP and 2000 should return an error EOPNOTSUPP for info levels higher than 0x104 but they do not.
http://msdn.microsoft.com/en-us/library/cc246806(v=prot.13).aspx says that Windows XP does not handle SMB_FIND_FILE_ID_FULL_DIRECTORY_INFO.
"<66> Section 2.2.8.1.2: The SMB_FIND_FILE_ID_FULL_DIRECTORY_INFO Information Level was introduced in Windows Server 2003 and is not present in Windows 2000 Server and Windows XP."
Quoted 28 lines · Jeff Layton
Quoted 25 lines · On Thu, 29 Dec 2011 02:08:57 +0200 Konstantinos…
I mount via cifs a windows XP share, df gives me correct sizes, but when I ls the mount point i get input/output error.
strace: http://pastebin.com/WXf8M1numount --verbose -t cifs -o username=administrator,password=blahblah //192.168.0.11/jobs
/mnt/backups/montaz/jobs
mount.cifs kernel mount options:
ip=192.168.0.11,unc=\\192.168.0.11\jobs,,ver=1,user=administrator,pass=********df
//192.168.0.11/jobs 114464
105196 9268 92%/mnt/backups/montaz/jobsls
/mnt/backups/montaz/jobs/
ls: reading directory/mnt/backups/montaz/jobs/: Input/output error total 0the fun thing is that i can cd to a lower level directory, and ls works fine there! only the mount point has the problem
ls
/mnt/backups/montaz/jobs/test
total 44K
drwxr-xr-x 1 root root 0 Apr 30 2010 blah blah/
......
Quoted 8 lines
Quoted 6 lines
kernel version 3.2rc7
this seems to be related to :
https://lkml.org/lkml/2011/8/1/427
Re: [3.0.0+][Regression][Bisected] CIFS:getdents()broken for large dirsHmmm, maybe. What makes you think that it's related? What sort of server are you seeing this against?
Windows XP service pack 2 (greek)
Quoted 7 lines
What would be most helpful is a capture and/or some cifsFYI info. Instructions on how to capture that are here:
http://wiki.samba.org/index.php/LinuxCIFS_troubleshooting
Would you be able to provide those?
Quoted 2 lines
Thanks,
Quoted 31 lines · On Thu, 29 Dec 2011 12:30:18 +0200 Konstantinos Skarlatos…
Quoted 30 lines · Jeff Layton
Quoted 27 lines · On Thu, 29 Dec 2011 02:08:57 +0200 Konstantinos…
I mount via cifs a windows XP share, df gives me correct sizes, but when I ls the mount point i get input/output error.
strace: http://pastebin.com/WXf8M1numount --verbose -t cifs -o username=administrator,password=blahblah //192.168.0.11/jobs
/mnt/backups/montaz/jobs
mount.cifs kernel mount options:
ip=192.168.0.11,unc=\\192.168.0.11\jobs,,ver=1,user=administrator,pass=********df
//192.168.0.11/jobs 114464
105196 9268 92%/mnt/backups/montaz/jobsls
/mnt/backups/montaz/jobs/
ls: reading directory/mnt/backups/montaz/jobs/: Input/output error total 0the fun thing is that i can cd to a lower level directory, and ls works fine there! only the mount point has the problem
ls
/mnt/backups/montaz/jobs/test
total 44K
drwxr-xr-x 1 root root 0 Apr 30 2010 blah blah/
......kernel version 3.2rc7
Quoted 7 lines
Quoted 6 lines
Quoted 4 lines
this seems to be related to :
https://lkml.org/lkml/2011/8/1/427
Re: [3.0.0+][Regression][Bisected] CIFS:getdents()broken for large dirsHmmm, maybe. What makes you think that it's related? What sort of server are you seeing this against?
Windows XP service pack 2 (greek)
How many files are in the directory?
Signature · 1 line
Jeff Layton <[email protected]>
Quoted 38 lines · Jeff Layton
Quoted 35 lines · On Thu, 29 Dec 2011 12:30:18 +0200 Konstantinos…
Quoted 34 lines · Jeff Layton
Quoted 31 lines · On Thu, 29 Dec 2011 02:08:57 +0200 Konstantinos…
I mount via cifs a windows XP share, df gives me correct sizes, but when I ls the mount point i get input/output error.
strace: http://pastebin.com/WXf8M1numount --verbose -t cifs -o username=administrator,password=blahblah //192.168.0.11/jobs
/mnt/backups/montaz/jobs
mount.cifs kernel mount options:
ip=192.168.0.11,unc=\\192.168.0.11\jobs,,ver=1,user=administrator,pass=********df
//192.168.0.11/jobs 114464
105196 9268 92%/mnt/backups/montaz/jobsls
/mnt/backups/montaz/jobs/
ls: reading directory/mnt/backups/montaz/jobs/: Input/output error total 0the fun thing is that i can cd to a lower level directory, and ls works fine there! only the mount point has the problem
ls
/mnt/backups/montaz/jobs/test
total 44K
drwxr-xr-x 1 root root 0 Apr 30 2010 blah blah/
......kernel version 3.2rc7
this seems to be related to :
https://lkml.org/lkml/2011/8/1/427
Re: [3.0.0+][Regression][Bisected] CIFS:getdents()broken for large dirs
Quoted 7 lines
Quoted 3 lines
Quoted 2 lines
Hmmm, maybe. What makes you think that it's related? What sort of server are you seeing this against?
Windows XP service pack 2 (greek)
How many files are in the directory?
140 folders and 20 files
Quoted 40 lines · Konstantinos Skarlatos
Quoted 39 lines · Jeff Layton
Quoted 36 lines · On Thu, 29 Dec 2011 12:30:18 +0200 Konstantinos…
Quoted 35 lines · Jeff Layton
Quoted 32 lines · On Thu, 29 Dec 2011 02:08:57 +0200 Konstantinos…
I mount via cifs a windows XP share, df gives me correct sizes, but when I ls the mount point i get input/output error.
strace: http://pastebin.com/WXf8M1numount --verbose -t cifs -o username=administrator,password=blahblah //192.168.0.11/jobs
/mnt/backups/montaz/jobs
mount.cifs kernel mount options:
ip=192.168.0.11,unc=\\192.168.0.11\jobs,,ver=1,user=administrator,pass=********df
//192.168.0.11/jobs 114464
105196 9268 92%/mnt/backups/montaz/jobsls
/mnt/backups/montaz/jobs/
ls: reading directory/mnt/backups/montaz/jobs/: Input/output error total 0the fun thing is that i can cd to a lower level directory, and ls works fine there! only the mount point has the problem
ls
/mnt/backups/montaz/jobs/test
total 44K
drwxr-xr-x 1 root root 0 Apr 30 2010 blah blah/
......kernel version 3.2rc7
this seems to be related to :
https://lkml.org/lkml/2011/8/1/427
Re: [3.0.0+][Regression][Bisected] CIFS:getdents()broken for large dirs
Quoted 9 lines
Quoted 7 lines
Quoted 3 lines
Quoted 2 lines
Hmmm, maybe. What makes you think that it's related? What sort of server are you seeing this against?
Windows XP service pack 2 (greek)
How many files are in the directory?
140 folders and 20 files
Attached is a tcp dump of my session.
Quoted 46 lines · On Fri, 30 Dec 2011 11:04:59 +0200 Konstantinos Skarlatos…
Quoted 45 lines · Konstantinos Skarlatos
Quoted 44 lines · Jeff Layton
Quoted 41 lines · On Thu, 29 Dec 2011 12:30:18 +0200 Konstantinos…
Quoted 40 lines · Jeff Layton
Quoted 28 lines · On Thu, 29 Dec 2011 02:08:57 +0200 Konstantinos…
I mount via cifs a windows XP share, df gives me correct sizes, but when
I ls the mount point i get input/output error.
strace: http://pastebin.com/WXf8M1numount --verbose -t cifs -o username=administrator,password=blahblah //192.168.0.11/jobs
/mnt/backups/montaz/jobs
mount.cifs kernel mount options:
ip=192.168.0.11,unc=\\192.168.0.11\jobs,,ver=1,user=administrator,pass=********df
//192.168.0.11/jobs 114464
105196 9268 92%/mnt/backups/montaz/jobsls
/mnt/backups/montaz/jobs/
ls: reading directory/mnt/backups/montaz/jobs/: Input/output error total 0the fun thing is that i can cd to a lower level directory, and ls works
fine there! only the mount point has the problemls
/mnt/backups/montaz/jobs/test
total 44K
drwxr-xr-x 1 root root 0 Apr 30 2010 blah blah/
......Quoted 7 lines
kernel version 3.2rc7
this seems to be related to :
https://lkml.org/lkml/2011/8/1/427
Re: [3.0.0+][Regression][Bisected] CIFS:getdents()broken for large dirsHmmm, maybe. What makes you think that it's related? What sort of server are you seeing this against?
Quoted 8 lines
Quoted 7 lines
Quoted 5 lines
Quoted 1 line
Windows XP service pack 2 (greek)
How many files are in the directory?
140 folders and 20 files
Attached is a tcp dump of my session.
I tried reproducing this here, but wasn't able to. Testing against my xp box worked fine.
Most likely, the FIND_FILE responses are falling afoul of the code in coalesce_t2 or check2ndT2. Unfortunately that code is pretty complicated and I'm not certain what the problem actually is...
One thing that's interesting is that the total data being sent in the request is rather large (16336 bytes). I think that's legit, but maybe it's exceeding the end of the buffer once we try to coalesce it.
Would it be possible to get the cFYI output from this test?
Is this a regression? Did it work with earlier kernels and only recently start failing?
Signature · 1 line
Jeff Layton <[email protected]>
Quoted 63 lines · Jeff Layton
Quoted 46 lines · On Fri, 30 Dec 2011 11:04:59 +0200 Konstantinos…
Quoted 45 lines · Konstantinos Skarlatos
Quoted 44 lines · Jeff Layton
Quoted 41 lines · On Thu, 29 Dec 2011 12:30:18 +0200 Konstantinos…
Quoted 40 lines · Jeff Layton
Quoted 28 lines · On Thu, 29 Dec 2011 02:08:57 +0200 Konstantinos…
I mount via cifs a windows XP share, df gives me correct sizes, but when
I ls the mount point i get input/output error.
strace: http://pastebin.com/WXf8M1numount --verbose -t cifs -o username=administrator,password=blahblah //192.168.0.11/jobs
/mnt/backups/montaz/jobs
mount.cifs kernel mount options:
ip=192.168.0.11,unc=\\192.168.0.11\jobs,,ver=1,user=administrator,pass=********df
//192.168.0.11/jobs 114464
105196 9268 92%/mnt/backups/montaz/jobsls
/mnt/backups/montaz/jobs/
ls: reading directory/mnt/backups/montaz/jobs/: Input/output error total 0the fun thing is that i can cd to a lower level directory, and ls works
fine there! only the mount point has the problemls
/mnt/backups/montaz/jobs/test
total 44K
drwxr-xr-x 1 root root 0 Apr 30 2010 blah blah/
......Quoted 7 lines
kernel version 3.2rc7
this seems to be related to :
https://lkml.org/lkml/2011/8/1/427
Re: [3.0.0+][Regression][Bisected] CIFS:getdents()broken for large dirsHmmm, maybe. What makes you think that it's related? What sort of server are you seeing this against?
Quoted 7 lines
Quoted 6 lines
Quoted 4 lines
Quoted 1 line
Windows XP service pack 2 (greek)
How many files are in the directory?
140 folders and 20 files
Attached is a tcp dump of my session.
I tried reproducing this here, but wasn't able to. Testing against my xp box worked fine.
Most likely, the
FIND_FILEresponses are falling afoul of the code incoalesce_t2or check2ndT2. Unfortunately that code is pretty complicated and I'm not certain what the problem actually is...
Quoted 5 lines
One thing that's interesting is that the total data being sent in the request is rather large (16336 bytes). I think that's legit, but maybe it's exceeding the end of the buffer once we try to coalesce it.
Would it be possible to get the cFYI output from this test?
I did not get a cFYI output from that test, but i redid a mount-ls-umount and am attaching the tcpdump Also here http://pastebin.com/J20uC6kU you can find the cifsFYI and the contents of /proc/fs/cifs/DebugData form the same test
Quoted 4 lines
Is this a regression? Did it work with earlier kernels and only recently start failing?
I do not know, and i am a bit afraid to downgrade this machine below 3.0 due to some changes arch linux has introduced recently. I can always set up a few virtual machines though, and i can even request permission from my company to give you shell access if you like. Which kernel versions would you like me to test?
Quoted 69 lines · Konstantinos Skarlatos
Quoted 68 lines · Jeff Layton
Quoted 47 lines · On Fri, 30 Dec 2011 11:04:59 +0200 Konstantinos…
Quoted 46 lines · Konstantinos Skarlatos
Quoted 45 lines · Jeff Layton
Quoted 42 lines · On Thu, 29 Dec 2011 12:30:18 +0200 Konstantinos…
Quoted 41 lines · Jeff Layton
Quoted 29 lines · On Thu, 29 Dec 2011 02:08:57 +0200 Konstantinos…
I mount via cifs a windows XP share, df gives me correct sizes, but when
I ls the mount point i get input/output error.
strace: http://pastebin.com/WXf8M1numount --verbose -t cifs -o username=administrator,password=blahblah //192.168.0.11/jobs
/mnt/backups/montaz/jobs
mount.cifs kernel mount options:
ip=192.168.0.11,unc=\\192.168.0.11\jobs,,ver=1,user=administrator,pass=********df
//192.168.0.11/jobs 114464
105196 9268 92%/mnt/backups/montaz/jobsls
/mnt/backups/montaz/jobs/
ls: reading directory/mnt/backups/montaz/jobs/: Input/output error total 0the fun thing is that i can cd to a lower level directory, and ls works
fine there! only the mount point has the problemls
/mnt/backups/montaz/jobs/test
total 44K
drwxr-xr-x 1 root root 0 Apr 30 2010 blah blah/
......Quoted 7 lines
kernel version 3.2rc7
this seems to be related to :
https://lkml.org/lkml/2011/8/1/427
Re: [3.0.0+][Regression][Bisected] CIFS:getdents()broken for large dirsHmmm, maybe. What makes you think that it's related? What sort of server are you seeing this against?
Quoted 7 lines
Quoted 6 lines
Quoted 4 lines
Quoted 1 line
Windows XP service pack 2 (greek)
How many files are in the directory?
140 folders and 20 files
Attached is a tcp dump of my session.
I tried reproducing this here, but wasn't able to. Testing against my xp box worked fine.
Most likely, the
FIND_FILEresponses are falling afoul of the code incoalesce_t2or check2ndT2. Unfortunately that code is pretty complicated and I'm not certain what the problem actually is...One thing that's interesting is that the total data being sent in the request is rather large (16336 bytes). I think that's legit, but maybe it's exceeding the end of the buffer once we try to coalesce it.
Quoted 8 lines
Quoted 1 line
Would it be possible to get the cFYI output from this test?
I did not get a cFYI output from that test, but i redid a mount-ls-umount and am attaching the tcpdump Also here http://pastebin.com/J20uC6kU you can find the cifsFYI and the contents of
/proc/fs/cifs/DebugDataform the same testQuoted 4 lines
Is this a regression? Did it work with earlier kernels and only recently start failing?
I do not know, and i am a bit afraid to downgrade this machine below 3.0 due to some changes arch linux has introduced recently. I can always set up a few virtual machines though, and i can even request permission from my company to give you shell access if you like. Which kernel versions would you like me to test?
I just tested 3.1.5-1-ARCH on a virtual machine and it works, so it is probably a regression... On the same virtual machine 3.2-rc7 produces input/output error. The virtual machine is a fresh install of arch linux. here is the relevant pastebin http://pastebin.com/BwX2DqJC
and attached is the tcpdump
As a am a noob to all this, what should I do next in order to help you? maybe compile a 3.1 kernel from official sources to make sure no patches from arch linux interfere?
Quoted 95 lines · Konstantinos Skarlatos
Quoted 85 lines · Konstantinos Skarlatos
Quoted 71 lines · Jeff Layton
Quoted 51 lines · On Fri, 30 Dec 2011 11:04:59 +0200 Konstantinos…
Quoted 50 lines · Konstantinos Skarlatos
Quoted 48 lines · Jeff Layton
Quoted 44 lines · On Thu, 29 Dec 2011 12:30:18 +0200 Konstantinos…
Quoted 42 lines · Jeff Layton
Quoted 29 lines · On Thu, 29 Dec 2011 02:08:57 +0200 Konstantinos…
I mount via cifs a windows XP share, df gives me correct sizes, but when
I ls the mount point i get input/output error.
strace: http://pastebin.com/WXf8M1numount --verbose -t cifs -o username=administrator,password=blahblah //192.168.0.11/jobs
/mnt/backups/montaz/jobs
mount.cifs kernel mount options:ip=192.168.0.11,unc=\\192.168.0.11\jobs,,ver=1,user=administrator,pass=********
df
//192.168.0.11/jobs 114464 105196 9268 92%/mnt/backups/montaz/jobsls
/mnt/backups/montaz/jobs/
ls: reading directory/mnt/backups/montaz/jobs/: Input/output error total 0the fun thing is that i can cd to a lower level directory, and ls works
fine there! only the mount point has the problemls
/mnt/backups/montaz/jobs/test
total 44K
drwxr-xr-x 1 root root 0 Apr 30 2010 blah blah/
......Quoted 7 lines
kernel version 3.2rc7
this seems to be related to :
https://lkml.org/lkml/2011/8/1/427
Re: [3.0.0+][Regression][Bisected] CIFS:getdents()broken for large dirsHmmm, maybe. What makes you think that it's related? What sort of server are you seeing this against?
Quoted 8 lines
Quoted 7 lines
Quoted 5 lines
Quoted 1 line
Windows XP service pack 2 (greek)
How many files are in the directory?
140 folders and 20 files
Attached is a tcp dump of my session.
I tried reproducing this here, but wasn't able to. Testing against my xp box worked fine.
Most likely, the
FIND_FILEresponses are falling afoul of the code incoalesce_t2or check2ndT2. Unfortunately that code is pretty complicated and I'm not certain what the problem actually is...Quoted 5 lines
One thing that's interesting is that the total data being sent in the request is rather large (16336 bytes). I think that's legit, but maybe it's exceeding the end of the buffer once we try to coalesce it.
Would it be possible to get the cFYI output from this test?
I did not get a cFYI output from that test, but i redid a mount-ls-umount and am attaching the tcpdump
Also here http://pastebin.com/J20uC6kU you can find the cifsFYI and the contents of/proc/fs/cifs/DebugDataform the same testQuoted 2 lines
Quoted 8 lines
Quoted 3 lines
Is this a regression? Did it work with earlier kernels and only recently start failing?
I do not know, and i am a bit afraid to downgrade this machine below 3.0 due to some changes arch linux has introduced recently. I can always set up a few virtual machines though, and i can even request permission from my company to give you shell access if you like. Which kernel versions would you like me to test?
Quoted 9 lines
I just tested 3.1.5-1-ARCH on a virtual machine and it works, so it is probably a regression... On the same virtual machine 3.2-rc7 produces input/output error. The virtual machine is a fresh install of arch linux. here is the relevant pastebin http://pastebin.com/BwX2DqJC
and attached is the tcpdump
As a am a noob to all this, what should I do next in order to help you? maybe compile a 3.1 kernel from official sources to make sure no patches from arch linux interfere?
It is the same request from the cifs client in both the successful and unsuccessful cases. trans2, findfirst2, infolevel of 261 and search count of 150.
So nothing has changed from what is emanating from cifs client.
In the error case, Windows XP responds with search count of 142 whereas in successful case, Windows XP responds with search count of significantly less number, e.g. 36.
Are files/directories in jobs directory less now i.e. has anything changed in this directory on the server 192.168.0.11 i.e. have files been deleted, directories removed etc. since the failure was noticed the very first time?
Quoted 100 lines · Shirish Pargaonkar
Quoted 84 lines · Konstantinos Skarlatos
Quoted 75 lines · Konstantinos Skarlatos
Quoted 64 lines · Jeff Layton
Quoted 47 lines · On Fri, 30 Dec 2011 11:04:59 +0200 Konstantinos…
Quoted 46 lines · Konstantinos Skarlatos
Quoted 45 lines · Jeff Layton
Quoted 42 lines · On Thu, 29 Dec 2011 12:30:18 +0200 Konstantinos…
Quoted 41 lines · Jeff Layton
Quoted 29 lines · On Thu, 29 Dec 2011 02:08:57 +0200 Konstantinos…
I mount via cifs a windows XP share, df gives me correct sizes, but when
I ls the mount point i get input/output error.
strace: http://pastebin.com/WXf8M1numount --verbose -t cifs -o username=administrator,password=blahblah //192.168.0.11/jobs
/mnt/backups/montaz/jobs
mount.cifs kernel mount options:ip=192.168.0.11,unc=\\192.168.0.11\jobs,,ver=1,user=administrator,pass=********
df
//192.168.0.11/jobs 114464
105196 9268 92%/mnt/backups/montaz/jobsls
/mnt/backups/montaz/jobs/
ls: reading directory/mnt/backups/montaz/jobs/: Input/output error total 0the fun thing is that i can cd to a lower level directory, and ls works
fine there! only the mount point has the problemls
/mnt/backups/montaz/jobs/test
total 44K
drwxr-xr-x 1 root root 0 Apr 30 2010 blah blah/
......Quoted 7 lines
kernel version 3.2rc7
this seems to be related to :
https://lkml.org/lkml/2011/8/1/427
Re: [3.0.0+][Regression][Bisected] CIFS:getdents()broken for large dirsHmmm, maybe. What makes you think that it's related? What sort of server are you seeing this against?
Quoted 7 lines
Quoted 6 lines
Quoted 4 lines
Quoted 1 line
Windows XP service pack 2 (greek)
How many files are in the directory?
140 folders and 20 files
Attached is a tcp dump of my session.
I tried reproducing this here, but wasn't able to. Testing against my xp box worked fine.
Most likely, the
FIND_FILEresponses are falling afoul of the code incoalesce_t2or check2ndT2. Unfortunately that code is pretty complicated and I'm not certain what the problem actually is...Quoted 5 lines
One thing that's interesting is that the total data being sent in the request is rather large (16336 bytes). I think that's legit, but maybe it's exceeding the end of the buffer once we try to coalesce it.
Would it be possible to get the cFYI output from this test?
I did not get a cFYI output from that test, but i redid a mount-ls-umount and am attaching the tcpdump
Also here http://pastebin.com/J20uC6kU you can find the cifsFYI and the contents of/proc/fs/cifs/DebugDataform the same testQuoted 1 line
Quoted 8 lines
Quoted 3 lines
Is this a regression? Did it work with earlier kernels and only recently start failing?
I do not know, and i am a bit afraid to downgrade this machine below 3.0 due to some changes arch linux has introduced recently. I can always set up a few virtual machines though, and i can even request permission from my company to give you shell access if you like. Which kernel versions would you like me to test?
Quoted 9 lines
I just tested 3.1.5-1-ARCH on a virtual machine and it works, so it is probably a regression... On the same virtual machine 3.2-rc7 produces input/output error. The virtual machine is a fresh install of arch linux. here is the relevant pastebin http://pastebin.com/BwX2DqJC
and attached is the tcpdump
As a am a noob to all this, what should I do next in order to help you? maybe compile a 3.1 kernel from official sources to make sure no patches from arch linux interfere?It is the same request from the cifs client in both the successful and unsuccessful cases. trans2, findfirst2, infolevel of 261 and search count of 150.
So nothing has changed from what is emanating from cifs client.
Quoted 7 lines
In the error case, Windows XP responds with search count of 142 whereas in successful case, Windows XP responds with search count of significantly less number, e.g. 36.
Are files/directories in jobs directory less now i.e. has anything changed in this directory on the server 192.168.0.11 i.e. have files been deleted, directories removed etc. since the failure was noticed the very first time?
there are some changes in the jobs directory since the failure was noticed, but they are quite minor. at most one folder was deleted and one was added, and the pdf files in the root dir (1_B_MAYRO.pdf 1XR.pdf ....7_B_MAYRO.pdf ) were overwritten, +- one or two files.
Quoted 70 lines · Konstantinos Skarlatos
Quoted 69 lines · Shirish Pargaonkar
Quoted 67 lines · Konstantinos Skarlatos
Quoted 66 lines · Konstantinos Skarlatos
Quoted 65 lines · Jeff Layton
Quoted 48 lines · On Fri, 30 Dec 2011 11:04:59 +0200 Konstantinos…
Quoted 47 lines · Konstantinos Skarlatos
Quoted 46 lines · Jeff Layton
Quoted 43 lines · On Thu, 29 Dec 2011 12:30:18 +0200 Konstantinos…
Quoted 42 lines · Jeff Layton
Quoted 30 lines · On Thu, 29 Dec 2011 02:08:57 +0200 Konstantinos…
I mount via cifs a windows XP share, df gives me correct sizes, but when
I ls the mount point i get input/output error.
strace: http://pastebin.com/WXf8M1numount --verbose -t cifs -o username=administrator,password=blahblah //192.168.0.11/jobs
/mnt/backups/montaz/jobs
mount.cifs kernel mount options:ip=192.168.0.11,unc=\\192.168.0.11\jobs,,ver=1,user=administrator,pass=********
df
//192.168.0.11/jobs 114464
105196 9268 92%/mnt/backups/montaz/jobsls
/mnt/backups/montaz/jobs/
ls: reading directory/mnt/backups/montaz/jobs/: Input/output error total 0the fun thing is that i can cd to a lower level directory, and ls works
fine there! only the mount point has the problemls
/mnt/backups/montaz/jobs/test
total 44K
drwxr-xr-x 1 root root 0 Apr 30 2010 blah blah/
......Quoted 7 lines
kernel version 3.2rc7
this seems to be related to :
https://lkml.org/lkml/2011/8/1/427
Re: [3.0.0+][Regression][Bisected] CIFS:getdents()broken for large dirsHmmm, maybe. What makes you think that it's related? What sort of server are you seeing this against?
Quoted 7 lines
Quoted 6 lines
Quoted 4 lines
Quoted 1 line
Windows XP service pack 2 (greek)
How many files are in the directory?
140 folders and 20 files
Attached is a tcp dump of my session.
I tried reproducing this here, but wasn't able to. Testing against my xp box worked fine.
Most likely, the
FIND_FILEresponses are falling afoul of the code incoalesce_t2or check2ndT2. Unfortunately that code is pretty complicated and I'm not certain what the problem actually is...
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 5 lines
One thing that's interesting is that the total data being sent in the request is rather large (16336 bytes). I think that's legit, but maybe it's exceeding the end of the buffer once we try to coalesce it.
Would it be possible to get the cFYI output from this test?
I did not get a cFYI output from that test, but i redid a mount-ls-umount and am attaching the tcpdump
Also here http://pastebin.com/J20uC6kU you can find the cifsFYI and the
i forgot to mention that since i got this pastebin and the attached tcpdump the contents of jobs have not changed at all, so you can compare with these if you like
Quoted 33 lines
Quoted 24 lines
Quoted 10 lines
Quoted 10 lines
contents of
/proc/fs/cifs/DebugDataform the same testQuoted 4 lines
Is this a regression? Did it work with earlier kernels and only recently start failing?
I do not know, and i am a bit afraid to downgrade this machine below 3.0 due to some changes arch linux has introduced recently. I can always set up a few virtual machines though, and i can even request permission from my company to give you shell access if you like. Which kernel versions would you like me to test?
Quoted 9 lines
I just tested 3.1.5-1-ARCH on a virtual machine and it works, so it is probably a regression... On the same virtual machine 3.2-rc7 produces input/output error. The virtual machine is a fresh install of arch linux. here is the relevant pastebin http://pastebin.com/BwX2DqJC
and attached is the tcpdump
As a am a noob to all this, what should I do next in order to help you? maybe compile a 3.1 kernel from official sources to make sure no patches from arch linux interfere?It is the same request from the cifs client in both the successful and unsuccessful cases. trans2, findfirst2, infolevel of 261 and search count of 150.
So nothing has changed from what is emanating from cifs client.Quoted 7 lines
In the error case, Windows XP responds with search count of 142 whereas in successful case, Windows XP responds with search count of significantly less number, e.g. 36.
Are files/directories in jobs directory less now i.e. has anything changed in this directory on the server 192.168.0.11 i.e. have files been deleted, directories removed etc. since the failure was noticed the very first time?
there are some changes in the jobs directory since the failure was noticed, but they are quite minor. at most one folder was deleted and one was added, and the pdf files in the root dir (1_B_MAYRO.pdf 1XR.pdf ....7_B_MAYRO.pdf ) were overwritten, +- one or two files.
Quoted 80 lines · Konstantinos Skarlatos
Quoted 79 lines · Shirish Pargaonkar
Quoted 77 lines · Konstantinos Skarlatos
Quoted 76 lines · Konstantinos Skarlatos
Quoted 75 lines · Jeff Layton
Quoted 53 lines · On Fri, 30 Dec 2011 11:04:59 +0200 Konstantinos…
Quoted 52 lines · Konstantinos Skarlatos
Quoted 51 lines · Jeff Layton
Quoted 48 lines · On Thu, 29 Dec 2011 12:30:18 +0200 Konstantinos…
Quoted 47 lines · Jeff Layton
Quoted 34 lines · On Thu, 29 Dec 2011 02:08:57 +0200 Konstantinos…
I mount via cifs a windows XP share, df gives me correct sizes, but when
I ls the mount point i get input/output error.
strace: http://pastebin.com/WXf8M1numount --verbose -t cifs -o
username=administrator,password=blahblah
//192.168.0.11/jobs/mnt/backups/montaz/jobs
mount.cifs kernel mount options:ip=192.168.0.11,unc=\\192.168.0.11\jobs,,ver=1,user=administrator,pass=********
df
//192.168.0.11/jobs
114464
105196 9268 92%/mnt/backups/montaz/jobsls
/mnt/backups/montaz/jobs/
ls: reading directory/mnt/backups/montaz/jobs/:
Input/output error
total 0the fun thing is that i can cd to a lower level directory, and ls
works
fine there! only the mount point has the problemls
/mnt/backups/montaz/jobs/test
total 44K
drwxr-xr-x 1 root root 0 Apr 30 2010 blah blah/
......Quoted 7 lines
kernel version 3.2rc7
this seems to be related to :
https://lkml.org/lkml/2011/8/1/427
Re: [3.0.0+][Regression][Bisected] CIFS:getdents()broken for large dirsHmmm, maybe. What makes you think that it's related? What sort of
server are you seeing this against?Quoted 7 lines
Quoted 6 lines
Quoted 4 lines
Quoted 1 line
Windows XP service pack 2 (greek)
How many files are in the directory?
140 folders and 20 files
Attached is a tcp dump of my session.
I tried reproducing this here, but wasn't able to. Testing against my xp box worked fine.
Most likely, the
FIND_FILEresponses are falling afoul of the code incoalesce_t2or check2ndT2. Unfortunately that code is pretty complicated and I'm not certain what the problem actually is...One thing that's interesting is that the total data being sent in the request is rather large (16336 bytes). I think that's legit, but maybe
it's exceeding the end of the buffer once we try to coalesce it.
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 1 line
Would it be possible to get the cFYI output from this test?
I did not get a cFYI output from that test, but i redid a mount-ls-umount
and am attaching the tcpdump
Also here http://pastebin.com/J20uC6kU you can find the cifsFYI and the
i forgot to mention that since i got this pastebin and the attached tcpdump the contents of jobs have not changed at all, so you can compare with these if you like
Quoted 42 lines
Quoted 36 lines
Quoted 14 lines
Quoted 5 lines
contents of
/proc/fs/cifs/DebugDataform the same testQuoted 4 lines
Is this a regression? Did it work with earlier kernels and only recently start failing?
Quoted 9 lines
I do not know, and i am a bit afraid to downgrade this machine below 3.0
due to some changes arch linux has introduced recently. I can always set up
a few virtual machines though, and i can even request permission from my
company to give you shell access if you like. Which kernel versions would
you like me to test?Quoted 11 lines
I just tested 3.1.5-1-ARCH on a virtual machine and it works, so it is probably a regression... On the same virtual machine 3.2-rc7 produces input/output error. The virtual machine is a fresh install of arch linux.
here is the relevant pastebin http://pastebin.com/BwX2DqJC
and attached is the tcpdump
As a am a noob to all this, what should I do next in order to help you? maybe compile a 3.1 kernel from official sources to make sure no patches
from arch linux interfere?It is the same request from the cifs client in both the successful and unsuccessful cases. trans2, findfirst2, infolevel of 261 and search count
of 150.
So nothing has changed from what is emanating from cifs client.In the error case, Windows XP responds with search count of 142 whereas in successful case, Windows XP responds with search count of significantly
less number, e.g. 36.Quoted 6 lines
Are files/directories in jobs directory less now i.e. has anything changed
in this directory on the server 192.168.0.11 i.e. have files been deleted,
directories removed etc. since the failure was noticed the very first time?
Quoted 5 lines
there are some changes in the jobs directory since the failure was noticed, but they are quite minor. at most one folder was deleted and one was added, and the pdf files in the root dir (1_B_MAYRO.pdf 1XR.pdf ....7_B_MAYRO.pdf ) were overwritten, +- one or two files.
(resent as non-html)
Quoted 86 lines · Konstantinos Skarlatos
Quoted 77 lines · Konstantinos Skarlatos
Quoted 66 lines · Jeff Layton
Quoted 49 lines · On Fri, 30 Dec 2011 11:04:59 +0200 Konstantinos…
Quoted 48 lines · Konstantinos Skarlatos
Quoted 47 lines · Jeff Layton
Quoted 44 lines · On Thu, 29 Dec 2011 12:30:18 +0200 Konstantinos…
Quoted 43 lines · Jeff Layton
Quoted 31 lines · On Thu, 29 Dec 2011 02:08:57 +0200 Konstantinos…
I mount via cifs a windows XP share, df gives me correct sizes, but when
I ls the mount point i get input/output error.
strace: http://pastebin.com/WXf8M1numount --verbose -t cifs -o username=administrator,password=blahblah
//192.168.0.11/jobs/mnt/backups/montaz/jobs
mount.cifs kernel mount options:
ip=192.168.0.11,unc=\\192.168.0.11\jobs,,ver=1,user=administrator,pass=********df
//192.168.0.11/jobs 114464
105196 9268 92%/mnt/backups/montaz/jobsls
/mnt/backups/montaz/jobs/
ls: reading directory/mnt/backups/montaz/jobs/: Input/output error
total 0the fun thing is that i can cd to a lower level directory, and ls works
fine there! only the mount point has the problemls
/mnt/backups/montaz/jobs/test
total 44K
drwxr-xr-x 1 root root 0 Apr 30 2010 blah blah/
......Quoted 7 lines
kernel version 3.2rc7
this seems to be related to :
https://lkml.org/lkml/2011/8/1/427
Re: [3.0.0+][Regression][Bisected] CIFS:getdents()broken for large dirsHmmm, maybe. What makes you think that it's related? What sort of server are you seeing this against?
Quoted 7 lines
Quoted 6 lines
Quoted 4 lines
Quoted 1 line
Windows XP service pack 2 (greek)
How many files are in the directory?
140 folders and 20 files
Attached is a tcp dump of my session.
I tried reproducing this here, but wasn't able to. Testing against my xp box worked fine.
Most likely, the
FIND_FILEresponses are falling afoul of the code incoalesce_t2or check2ndT2. Unfortunately that code is pretty complicated and I'm not certain what the problem actually is...Quoted 5 lines
One thing that's interesting is that the total data being sent in the request is rather large (16336 bytes). I think that's legit, but maybe it's exceeding the end of the buffer once we try to coalesce it.
Would it be possible to get the cFYI output from this test?
I did not get a cFYI output from that test, but i redid a mount-ls-umount and am attaching the tcpdump
Also here http://pastebin.com/J20uC6kU you can find the cifsFYI and the contents of/proc/fs/cifs/DebugDataform the same testQuoted 1 line
Quoted 8 lines
Quoted 3 lines
Is this a regression? Did it work with earlier kernels and only recently start failing?
I do not know, and i am a bit afraid to downgrade this machine below 3.0 due to some changes arch linux has introduced recently. I can always set up a few virtual machines though, and i can even request permission from my company to give you shell access if you like. Which kernel versions would you like me to test?
Quoted 9 lines
I just tested 3.1.5-1-ARCH on a virtual machine and it works, so it is probably a regression... On the same virtual machine 3.2-rc7 produces input/output error. The virtual machine is a fresh install of arch linux. here is the relevant pastebin http://pastebin.com/BwX2DqJC
and attached is the tcpdump
As a am a noob to all this, what should I do next in order to help you? maybe compile a 3.1 kernel from official sources to make sure no patches from arch linux interfere?
After some testing, vanilla 3.1.6 does not have this bug and 3.2.0-rc1 has it.
Quoted 109 lines · Konstantinos Skarlatos
Quoted 99 lines · Konstantinos Skarlatos
Quoted 88 lines · Konstantinos Skarlatos
Quoted 74 lines · Jeff Layton
Quoted 54 lines · On Fri, 30 Dec 2011 11:04:59 +0200 Konstantinos…
Quoted 53 lines · Konstantinos Skarlatos
Quoted 51 lines · Jeff Layton
Quoted 47 lines · On Thu, 29 Dec 2011 12:30:18 +0200 Konstantinos…
Quoted 45 lines · Jeff Layton
Quoted 32 lines · On Thu, 29 Dec 2011 02:08:57 +0200 Konstantinos…
I mount via cifs a windows XP share, df gives me correct sizes, but when
I ls the mount point i get input/output error.
strace: http://pastebin.com/WXf8M1numount --verbose -t cifs -o
username=administrator,password=blahblah
//192.168.0.11/jobs/mnt/backups/montaz/jobs
mount.cifs kernel mount options:ip=192.168.0.11,unc=\\192.168.0.11\jobs,,ver=1,user=administrator,pass=********
df
//192.168.0.11/jobs 114464
105196 9268 92%/mnt/backups/montaz/jobsls
/mnt/backups/montaz/jobs/
ls: reading directory/mnt/backups/montaz/jobs/: Input/output error
total 0the fun thing is that i can cd to a lower level directory, and ls works
fine there! only the mount point has the problemls
/mnt/backups/montaz/jobs/test
total 44K
drwxr-xr-x 1 root root 0 Apr 30 2010 blah blah/
......Quoted 7 lines
kernel version 3.2rc7
this seems to be related to :
https://lkml.org/lkml/2011/8/1/427
Re: [3.0.0+][Regression][Bisected] CIFS:getdents()broken for large dirsHmmm, maybe. What makes you think that it's related? What sort of server are you seeing this against?
Quoted 8 lines
Quoted 7 lines
Quoted 5 lines
Quoted 1 line
Windows XP service pack 2 (greek)
How many files are in the directory?
140 folders and 20 files
Attached is a tcp dump of my session.
I tried reproducing this here, but wasn't able to. Testing against my xp box worked fine.
Most likely, the
FIND_FILEresponses are falling afoul of the code incoalesce_t2or check2ndT2. Unfortunately that code is pretty complicated and I'm not certain what the problem actually is...Quoted 5 lines
One thing that's interesting is that the total data being sent in the request is rather large (16336 bytes). I think that's legit, but maybe it's exceeding the end of the buffer once we try to coalesce it.
Would it be possible to get the cFYI output from this test?
I did not get a cFYI output from that test, but i redid a mount-ls-umount and am attaching the tcpdump
Also here http://pastebin.com/J20uC6kU you can find the cifsFYI and the contents of/proc/fs/cifs/DebugDataform the same testQuoted 2 lines
Quoted 8 lines
Quoted 3 lines
Is this a regression? Did it work with earlier kernels and only recently start failing?
I do not know, and i am a bit afraid to downgrade this machine below 3.0 due to some changes arch linux has introduced recently. I can always set up a few virtual machines though, and i can even request permission from my company to give you shell access if you like. Which kernel versions would you like me to test?
Quoted 9 lines
I just tested 3.1.5-1-ARCH on a virtual machine and it works, so it is probably a regression... On the same virtual machine 3.2-rc7 produces input/output error. The virtual machine is a fresh install of arch linux. here is the relevant pastebin http://pastebin.com/BwX2DqJC
and attached is the tcpdump
As a am a noob to all this, what should I do next in order to help you? maybe compile a 3.1 kernel from official sources to make sure no patches from arch linux interfere?
Quoted 2 lines
After some testing, vanilla 3.1.6 does not have this bug and 3.2.0-rc1 has it.
I suspect 3.2.0-rc1 cifs will work correctly against a Windows 2003 or Windows 7 or Window 2008 server.
Quoted 109 lines · Konstantinos Skarlatos
Quoted 99 lines · Konstantinos Skarlatos
Quoted 88 lines · Konstantinos Skarlatos
Quoted 74 lines · Jeff Layton
Quoted 54 lines · On Fri, 30 Dec 2011 11:04:59 +0200 Konstantinos…
Quoted 53 lines · Konstantinos Skarlatos
Quoted 51 lines · Jeff Layton
Quoted 47 lines · On Thu, 29 Dec 2011 12:30:18 +0200 Konstantinos…
Quoted 45 lines · Jeff Layton
Quoted 32 lines · On Thu, 29 Dec 2011 02:08:57 +0200 Konstantinos…
I mount via cifs a windows XP share, df gives me correct sizes, but when
I ls the mount point i get input/output error.
strace: http://pastebin.com/WXf8M1numount --verbose -t cifs -o
username=administrator,password=blahblah
//192.168.0.11/jobs/mnt/backups/montaz/jobs
mount.cifs kernel mount options:ip=192.168.0.11,unc=\\192.168.0.11\jobs,,ver=1,user=administrator,pass=********
df
//192.168.0.11/jobs 114464
105196 9268 92%/mnt/backups/montaz/jobsls
/mnt/backups/montaz/jobs/
ls: reading directory/mnt/backups/montaz/jobs/: Input/output error
total 0the fun thing is that i can cd to a lower level directory, and ls works
fine there! only the mount point has the problemls
/mnt/backups/montaz/jobs/test
total 44K
drwxr-xr-x 1 root root 0 Apr 30 2010 blah blah/
......Quoted 7 lines
kernel version 3.2rc7
this seems to be related to :
https://lkml.org/lkml/2011/8/1/427
Re: [3.0.0+][Regression][Bisected] CIFS:getdents()broken for large dirsHmmm, maybe. What makes you think that it's related? What sort of server are you seeing this against?
Quoted 8 lines
Quoted 7 lines
Quoted 5 lines
Quoted 1 line
Windows XP service pack 2 (greek)
How many files are in the directory?
140 folders and 20 files
Attached is a tcp dump of my session.
I tried reproducing this here, but wasn't able to. Testing against my xp box worked fine.
Most likely, the
FIND_FILEresponses are falling afoul of the code incoalesce_t2or check2ndT2. Unfortunately that code is pretty complicated and I'm not certain what the problem actually is...Quoted 5 lines
One thing that's interesting is that the total data being sent in the request is rather large (16336 bytes). I think that's legit, but maybe it's exceeding the end of the buffer once we try to coalesce it.
Would it be possible to get the cFYI output from this test?
I did not get a cFYI output from that test, but i redid a mount-ls-umount and am attaching the tcpdump
Also here http://pastebin.com/J20uC6kU you can find the cifsFYI and the contents of/proc/fs/cifs/DebugDataform the same testQuoted 2 lines
Quoted 8 lines
Quoted 3 lines
Is this a regression? Did it work with earlier kernels and only recently start failing?
I do not know, and i am a bit afraid to downgrade this machine below 3.0 due to some changes arch linux has introduced recently. I can always set up a few virtual machines though, and i can even request permission from my company to give you shell access if you like. Which kernel versions would you like me to test?
Quoted 9 lines
I just tested 3.1.5-1-ARCH on a virtual machine and it works, so it is probably a regression... On the same virtual machine 3.2-rc7 produces input/output error. The virtual machine is a fresh install of arch linux. here is the relevant pastebin http://pastebin.com/BwX2DqJC
and attached is the tcpdump
As a am a noob to all this, what should I do next in order to help you? maybe compile a 3.1 kernel from official sources to make sure no patches from arch linux interfere?
Quoted 2 lines
After some testing, vanilla 3.1.6 does not have this bug and 3.2.0-rc1 has it.
different responses by Windows XP is causing the error. cifs does not send different requests in 3.16 and 3.2.0 (to receive different responses from Windows XP).
I have 3.1.0-1-default-1+ and I see this error against a Windows XP share when number of file system objects are large in the directory.
Quoted 74 lines · On Fri, 30 Dec 2011 20:00:46 +0200 Konstantinos Skarlatos…
Quoted 63 lines · Jeff Layton
Quoted 46 lines · On Fri, 30 Dec 2011 11:04:59 +0200 Konstantinos…
Quoted 45 lines · Konstantinos Skarlatos
Quoted 44 lines · Jeff Layton
Quoted 41 lines · On Thu, 29 Dec 2011 12:30:18 +0200 Konstantinos…
Quoted 40 lines · Jeff Layton
Quoted 28 lines · On Thu, 29 Dec 2011 02:08:57 +0200 Konstantinos…
I mount via cifs a windows XP share, df gives me correct sizes, but when
I ls the mount point i get input/output error.
strace: http://pastebin.com/WXf8M1numount --verbose -t cifs -o username=administrator,password=blahblah //192.168.0.11/jobs
/mnt/backups/montaz/jobs
mount.cifs kernel mount options:
ip=192.168.0.11,unc=\\192.168.0.11\jobs,,ver=1,user=administrator,pass=********df
//192.168.0.11/jobs 114464
105196 9268 92%/mnt/backups/montaz/jobsls
/mnt/backups/montaz/jobs/
ls: reading directory/mnt/backups/montaz/jobs/: Input/output error total 0the fun thing is that i can cd to a lower level directory, and ls works
fine there! only the mount point has the problemls
/mnt/backups/montaz/jobs/test
total 44K
drwxr-xr-x 1 root root 0 Apr 30 2010 blah blah/
......Quoted 7 lines
kernel version 3.2rc7
this seems to be related to :
https://lkml.org/lkml/2011/8/1/427
Re: [3.0.0+][Regression][Bisected] CIFS:getdents()broken for large dirsHmmm, maybe. What makes you think that it's related? What sort of server are you seeing this against?
Quoted 7 lines
Quoted 6 lines
Quoted 4 lines
Quoted 1 line
Windows XP service pack 2 (greek)
How many files are in the directory?
140 folders and 20 files
Attached is a tcp dump of my session.
I tried reproducing this here, but wasn't able to. Testing against my xp box worked fine.
Most likely, the
FIND_FILEresponses are falling afoul of the code incoalesce_t2or check2ndT2. Unfortunately that code is pretty complicated and I'm not certain what the problem actually is...Quoted 5 lines
One thing that's interesting is that the total data being sent in the request is rather large (16336 bytes). I think that's legit, but maybe it's exceeding the end of the buffer once we try to coalesce it.
Would it be possible to get the cFYI output from this test?
I did not get a cFYI output from that test, but i redid a mount-ls-umount and am attaching the tcpdump
Also here http://pastebin.com/J20uC6kU you can find the cifsFYI and the contents of/proc/fs/cifs/DebugDataform the same testQuoted 1 line
Quoted 8 lines
Quoted 3 lines
Is this a regression? Did it work with earlier kernels and only recently start failing?
I do not know, and i am a bit afraid to downgrade this machine below 3.0 due to some changes arch linux has introduced recently. I can always set up a few virtual machines though, and i can even request permission from my company to give you shell access if you like. Which kernel versions would you like me to test?
Ok, that tells us a little:
-------------------[snip]---------------------
[96268.787078] fs/cifs/cifssmb.c: In FindFirst for [96268.787083] fs/cifs/transport.c: For smb_command 50 [96268.787086] fs/cifs/transport.c: Sending smb: total_len 88
...FIND_FIRST command is sent
[96268.787690] fs/cifs/connect.c: RFC1002 header 0x1104 [96268.787697] fs/cifs/connect.c: missing 12048 bytes from transact2, check next response [96268.787865] fs/cifs/connect.c: RFC1002 header 0x1104 [96268.787870] fs/cifs/connect.c: missing 12036 bytes from transact2, check next response [96268.788037] fs/cifs/connect.c: RFC1002 header 0x1104 [96268.788042] fs/cifs/connect.c: missing 12036 bytes from transact2, check next response [96268.788371] fs/cifs/connect.c: RFC1002 header 0xdb0 [96268.788375] fs/cifs/connect.c: missing 12888 bytes from transact2, check next response
...all four parts of the response are collected here
[96268.788391] fs/cifs/transport.c: cifs_sync_mid_result: cmd=50 mid=12 state=16
...but the state at this point is MID_RESPONSE_MALFORMED
[96268.788395] fs/cifs/cifssmb.c: Error in FindFirst = -5 [96268.788397] fs/cifs/readdir.c: initiate cifs search rc -5 [96268.788398] fs/cifs/readdir.c: CIFS VFS: leaving cifs_readdir (xid = 737644) rc = -5
...which makes readdir return -EIO
-------------------[snip]---------------------
Based on that, it looks like something in one of these frames caused coalesce_t2() to return an error. I don't see the problem right offhand in the capture, but T2 response handling is pretty complex so it can be hard to see.
Would it be possible for you to rebuild your kernel (or just cifs.ko) with this patch? Once you do that, rerun the test with cFYI turned up, and it should help point out what the problem is.
Thanks,
Signature · 1 line
Jeff Layton <[email protected]>
Quoted 1 line
From
3ae6fb6404d150c5ac026b7b16cd1c6b17e6f1adMon Sep 17 00:00:00 2001
From: Jeff Layton <[email protected]>
Date: Sat, 31 Dec 2011 06:53:26 -0500
Subject: [PATCH] cifs: better instrumentation for coalesce_t2
When coalesce_t2 returns an error, have it throw a cFYI message that explains the reason.
--- fs/cifs/connect.c | 29 +++++++++++++++++++++-------- 1 files changed, 21 insertions(+), 8 deletions(-)
fs/cifs/connect.c +21 -8
@@ -243,16 +243,21 @@ static int coalesce_t2(struct smb_hdr *psecond, struct smb_hdr *pTargetSMB) remaining = total_data_size - total_in_buf; -if (remaining <0 ) +if (remaining <0 ) { +cFYI (1 ,"Server sent too much data. total_data_size=%hu " +"total_in_buf=%hu" , total_data_size, total_in_buf);return -EPROTO ; + } -if (remaining ==0 )/* nothing to do, ignore */ +if (remaining ==0 ) { +/* nothing to do, ignore */ +cFYI (1 ,"no more data remains" );return 0 ; + } total_in_buf2 =get_unaligned_le16 (&pSMB2->t2_rsp.DataCount); -if (remaining < total_in_buf2) { +if (remaining < total_in_buf2)cFYI (1 ,"transact2 2nd response contains too much data" ); - }/* find end of first SMB data area */ data_area_of_target = (char *)&pSMBt->hdr.Protocol +
@@ -267,23 +272,29 @@ static int coalesce_t2(struct smb_hdr *psecond, struct smb_hdr *pTargetSMB)/* copy second buffer into end of first buffer */ total_in_buf += total_in_buf2;/* is the result too big for the field? */ -if (total_in_buf >USHRT_MAX ) +if (total_in_buf >USHRT_MAX ) { +cFYI (1 ,"coalesced total_in_buf too large (%u)" , total_in_buf);return -EPROTO ; + }put_unaligned_le16 (total_in_buf, &pSMBt->t2_rsp.DataCount);/* fix up the BCC */ byte_count =get_bcc (pTargetSMB); byte_count += total_in_buf2;/* is the result too big for the field? */ -if (byte_count >USHRT_MAX ) +if (byte_count >USHRT_MAX ) { +cFYI (1 ,"coalesced BCC too large (%u)" , byte_count);return -EPROTO ; + }put_bcc (byte_count, pTargetSMB); byte_count =be32_to_cpu (pTargetSMB->smb_buf_length); byte_count += total_in_buf2;/* don't allow buffer to overflow */ -if (byte_count > CIFSMaxBufSize) +if (byte_count > CIFSMaxBufSize) { +cFYI (1 ,"coalesced BCC exceeds buffer size (%u)" , byte_count);return -ENOBUFS ; + } pTargetSMB->smb_buf_length =cpu_to_be32 (byte_count);memcpy (data_area_of_target, data_area_of_buf2, total_in_buf2);
@@ -291,8 +302,10 @@ static int coalesce_t2(struct smb_hdr *psecond, struct smb_hdr *pTargetSMB)if (remaining == total_in_buf2) {cFYI (1 ,"found the last secondary response" );return 0 ;/* we are done */ - }else /* more responses to go */ + }else {/* more responses to go */ +cFYI (1 ,"waiting for more secondary responses" );return 1 ; + } } static void
Signature · 1 line
1.7.7.4
Quoted 124 lines · Jeff Layton
Quoted 74 lines · On Fri, 30 Dec 2011 20:00:46 +0200 Konstantinos…
Quoted 63 lines · Jeff Layton
Quoted 46 lines · On Fri, 30 Dec 2011 11:04:59 +0200 Konstantinos…
Quoted 45 lines · Konstantinos Skarlatos
Quoted 44 lines · Jeff Layton
Quoted 41 lines · On Thu, 29 Dec 2011 12:30:18 +0200 Konstantinos…
Quoted 40 lines · Jeff Layton
Quoted 28 lines · On Thu, 29 Dec 2011 02:08:57 +0200 Konstantinos…
I mount via cifs a windows XP share, df gives me correct sizes, but when
I ls the mount point i get input/output error.
strace: http://pastebin.com/WXf8M1numount --verbose -t cifs -o username=administrator,password=blahblah //192.168.0.11/jobs
/mnt/backups/montaz/jobs
mount.cifs kernel mount options:
ip=192.168.0.11,unc=\\192.168.0.11\jobs,,ver=1,user=administrator,pass=********df
//192.168.0.11/jobs 114464
105196 9268 92%/mnt/backups/montaz/jobsls
/mnt/backups/montaz/jobs/
ls: reading directory/mnt/backups/montaz/jobs/: Input/output error total 0the fun thing is that i can cd to a lower level directory, and ls works
fine there! only the mount point has the problemls
/mnt/backups/montaz/jobs/test
total 44K
drwxr-xr-x 1 root root 0 Apr 30 2010 blah blah/
......Quoted 7 lines
kernel version 3.2rc7
this seems to be related to :
https://lkml.org/lkml/2011/8/1/427
Re: [3.0.0+][Regression][Bisected] CIFS:getdents()broken for large dirsHmmm, maybe. What makes you think that it's related? What sort of server are you seeing this against?
Quoted 7 lines
Quoted 6 lines
Quoted 4 lines
Quoted 1 line
Windows XP service pack 2 (greek)
How many files are in the directory?
140 folders and 20 files
Attached is a tcp dump of my session.
I tried reproducing this here, but wasn't able to. Testing against my xp box worked fine.
Most likely, the
FIND_FILEresponses are falling afoul of the code incoalesce_t2or check2ndT2. Unfortunately that code is pretty complicated and I'm not certain what the problem actually is...Quoted 5 lines
One thing that's interesting is that the total data being sent in the request is rather large (16336 bytes). I think that's legit, but maybe it's exceeding the end of the buffer once we try to coalesce it.
Would it be possible to get the cFYI output from this test?
I did not get a cFYI output from that test, but i redid a mount-ls-umount and am attaching the tcpdump
Also here http://pastebin.com/J20uC6kU you can find the cifsFYI and the contents of/proc/fs/cifs/DebugDataform the same testQuoted 1 line
Quoted 8 lines
Quoted 3 lines
Is this a regression? Did it work with earlier kernels and only recently start failing?
I do not know, and i am a bit afraid to downgrade this machine below 3.0 due to some changes arch linux has introduced recently. I can always set up a few virtual machines though, and i can even request permission from my company to give you shell access if you like. Which kernel versions would you like me to test?
Ok, that tells us a little:
-------------------[snip]---------------------
[96268.787078] fs/cifs/cifssmb.c: In FindFirst for [96268.787083] fs/cifs/transport.c: For smb_command 50 [96268.787086] fs/cifs/transport.c: Sending smb: total_len 88...FIND_FIRST command is sent
[96268.787690] fs/cifs/connect.c: RFC1002 header 0x1104 [96268.787697] fs/cifs/connect.c: missing 12048 bytes from transact2, check next response [96268.787865] fs/cifs/connect.c: RFC1002 header 0x1104 [96268.787870] fs/cifs/connect.c: missing 12036 bytes from transact2, check next response [96268.788037] fs/cifs/connect.c: RFC1002 header 0x1104 [96268.788042] fs/cifs/connect.c: missing 12036 bytes from transact2, check next response [96268.788371] fs/cifs/connect.c: RFC1002 header 0xdb0 [96268.788375] fs/cifs/connect.c: missing 12888 bytes from transact2, check next response...all four parts of the response are collected here
[96268.788391] fs/cifs/transport.c: cifs_sync_mid_result: cmd=50 mid=12 state=16...but the state at this point is
MID_RESPONSE_MALFORMED[96268.788395] fs/cifs/cifssmb.c: Error in FindFirst = -5 [96268.788397] fs/cifs/readdir.c: initiate cifs search rc -5 [96268.788398] fs/cifs/readdir.c: CIFS VFS: leaving cifs_readdir (xid = 737644) rc = -5...which makes readdir return
-EIO-------------------[snip]---------------------
Based on that, it looks like something in one of these frames caused
coalesce_t2()to return an error. I don't see the problem right offhand in the capture, but T2 response handling is pretty complex so it can be hard to see.
Quoted 5 lines
Would it be possible for you to rebuild your kernel (or just cifs.ko) with this patch? Once you do that, rerun the test with cFYI turned up, and it should help point out what the problem is.
Thanks,
Ok i am now rebuilding the kernel and will report when i have results.
Quoted 134 lines · Konstantinos Skarlatos
Quoted 133 lines · Jeff Layton
Quoted 77 lines · On Fri, 30 Dec 2011 20:00:46 +0200 Konstantinos…
Quoted 66 lines · Jeff Layton
Quoted 49 lines · On Fri, 30 Dec 2011 11:04:59 +0200 Konstantinos…
Quoted 48 lines · Konstantinos Skarlatos
Quoted 47 lines · Jeff Layton
Quoted 44 lines · On Thu, 29 Dec 2011 12:30:18 +0200 Konstantinos…
Quoted 43 lines · Jeff Layton
Quoted 31 lines · On Thu, 29 Dec 2011 02:08:57 +0200 Konstantinos…
I mount via cifs a windows XP share, df gives me correct sizes, but when
I ls the mount point i get input/output error.
strace: http://pastebin.com/WXf8M1numount --verbose -t cifs -o username=administrator,password=blahblah
//192.168.0.11/jobs/mnt/backups/montaz/jobs
mount.cifs kernel mount options:
ip=192.168.0.11,unc=\\192.168.0.11\jobs,,ver=1,user=administrator,pass=********df
//192.168.0.11/jobs 114464
105196 9268 92%/mnt/backups/montaz/jobsls
/mnt/backups/montaz/jobs/
ls: reading directory/mnt/backups/montaz/jobs/: Input/output error
total 0the fun thing is that i can cd to a lower level directory, and ls works
fine there! only the mount point has the problemls
/mnt/backups/montaz/jobs/test
total 44K
drwxr-xr-x 1 root root 0 Apr 30 2010 blah blah/
......Quoted 7 lines
kernel version 3.2rc7
this seems to be related to :
https://lkml.org/lkml/2011/8/1/427
Re: [3.0.0+][Regression][Bisected] CIFS:getdents()broken for large dirsHmmm, maybe. What makes you think that it's related? What sort of server are you seeing this against?
Quoted 7 lines
Quoted 6 lines
Quoted 4 lines
Quoted 1 line
Windows XP service pack 2 (greek)
How many files are in the directory?
140 folders and 20 files
Attached is a tcp dump of my session.
I tried reproducing this here, but wasn't able to. Testing against my xp box worked fine.
Most likely, the
FIND_FILEresponses are falling afoul of the code incoalesce_t2or check2ndT2. Unfortunately that code is pretty complicated and I'm not certain what the problem actually is...Quoted 5 lines
One thing that's interesting is that the total data being sent in the request is rather large (16336 bytes). I think that's legit, but maybe it's exceeding the end of the buffer once we try to coalesce it.
Would it be possible to get the cFYI output from this test?
I did not get a cFYI output from that test, but i redid a mount-ls-umount and am attaching the tcpdump
Also here http://pastebin.com/J20uC6kU you can find the cifsFYI and the contents of/proc/fs/cifs/DebugDataform the same testQuoted 1 line
Quoted 8 lines
Quoted 3 lines
Is this a regression? Did it work with earlier kernels and only recently start failing?
I do not know, and i am a bit afraid to downgrade this machine below 3.0 due to some changes arch linux has introduced recently. I can always set up a few virtual machines though, and i can even request permission from my company to give you shell access if you like. Which kernel versions would you like me to test?
Ok, that tells us a little:
-------------------[snip]---------------------
[96268.787078] fs/cifs/cifssmb.c: In FindFirst for [96268.787083] fs/cifs/transport.c: For smb_command 50 [96268.787086] fs/cifs/transport.c: Sending smb: total_len 88...FIND_FIRST command is sent
[96268.787690] fs/cifs/connect.c: RFC1002 header 0x1104 [96268.787697] fs/cifs/connect.c: missing 12048 bytes from transact2, check next response [96268.787865] fs/cifs/connect.c: RFC1002 header 0x1104 [96268.787870] fs/cifs/connect.c: missing 12036 bytes from transact2, check next response [96268.788037] fs/cifs/connect.c: RFC1002 header 0x1104 [96268.788042] fs/cifs/connect.c: missing 12036 bytes from transact2, check next response [96268.788371] fs/cifs/connect.c: RFC1002 header 0xdb0 [96268.788375] fs/cifs/connect.c: missing 12888 bytes from transact2,check next response
...all four parts of the response are collected here
[96268.788391] fs/cifs/transport.c: cifs_sync_mid_result: cmd=50mid=12 state=16
...but the state at this point is
MID_RESPONSE_MALFORMED[96268.788395] fs/cifs/cifssmb.c: Error in FindFirst = -5 [96268.788397] fs/cifs/readdir.c: initiate cifs search rc -5 [96268.788398] fs/cifs/readdir.c: CIFS VFS: leaving cifs_readdir (xid= 737644) rc = -5
...which makes readdir return
-EIO-------------------[snip]---------------------
Based on that, it looks like something in one of these frames caused
coalesce_t2()to return an error. I don't see the problem right offhand in the capture, but T2 response handling is pretty complex so it can be hard to see.
Quoted 7 lines
Quoted 5 lines
Would it be possible for you to rebuild your kernel (or just cifs.ko) with this patch? Once you do that, rerun the test with cFYI turned up, and it should help point out what the problem is.
Thanks,
Ok i am now rebuilding the kernel and will report when i have results.
http://pastebin.com/scgyDjhT and attached tcpdump
Quoted 135 lines · On Sat, 31 Dec 2011 15:41:32 +0200 Konstantinos Skarlatos…
Quoted 134 lines · Konstantinos Skarlatos
Quoted 133 lines · Jeff Layton
Quoted 77 lines · On Fri, 30 Dec 2011 20:00:46 +0200 Konstantinos…
Quoted 66 lines · Jeff Layton
Quoted 49 lines · On Fri, 30 Dec 2011 11:04:59 +0200 Konstantinos…
Quoted 48 lines · Konstantinos Skarlatos
Quoted 47 lines · Jeff Layton
Quoted 44 lines · On Thu, 29 Dec 2011 12:30:18 +0200 Konstantinos…
Quoted 43 lines · Jeff Layton
Quoted 31 lines · On Thu, 29 Dec 2011 02:08:57 +0200 Konstantinos…
I mount via cifs a windows XP share, df gives me correct sizes, but when
I ls the mount point i get input/output error.
strace: http://pastebin.com/WXf8M1numount --verbose -t cifs -o username=administrator,password=blahblah
//192.168.0.11/jobs/mnt/backups/montaz/jobs
mount.cifs kernel mount options:
ip=192.168.0.11,unc=\\192.168.0.11\jobs,,ver=1,user=administrator,pass=********df
//192.168.0.11/jobs 114464
105196 9268 92%/mnt/backups/montaz/jobsls
/mnt/backups/montaz/jobs/
ls: reading directory/mnt/backups/montaz/jobs/: Input/output error
total 0the fun thing is that i can cd to a lower level directory, and ls works
fine there! only the mount point has the problemls
/mnt/backups/montaz/jobs/test
total 44K
drwxr-xr-x 1 root root 0 Apr 30 2010 blah blah/
......Quoted 7 lines
kernel version 3.2rc7
this seems to be related to :
https://lkml.org/lkml/2011/8/1/427
Re: [3.0.0+][Regression][Bisected] CIFS:getdents()broken for large dirsHmmm, maybe. What makes you think that it's related? What sort of server are you seeing this against?
Quoted 7 lines
Quoted 6 lines
Quoted 4 lines
Quoted 1 line
Windows XP service pack 2 (greek)
How many files are in the directory?
140 folders and 20 files
Attached is a tcp dump of my session.
I tried reproducing this here, but wasn't able to. Testing against my xp box worked fine.
Most likely, the
FIND_FILEresponses are falling afoul of the code incoalesce_t2or check2ndT2. Unfortunately that code is pretty complicated and I'm not certain what the problem actually is...Quoted 5 lines
One thing that's interesting is that the total data being sent in the request is rather large (16336 bytes). I think that's legit, but maybe it's exceeding the end of the buffer once we try to coalesce it.
Would it be possible to get the cFYI output from this test?
I did not get a cFYI output from that test, but i redid a mount-ls-umount and am attaching the tcpdump
Also here http://pastebin.com/J20uC6kU you can find the cifsFYI and the contents of/proc/fs/cifs/DebugDataform the same testQuoted 1 line
Quoted 8 lines
Quoted 3 lines
Is this a regression? Did it work with earlier kernels and only recently start failing?
I do not know, and i am a bit afraid to downgrade this machine below 3.0 due to some changes arch linux has introduced recently. I can always set up a few virtual machines though, and i can even request permission from my company to give you shell access if you like. Which kernel versions would you like me to test?
Ok, that tells us a little:
-------------------[snip]---------------------
[96268.787078] fs/cifs/cifssmb.c: In FindFirst for [96268.787083] fs/cifs/transport.c: For smb_command 50 [96268.787086] fs/cifs/transport.c: Sending smb: total_len 88...FIND_FIRST command is sent
[96268.787690] fs/cifs/connect.c: RFC1002 header 0x1104 [96268.787697] fs/cifs/connect.c: missing 12048 bytes from transact2, check next response [96268.787865] fs/cifs/connect.c: RFC1002 header 0x1104 [96268.787870] fs/cifs/connect.c: missing 12036 bytes from transact2, check next response [96268.788037] fs/cifs/connect.c: RFC1002 header 0x1104 [96268.788042] fs/cifs/connect.c: missing 12036 bytes from transact2, check next response [96268.788371] fs/cifs/connect.c: RFC1002 header 0xdb0 [96268.788375] fs/cifs/connect.c: missing 12888 bytes from transact2,check next response
...all four parts of the response are collected here
[96268.788391] fs/cifs/transport.c: cifs_sync_mid_result: cmd=50mid=12 state=16
...but the state at this point is
MID_RESPONSE_MALFORMED[96268.788395] fs/cifs/cifssmb.c: Error in FindFirst = -5 [96268.788397] fs/cifs/readdir.c: initiate cifs search rc -5 [96268.788398] fs/cifs/readdir.c: CIFS VFS: leaving cifs_readdir (xid= 737644) rc = -5
...which makes readdir return
-EIO-------------------[snip]---------------------
Based on that, it looks like something in one of these frames caused
coalesce_t2()to return an error. I don't see the problem right offhand in the capture, but T2 response handling is pretty complex so it can be hard to see.
Quoted 8 lines
Quoted 7 lines
Quoted 5 lines
Would it be possible for you to rebuild your kernel (or just cifs.ko) with this patch? Once you do that, rerun the test with cFYI turned up, and it should help point out what the problem is.
Thanks,
Ok i am now rebuilding the kernel and will report when i have results.
http://pastebin.com/scgyDjhT and attached tcpdump
Very helpful -- I think I see the bug now. Can you test this patch and let me know if it fixes the problem? You'll need to back out the debug patch first, and then apply this one. If it works, I'll plan to clean this up and send to Steve for inclusion.
Thanks...
----------------------[snip]---------------------
[PATCH] cifs: fix buffer length check and better instrumentation for coalesce_t2
Fix the buffer length check in coalesce_t2. The current one checks to see that the rfc1002 length is not larger than the buffer that will hold it. It gets the check wrong though. The buffer is MAX_CIFS_HDR_SIZE larger than the current check allows for.
Also, to make it easier to debug problems of this sort, when coalesce_t2 returns an error, have it throw a cFYI message that explains the reason.
--- fs/cifs/connect.c | 29 +++++++++++++++++++++-------- 1 files changed, 21 insertions(+), 8 deletions(-)
fs/cifs/connect.c +21 -8
@@ -243,16 +243,21 @@ static int coalesce_t2(struct smb_hdr *psecond, struct smb_hdr *pTargetSMB) remaining = total_data_size - total_in_buf; -if (remaining <0 ) +if (remaining <0 ) { +cFYI (1 ,"Server sent too much data. total_data_size=%hu " +"total_in_buf=%hu" , total_data_size, total_in_buf);return -EPROTO ; + } -if (remaining ==0 )/* nothing to do, ignore */ +if (remaining ==0 ) { +/* nothing to do, ignore */ +cFYI (1 ,"no more data remains" );return 0 ; + } total_in_buf2 =get_unaligned_le16 (&pSMB2->t2_rsp.DataCount); -if (remaining < total_in_buf2) { +if (remaining < total_in_buf2)cFYI (1 ,"transact2 2nd response contains too much data" ); - }/* find end of first SMB data area */ data_area_of_target = (char *)&pSMBt->hdr.Protocol +
@@ -267,23 +272,29 @@ static int coalesce_t2(struct smb_hdr *psecond, struct smb_hdr *pTargetSMB)/* copy second buffer into end of first buffer */ total_in_buf += total_in_buf2;/* is the result too big for the field? */ -if (total_in_buf >USHRT_MAX ) +if (total_in_buf >USHRT_MAX ) { +cFYI (1 ,"coalesced total_in_buf too large (%u)" , total_in_buf);return -EPROTO ; + }put_unaligned_le16 (total_in_buf, &pSMBt->t2_rsp.DataCount);/* fix up the BCC */ byte_count =get_bcc (pTargetSMB); byte_count += total_in_buf2;/* is the result too big for the field? */ -if (byte_count >USHRT_MAX ) +if (byte_count >USHRT_MAX ) { +cFYI (1 ,"coalesced BCC too large (%u)" , byte_count);return -EPROTO ; + }put_bcc (byte_count, pTargetSMB); byte_count =be32_to_cpu (pTargetSMB->smb_buf_length); byte_count += total_in_buf2;/* don't allow buffer to overflow */ -if (byte_count > CIFSMaxBufSize) +if (byte_count > CIFSMaxBufSize +MAX_CIFS_HDR_SIZE ) { +cFYI (1 ,"coalesced BCC exceeds buffer size (%u)" , byte_count);return -ENOBUFS ; + } pTargetSMB->smb_buf_length =cpu_to_be32 (byte_count);memcpy (data_area_of_target, data_area_of_buf2, total_in_buf2);
@@ -291,8 +302,10 @@ static int coalesce_t2(struct smb_hdr *psecond, struct smb_hdr *pTargetSMB)if (remaining == total_in_buf2) {cFYI (1 ,"found the last secondary response" );return 0 ;/* we are done */ - }else /* more responses to go */ + }else {/* more responses to go */ +cFYI (1 ,"waiting for more secondary responses" );return 1 ; + } } static void
Signature · 1 line
1.7.7.4
Quoted 250 lines · Jeff Layton
Quoted 135 lines · On Sat, 31 Dec 2011 15:41:32 +0200 Konstantinos Skarlatos…
Quoted 134 lines · Konstantinos Skarlatos
Quoted 133 lines · Jeff Layton
Quoted 77 lines · On Fri, 30 Dec 2011 20:00:46 +0200 Konstantinos…
Quoted 66 lines · Jeff Layton
Quoted 49 lines · On Fri, 30 Dec 2011 11:04:59 +0200 Konstantinos…
Quoted 48 lines · Konstantinos Skarlatos
Quoted 47 lines · Jeff Layton
Quoted 44 lines · On Thu, 29 Dec 2011 12:30:18 +0200 Konstantinos…
Quoted 43 lines · Jeff Layton
Quoted 31 lines · On Thu, 29 Dec 2011 02:08:57 +0200 Konstantinos…
I mount via cifs a windows XP share, df gives me correct sizes, but when
I ls the mount point i get input/output error.
strace: http://pastebin.com/WXf8M1numount --verbose -t cifs -o username=administrator,password=blahblah
//192.168.0.11/jobs/mnt/backups/montaz/jobs
mount.cifs kernel mount options:
ip=192.168.0.11,unc=\\192.168.0.11\jobs,,ver=1,user=administrator,pass=********df
//192.168.0.11/jobs 114464
105196 9268 92%/mnt/backups/montaz/jobsls
/mnt/backups/montaz/jobs/
ls: reading directory/mnt/backups/montaz/jobs/: Input/output error
total 0the fun thing is that i can cd to a lower level directory, and ls works
fine there! only the mount point has the problemls
/mnt/backups/montaz/jobs/test
total 44K
drwxr-xr-x 1 root root 0 Apr 30 2010 blah blah/
......Quoted 7 lines
kernel version 3.2rc7
this seems to be related to :
https://lkml.org/lkml/2011/8/1/427
Re: [3.0.0+][Regression][Bisected] CIFS:getdents()broken for large dirsHmmm, maybe. What makes you think that it's related? What sort of server are you seeing this against?
Quoted 7 lines
Quoted 6 lines
Quoted 4 lines
Quoted 1 line
Windows XP service pack 2 (greek)
How many files are in the directory?
140 folders and 20 files
Attached is a tcp dump of my session.
I tried reproducing this here, but wasn't able to. Testing against my xp box worked fine.
Most likely, the
FIND_FILEresponses are falling afoul of the code incoalesce_t2or check2ndT2. Unfortunately that code is pretty complicated and I'm not certain what the problem actually is...Quoted 5 lines
One thing that's interesting is that the total data being sent in the request is rather large (16336 bytes). I think that's legit, but maybe it's exceeding the end of the buffer once we try to coalesce it.
Would it be possible to get the cFYI output from this test?
I did not get a cFYI output from that test, but i redid a mount-ls-umount and am attaching the tcpdump
Also here http://pastebin.com/J20uC6kU you can find the cifsFYI and the contents of/proc/fs/cifs/DebugDataform the same testQuoted 1 line
Quoted 8 lines
Quoted 3 lines
Is this a regression? Did it work with earlier kernels and only recently start failing?
I do not know, and i am a bit afraid to downgrade this machine below 3.0 due to some changes arch linux has introduced recently. I can always set up a few virtual machines though, and i can even request permission from my company to give you shell access if you like. Which kernel versions would you like me to test?
Ok, that tells us a little:
-------------------[snip]---------------------
[96268.787078] fs/cifs/cifssmb.c: In FindFirst for [96268.787083] fs/cifs/transport.c: For smb_command 50 [96268.787086] fs/cifs/transport.c: Sending smb: total_len 88...FIND_FIRST command is sent
[96268.787690] fs/cifs/connect.c: RFC1002 header 0x1104 [96268.787697] fs/cifs/connect.c: missing 12048 bytes from transact2, check next response [96268.787865] fs/cifs/connect.c: RFC1002 header 0x1104 [96268.787870] fs/cifs/connect.c: missing 12036 bytes from transact2, check next response [96268.788037] fs/cifs/connect.c: RFC1002 header 0x1104 [96268.788042] fs/cifs/connect.c: missing 12036 bytes from transact2, check next response [96268.788371] fs/cifs/connect.c: RFC1002 header 0xdb0 [96268.788375] fs/cifs/connect.c: missing 12888 bytes from transact2,check next response
...all four parts of the response are collected here
[96268.788391] fs/cifs/transport.c: cifs_sync_mid_result: cmd=50mid=12 state=16
...but the state at this point is
MID_RESPONSE_MALFORMED[96268.788395] fs/cifs/cifssmb.c: Error in FindFirst = -5 [96268.788397] fs/cifs/readdir.c: initiate cifs search rc -5 [96268.788398] fs/cifs/readdir.c: CIFS VFS: leaving cifs_readdir (xid= 737644) rc = -5
...which makes readdir return
-EIO-------------------[snip]---------------------
Based on that, it looks like something in one of these frames caused
coalesce_t2()to return an error. I don't see the problem right offhand in the capture, but T2 response handling is pretty complex so it can be hard to see.Quoted 8 lines
Quoted 7 lines
Quoted 5 lines
Would it be possible for you to rebuild your kernel (or just cifs.ko) with this patch? Once you do that, rerun the test with cFYI turned up, and it should help point out what the problem is.
Thanks,
Ok i am now rebuilding the kernel and will report when i have results.
http://pastebin.com/scgyDjhT and attached tcpdump
Very helpful -- I think I see the bug now. Can you test this patch and let me know if it fixes the problem? You'll need to back out the debug patch first, and then apply this one. If it works, I'll plan to clean this up and send to Steve for inclusion.
Thanks...
----------------------[snip]---------------------
[PATCH] cifs: fix buffer length check and better instrumentation for
coalesce_t2Fix the buffer length check in
coalesce_t2. The current one checks to see that the rfc1002 length is not larger than the buffer that will hold it. It gets the check wrong though. The buffer isMAX_CIFS_HDR_SIZElarger than the current check allows for.Also, to make it easier to debug problems of this sort, when
coalesce_t2returns an error, have it throw a cFYI message that explains the reason.Signed-off-by Jeff Layton <[email protected]>---
fs/cifs/connect.c | 29 +++++++++++++++++++++--------1 files changed, 21 insertions(+), 8 deletions(-)fs/cifs/connect.c +9
-4@@ -243,16 +243,21 @@ static int coalesce_t2(struct smb_hdr *psecond, struct smb_hdr *pTargetSMB) remaining = total_data_size - total_in_buf; - if (remaining < 0) + if (remaining < 0) { + cFYI(1, "Server sent too much data. total_data_size=%hu " + "total_in_buf=%hu", total_data_size, total_in_buf); return -EPROTO; + } - if (remaining == 0) /* nothing to do, ignore */ + if (remaining == 0) { + /* nothing to do, ignore */ + cFYI(1, "no more data remains"); return 0; + } total_in_buf2 = get_unaligned_le16(&pSMB2->t2_rsp.DataCount); - if (remaining < total_in_buf2) { + if (remaining < total_in_buf2) cFYI(1, "transact2 2nd response contains too much data"); - } /* find end of first SMB data area */data_area_of_target = (char *)&pSMBt->hdr.Protocol +
fs/cifs/connect.c +9
-3@@ -267,23 +272,29 @@ static int coalesce_t2(struct smb_hdr *psecond, struct smb_hdr *pTargetSMB) /* copy second buffer into end of first buffer */ total_in_buf += total_in_buf2; /* is the result too big for the field? */ - if (total_in_buf > USHRT_MAX) + if (total_in_buf > USHRT_MAX) { + cFYI(1, "coalesced total_in_buf too large (%u)", total_in_buf); return -EPROTO; + } put_unaligned_le16(total_in_buf, &pSMBt->t2_rsp.DataCount); /* fix up the BCC */ byte_count = get_bcc(pTargetSMB); byte_count += total_in_buf2; /* is the result too big for the field? */ - if (byte_count > USHRT_MAX) + if (byte_count > USHRT_MAX) { + cFYI(1, "coalesced BCC too large (%u)", byte_count); return -EPROTO; + } put_bcc(byte_count, pTargetSMB); byte_count = be32_to_cpu(pTargetSMB->smb_buf_length); byte_count += total_in_buf2; /* don't allow buffer to overflow */ - if (byte_count > CIFSMaxBufSize) + if (byte_count > CIFSMaxBufSize + MAX_CIFS_HDR_SIZE) { + cFYI(1, "coalesced BCC exceeds buffer size (%u)", byte_count); return -ENOBUFS; + } pTargetSMB->smb_buf_length = cpu_to_be32(byte_count); memcpy(data_area_of_target, data_area_of_buf2, total_in_buf2);@@ -291,8 +302,10 @@ static int coalesce_t2(struct smb_hdr *psecond, struct smb_hdr *pTargetSMB) if (remaining == total_in_buf2) { cFYI(1, "found the last secondary response");return 0; /* we are done */
- } else /* more responses to go */
+ } else {/* more responses to go */
+cFYI(1, "waiting for more secondary responses"); return 1;
+ }
}static void
--
1.7.7.4
Quoted 4 lines
--
To unsubscribe from this list: send the line "unsubscribe linux-cifs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
what I do not understand is,
a)
Why does Windows XP send different search counts to a findfirst2 request.
b)
This bug also does not manifest against a Windows 2003 server, Windows 2008 server and Windows 7.
c)
As per this document,
http://msdn.microsoft.com/en-us/library/cc246806(v=prot.13).aspx
"<66> Section 2.2.8.1.2: The SMB_FIND_FILE_ID_FULL_DIRECTORY_INFO Information Level was introduced in Windows Server 2003 and is not present in Windows 2000 Server and Windows XP."
then why does Windows XP respond to
trans2/find first2/SMB_FIND_FILE_ID_FULL_DIRECTORY_INFO request with an error.
Quoted 269 lines · On Sun, 1 Jan 2012 01:22:59 -0600 Shirish Pargaonkar…
Quoted 250 lines · Jeff Layton
Quoted 135 lines · On Sat, 31 Dec 2011 15:41:32 +0200 Konstantinos Skarlatos…
Quoted 134 lines · Konstantinos Skarlatos
Quoted 133 lines · Jeff Layton
Quoted 77 lines · On Fri, 30 Dec 2011 20:00:46 +0200 Konstantinos…
Quoted 66 lines · Jeff Layton
Quoted 49 lines · On Fri, 30 Dec 2011 11:04:59 +0200 Konstantinos…
Quoted 48 lines · Konstantinos Skarlatos
Quoted 47 lines · Jeff Layton
Quoted 44 lines · On Thu, 29 Dec 2011 12:30:18 +0200 Konstantinos…
Quoted 43 lines · Jeff Layton
Quoted 38 lines · On Thu, 29 Dec 2011 02:08:57 +0200 Konstantinos…
I mount via cifs a windows XP share, df gives me correct sizes,
but when
I ls the mount point i get input/output error.
strace: http://pastebin.com/WXf8M1numount --verbose -t cifs -o
username=administrator,password=blahblah
//192.168.0.11/jobs /mnt/backups/montaz/jobs
mount.cifs kernel mount options:
ip=192.168.0.11,unc=\\192.168.0.11\jobs,,ver=1,user=administrator,pass=********df
//192.168.0.11/jobs 114464
105196 9268 92% /mnt/backups/montaz/jobsls /mnt/backups/montaz/jobs/
ls: reading directory /mnt/backups/montaz/jobs/: Input/output
error
total 0the fun thing is that i can cd to a lower level directory, and ls
works
fine there! only the mount point has the problemls /mnt/backups/montaz/jobs/test
total 44K
drwxr-xr-x 1 root root 0 Apr 30 2010 blah blah/
......kernel version 3.2rc7
this seems to be related to :
https://lkml.org/lkml/2011/8/1/427
Re: [3.0.0+][Regression][Bisected] CIFS: getdents() broken for
large dirsHmmm, maybe. What makes you think that it's related? What sort of server are you seeing this against?
Quoted 7 lines
Quoted 6 lines
Quoted 4 lines
Quoted 1 line
Windows XP service pack 2 (greek)
How many files are in the directory?
140 folders and 20 files
Attached is a tcp dump of my session.
I tried reproducing this here, but wasn't able to. Testing against my xp box worked fine.
Most likely, the
FIND_FILEresponses are falling afoul of the code incoalesce_t2or check2ndT2. Unfortunately that code is pretty complicated and I'm not certain what the problem actually is...Quoted 5 lines
One thing that's interesting is that the total data being sent in the request is rather large (16336 bytes). I think that's legit, but maybe it's exceeding the end of the buffer once we try to coalesce it.
Would it be possible to get the cFYI output from this test?
I did not get a cFYI output from that test, but i redid a mount-ls-umount and am attaching the tcpdump
Also here http://pastebin.com/J20uC6kU you can find the cifsFYI and the contents of/proc/fs/cifs/DebugDataform the same testQuoted 1 line
Quoted 8 lines
Quoted 3 lines
Is this a regression? Did it work with earlier kernels and only recently start failing?
I do not know, and i am a bit afraid to downgrade this machine below 3.0 due to some changes arch linux has introduced recently. I can always set up a few virtual machines though, and i can even request permission from my company to give you shell access if you like. Which kernel versions would you like me to test?
Ok, that tells us a little:
-------------------[snip]---------------------
[96268.787078] fs/cifs/cifssmb.c: In FindFirst for [96268.787083] fs/cifs/transport.c: For smb_command 50 [96268.787086] fs/cifs/transport.c: Sending smb: total_len 88...FIND_FIRST command is sent
[96268.787690] fs/cifs/connect.c: RFC1002 header 0x1104 [96268.787697] fs/cifs/connect.c: missing 12048 bytes from transact2, check next response [96268.787865] fs/cifs/connect.c: RFC1002 header 0x1104 [96268.787870] fs/cifs/connect.c: missing 12036 bytes from transact2, check next response [96268.788037] fs/cifs/connect.c: RFC1002 header 0x1104 [96268.788042] fs/cifs/connect.c: missing 12036 bytes from transact2, check next response [96268.788371] fs/cifs/connect.c: RFC1002 header 0xdb0 [96268.788375] fs/cifs/connect.c: missing 12888 bytes from transact2,check next response
...all four parts of the response are collected here
[96268.788391] fs/cifs/transport.c: cifs_sync_mid_result: cmd=50mid=12 state=16
...but the state at this point is
MID_RESPONSE_MALFORMED[96268.788395] fs/cifs/cifssmb.c: Error in FindFirst = -5 [96268.788397] fs/cifs/readdir.c: initiate cifs search rc -5 [96268.788398] fs/cifs/readdir.c: CIFS VFS: leaving cifs_readdir (xid= 737644) rc = -5
...which makes readdir return
-EIO-------------------[snip]---------------------
Based on that, it looks like something in one of these frames caused
coalesce_t2()to return an error. I don't see the problem right offhand in the capture, but T2 response handling is pretty complex so it can be hard to see.Quoted 8 lines
Quoted 7 lines
Quoted 5 lines
Would it be possible for you to rebuild your kernel (or just cifs.ko) with this patch? Once you do that, rerun the test with cFYI turned up, and it should help point out what the problem is.
Thanks,
Ok i am now rebuilding the kernel and will report when i have results.
http://pastebin.com/scgyDjhT and attached tcpdump
Very helpful -- I think I see the bug now. Can you test this patch and let me know if it fixes the problem? You'll need to back out the debug patch first, and then apply this one. If it works, I'll plan to clean this up and send to Steve for inclusion.
Thanks...
----------------------[snip]---------------------
[PATCH] cifs: fix buffer length check and better instrumentation for
coalesce_t2Fix the buffer length check in
coalesce_t2. The current one checks to see that the rfc1002 length is not larger than the buffer that will hold it. It gets the check wrong though. The buffer isMAX_CIFS_HDR_SIZElarger than the current check allows for.Also, to make it easier to debug problems of this sort, when
coalesce_t2returns an error, have it throw a cFYI message that explains the reason.Signed-off-by Jeff Layton <[email protected]>---
fs/cifs/connect.c | 29 +++++++++++++++++++++--------1 files changed, 21 insertions(+), 8 deletions(-)fs/cifs/connect.c +9
-4@@ -243,16 +243,21 @@ static int coalesce_t2(struct smb_hdr *psecond, struct smb_hdr *pTargetSMB) remaining = total_data_size - total_in_buf; - if (remaining < 0) + if (remaining < 0) { + cFYI(1, "Server sent too much data. total_data_size=%hu " + "total_in_buf=%hu", total_data_size, total_in_buf); return -EPROTO; + } - if (remaining == 0) /* nothing to do, ignore */ + if (remaining == 0) { + /* nothing to do, ignore */ + cFYI(1, "no more data remains"); return 0; + } total_in_buf2 = get_unaligned_le16(&pSMB2->t2_rsp.DataCount); - if (remaining < total_in_buf2) { + if (remaining < total_in_buf2) cFYI(1, "transact2 2nd response contains too much data"); - } /* find end of first SMB data area */data_area_of_target = (char *)&pSMBt->hdr.Protocol +
fs/cifs/connect.c +9
-3@@ -267,23 +272,29 @@ static int coalesce_t2(struct smb_hdr *psecond, struct smb_hdr *pTargetSMB) /* copy second buffer into end of first buffer */ total_in_buf += total_in_buf2; /* is the result too big for the field? */ - if (total_in_buf > USHRT_MAX) + if (total_in_buf > USHRT_MAX) { + cFYI(1, "coalesced total_in_buf too large (%u)", total_in_buf); return -EPROTO; + } put_unaligned_le16(total_in_buf, &pSMBt->t2_rsp.DataCount); /* fix up the BCC */ byte_count = get_bcc(pTargetSMB); byte_count += total_in_buf2; /* is the result too big for the field? */ - if (byte_count > USHRT_MAX) + if (byte_count > USHRT_MAX) { + cFYI(1, "coalesced BCC too large (%u)", byte_count); return -EPROTO; + } put_bcc(byte_count, pTargetSMB); byte_count = be32_to_cpu(pTargetSMB->smb_buf_length); byte_count += total_in_buf2; /* don't allow buffer to overflow */ - if (byte_count > CIFSMaxBufSize) + if (byte_count > CIFSMaxBufSize + MAX_CIFS_HDR_SIZE) { + cFYI(1, "coalesced BCC exceeds buffer size (%u)", byte_count); return -ENOBUFS; + } pTargetSMB->smb_buf_length = cpu_to_be32(byte_count); memcpy(data_area_of_target, data_area_of_buf2, total_in_buf2);@@ -291,8 +302,10 @@ static int coalesce_t2(struct smb_hdr *psecond, struct smb_hdr *pTargetSMB) if (remaining == total_in_buf2) { cFYI(1, "found the last secondary response");return 0; /* we are done */
- } else /* more responses to go */
+ } else {/* more responses to go */
+cFYI(1, "waiting for more secondary responses"); return 1;
+ }
}static void
--
1.7.7.4Quoted 4 lines
--
To unsubscribe from this list: send the line "unsubscribe linux-cifs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.htmlwhat I do not understand is,
a)
Why does Windows XP send different search counts to a findfirst2 request.b)
This bug also does not manifest against a Windows 2003 server, Windows 2008 server and Windows 7.c)
As per this document,
http://msdn.microsoft.com/en-us/library/cc246806(v=prot.13).aspx
Quoted 7 lines
"<66> Section 2.2.8.1.2: The
SMB_FIND_FILE_ID_FULL_DIRECTORY_INFOInformation Level was introduced in Windows Server 2003 and is not present in Windows 2000 Server and Windows XP."then why does Windows XP respond to
trans2/find first2/SMB_FIND_FILE_ID_FULL_DIRECTORY_INFO request with an error.
I've not personally seen the behavior you describe. I think you're talking about a different problem altogether.
This bug was probably due to commit c974befa. Prior to that, the client would artificially cap the MaxDataCount for the FindFirst at the same level as that of the server. That papered over this bug in coalesce_t2.
Signature · 1 line
Jeff Layton <[email protected]>
Quoted 145 lines · On Sat, 31 Dec 2011 20:23:31 -0500 Jeff Layton…
Quoted 142 lines · On Sat, 31 Dec 2011 15:41:32 +0200 Konstantinos Skarlatos…
Quoted 141 lines · Konstantinos Skarlatos
Quoted 133 lines · Jeff Layton
Quoted 77 lines · On Fri, 30 Dec 2011 20:00:46 +0200 Konstantinos…
Quoted 66 lines · Jeff Layton
Quoted 49 lines · On Fri, 30 Dec 2011 11:04:59 +0200 Konstantinos…
Quoted 48 lines · Konstantinos Skarlatos
Quoted 47 lines · Jeff Layton
Quoted 44 lines · On Thu, 29 Dec 2011 12:30:18 +0200 Konstantinos…
Quoted 43 lines · Jeff Layton
Quoted 31 lines · On Thu, 29 Dec 2011 02:08:57 +0200 Konstantinos…
I mount via cifs a windows XP share, df gives me correct sizes, but when
I ls the mount point i get input/output error.
strace: http://pastebin.com/WXf8M1numount --verbose -t cifs -o username=administrator,password=blahblah
//192.168.0.11/jobs/mnt/backups/montaz/jobs
mount.cifs kernel mount options:
ip=192.168.0.11,unc=\\192.168.0.11\jobs,,ver=1,user=administrator,pass=********df
//192.168.0.11/jobs 114464
105196 9268 92%/mnt/backups/montaz/jobsls
/mnt/backups/montaz/jobs/
ls: reading directory/mnt/backups/montaz/jobs/: Input/output error
total 0the fun thing is that i can cd to a lower level directory, and ls works
fine there! only the mount point has the problemls
/mnt/backups/montaz/jobs/test
total 44K
drwxr-xr-x 1 root root 0 Apr 30 2010 blah blah/
......Quoted 7 lines
kernel version 3.2rc7
this seems to be related to :
https://lkml.org/lkml/2011/8/1/427
Re: [3.0.0+][Regression][Bisected] CIFS:getdents()broken for large dirsHmmm, maybe. What makes you think that it's related? What sort of server are you seeing this against?
Quoted 7 lines
Quoted 6 lines
Quoted 4 lines
Quoted 1 line
Windows XP service pack 2 (greek)
How many files are in the directory?
140 folders and 20 files
Attached is a tcp dump of my session.
I tried reproducing this here, but wasn't able to. Testing against my xp box worked fine.
Most likely, the
FIND_FILEresponses are falling afoul of the code incoalesce_t2or check2ndT2. Unfortunately that code is pretty complicated and I'm not certain what the problem actually is...Quoted 5 lines
One thing that's interesting is that the total data being sent in the request is rather large (16336 bytes). I think that's legit, but maybe it's exceeding the end of the buffer once we try to coalesce it.
Would it be possible to get the cFYI output from this test?
I did not get a cFYI output from that test, but i redid a mount-ls-umount and am attaching the tcpdump
Also here http://pastebin.com/J20uC6kU you can find the cifsFYI and the contents of/proc/fs/cifs/DebugDataform the same testQuoted 1 line
Quoted 8 lines
Quoted 3 lines
Is this a regression? Did it work with earlier kernels and only recently start failing?
I do not know, and i am a bit afraid to downgrade this machine below 3.0 due to some changes arch linux has introduced recently. I can always set up a few virtual machines though, and i can even request permission from my company to give you shell access if you like. Which kernel versions would you like me to test?
Ok, that tells us a little:
-------------------[snip]---------------------
[96268.787078] fs/cifs/cifssmb.c: In FindFirst for [96268.787083] fs/cifs/transport.c: For smb_command 50 [96268.787086] fs/cifs/transport.c: Sending smb: total_len 88...FIND_FIRST command is sent
[96268.787690] fs/cifs/connect.c: RFC1002 header 0x1104 [96268.787697] fs/cifs/connect.c: missing 12048 bytes from transact2, check next response [96268.787865] fs/cifs/connect.c: RFC1002 header 0x1104 [96268.787870] fs/cifs/connect.c: missing 12036 bytes from transact2, check next response [96268.788037] fs/cifs/connect.c: RFC1002 header 0x1104 [96268.788042] fs/cifs/connect.c: missing 12036 bytes from transact2, check next response [96268.788371] fs/cifs/connect.c: RFC1002 header 0xdb0 [96268.788375] fs/cifs/connect.c: missing 12888 bytes from transact2,check next response
...all four parts of the response are collected here
[96268.788391] fs/cifs/transport.c: cifs_sync_mid_result: cmd=50mid=12 state=16
...but the state at this point is
MID_RESPONSE_MALFORMED[96268.788395] fs/cifs/cifssmb.c: Error in FindFirst = -5 [96268.788397] fs/cifs/readdir.c: initiate cifs search rc -5 [96268.788398] fs/cifs/readdir.c: CIFS VFS: leaving cifs_readdir (xid= 737644) rc = -5
...which makes readdir return
-EIO-------------------[snip]---------------------
Based on that, it looks like something in one of these frames caused
coalesce_t2()to return an error. I don't see the problem right offhand in the capture, but T2 response handling is pretty complex so it can be hard to see.Quoted 5 lines
Would it be possible for you to rebuild your kernel (or just cifs.ko) with this patch? Once you do that, rerun the test with cFYI turned up, and it should help point out what the problem is.
Thanks,
Ok i am now rebuilding the kernel and will report when i have results.
Quoted 9 lines
Quoted 1 line
http://pastebin.com/scgyDjhT and attached tcpdump
Very helpful -- I think I see the bug now. Can you test this patch and let me know if it fixes the problem? You'll need to back out the debug patch first, and then apply this one. If it works, I'll plan to clean this up and send to Steve for inclusion.
Thanks...
Actually...that patch is off by 4 bytes. Since we're so late in the cycle too, we should probably keep extraneous changes to a minimum. I'll probably propose this patch for 3.2 once you've tested it, and will spin up a patch to clean up coalesce_t2 for 3.3.
If you're already testing the other one, then it's probably good enough to verify the fix. If you haven't tested it yet though, then you should probably test this one instead. Either way, let me know soon if you can so we can get this into 3.2.
---------------------------[snip]----------------------------
[PATCH] cifs: fix bad buffer length check in coalesce_t2
The current check looks to see if the RFC1002 length is larger than CIFSMaxBufSize and fails if it is. The buffer is actually larger than that by MAX_CIFS_HDR_SIZE. Fix the check so that it doesn't fail due to this.
This bug has been around for a long time, but the fact that we used to cap the clients MaxBufferSize at the same level as the server tended to paper over it. Commit c974befa changed that however.
--- fs/cifs/connect.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
fs/cifs/connect.c +1 -1
@@ -282,7 +282,7 @@ static int coalesce_t2(struct smb_hdr *psecond, struct smb_hdr *pTargetSMB) byte_count =be32_to_cpu (pTargetSMB->smb_buf_length); byte_count += total_in_buf2;/* don't allow buffer to overflow */ -if (byte_count > CIFSMaxBufSize) +if (byte_count > CIFSMaxBufSize +MAX_CIFS_HDR_SIZE -4 )return -ENOBUFS ; pTargetSMB->smb_buf_length =cpu_to_be32 (byte_count);
Signature · 1 line
1.7.7.4
Quoted 163 lines · Jeff Layton
Quoted 145 lines · On Sat, 31 Dec 2011 20:23:31 -0500 Jeff Layton<jlayton-H+wXaHxf7aLQT0dZR+AlfA@p…
Quoted 142 lines · On Sat, 31 Dec 2011 15:41:32 +0200 Konstantinos…
Quoted 141 lines · Konstantinos Skarlatos
Quoted 133 lines · Jeff Layton
Quoted 77 lines · On Fri, 30 Dec 2011 20:00:46 +0200 Konstantinos…
Quoted 66 lines · Jeff Layton
Quoted 49 lines · On Fri, 30 Dec 2011 11:04:59 +0200 Konstantinos…
Quoted 48 lines · Konstantinos Skarlatos
Quoted 47 lines · Jeff Layton
Quoted 44 lines · On Thu, 29 Dec 2011 12:30:18 +0200 Konstantinos…
Quoted 43 lines · Jeff Layton
Quoted 38 lines · On Thu, 29 Dec 2011 02:08:57 +0200 Konstantinos…
I mount via cifs a windows XP share, df gives me correct sizes,
but when
I ls the mount point i get input/output error.
strace: http://pastebin.com/WXf8M1numount --verbose -t cifs -o
username=administrator,password=blahblah
//192.168.0.11/jobs /mnt/backups/montaz/jobs
mount.cifs kernel mount options:
ip=192.168.0.11,unc=\\192.168.0.11\jobs,,ver=1,user=administrator,pass=********df
//192.168.0.11/jobs 114464
105196 9268 92% /mnt/backups/montaz/jobsls /mnt/backups/montaz/jobs/
ls: reading directory /mnt/backups/montaz/jobs/: Input/output
error
total 0the fun thing is that i can cd to a lower level directory, and ls
works
fine there! only the mount point has the problemls /mnt/backups/montaz/jobs/test
total 44K
drwxr-xr-x 1 root root 0 Apr 30 2010 blah blah/
......kernel version 3.2rc7
this seems to be related to :
https://lkml.org/lkml/2011/8/1/427
Re: [3.0.0+][Regression][Bisected] CIFS: getdents() broken for
large dirsHmmm, maybe. What makes you think that it's related? What sort of server are you seeing this against?
Quoted 7 lines
Quoted 6 lines
Quoted 4 lines
Quoted 1 line
Windows XP service pack 2 (greek)
How many files are in the directory?
140 folders and 20 files
Attached is a tcp dump of my session.
I tried reproducing this here, but wasn't able to. Testing against my xp box worked fine.
Most likely, the
FIND_FILEresponses are falling afoul of the code incoalesce_t2or check2ndT2. Unfortunately that code is pretty complicated and I'm not certain what the problem actually is...Quoted 5 lines
One thing that's interesting is that the total data being sent in the request is rather large (16336 bytes). I think that's legit, but maybe it's exceeding the end of the buffer once we try to coalesce it.
Would it be possible to get the cFYI output from this test?
I did not get a cFYI output from that test, but i redid a mount-ls-umount and am attaching the tcpdump
Also here http://pastebin.com/J20uC6kU you can find the cifsFYI and the contents of/proc/fs/cifs/DebugDataform the same testQuoted 1 line
Quoted 8 lines
Quoted 3 lines
Is this a regression? Did it work with earlier kernels and only recently start failing?
I do not know, and i am a bit afraid to downgrade this machine below 3.0 due to some changes arch linux has introduced recently. I can always set up a few virtual machines though, and i can even request permission from my company to give you shell access if you like. Which kernel versions would you like me to test?
Ok, that tells us a little:
-------------------[snip]---------------------
[96268.787078] fs/cifs/cifssmb.c: In FindFirst for [96268.787083] fs/cifs/transport.c: For smb_command 50 [96268.787086] fs/cifs/transport.c: Sending smb: total_len 88...FIND_FIRST command is sent
[96268.787690] fs/cifs/connect.c: RFC1002 header 0x1104 [96268.787697] fs/cifs/connect.c: missing 12048 bytes from transact2, check next response [96268.787865] fs/cifs/connect.c: RFC1002 header 0x1104 [96268.787870] fs/cifs/connect.c: missing 12036 bytes from transact2, check next response [96268.788037] fs/cifs/connect.c: RFC1002 header 0x1104 [96268.788042] fs/cifs/connect.c: missing 12036 bytes from transact2, check next response [96268.788371] fs/cifs/connect.c: RFC1002 header 0xdb0 [96268.788375] fs/cifs/connect.c: missing 12888 bytes from transact2,check next response
...all four parts of the response are collected here
[96268.788391] fs/cifs/transport.c: cifs_sync_mid_result: cmd=50mid=12 state=16
...but the state at this point is
MID_RESPONSE_MALFORMED[96268.788395] fs/cifs/cifssmb.c: Error in FindFirst = -5 [96268.788397] fs/cifs/readdir.c: initiate cifs search rc -5 [96268.788398] fs/cifs/readdir.c: CIFS VFS: leaving cifs_readdir (xid= 737644) rc = -5
...which makes readdir return
-EIO-------------------[snip]---------------------
Based on that, it looks like something in one of these frames caused
coalesce_t2()to return an error. I don't see the problem right offhand in the capture, but T2 response handling is pretty complex so it can be hard to see.Quoted 5 lines
Would it be possible for you to rebuild your kernel (or just cifs.ko) with this patch? Once you do that, rerun the test with cFYI turned up, and it should help point out what the problem is.
Thanks,
Ok i am now rebuilding the kernel and will report when i have results.
Quoted 9 lines
Quoted 1 line
http://pastebin.com/scgyDjhT and attached tcpdump
Very helpful -- I think I see the bug now. Can you test this patch and let me know if it fixes the problem? You'll need to back out the debug patch first, and then apply this one. If it works, I'll plan to clean this up and send to Steve for inclusion.
Thanks...
Actually...that patch is off by 4 bytes. Since we're so late in the cycle too, we should probably keep extraneous changes to a minimum. I'll probably propose this patch for 3.2 once you've tested it, and will spin up a patch to clean up
coalesce_t2for 3.3.
Quoted 4 lines
If you're already testing the other one, then it's probably good enough to verify the fix. If you haven't tested it yet though, then you should probably test this one instead. Either way, let me know soon if you can so we can get this into 3.2.
Hello, i just woke up from the new year celebration :) . I think that i will be able to test this patch in an hour or so.
Regards
Quoted 34 lines
---------------------------[snip]----------------------------
[PATCH] cifs: fix bad buffer length check in
coalesce_t2The current check looks to see if the RFC1002 length is larger than CIFSMaxBufSize and fails if it is. The buffer is actually larger than that by
MAX_CIFS_HDR_SIZE. Fix the check so that it doesn't fail due to this.This bug has been around for a long time, but the fact that we used to cap the clients MaxBufferSize at the same level as the server tended to paper over it. Commit
c974befachanged that however.Reported-by Konstantinos Skarlatos<[email protected]>Signed-off-by Jeff Layton<[email protected]>--- fs/cifs/connect.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)fs/cifs/connect.c +1
-1@@ -282,7 +282,7 @@ static int coalesce_t2(struct smb_hdr *psecond, struct smb_hdr *pTargetSMB) byte_count =be32_to_cpu (pTargetSMB->smb_buf_length); byte_count += total_in_buf2;/* don't allow buffer to overflow */ -if (byte_count> CIFSMaxBufSize) +if (byte_count> CIFSMaxBufSize +MAX_CIFS_HDR_SIZE -4 )return -ENOBUFS ; pTargetSMB->smb_buf_length =cpu_to_be32 (byte_count);
Konstantinos Skarlatos
Tested-byQuoted 163 lines · Jeff Layton
Quoted 145 lines · On Sat, 31 Dec 2011 20:23:31 -0500 Jeff Layton<jlayton-H+wXaHxf7aLQT0dZR+AlfA@p…
Quoted 142 lines · On Sat, 31 Dec 2011 15:41:32 +0200 Konstantinos…
Quoted 141 lines · Konstantinos Skarlatos
Quoted 133 lines · Jeff Layton
Quoted 77 lines · On Fri, 30 Dec 2011 20:00:46 +0200 Konstantinos…
Quoted 66 lines · Jeff Layton
Quoted 49 lines · On Fri, 30 Dec 2011 11:04:59 +0200 Konstantinos…
Quoted 48 lines · Konstantinos Skarlatos
Quoted 47 lines · Jeff Layton
Quoted 44 lines · On Thu, 29 Dec 2011 12:30:18 +0200 Konstantinos…
Quoted 43 lines · Jeff Layton
Quoted 38 lines · On Thu, 29 Dec 2011 02:08:57 +0200 Konstantinos…
I mount via cifs a windows XP share, df gives me correct sizes,
but when
I ls the mount point i get input/output error.
strace: http://pastebin.com/WXf8M1numount --verbose -t cifs -o
username=administrator,password=blahblah
//192.168.0.11/jobs /mnt/backups/montaz/jobs
mount.cifs kernel mount options:
ip=192.168.0.11,unc=\\192.168.0.11\jobs,,ver=1,user=administrator,pass=********df
//192.168.0.11/jobs 114464
105196 9268 92% /mnt/backups/montaz/jobsls /mnt/backups/montaz/jobs/
ls: reading directory /mnt/backups/montaz/jobs/: Input/output
error
total 0the fun thing is that i can cd to a lower level directory, and ls
works
fine there! only the mount point has the problemls /mnt/backups/montaz/jobs/test
total 44K
drwxr-xr-x 1 root root 0 Apr 30 2010 blah blah/
......kernel version 3.2rc7
this seems to be related to :
https://lkml.org/lkml/2011/8/1/427
Re: [3.0.0+][Regression][Bisected] CIFS: getdents() broken for
large dirsHmmm, maybe. What makes you think that it's related? What sort of server are you seeing this against?
Quoted 7 lines
Quoted 6 lines
Quoted 4 lines
Quoted 1 line
Windows XP service pack 2 (greek)
How many files are in the directory?
140 folders and 20 files
Attached is a tcp dump of my session.
I tried reproducing this here, but wasn't able to. Testing against my xp box worked fine.
Most likely, the
FIND_FILEresponses are falling afoul of the code incoalesce_t2or check2ndT2. Unfortunately that code is pretty complicated and I'm not certain what the problem actually is...Quoted 5 lines
One thing that's interesting is that the total data being sent in the request is rather large (16336 bytes). I think that's legit, but maybe it's exceeding the end of the buffer once we try to coalesce it.
Would it be possible to get the cFYI output from this test?
I did not get a cFYI output from that test, but i redid a mount-ls-umount and am attaching the tcpdump
Also here http://pastebin.com/J20uC6kU you can find the cifsFYI and the contents of/proc/fs/cifs/DebugDataform the same testQuoted 1 line
Quoted 8 lines
Quoted 3 lines
Is this a regression? Did it work with earlier kernels and only recently start failing?
I do not know, and i am a bit afraid to downgrade this machine below 3.0 due to some changes arch linux has introduced recently. I can always set up a few virtual machines though, and i can even request permission from my company to give you shell access if you like. Which kernel versions would you like me to test?
Ok, that tells us a little:
-------------------[snip]---------------------
[96268.787078] fs/cifs/cifssmb.c: In FindFirst for [96268.787083] fs/cifs/transport.c: For smb_command 50 [96268.787086] fs/cifs/transport.c: Sending smb: total_len 88...FIND_FIRST command is sent
[96268.787690] fs/cifs/connect.c: RFC1002 header 0x1104 [96268.787697] fs/cifs/connect.c: missing 12048 bytes from transact2, check next response [96268.787865] fs/cifs/connect.c: RFC1002 header 0x1104 [96268.787870] fs/cifs/connect.c: missing 12036 bytes from transact2, check next response [96268.788037] fs/cifs/connect.c: RFC1002 header 0x1104 [96268.788042] fs/cifs/connect.c: missing 12036 bytes from transact2, check next response [96268.788371] fs/cifs/connect.c: RFC1002 header 0xdb0 [96268.788375] fs/cifs/connect.c: missing 12888 bytes from transact2,check next response
...all four parts of the response are collected here
[96268.788391] fs/cifs/transport.c: cifs_sync_mid_result: cmd=50mid=12 state=16
...but the state at this point is
MID_RESPONSE_MALFORMED[96268.788395] fs/cifs/cifssmb.c: Error in FindFirst = -5 [96268.788397] fs/cifs/readdir.c: initiate cifs search rc -5 [96268.788398] fs/cifs/readdir.c: CIFS VFS: leaving cifs_readdir (xid= 737644) rc = -5
...which makes readdir return
-EIO-------------------[snip]---------------------
Based on that, it looks like something in one of these frames caused
coalesce_t2()to return an error. I don't see the problem right offhand in the capture, but T2 response handling is pretty complex so it can be hard to see.Quoted 5 lines
Would it be possible for you to rebuild your kernel (or just cifs.ko) with this patch? Once you do that, rerun the test with cFYI turned up, and it should help point out what the problem is.
Thanks,
Ok i am now rebuilding the kernel and will report when i have results.
Quoted 9 lines
Quoted 1 line
http://pastebin.com/scgyDjhT and attached tcpdump
Very helpful -- I think I see the bug now. Can you test this patch and let me know if it fixes the problem? You'll need to back out the debug patch first, and then apply this one. If it works, I'll plan to clean this up and send to Steve for inclusion.
Thanks...
Actually...that patch is off by 4 bytes. Since we're so late in the cycle too, we should probably keep extraneous changes to a minimum. I'll probably propose this patch for 3.2 once you've tested it, and will spin up a patch to clean up
coalesce_t2for 3.3.
Quoted 4 lines
If you're already testing the other one, then it's probably good enough to verify the fix. If you haven't tested it yet though, then you should probably test this one instead. Either way, let me know soon if you can so we can get this into 3.2.
Ok, I tested and it works fine for me. Thanks!
PS: would you like me to send any cFYI or tcpdumps?
Quoted 34 lines
---------------------------[snip]----------------------------
[PATCH] cifs: fix bad buffer length check in
coalesce_t2The current check looks to see if the RFC1002 length is larger than CIFSMaxBufSize and fails if it is. The buffer is actually larger than that by
MAX_CIFS_HDR_SIZE. Fix the check so that it doesn't fail due to this.This bug has been around for a long time, but the fact that we used to cap the clients MaxBufferSize at the same level as the server tended to paper over it. Commit
c974befachanged that however.Reported-by Konstantinos Skarlatos<[email protected]>Signed-off-by Jeff Layton<[email protected]>--- fs/cifs/connect.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)fs/cifs/connect.c +1
-1@@ -282,7 +282,7 @@ static int coalesce_t2(struct smb_hdr *psecond, struct smb_hdr *pTargetSMB) byte_count =be32_to_cpu (pTargetSMB->smb_buf_length); byte_count += total_in_buf2;/* don't allow buffer to overflow */ -if (byte_count> CIFSMaxBufSize) +if (byte_count> CIFSMaxBufSize +MAX_CIFS_HDR_SIZE -4 )return -ENOBUFS ; pTargetSMB->smb_buf_length =cpu_to_be32 (byte_count);