Read speed
Subsystems (3)
Hi,
On a RHEL 6.3 box talking to a Windows 7 Enterprise box, I am seeing approximately 1/4th the speed with mount.cifs as I am with smbclient 'get'. RHEL 6.3 currently has CIFS 1.68.
After about a half hour of reading forum threads for the last few years, it seems this is very well known and has been the case for a long time.
I have tried using CIFSMaxBufSize=61440 with rsize=61140 at mount-time and it doesn't really buy me much.
Is there any sort of public-facing summary of the state of the read performance issues. I saw no mention of it in the BUGS section of the mount.cifs man page or in the README for the kernel module.
Is the cause known?
Has already been fixed since 1.68 by chance? If so, what assembly of pieces will overcome the issue? Should I just open a RHEL bug through our support channel and get them involved in this effort somehow?
Any guidance would be welcome at this point.
Jeff
You will need a more recent kernel (probably based on the 3.2 kernel or later, 3.2 was released a year or so ago) to see the dramatic improvements in cifs read speeds (with the redesign of the read code to add more parallelism on i/o to the same file) although RedHat may have backported some of Jeff's excellent performance improvements to some of the older distros. See slides 21 through 26 of my presentation at
Slides 23 and 24 list the cifs performance and functional enhancements by kernel release. Buffered, sequential read (e.g. file copy from a server) got much faster in 3.2 kernel, especially to Samba and other server which support the Unix extensions (due to support for larger i/o sizes than 64K).
Similarly note that cifs write speed was dramatically improved starting at kernel version 3.0 (1.5 to 2 years ago) due to the addition of more async parallelism to the design of the cifs write code (writing to the server from the cifs client) by making the i/o sizes larger and allowing more async dispatch of writes (previously to use a network interface fully you would need to be reading and or writing to multiple different files simultaneously).
Quoted 33 lines · Jeff Blaine
Hi,
On a RHEL 6.3 box talking to a Windows 7 Enterprise box, I am seeing approximately 1/4th the speed with mount.cifs as I am with smbclient 'get'. RHEL 6.3 currently has CIFS 1.68.
After about a half hour of reading forum threads for the last few years, it seems this is very well known and has been the case for a long time.
I have tried using CIFSMaxBufSize=61440 with rsize=61140 at mount-time and it doesn't really buy me much.
Is there any sort of public-facing summary of the state of the read performance issues. I saw no mention of it in the BUGS section of the mount.cifs man page or in the README for the kernel module.
Is the cause known?
Has already been fixed since 1.68 by chance? If so, what assembly of pieces will overcome the issue? Should I just open a RHEL bug through our support channel and get them involved in this effort somehow?
Any guidance would be welcome at this point.
Jeff
--
To unsubscribe from this list: send the line "unsubscribe linux-cifs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Signature · 3 lines
Thanks, Steve
A message that is not in the archive (sent off-list or not yet received): CAH2r5mvbbuo9e30kuRFp9eegJqq_9HLR9=oaGvFqyMi-aBKXTQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org
Thank you for the thorough reply, Steve. It's nice to read of the progress in 2012, but RHEL 6.3 and what is supported there is what we have to work with. It is the latest supported version, as you surely know, so it seems like we'll have to wait quite awhile before getting the module's read performance increases.
We're dealing with 9 US sites, nightly time window (which we're exceeding) for transferring large amounts of data over 1.5Mbps links).
Back to the drawing board.
Thanks again.
Quoted 61 lines · Steve French
You will need a more recent kernel (probably based on the 3.2 kernel or later, 3.2 was released a year or so ago) to see the dramatic improvements in cifs read speeds (with the redesign of the read code to add more parallelism on i/o to the same file) although RedHat may have backported some of Jeff's excellent performance improvements to some of the older distros. See slides 21 through 26 of my presentation at
Slides 23 and 24 list the cifs performance and functional enhancements by kernel release. Buffered, sequential read (e.g. file copy from a server) got much faster in 3.2 kernel, especially to Samba and other server which support the Unix extensions (due to support for larger i/o sizes than 64K).
Similarly note that cifs write speed was dramatically improved starting at kernel version 3.0 (1.5 to 2 years ago) due to the addition of more async parallelism to the design of the cifs write code (writing to the server from the cifs client) by making the i/o sizes larger and allowing more async dispatch of writes (previously to use a network interface fully you would need to be reading and or writing to multiple different files simultaneously).
Quoted 33 lines · Jeff Blaine
Hi,
On a RHEL 6.3 box talking to a Windows 7 Enterprise box, I am seeing approximately 1/4th the speed with mount.cifs as I am with smbclient 'get'. RHEL 6.3 currently has CIFS 1.68.
After about a half hour of reading forum threads for the last few years, it seems this is very well known and has been the case for a long time.
I have tried using CIFSMaxBufSize=61440 with rsize=61140 at mount-time and it doesn't really buy me much.
Is there any sort of public-facing summary of the state of the read performance issues. I saw no mention of it in the BUGS section of the mount.cifs man page or in the README for the kernel module.
Is the cause known?
Has already been fixed since 1.68 by chance? If so, what assembly of pieces will overcome the issue? Should I just open a RHEL bug through our support channel and get them involved in this effort somehow?
Any guidance would be welcome at this point.
Jeff
--
To unsubscribe from this list: send the line "unsubscribe linux-cifs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 7 lines · On Tue, 05 Feb 2013 12:12:22 -0500 Jeff Blaine…
Thank you for the thorough reply, Steve. It's nice to read of the progress in 2012, but RHEL 6.3 and what is supported there is what we have to work with. It is the latest supported version, as you surely know, so it seems like we'll have to wait quite awhile before getting the module's read performance increases.
Quoted 8 lines
We're dealing with 9 US sites, nightly time window (which we're exceeding) for transferring large amounts of data over 1.5Mbps links).
Back to the drawing board.
Thanks again.
I think you're basically out of luck for now...
Long haul links are really a pessimal case for the older code that did synchronous reads and writes. You spend most of your time waiting around for the calls to go back and forth.
I believe Sachin is looking at backporting the async read patches for 6.5. The async write code is already in 6.3 (I think). You should definitely open a support case since documented customer demand helps us make the case for including these sorts of changes in an update.
Signature · 1 line
Jeff Layton <[email protected]>
A message that is not in the archive (sent off-list or not yet received): 20130205142058.7618dfcf-4QP7MXygkU+dMjc06nkz3ljfA9RmPOcC@public.gmane.org
Quoted 21 lines · On Tue, 5 Feb 2013 14:20:58 -0500 Jeff Layton…
Quoted 7 lines · On Tue, 05 Feb 2013 12:12:22 -0500 Jeff Blaine…
Thank you for the thorough reply, Steve. It's nice to read of the progress in 2012, but RHEL 6.3 and what is supported there is what we have to work with. It is the latest supported version, as you surely know, so it seems like we'll have to wait quite awhile before getting the module's read performance increases.
Quoted 8 lines
We're dealing with 9 US sites, nightly time window (which we're exceeding) for transferring large amounts of data over 1.5Mbps links).
Back to the drawing board.
Thanks again.
I think you're basically out of luck for now...
Quoted 9 lines
Long haul links are really a pessimal case for the older code that did synchronous reads and writes. You spend most of your time waiting around for the calls to go back and forth.
I believe Sachin is looking at backporting the async read patches for 6.5. The async write code is already in 6.3 (I think). You should definitely open a support case since documented customer demand helps us make the case for including these sorts of changes in an update.
...oh, and if you haven't already, you should try to make some time for testing a more recent kernel to ensure that this has been addressed there. Consider doing something like installing Fedora and testing the read speeds over your long haul link.
It wouldn't do you any good for us to backport those changes if they don't actually help your use-case.
Signature · 1 line
Jeff Layton <[email protected]>
A message that is not in the archive (sent off-list or not yet received): 20130205142307.58d87a21-4QP7MXygkU+dMjc06nkz3ljfA9RmPOcC@public.gmane.org
Quoted 4 lines
Quoted 4 lines
I believe Sachin is looking at backporting the async read patches for 6.5. The async write code is already in 6.3 (I think). You should definitely open a support case since documented customer demand helps us make the case for including these sorts of changes in an update.
I opened one an hour or so ago, so that's done.
Should I test with Fedora 17 or Fedora 18 for the sake of this? Just because 18 is newer doesn't mean it's the most appropriate for this test need, eh?
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 10 lines · On Tue, 05 Feb 2013 15:02:52 -0500 Jeff Blaine…
Quoted 4 lines
Quoted 4 lines
I believe Sachin is looking at backporting the async read patches for 6.5. The async write code is already in 6.3 (I think). You should definitely open a support case since documented customer demand helps us make the case for including these sorts of changes in an update.
I opened one an hour or so ago, so that's done.
Should I test with Fedora 17 or Fedora 18 for the sake of this? Just because 18 is newer doesn't mean it's the most appropriate for this test need, eh?
In general, later is better, but either should be fine. The async read code went into mainline about a year or so ago, so both should have it.
Signature · 1 line
Jeff Layton <[email protected]>
[ suggestion by Jeff Layton to try to ensure that a modern ] [ kernel addresses the issues we were seeing over WAN links ]
Just following up.
* Fedora 18 CIFS-over-WAN read performance, as recorded with the same testing that was done previously, is ~3.4x that of RHEL 6.3 (or any common production distro running a pre-3.2 kernel).
So yes, the work done to CIFS addresses our situation directly.
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 10 lines · Jeff Blaine
[ suggestion by Jeff Layton to try to ensure that a modern ] [ kernel addresses the issues we were seeing over WAN links ]
Just following up.
* Fedora 18 CIFS-over-WAN read performance, as recorded with the same testing that was done previously, is ~3.4x that of RHEL 6.3 (or any common production distro running a pre-3.2 kernel).
So yes, the work done to CIFS addresses our situation directly.
That is great - hope the SMB2.1 leases (and SMB3 directory leases) will help even more over WAN as we look to the future
Signature · 3 lines
Thanks, Steve