USB-Storage, slow speed with sync option
Hi folks,
I'm not sure if this is the correct list for my problem. So bare with me.
When I copy files to my USB 2.0 Stick , my transferspeed are horrible in case I mount the device with sync option, the rates are around 50KB/s, doesn't really matter how big the files are.
If I don't mount with option sync, then the speeds are as expected a couple of MB/s
I don't think this is the default behaviour, right?
Anybody got an idea what could be wrong?
Filesystem is vfat.
I tried with different kernels and a live cd(knoppix), currently I use kernel 2.6.13.
Motherboard is an NFORCE 4 based one:
0000:00:02.0 USB Controller: nVidia Corporation CK804 USB Controller (rev a2)
0000:00:02.1 USB Controller: nVidia Corporation CK804 USB Controller (rev a3)
USB Info shows the device using ehci_hcd module
Thanks in advance, Stefan
___________________________________________________________
Gesendet von Yahoo! Mail - Jetzt mit 1GB Speicher kostenlos - Hier anmelden: http://mail.yahoo.de
Quoted 5 lines · Stefan
When I copy files to my USB 2.0 Stick , my transferspeed are horrible in case I mount the device with sync option, the rates are around 50KB/s, doesn't really matter how big the files are.
If I don't mount with option sync, then the speeds are as expected a couple of MB/s
Now do a complete benchmark and type "sync" after the cp command.
Quoted 11 lines · Erik Slagter
Quoted 7 lines · Stefan
When I copy files to my USB 2.0 Stick , my transferspeed are horrible in case I mount the device with sync option, the rates are around 50KB/s, doesn't really matter how big the files are.
If I don't mount with option sync, then the speeds are as expected a couple of MB/s
Quoted 3 lines
Now do a complete benchmark and type "sync" after the cp command.
I already did that:-)
With sync option, a test directory takes about 15min to copy. Without sync option about 20s and running command sync afterwards takes about 5-10s.
So roughly 15min compared to 30s
___________________________________________________________
Gesendet von Yahoo! Mail - Jetzt mit 1GB Speicher kostenlos - Hier anmelden: http://mail.yahoo.de
Quoted 7 lines · Stefan
Quoted 6 lines
Quoted 5 lines
When I copy files to my USB 2.0 Stick , my transferspeed are horrible in case I mount the device with sync option, the rates are around 50KB/s, doesn't really matter how big the files are.
If I don't mount with option sync, then the speeds are as expected a couple of MB/sNow do a complete benchmark and type "sync" after the cp command.
Quoted 7 lines
I already did that:-)
With sync option, a test directory takes about 15min to copy. Without sync option about 20s and running command sync afterwards takes about 5-10s.
So roughly 15min compared to 30s
The problem is that with the "sync" option the metadata on the stick has be written on every block write. That is not only terribly slow (especially on a flash device), it also wears out certain sectors. Compare it to using floppy's on a MS/DOS computer.
Why would you _ever_ want to use the sync mount option on a flash device?
Quoted 26 lines · Erik Slagter
Quoted 13 lines · Stefan
Quoted 10 lines
Quoted 7 lines
When I copy files to my USB 2.0 Stick , my transferspeed are horrible in case I mount the device with sync option, the rates are around 50KB/s, doesn't really matter how big the files are.
If I don't mount with option sync, then the speeds are as expected a couple of MB/sNow do a complete benchmark and type "sync" after the cp command.
Quoted 9 lines
I already did that:-)
With sync option, a test directory takes about 15min to copy. Without sync option about 20s and running command sync afterwards takes about 5-10s.
So roughly 15min compared to 30s
Quoted 9 lines
The problem is that with the "sync" option the metadata on the stick has be written on every block write. That is not only terribly slow (especially on a flash device), it also wears out certain sectors. Compare it to using floppy's on a MS/DOS computer.
Why would you _ever_ want to use the sync mount option on a flash device?
Don't ask me:-)
I used to mount without sync option, but I switched to hal + dbus now and with gentoo it automatically sets sync option in fstab :-(
Your explanation makes sense and I kind of had the same thought at first, but also was unsure since somebody must have put it there, therefore I'm going to check how to remove sync from those scripts that set the mountpoint in fstab and discuss with the gentoo dev who set this up.
Thanks for your help Erik
___________________________________________________________
Gesendet von Yahoo! Mail - Jetzt mit 1GB Speicher kostenlos - Hier anmelden: http://mail.yahoo.de
Stefan wrote the following on 14.09.2005 12:43 :
Quoted 10 lines
Quoted 7 lines · Erik Slagter
Why would you _ever_ want to use the sync mount option on a flash device?
Quoted 6 lines
Don't ask me:-)
I used to mount without sync option, but I switched to hal + dbus now and with gentoo it automatically sets sync option in fstab :-(
This is purely a PBKAK :
users want to remove the device as soon as they get feedback from their UI telling them that their transfer is done (in fact they even screw up and remove the device before it is done). sync in fstab isn't particularly clever, but sync with automount is more useful : when properly configured, as soon as the transfer is done the automounter can cleanly umount the block device which is nearly instantaneous when sync is used. Without sync there can be some awfully long delays with flash and huge caches (on a 512M slow SD, I saw 30+ seconds after transfering 10s or 100s of MB).
This is understandable, for all the removable devices without a clean "eject" and VFAT formatted, using sync is probably the best safeguard against badly screwed up filesystems.
Lionel.
Quoted 18 lines · Lionel Bouton
Quoted 6 lines
Quoted 2 lines
Why would you _ever_ want to use the sync mount option on a flash device?
Don't ask me:-)
I used to mount without sync option, but I switched to hal + dbus now and with gentoo it automatically sets sync option in fstab :-(
This is purely a PBKAK :
users want to remove the device as soon as they get feedback from their UI telling them that their transfer is done (in fact they even screw up and remove the device before it is done). sync in fstab isn't particularly clever, but sync with automount is more useful : when properly configured, as soon as the transfer is done the automounter can cleanly umount the block device which is nearly instantaneous when sync is used. Without sync there can be some awfully long delays with flash and huge caches (on a 512M slow SD, I saw 30+ seconds after transfering 10s or 100s of MB).
Quoted 3 lines
This is understandable, for all the removable devices without a clean "eject" and VFAT formatted, using sync is probably the best safeguard against badly screwed up filesystems.
It's a very bad solution, because it's slow, but worse, it wears out particular sectors, although the only good option (make the stick non-removable when in use is not quite feasible).
People must be instructed to eject() before plug out. Just like floppies.
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 2 lines · Greg Freemyer
Maybe a bad idea, but SuSE for one has been doing sync on automount of USB devices since at least 9.1 (18 months ago)
It seems a better idea to me to tune bd/pdflush/kflushd parameters to make it flush buffers sooner.
Grmbl, back to windows semantics :-(
Quoted 4 lines · Erik Slagter
It's a very bad solution, because it's slow, but worse, it wears out particular sectors, although the only good option (make the stick non-removable when in use is not quite feasible).
Bad, YES. Slow, VERY.
Wears out sectors? Not really.
These flash sticks, along with CF and SD cards, all include built-in controllers that do automatic wear-leveling.
So rewriting a small group of "logical" sectors over and over does not, in fact, rewrite the same physical pages in flash.
But SYNC is still horrible for this application!
Mark Lord wrote the following on 14.09.2005 17:26 :
Quoted 8 lines
Quoted 4 lines · Erik Slagter
It's a very bad solution, because it's slow, but worse, it wears out particular sectors, although the only good option (make the stick non-removable when in use is not quite feasible).
Quoted 5 lines
Bad, YES. Slow, VERY.
Wears out sectors? Not really.
These flash sticks, along with CF and SD cards, all include built-in controllers that do automatic wear-leveling.
Do they all have such controllers now? Last time I checked, low quality parts didn't have them and it wasn't easy to verify which ones did.
Quoted 5 lines · Lionel Bouton
Mark Lord wrote the following on 14.09.2005 17:26 :
Quoted 2 lines
These flash sticks, along with CF and SD cards, all include built-in controllers that do automatic wear-leveling.
Do they all have such controllers now? Last time I checked, low quality parts didn't have them and it wasn't easy to verify which ones did.
Pretty much all of them use NAND flash, which comes with 1-3% bad blocks from the factory. So the onboard controllers already have to deal with *that* issue, and the same remapping logic is normally extended to allow them to do wear leveling as well.
Wear leveling is imperfect, in that they devices can only include sectors *known* by the *device* to be inactive, so as files are written and deleted many sectors are removed from the wear rotation until the next time they are logically rewritten (or until a CFA ERASE command, which Linux never does. Sad.).
Cannot say for sure whether 100% of controllers are adequately intelligent, but that kind of control logic is pretty pervasive these days. It would be a very rare device indeed that lacks it.
Cheers
Quoted 19 lines · Lionel Bouton
Stefan wrote the following on 14.09.2005 12:43 :
Quoted 10 lines
Quoted 6 lines · Erik Slagter
Why would you _ever_ want to use the sync mount option on a flash device?
Quoted 6 lines
Don't ask me:-)
I used to mount without sync option, but I switched to hal + dbus now and with gentoo it automatically sets sync option in fstab :-(
Quoted 8 lines
This is purely a PBKAK :
users want to remove the device as soon as they get feedback from their UI telling them that their transfer is done (in fact they even screw up and remove the device before it is done). sync in fstab isn't particularly clever, but sync with automount is more useful : when properly configured, as soon as the transfer is done the automounter can cleanly umount the block device which is nearly instantaneous when sync is used.
I guess that is what suse must have setup, since I just checked there and it works smooth although they use sync option.
Quoted 10 lines
Without sync there can be some awfully long delays with flash and huge caches (on a 512M slow SD, I saw 30+ seconds after transfering 10s or 100s of MB).
This is understandable, for all the removable devices without a clean "eject" and VFAT formatted, using sync is probably the best safeguard against badly screwed up filesystems.
Lionel.
___________________________________________________________
Gesendet von Yahoo! Mail - Jetzt mit 1GB Speicher kostenlos - Hier anmelden: http://mail.yahoo.de