Skip to contentSkip to conversation

very poor ext3 write performance on big filesystems?

DISCUSSION Paul Slootman 1 participant 1 message Feb 19, 2008 6 messages not in the archive

lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): [email protected]

#6 lore raw
Quoted 11 lines · Andi Kleen
Quoted 6 lines · Theodore Tso
Quoted 3 lines · Tomasz Chmielewski

I tried to copy that filesystem once (when it was much smaller) with "rsync -a -H", but after 3 days, rsync was still building an index and didn't copy any file.

If you're going to copy the whole filesystem don't use rsync!

Yes, I managed to kill systems (drive them really badly into oom and get very long swap storms) with rsync -H in the past too. Something is very wrong with the rsync implementation of this.

Note that the soon-to-be-released version 3.0.0 of rsync has very much improved performance here, both in speed and memory usage, thanks to a new incremental transfer protocol (before it read in the complete file list, first on the source, then on the target, and then only started to do the actual work).

Paul Slootman