fixing slow sync(2)
The issue with sync(2) being slow has been known for a while now (I can find discussions as early as July 2010). But nobody seems to be willing to fix it or even figure out out what the actual problem is. I use btrfs on two hosts, and the sync(2) syscall is consistently slow. By slow I mean 20 seconds on one and 45(!) seconds on the other. Every single time, no exceptions.
Is there anything I can do to help track down the cause? I can test patches, trace the kernel or use whatever tool is most helpful.
tom
Quoted 10 lines · Tomas Carnecky
The issue with sync(2) being slow has been known for a while now (I can find discussions as early as July 2010). But nobody seems to be willing to fix it or even figure out out what the actual problem is. I use btrfs on two hosts, and the sync(2) syscall is consistently slow. By slow I mean 20 seconds on one and 45(!) seconds on the other. Every single time, no exceptions.
Is there anything I can do to help track down the cause? I can test patches, trace the kernel or use whatever tool is most helpful.
I think I fixed this, try my git tree
git://git.kernel.org/pub/scm/linux/kernel/git/josef/btrfs-work.git
Let me know if it helps. And what are you doing when you call sync? I've not been able to reproduce this problem so I'm having a hard time nailing down what it is, so if I can get a reliable way to reproduce it I'll try and figure it out. Thanks,
Josef
A message that is not in the archive (sent off-list or not yet received): CAE=-PDhvddavk9DyyF07gi0uq=a+WfGbXN89r28SRK9SYPaZfA@mail.gmail.com
Resending as plain text.
---------- Forwarded message ----------
=46rom: John <[email protected]>
Date: Mon, Oct 10, 2011 at 9:34 PM
Subject: Re: fixing slow sync(2)
To: Josef Bacik <[email protected]>
Cc: Tomas Carnecky <[email protected]>, [email protected]
Quoted 6 lines · Josef Bacik
I think I fixed this, try my git tree
git://git.kernel.org/pub/scm/linux/kernel/git/josef/btrfs-work.gitLet me know if it helps. =A0And what are you doing when you call sync=
? =A0I've not
Quoted 1 line
been able to reproduce this problem so I'm having a hard time nailing=
down what
Quoted 1 line
it is, so if I can get a reliable way to reproduce it I'll try and fi=
gure it
Quoted 1 line
out. =A0Thanks,
I've been seeing very slow syncs (and unmounts) and this helps. Under 3.0.0 if I mounted the FS (which is 200G, about 97G used with ~720 snapshots) it took about 3 minutes 40 seconds. If I did another sync right away it took the same amount of time. With a kernel compiled from your repo the first time I sync it takes about 1 minute, as soon as I sync a=A0btrfs-endio-met and=A0btrfs-cache-0 process show up in to= p
and a lot of IO happens. Once those go away (which takes a couple minutes) then it takes well under a second. If I unmount and remount it acts exactly the same. Will those changes be up in 3.1 or 3.2? If you want me to test anything else, I'm happy to do so. Thanks much.
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" = in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 16 lines · John
Quoted 5 lines · Josef Bacik
I think I fixed this, try my git tree
git://git.kernel.org/pub/scm/linux/kernel/git/josef/btrfs-work.gitQuoted 8 lines
Let me know if it helps. And what are you doing when you call sync? I've not
been able to reproduce this problem so I'm having a hard time nailing down what
it is, so if I can get a reliable way to reproduce it I'll try and figure it
out. Thanks,
Quoted 10 lines
I've been seeing very slow syncs (and unmounts) and this helps. Under 3.0.0 if I mounted the FS (which is 200G, about 97G used with ~720 snapshots) it took about 3 minutes 40 seconds. If I did another sync right away it took the same amount of time. With a kernel compiled from your repo the first time I sync it takes about 1 minute, as soon as I sync a btrfs-endio-met and btrfs-cache-0 process show up in top and a lot of IO happens. Once those go away (which takes a couple minutes) then it takes well under a second. If I unmount and remount it acts exactly the same. Will those changes be up in 3.1 or 3.2? If you want me to test anything else, I'm happy to do so.
They'll show up in 3.2. Thanks for testing,
Josef
Quoted 3 lines · Josef Bacik
I think I fixed this, try my git tree
git://git.kernel.org/pub/scm/linux/kernel/git/josef/btrfs-work.git
I wanted to Ack this as well - 3.1-rc4 was completely unusable when firefox was running (30+ second pauses to read directories, btrfs threads were constantly running, even the mouse was jerky due to the load)
Built from your tree (fa5cf66) and everything works like it should again. No load, fast response to IO requests.