short term task list for Reiser4
Please feel free to comment on this list and the order of its tasks:
0) fix all bugs as they arise
1) get batch_write into the -mm kernel --- small task
2) get read optimization code into the -mm kernel (coded and probably debugged but not fully tested and not sent in yet) --- small task
3) get EVERYTHING into wiki (migration has started already, thanks flx).
4) review complaints of pauses while using reiser4 --- size of task unknown, and it is also unknown how much we may have fixed it while writing recent patches.
5) review crypt-compress code --- full code review --- substantive task
6) optimize fsync --- substantive task which requires using fixed area for write twice logging, and using write twice logging for fsync'd data. It might require creating mount options to choose whether to optimize for serialized sequential fsyncs vs. lazy fsyncs.
7) review all of our installation instructions --- I am already doing that, but volunteers who will help out our wiki would be sorely appreciated. Installing reiser4 as the root for each distro needs step-by-step instructions.
8) review our kernel documentation --- I should do that but when will I have time?
Unfortunately, our code stability is going to decrease for a bit due to all these changes to the read and write code --- no way to cure that but
passage of time. On the other hand, our CPU usage went way down.
Reiser4's only performance weakness now is fsync.
Once the crypt-compress code is ready, we will release Reiser4.1-beta (with plugins, releasing a beta means telling users that if they mount -o reiser4.1-beta then cryptcompress will be their default plugin, and if they don't, then they are using Reiser4.0 still). Doubling our performance and halving our disk usage is going to be fun.
Quoted 15 lines · Hans Reiser
Please feel free to comment on this list and the order of its tasks:
0) fix all bugs as they arise
1) get
batch_writeinto the -mm kernel --- small task2) get read optimization code into the -mm kernel (coded and probably debugged but not fully tested and not sent in yet) --- small task
3) get EVERYTHING into wiki (migration has started already, thanks flx).
4) review complaints of pauses while using reiser4 --- size of task unknown, and it is also unknown how much we may have fixed it while writing recent patches.
Quoted 6 lines
5) review crypt-compress code --- full code review --- substantive task
6) optimize fsync --- substantive task which requires using fixed area for write twice logging, and using write twice logging for fsync'd data. It might require creating mount options to choose whether to optimize for serialized sequential fsyncs vs. lazy fsyncs.
With the serialized sequential fsync, is that essentially what I was talking about earlier with slowly streaming dirty writes to disk when the HDD is idle? If that's the case, I don't see the advantage in having lazy fsyncs except in situations where you want to keep the HDD spun down as much as possible. If you keep as much of the writes in RAM as you would have if you used lazy fsyncs, then you get the same temporal locality speed up, with the added advantages that you can clear the RAM under memory pressure immediately and crashes result in lower likelyhood of data loss than with lazy fsync. I suppose it isn't a bad idea to give people more options (unless you're a GNOME UI developer :-P), but at the very least I think that slow streaming to disk would be a very wise default option.
My CS focus is Human Interfaces, so I may be way out of my league here with FSs, but I thought I'd still throw in my two cents.
Quoted 5 lines
7) review all of our installation instructions --- I am already doing that, but volunteers who will help out our wiki would be sorely appreciated. Installing reiser4 as the root for each distro needs step-by-step instructions.
I've been meaning to hose my laptop (assuming I fix one problem with my desktop), so I am willing to help write Gentoo install docs (or possibly Arch Linux). I can also test exsiting instructions.
Quoted 9 lines
8) review our kernel documentation --- I should do that but when will I have time?
Unfortunately, our code stability is going to decrease for a bit due to all these changes to the read and write code --- no way to cure that but
passage of time. On the other hand, our CPU usage went way down.
Reiser4's only performance weakness now is fsync.
Quoted 5 lines
Once the crypt-compress code is ready, we will release Reiser4.1-beta (with plugins, releasing a beta means telling users that if they mount -o reiser4.1-beta then cryptcompress will be their default plugin, and if they don't, then they are using Reiser4.0 still). Doubling our performance and halving our disk usage is going to be fun.
--Clay
Quoted 10 lines · Clay Barnes
Quoted 7 lines · Hans Reiser
6) optimize fsync --- substantive task which requires using fixed area for write twice logging, and using write twice logging for fsync'd data. It might require creating mount options to choose whether to optimize for serialized sequential fsyncs vs. lazy fsyncs.
Quoted 5 lines
With the serialized sequential fsync, is that essentially what I was talking about earlier with slowly streaming dirty writes to disk when the HDD is idle? If that's the case, I don't see the advantage in having lazy fsyncs
if you are optimizing throughput rather than latency, then you let things get to disk whenever they get there, and you let the app hang while it waits. A mailer processing many requests in parallel might find 30 seconds of latency to be just fine but a database might find 3 seconds of latency to be too much. (I make up these examples, mailer programmers please correct me.)
Quoted 3 lines
except in situations where you want to keep the HDD spun down as much as possible.
No, that is not when you do it.
Quoted 5 lines
I've been meaning to hose my laptop (assuming I fix one problem with my desktop), so I am willing to help write Gentoo install docs (or possibly Arch Linux). I can also test exsiting instructions.
That would be way cool.
<http://wiki.namesys.com/Reiser4-GettingStarted#preview>
Quoted 10 lines · Hans Reiser
Quoted 9 lines · Clay Barnes
Quoted 4 lines · Hans Reiser
6) optimize fsync --- substantive task which requires using fixed area for write twice logging, and using write twice logging for fsync'd data. It might require creating mount options to choose whether to optimize for serialized sequential fsyncs vs. lazy fsyncs.
With the serialized sequential fsync, is that essentially what I was talking about earlier with slowly streaming dirty writes to disk when the HDD is idle? If that's the case, I don't see the advantage in having lazy fsyncs
Quoted 6 lines
if you are optimizing throughput rather than latency, then you let things get to disk whenever they get there, and you let the app hang while it waits. A mailer processing many requests in parallel might find 30 seconds of latency to be just fine but a database might find 3 seconds of latency to be too much. (I make up these examples, mailer programmers please correct me.)
I see your point, but here's where I'm still uncertain:
If you have a lazy write policy, what exactly is gained by intentionally delaying writes (beyond a certain size that is necessary to make things like dancing trees actually effecient)? If you trickle some data to disk, then when memory pressure causes (or an app calls) a big sync, then you have less to actually write. What I'm suggesting, now, is not a major write policy change, but rather a light process that is limited to extremely low resource use (I/O, CPU, etc.). It would take some of the edge off of major syncs, and for many (most?) non-server users, it could wholly eliminate memory pressure-induced heavy syncs.
--Clay
Reiser4 generally could use some knobs controlling things like whether we trickle data slowly and continuously
Quoted 16 lines
If you have a lazy write policy, what exactly is gained by intentionally delaying writes (beyond a certain size that is necessary to make things like dancing trees actually effecient)? If you trickle some data to disk, then when memory pressure causes (or an app calls) a big sync, then you have less to actually write. What I'm suggesting, now, is not a major write policy change, but rather a light process that is limited to extremely low resource use (I/O, CPU, etc.). It would take some of the edge off of major syncs, and for many (most?) non-server users, it could wholly eliminate memory pressure-induced heavy syncs.
--Clay
Will there be any NUMA /SMP fixes? Reiserfs performance is severely impacted at higher processor counts by the mandatory centralized locking in both read and write paths in the filesystem.
Quoted 7 lines · Christoph Lameter
Will there be any NUMA /SMP fixes? Reiserfs performance is severely impacted at higher processor counts by the mandatory centralized locking in both read and write paths in the filesystem.
Reiserfs or Reiser4?
Quoted 6 lines · Hans Reiser
Quoted 3 lines · Christoph Lameter
Will there be any NUMA /SMP fixes? Reiserfs performance is severely impacted at higher processor counts by the mandatory centralized locking in both read and write paths in the filesystem.
Reiserfs or Reiser4?
Either.
Quoted 10 lines · Hans Reiser
Please feel free to comment on this list and the order of its tasks:
4) review complaints of pauses while using reiser4 --- size of task unknown, and it is also unknown how much we may have fixed it while writing recent patches.6) optimize fsync --- substantive task which requires using fixed area for write twice logging, and using write twice logging for fsync'd data. It might require creating mount options to choose whether to optimize for serialized sequential fsyncs vs. lazy fsyncs.
I believe those two are related. I'm having the "pauses" (of many minutes at times!) when writing to reiser4. It seems it is triggered mostly by the use of fsync(); NFS in sync mode manages to trigger it way often: I mount my old desktop's home from my new computer via synced NFS.
The pauses consist in the application being frozen and the other applications being slowed down on their IO operations, while the disks writes data continously during an interval that usually last a minute or so, but may last many times that. It often happens with small files (like, when sending a mail, as it passes through the MTA), so I believe it probably (re)writes to disk lots of stuff that doesn't need to be written to disk at all.
It only happens on reiser4 patches against 2.6.13 or newer, 2.6.12.x is fine.
Signature · 1 line
- Do you study theories? - Oh yes! Theories are fun. - How do you use them? - When I must act, theories are too slow. To act, I must know.
Quoted 15 lines · rvalles
Quoted 5 lines · Hans Reiser
Please feel free to comment on this list and the order of its tasks:
4) review complaints of pauses while using reiser4 --- size of task unknown, and it is also unknown how much we may have fixed it while writing recent patches.Quoted 7 lines
6) optimize fsync --- substantive task which requires using fixed area for write twice logging, and using write twice logging for fsync'd data. It might require creating mount options to choose whether to optimize for serialized sequential fsyncs vs. lazy fsyncs.
Quoted 16 lines
I believe those two are related. I'm having the "pauses" (of many minutes at times!) when writing to reiser4. It seems it is triggered mostly by the use of
fsync(); NFS in sync mode manages to trigger it way often: I mount my old desktop's home from my new computer via synced NFS.
The pauses consist in the application being frozen and the other applications being slowed down on their IO operations, while the disks writes data continously during an interval that usually last a minute or so, but may last many times that. It often happens with small files (like, when sending a mail, as it passes through the MTA), so I believe it probably (re)writes to disk lots of stuff that doesn't need to be written to disk at all.
It only happens on reiser4 patches against 2.6.13 or newer, 2.6.12.x is fine.
So the pauses are experienced by a process waiting on fsync to finish? If yes, then the problem is a very different issue from what I thought....
Hans
Quoted 13 lines · Christoph Lameter
Quoted 9 lines · Hans Reiser
Quoted 5 lines · Christoph Lameter
Will there be any NUMA /SMP fixes? Reiserfs performance is severely impacted at higher processor counts by the mandatory centralized locking in both read and write paths in the filesystem.
Quoted 10 lines
Quoted 3 lines
Reiserfs or Reiser4?
Either.
What big locks are you running into in V4? Please tell me more.
Hans
Quoted 17 lines · On Fri, 14 Jul 2006 00:01:49 -0700 Hans Reiser <[email protected]> wrote:
Quoted 6 lines · rvalles
I believe those two are related. I'm having the "pauses" (of many minutes at times!) when writing to reiser4. It seems it is triggered mostly by the use of
fsync(); NFS in sync mode manages to trigger it way often: I mount my old desktop's home from my new computer via synced NFS.
The pauses consist in the application being frozen and the otherQuoted 10 lines
applications being slowed down on their IO operations, while the disks writes data continously during an interval that usually last a minute or so, but may last many times that. It often happens with small files (like, when sending a mail, as it passes through the MTA), so I believe it probably (re)writes to disk lots of stuff that doesn't need to be written to disk at all.
It only happens on reiser4 patches against 2.6.13 or newer, 2.6.12.x is fine.
Quoted 3 lines
So the pauses are experienced by a process waiting on fsync to finish? If yes, then the problem is a very different issue from what I thought....
I am pretty sure I triggered the same a couple of times with aMule, which was downloading and uploading about 20-30kB per second in each direction, and randomly accessing larger files. Doing a sync then suddenly caused the disk seek like crazy and write about 500kB per second constantly. I waited about a minute, and then better killed aMule. Could it be possible that sync tries to sync the writes which arrive during the operation and cannot catch up?
Signature · 1 line
Jindrich Makovicka
Quoted 31 lines · Jindrich Makovicka
Quoted 21 lines · On Fri, 14 Jul 2006 00:01:49 -0700 Hans Reiser <[email protected]> wrote:
Quoted 6 lines · rvalles
I believe those two are related. I'm having the "pauses" (of many minutes at times!) when writing to reiser4. It seems it is triggered mostly by the use of
fsync(); NFS in sync mode manages to trigger it way often: I mount my old desktop's home from my new computer via synced NFS.
The pauses consist in the application being frozen and the otherQuoted 12 lines
applications being slowed down on their IO operations, while the disks writes data continously during an interval that usually last a minute or so, but may last many times that. It often happens with small files (like, when sending a mail, as it passes through the MTA), so I believe it probably (re)writes to disk lots of stuff that doesn't need to be written to disk at all.
It only happens on reiser4 patches against 2.6.13 or newer, 2.6.12.x is fine.Quoted 5 lines
So the pauses are experienced by a process waiting on fsync to finish? If yes, then the problem is a very different issue from what I thought....
Quoted 10 lines
I am pretty sure I triggered the same a couple of times with aMule, which was downloading and uploading about 20-30kB per second in each direction, and randomly accessing larger files. Doing a sync then suddenly caused the disk seek like crazy and write about 500kB per second constantly. I waited about a minute, and then better killed aMule. Could it be possible that sync tries to sync the writes which arrive during the operation and cannot catch up?
zam, can you answer this, because he is right that if the answer is yes, then we have a bit of a problem we need to address. I worry that, assuming the answer is yes, that one process doing a lot of fsync, could muck up the performance of everyone its atoms fuse with if we expire the atom with every fsync, but if we don't then.....
Jindrich, don't let us forget this issue before we fix it, ok? Thanks,
Hans
Just wanted to mention that I have experienced pauses also. I use gentoo, and have triggered the problem when saving a small text file in vim while emerging in the background. Upon save and exit (:wq) vim will just hang there anywhere from 5-30 seconds. Let me know if/how I can help.