Reiser4, will double Linux FS performance, pleaseapply
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Andreas" == Andreas Dilger <[email protected]> writes:
Andreas> I think the bdflush defaults are (were?) something like 5 Andreas> seconds for metadata, and 30 seconds for file data. reiser4 Andreas> should (if it doesn't already) use the parameters set by Andreas> sys_bdflush() to tune the writeout intervals.
...
Andreas> So, except for the very unusual case of files with lifespans Andreas> between 30 seconds and 300 seconds, or files that are written Andreas> to between those intervals, I would guess that you are not Andreas> gaining much extra benefit by deferring the writes another Andreas> 270 seconds.
Some benchmarking done at Berkeley showed that for development loads, 30seconds was too short to avoid excessive writes.
See Roselli, Lorch and Anderson, `A Comparison of File System Workloads' in Usenix 2000.
http://research.microsoft.com/~lorch/papers/fs-workloads/fs-workloads.html
Their observations (summarised) were that most blocks die because of overwriting, not because of file deletes. Their workloads show that for NT, the write timeout to avoid commits blocks that will soon become dead needs to be around a day; for typical Unix loads (web serving, research, software development), an hour is enough. To catch 75%, a timeout of around 11 minutes is needed. 30seconds worked only for webserving and undergraduate teaching workloads, and caught around 40% for those workloads; for a research workload and NT fileserving, 30seconds catches only 10-20% of the rewrites.
See especially figure 3 in that paper.
--
Dr Peter Chubb [email protected]
You are lost in a maze of BitKeeper repositories, all almost the same.
Quoted 19 lines · Peter Chubb
Some benchmarking done at Berkeley showed that for development loads, 30seconds was too short to avoid excessive writes.
See Roselli, Lorch and Anderson, `A Comparison of File System Workloads' in Usenix 2000.
http://research.microsoft.com/~lorch/papers/fs-workloads/fs-workloads.html
Their observations (summarised) were that most blocks die because of overwriting, not because of file deletes. Their workloads show that for NT, the write timeout to avoid commits blocks that will soon become dead needs to be around a day; for typical Unix loads (web serving, research, software development), an hour is enough. To catch 75%, a timeout of around 11 minutes is needed. 30seconds worked only for webserving and undergraduate teaching workloads, and caught around 40% for those workloads; for a research workload and NT fileserving, 30seconds catches only 10-20% of the rewrites.
Quoted 4 lines
See especially figure 3 in that paper.
There is also a longer PhD thesis by her. 10 minutes is about as much work as I personally am willing to lose and try to remember. Avoiding 75% of writes instead of 20% is a substantial performance gain worth paying a cost for. Unfortunately it is not easy to say if it is worth that much cost, but I suspect it is. An approach we are exploring is for blocks to reach disk earlier than that if the device is not congested, on the grounds that if not much IO is occuring, then performance is not important.
Hans
Am Mit, 2002-11-06 um 02.33 schrieb reiser:
Quoted 8 lines
There is also a longer PhD thesis by her. 10 minutes is about as much work as I personally am willing to lose and try to remember. Avoiding 75% of writes instead of 20% is a substantial performance gain worth paying a cost for. Unfortunately it is not easy to say if it is worth that much cost, but I suspect it is. An approach we are exploring is for blocks to reach disk earlier than that if the device is not congested, on the grounds that if not much IO is occuring, then performance is not important.
Assuming your 10 minutes are just a default and tunable by sysctl I hardly can see any problems at all. Paranoid people can set it to make any tradeoff between performance and speed they'd like including setting it to 0, no?
Signature · 2 lines
Servus,
DanielQuoted 8 lines · reiser
There is also a longer PhD thesis by her. 10 minutes is about as much work as I personally am willing to lose and try to remember. Avoiding 75% of writes instead of 20% is a substantial performance gain worth paying a cost for. Unfortunately it is not easy to say if it is worth that much cost, but I suspect it is. An approach we are exploring is for blocks to reach disk earlier than that if the device is not congested, on the grounds that if not much IO is occuring, then performance is not important.
I would certainly like to see that, lost data in case of problems is
more of a problem than performance for many people.
Particularly if (a) there is an idle CPU, (b) there are no blocks queued
for write to the device, and (c) there are dirty blocks to go to the device, it would be good to ignore the age of the block or use a firly low minimum age. If we dropped a few blocks onto the drive each time the conditions were met, I suspect that with many systems that would result in a lot more free write space in memory. The total blocks written to the drive would go up, but it shouldn't hurt performance.
My first thought is that the check would be done after finding no
runable normal processes and before running batch priority processes. If only a few blocks were written each time oldest first it shouldn't even hurt the batch processes.
Signature · 3 lines
bill davidsen <[email protected]> CTO, TMR Associates, Inc Doing interesting things with little computers since 1979.
Hi!
Quoted 9 lines
Quoted 8 lines
There is also a longer PhD thesis by her. 10 minutes is about as much work as I personally am willing to lose and try to remember. Avoiding 75% of writes instead of 20% is a substantial performance gain worth paying a cost for. Unfortunately it is not easy to say if it is worth that much cost, but I suspect it is. An approach we are exploring is for blocks to reach disk earlier than that if the device is not congested, on the grounds that if not much IO is occuring, then performance is not important.
Quoted 4 lines
Assuming your 10 minutes are just a default and tunable by sysctl I hardly can see any problems at all. Paranoid people can set it to make any tradeoff between performance and speed they'd like including setting it to 0, no?
It has traditionaly been 30 seconds, so I'd suggest default stays.
Pavel
Signature · 2 lines
Worst form of spam? Adding advertisment signatures ala sourceforge.net. What goes next? Inserting advertisment *into* email?