writeback bug fixes and simplifications take 2
cover writeback bug fixes and simplifications take 2
Andrew,
This patchset mainly polishes the writeback queuing policies. The main goals are:
(1) small files should not be starved by big dirty files
(2) sync as fast as possible for not-blocked inodes/pages
- don't leave them out; no congestion_wait() in between them
(3) avoid busy iowait for blocked inodes
- retry them in the next go of s_io(maybe at the next wakeup of pdflush)
The role of the queues:
s_dirty: park for dirtied_when expiration s_io: park for io submission s_more_io: for big dirty inodes, they will be retried in this run of pdflush
(it ensures fairness between small/large files)s_more_io_wait: for blocked inodes, they will be picked up in next run of s_io
This patchset is in better shape, but still not ready for merge. It begins with:
[PATCH 01/13] writeback: revert 2e6883bdf49abd0e7f0d9b6297fc3be7ebb2250b [PATCH 02/13] writeback: clear PAGECACHE_TAG_DIRTY for truncated page in block_write_full_page()
Introduces more_io/more_io_wait based policies:
[PATCH 03/13] writeback: introduce writeback_control.more_io [PATCH 04/13] writeback: introduce super_block.s_more_io_wait [PATCH 05/13] writeback: merge duplicate code into writeback_some_pages() [PATCH 06/13] writeback: defer writeback on not-all-pages-written [PATCH 07/13] writeback: defer writeback on locked inode [PATCH 08/13] writeback: defer writeback on locked buffers [PATCH 09/13] writeback: requeue_io() on redirtied inode
And finishes with some code cleanups:
[PATCH 10/13] writeback: introduce queue_dirty() [PATCH 11/13] writeback: queue_dirty() on memory-backed bdi [PATCH 12/13] writeback: remove redirty_tail() [PATCH 13/13] writeback: cleanup __sync_single_inode()
Diffstat:
fs/buffer.c | 2 fs/fs-writeback.c | 121 +++++++++++++++--------------------------- fs/super.c | 1 include/linux/fs.h | 1 mm/page-writeback.c | 46 +++++++-------- 5 files changed, 72 insertions(+), 99 deletions(-)
Regards,
Fengguang Wu
Quoted 3 lines · Fengguang Wu
Andrew,
This patchset mainly polishes the writeback queuing policies.
Anyone know which tree is this patched based out of?
Quoted 8 lines
The main goals are:
(1) small files should not be starved by big dirty files
(2) sync as fast as possible for not-blocked inodes/pages
- don't leave them out; nocongestion_wait()in between them
(3) avoid busy iowait for blocked inodes
- retry them in the next go ofs_io(maybe at the next wakeup of pdflush)
Fengguang do you have any specific tests for any of these cases? As I have posted earlier I am putting together a writeback test suite for test.kernel.org and if you have one (even if it's an ugly shell script) that would save me some time.
Also if you want any of mine let me know. :-)
mrubin
Quoted 45 lines
The role of the queues:
s_dirty: park for dirtied_when expiration s_io: park for io submission s_more_io: for big dirty inodes, they will be retried in this run of pdflush(it ensures fairness between small/large files)
s_more_io_wait: for blocked inodes, they will be picked up in next run ofs_ioThis patchset is in better shape, but still not ready for merge. It begins with:
[PATCH 01/13] writeback: revert
2e6883bdf49abd0e7f0d9b6297fc3be7ebb2250b
[PATCH 02/13] writeback: clearPAGECACHE_TAG_DIRTYfor truncated page inblock_write_full_page()Introduces more_io/more_io_wait based policies:
[PATCH 03/13] writeback: introduce writeback_control.more_io [PATCH 04/13] writeback: introduce super_block.s_more_io_wait [PATCH 05/13] writeback: merge duplicate code into
writeback_some_pages()[PATCH 06/13] writeback: defer writeback on not-all-pages-written [PATCH 07/13] writeback: defer writeback on locked inode [PATCH 08/13] writeback: defer writeback on locked buffers [PATCH 09/13] writeback:requeue_io()on redirtied inodeAnd finishes with some code cleanups:
[PATCH 10/13] writeback: introduce
queue_dirty()[PATCH 11/13] writeback:queue_dirty()on memory-backed bdi [PATCH 12/13] writeback: removeredirty_tail()[PATCH 13/13] writeback: cleanup__sync_single_inode()Diffstat:
fs/buffer.c | 2 fs/fs-writeback.c | 121 +++++++++++++++--------------------------- fs/super.c | 1 include/linux/fs.h | 1 mm/page-writeback.c | 46 +++++++-------- 5 files changed, 72 insertions(+), 99 deletions(-)Regards,
Fengguang Wu
--
Quoted 6 lines · Michael Rubin
Quoted 3 lines · Fengguang Wu
Andrew,
This patchset mainly polishes the writeback queuing policies.
Anyone know which tree is this patched based out of?
They are against the latest -mm tree, or 2.6.24-rc6-mm1.
Quoted 9 lines
Quoted 8 lines
The main goals are:
(1) small files should not be starved by big dirty files
(2) sync as fast as possible for not-blocked inodes/pages
- don't leave them out; nocongestion_wait()in between them
(3) avoid busy iowait for blocked inodes
- retry them in the next go ofs_io(maybe at the next wakeup of pdflush)
Quoted 4 lines
Fengguang do you have any specific tests for any of these cases? As I have posted earlier I am putting together a writeback test suite for test.kernel.org and if you have one (even if it's an ugly shell script) that would save me some time.
No, I just run tests with cp/dd etc. I analyze the code and debug traces a lot, and know that it works in the situations I can imagine. But dedicated test suites are good in the long term.
Quoted 1 line
Also if you want any of mine let me know. :-)
OK, thank you.
Fengguang
Quoted 6 lines · Michael Rubin
Quoted 3 lines · Fengguang Wu
Andrew,
This patchset mainly polishes the writeback queuing policies.
Anyone know which tree is this patched based out of?
They are against the latest -mm tree, or 2.6.24-rc6-mm1.
Quoted 9 lines
Quoted 8 lines
The main goals are:
(1) small files should not be starved by big dirty files
(2) sync as fast as possible for not-blocked inodes/pages
- don't leave them out; nocongestion_wait()in between them
(3) avoid busy iowait for blocked inodes
- retry them in the next go ofs_io(maybe at the next wakeup of pdflush)
Quoted 4 lines
Fengguang do you have any specific tests for any of these cases? As I have posted earlier I am putting together a writeback test suite for test.kernel.org and if you have one (even if it's an ugly shell script) that would save me some time.
No, I just run tests with cp/dd etc. I analyze the code and debug traces a lot, and know that it works in the situations I can imagine. But dedicated test suites are good in the long term.
Quoted 1 line
Also if you want any of mine let me know. :-)
OK, thank you.
Fengguang
Quoted 11 lines · Fengguang Wu
Andrew,
This patchset mainly polishes the writeback queuing policies. The main goals are:
(1) small files should not be starved by big dirty files
(2) sync as fast as possible for not-blocked inodes/pages
- don't leave them out; nocongestion_wait()in between them
(3) avoid busy iowait for blocked inodes
- retry them in the next go ofs_io(maybe at the next wakeup of pdflush)
Quoted 7 lines
The role of the queues:
s_dirty: park for dirtied_when expiration s_io: park for io submission s_more_io: for big dirty inodes, they will be retried in this run of pdflush(it ensures fairness between small/large files)
s_more_io_wait: for blocked inodes, they will be picked up in next run ofs_io
Quick question to make sure I get this. Each queue is sorted as such:
s_dirty - sorted by the dirtied_when fields_io - sorted by no explicit key but by the order we want to process in sync_sb_inodess_more_io - held for later they are sorted in the same manner as s_io
Is that it?
mrubin
Quoted 20 lines · Michael Rubin
Quoted 11 lines · Fengguang Wu
Andrew,
This patchset mainly polishes the writeback queuing policies. The main goals are:
(1) small files should not be starved by big dirty files
(2) sync as fast as possible for not-blocked inodes/pages
- don't leave them out; nocongestion_wait()in between them
(3) avoid busy iowait for blocked inodes
- retry them in the next go ofs_io(maybe at the next wakeup of pdflush)Quoted 7 lines
The role of the queues:
s_dirty: park for dirtied_when expiration s_io: park for io submission s_more_io: for big dirty inodes, they will be retried in this run of pdflush(it ensures fairness between small/large files)
s_more_io_wait: for blocked inodes, they will be picked up in next run ofs_io
Quoted 8 lines
Quick question to make sure I get this. Each queue is sorted as such:
s_dirty- sorted by thedirtied_whenfields_io- sorted by no explicit key but by the order we want to process insync_sb_inodess_more_io- held for later they are sorted in the same manner ass_ioIs that it?
Yes, exactly. s_io and s_more_io can be considered as one list broken up into two - to provide the cursor for sequential iteration. And s_more_io_wait is simply a container for blocked inodes.
Quoted 20 lines · Michael Rubin
Quoted 11 lines · Fengguang Wu
Andrew,
This patchset mainly polishes the writeback queuing policies. The main goals are:
(1) small files should not be starved by big dirty files
(2) sync as fast as possible for not-blocked inodes/pages
- don't leave them out; nocongestion_wait()in between them
(3) avoid busy iowait for blocked inodes
- retry them in the next go ofs_io(maybe at the next wakeup of pdflush)Quoted 7 lines
The role of the queues:
s_dirty: park for dirtied_when expiration s_io: park for io submission s_more_io: for big dirty inodes, they will be retried in this run of pdflush(it ensures fairness between small/large files)
s_more_io_wait: for blocked inodes, they will be picked up in next run ofs_io
Quoted 8 lines
Quick question to make sure I get this. Each queue is sorted as such:
s_dirty- sorted by thedirtied_whenfields_io- sorted by no explicit key but by the order we want to process insync_sb_inodess_more_io- held for later they are sorted in the same manner ass_ioIs that it?
Yes, exactly. s_io and s_more_io can be considered as one list broken up into two - to provide the cursor for sequential iteration. And s_more_io_wait is simply a container for blocked inodes.