some writeback experiments
Get series with b4
b4 am [email protected]Versions newest first
- v1 Latest45 patches 67 repliesIn review
Patches in v1 No applies recorded yet
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/45 | writeback: reduce calls to global_page_state in balance_dirty_pages()mm/ | +33 | 20 replies | |
| 2/45 | writeback: reduce calculation of bdi dirty thresholdsfs/ include/ mm/ +1 more | +43 | 0 replies | |
| 3/45 | ext4: remove unused parameter wbc from __ext4_journalled_writepage()fs/ | +1 | 0 replies | |
| 4/45 | writeback: remove unused nonblocking and congestion checksdrivers/ fs/ fs/ +4 more | +2 | 2 replies | |
| 5/45 | writeback: remove the always false bdi_cap_writeback_dirty() testfs/ | +0 | 0 replies | |
| 6/45 | writeback: use larger ratelimit when dirty_exceededmm/ | +5 | 2 replies | |
| 7/45 | writeback: dont redirty tail an inode with dirty pagesfs/ | +7 | 1 reply | |
| 8/45 | writeback: quit on wrap for .range_cyclic (write_cache_pages)mm/ | +5 | 0 replies | |
| 9/45 | writeback: quit on wrap for .range_cyclic (pohmelfs)drivers/ | +8 | 2 replies | |
| 10/45 | writeback: quit on wrap for .range_cyclic (btrfs)fs/ | +6 | 0 replies | |
| 11/45 | writeback: quit on wrap for .range_cyclic (cifs)fs/ | +4 | 0 replies | |
| 12/45 | writeback: quit on wrap for .range_cyclic (ext4)fs/ | +4 | 0 replies | |
| 13/45 | writeback: quit on wrap for .range_cyclic (gfs2)fs/ | +2 | 0 replies | |
| 14/45 | writeback: quit on wrap for .range_cyclic (afs)fs/ | +3 | 5 replies | |
| 15/45 | writeback: fix queue_io() orderingfs/ | +5 | 0 replies | |
| 16/45 | writeback: merge for_kupdate and !for_kupdate casesfs/ | +6 | 0 replies | |
| 17/45 | writeback: only allow two background writeback worksfs/ include/ | +27 | 0 replies | |
| 18/45 | writeback: introduce wait queue for balance_dirty_pages()fs/ include/ mm/ +1 more | +103 | 16 replies | |
| 19/45 | writeback: remove the loop in balance_dirty_pages()mm/ | +38 | 0 replies | |
| 20/45 | NFS: introduce writeback wait queuefs/ fs/ include/ | +73 | 6 replies | |
| 21/45 | writeback: estimate bdi write bandwidthfs/ include/ include/ +1 more | +35 | 2 replies | |
| 22/45 | writeback: show bdi write bandwidth in debugfsmm/ | +4 | 0 replies | |
| 23/45 | writeback: kill space in debugfs item namemm/ | +1 | 0 replies | |
| 24/45 | writeback: remove global nr_to_write and use timeout insteadfs/ include/ mm/ | +8 | 0 replies | |
| 25/45 | writeback: convert wbc.nr_to_write to per-file parameterfs/ include/ mm/ | +15 | 0 replies | |
| 26/45 | block: pass the non-rotational queue flag to backing_dev_infoblock/ block/ drivers/ +5 more | +28 | 0 replies | |
| 27/45 | writeback: introduce wbc.for_backgroundfs/ fs/ include/ | +3 | 0 replies | |
| 28/45 | writeback: introduce wbc.nr_segmentsfs/ fs/ include/ +2 more | +25 | 0 replies | |
| 29/45 | writeback: fix the shmem AOP_WRITEPAGE_ACTIVATE casemm/ mm/ | +12 | 2 replies | |
| 30/45 | vmscan: lumpy pageoutmm/ mm/ | +23 | 0 replies | |
| 31/45 | writeback: sync old inodes first in background writebackfs/ | +17 | 2 replies | |
| 32/45 | writeback: update kupdate expire timestamp on each scan of b_iofs/ | +3 | 0 replies | |
| 33/45 | writeback: sync livelock - introduce wbc.for_syncfs/ include/ include/ +1 more | +22 | 0 replies | |
| 34/45 | writeback: sync livelock - kick background writebackfs/ include/ mm/ | +17 | 0 replies | |
| 35/45 | writeback: sync livelock - use single timestamp for whole sync workfs/ | +8 | 0 replies | |
| 36/45 | writeback: sync livelock - curb dirty speed for inodes to be syncedfs/ include/ mm/ | +18 | 0 replies | |
| 37/45 | writeback: use timestamp to indicate dirty exceededinclude/ mm/ mm/ | +4 | 0 replies | |
| 38/45 | writeback: introduce queue b_more_io_waitfs/ include/ mm/ | +32 | 0 replies | |
| 39/45 | writeback: remove wbc.more_iofs/ include/ include/ | +3 | 0 replies | |
| 40/45 | writeback: requeue_io_wait() on I_SYNC locked inodefs/ | +3 | 0 replies | |
| 41/45 | writeback: requeue_io_wait() on pages_skipped inodefs/ | +1 | 0 replies | |
| 42/45 | writeback: requeue_io_wait() on blocked inodefs/ | +1 | 0 replies | |
| 43/45 | writeback: requeue_io_wait() on fs redirtied inodefs/ | +1 | 0 replies | |
| 44/45 | NFS: remove NFS_INO_FLUSHING lockfs/ fs/ include/ | +0 | 4 replies | |
| 45/45 | btrfs: fix race on syncing the btree inodefs/ | +13 | 0 replies |
Reviews trailers per patch
2 of 45 patches have a Reviewed-by or Acked-by.
| Patch | David Howells | Jan Kara |
|---|---|---|
| 1/45 | ||
| 2/45 | ||
| 3/45 | ||
| 4/45 | ||
| 5/45 | ||
| 6/45 | ||
| 7/45 | A-b | |
| 8/45 | ||
| 9/45 | ||
| 10/45 | ||
| 11/45 | ||
| 12/45 | ||
| 13/45 | ||
| 14/45 | A-b | |
| 15/45 | ||
| 16/45 | ||
| 17/45 | ||
| 18/45 | ||
| 19/45 | ||
| 20/45 | ||
| 21/45 | ||
| 22/45 | ||
| 23/45 | ||
| 24/45 | ||
| 25/45 | ||
| 26/45 | ||
| 27/45 | ||
| 28/45 | ||
| 29/45 | ||
| 30/45 | ||
| 31/45 | ||
| 32/45 | ||
| 33/45 | ||
| 34/45 | ||
| 35/45 | ||
| 36/45 | ||
| 37/45 | ||
| 38/45 | ||
| 39/45 | ||
| 40/45 | ||
| 41/45 | ||
| 42/45 | ||
| 43/45 | ||
| 44/45 | ||
| 45/45 |
Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.
Cover letter [PATCH 00/45] some writeback experiments
Hi all,
Here is a collection of writeback patches on
- larger writeback chunk sizes
- single per-bdi flush thread (killing the foreground throttling writeouts)
- lumpy pageout
- sync livelock prevention
- writeback scheduling
- random fixes
Sorry for posting a too big series - there are many direct or implicit dependencies, and one patch lead to another before I can stop..
The lumpy pageout and nr_segments support is not complete and do not cover all filesystems for now. It may be better to first convert some of the ->writepages to the generic routines to avoid duplicate work.
I managed to address many issues in past week, however there are still known problems. Hints from filesystem developers are highly appreciated. Thanks!
The estimated writeback bandwidth is about 1/2 the real throughput for ext2/3/4 and btrfs; noticeable bigger than real throughput for NFS; and cannot be estimated at all for XFS. Very interesting..
NFS writeback is very bumpy. The page numbers and network throughput "freeze" together from time to time:
# vmmon -d 1 nr_writeback nr_dirty nr_unstable # (per 1-second samples)
nr_writeback nr_dirty nr_unstable
11227 41463 38044
11227 41463 38044
11227 41463 38044
11227 41463 38044
11045 53987 6490
11033 53120 8145
11195 52143 10886
11211 52144 10913
11211 52144 10913
11211 52144 10913btrfs seems to maintain a private pool of writeback pages, which can go out of control:
nr_writeback nr_dirty
261075 132
252891 195
244795 187
236851 187
228830 187
221040 218
212674 237
204981 237XFS has very interesting "bumpy writeback" behavior: it tends to wait collect enough pages and then write the whole world.
nr_writeback nr_dirty
80781 0
37117 37703
37117 43933
81044 6
81050 0
43943 10199
43930 36355
43930 36355
80293 0
80285 0
80285 0Thanks,
Fengguang