writeback: bdi write bandwidth estimation
Subsystems (4)
PATCH 6/13 writeback: bdi write bandwidth estimation
Andrew,
References: <[email protected]>
Content-Disposition: inline; filename=writeback-bandwidth-estimation-in-flusher.patch
The estimation value will start from 100MB/s and adapt to the real bandwidth in seconds. It's pretty accurate for common filesystems.
As the first use case, it replaces the fixed 100MB/s value used for throttle bandwidth calculation in balance_dirty_pages().
The overheads won't be high because the bdi bandwidth udpate only occurs in >10ms intervals.
Initially it's only estimated in balance_dirty_pages() because this is the most reliable place to get reasonable large bandwidth -- the bdi is normally fully utilized when bdi_thresh is reached.
Then Shaohua recommends to also do it in the flusher thread, to keep the value updated when there are only periodic/background writeback and no tasks throttled.
The estimation cannot be done purely in the flusher thread because it's not sufficient for NFS. NFS writeback won't block at get_request_wait(), so tend to complete quickly. Another problem is, slow devices may take dozens of seconds to write the initial 64MB chunk (write_bandwidth starts with 100MB/s, this translates to 64MB nr_to_write). So it may take more than 1 minute to adapt to the smallish bandwidth if the bandwidth is only updated in the flusher thread.
--- fs/fs-writeback.c | 5 ++++ include/linux/backing-dev.h | 2 + include/linux/writeback.h | 3 ++ mm/backing-dev.c | 1 mm/page-writeback.c | 41 +++++++++++++++++++++++++++++++++- 5 files changed, 51 insertions(+), 1 deletion(-)
include/linux/backing-dev.h +2 -0
@@ -75,6 +75,8 @@ struct backing_dev_info {struct percpu_counter bdi_stat[NR_BDI_STAT_ITEMS ];struct prop_local_percpu completions; +unsigned long write_bandwidth_update_time; +int write_bandwidth;int dirty_exceeded;unsigned int min_ratio;
mm/backing-dev.c +1 -0
@@ -660,6 +660,7 @@ int bdi_init(struct backing_dev_info *bd goto err; } + bdi->write_bandwidth =100 <<20 ; bdi->dirty_exceeded =0 ; err =prop_local_init_percpu (&bdi->completions);
fs/fs-writeback.c +5 -0
@@ -635,6 +635,8 @@ static long wb_writeback(struct bdi_writ .range_cyclic = work->range_cyclic, };unsigned long oldest_jif; +unsigned long bw_time; +s64 bw_written =0 ;long wrote =0 ;long write_chunk;struct inode *inode;
@@ -668,6 +670,8 @@ static long wb_writeback(struct bdi_writ write_chunk =LONG_MAX ; wbc.wb_start = jiffies;/* livelock avoidance */ +bdi_update_write_bandwidth (wb->bdi, &bw_time, &bw_written); +for (;;) {/* * Stop writeback when nr_pages has been consumed
@@ -702,6 +706,7 @@ static long wb_writeback(struct bdi_writelse writeback_inodes_wb (wb, &wbc);trace_wbc_writeback_written (&wbc, wb->bdi); +bdi_update_write_bandwidth (wb->bdi, &bw_time, &bw_written); work->nr_pages -= write_chunk - wbc.nr_to_write; wrote += write_chunk - wbc.nr_to_write;
mm/page-writeback.c +40 -1
67 lines not on this page · raw message
include/linux/writeback.h +3 -0
@@ -137,6 +137,9 @@ int dirty_writeback_centisecs_handler(stvoid global_dirty_limits (unsigned long *pbackground,unsigned long *pdirty);unsigned long bdi_dirty_limit (struct backing_dev_info *bdi,unsigned long dirty); +void bdi_update_write_bandwidth (struct backing_dev_info *bdi, +unsigned long *bw_time, +s64 *bw_written);void page_writeback_init (void ); void balance_dirty_pages_ratelimited_nr(struct address_space *mapping,
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom policy in Canada: sign http://dissolvethecrtc.ca/
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>