Skip to contentSkip to conversation

trivial writeback cleanups/fixes

PATCH In review Wu Fengguang 7 participants 31 messages Nov 18, 2009 – Nov 24, 2009

Subsystems (11)
lore
Share & tools
lore
#12 lore raw

PATCH 6/12 writeback: remove unused nonblocking and congestion checks (gfs2)

In reply to Wu Fengguang

No one is calling wb_writeback and write_cache_pages with wbc.nonblocking=1 any more. And lumpy pageout will want to do nonblocking writeback without the congestion wait.

CC Steven Whitehouse <[email protected]>
---
 fs/gfs2/aops.c |   10 ----------
 1 file changed, 10 deletions(-)
fs/gfs2/aops.c +0 -10
@@ -313,11 +313,6 @@ static int gfs2_write_jdata_pagevec(stru
 
 		if (ret || (--(wbc->nr_to_write) <= 0))
 			ret = 1;
-		if (wbc->nonblocking && bdi_write_congested(bdi)) {
-			wbc->encountered_congestion = 1;
-			ret = 1;
-		}
-
 	}
 	gfs2_trans_end(sdp);
 	return ret;
@@ -348,11 +343,6 @@ static int gfs2_write_cache_jdata(struct
 	int scanned = 0;
 	int range_whole = 0;
 
-	if (wbc->nonblocking && bdi_write_congested(bdi)) {
-		wbc->encountered_congestion = 1;
-		return 0;
-	}
-
 	pagevec_init(&pvec, 0);
 	if (wbc->range_cyclic) {
 		index = mapping->writeback_index; /* Start from prev offset */
#25 lore raw

PATCH 11/12 writeback: introduce wbc.for_background

In reply to Wu Fengguang

It will lower the flush priority for NFS, and maybe more in future.

CC Trond Myklebust <[email protected]>
---
 fs/fs-writeback.c         |    1 +
 fs/nfs/write.c            |    2 +-
 include/linux/writeback.h |    1 +
 3 files changed, 3 insertions(+), 1 deletion(-)
fs/fs-writeback.c +1 -0
@@ -713,6 +713,7 @@ static long wb_writeback(struct bdi_writ
 		.sync_mode		= args->sync_mode,
 		.older_than_this	= NULL,
 		.for_kupdate		= args->for_kupdate,
+		.for_background		= args->for_background,
 		.range_cyclic		= args->range_cyclic,
 	};
 	unsigned long oldest_jif;
include/linux/writeback.h +1 -0
@@ -76,6 +76,7 @@ struct writeback_control {
 	unsigned nonblocking:1;		/* Don't get stuck on request queues */
 	unsigned encountered_congestion:1; /* An output: a queue is full */
 	unsigned for_kupdate:1;		/* A kupdate writeback */
+	unsigned for_background:1;	/* A background writeback */
 	unsigned for_reclaim:1;		/* Invoked from the page allocator */
 	unsigned range_cyclic:1;	/* range_start is cyclic */
 	unsigned more_io:1;		/* more io to be dispatched */
fs/nfs/write.c +1 -1
@@ -178,7 +178,7 @@ static int wb_priority(struct writeback_
 {
 	if (wbc->for_reclaim)
 		return FLUSH_HIGHPRI | FLUSH_STABLE;
-	if (wbc->for_kupdate)
+	if (wbc->for_kupdate || wbc->for_background)
 		return FLUSH_LOWPRI;
 	return 0;
 }
#3 lore raw

PATCH 3/12 writeback: remove unused nonblocking and congestion checks (pohmelfs)

In reply to Wu Fengguang

No one is calling wb_writeback and write_cache_pages with wbc.nonblocking=1 any more. And lumpy pageout will want to do nonblocking writeback without the congestion wait.

CC Evgeniy Polyakov <[email protected]>
---
 drivers/staging/pohmelfs/inode.c |    9 ---------
 1 file changed, 9 deletions(-)
drivers/staging/pohmelfs/inode.c +0 -9
@@ -152,11 +152,6 @@ static int pohmelfs_writepages(struct ad
 	int scanned = 0;
 	int range_whole = 0;
 
-	if (wbc->nonblocking && bdi_write_congested(bdi)) {
-		wbc->encountered_congestion = 1;
-		return 0;
-	}
-
 	if (wbc->range_cyclic) {
 		index = mapping->writeback_index; /* Start from prev offset */
 		end = -1;
@@ -248,10 +243,6 @@ retry:
 
 			if (wbc->nr_to_write <= 0)
 				done = 1;
-			if (wbc->nonblocking && bdi_write_congested(bdi)) {
-				wbc->encountered_congestion = 1;
-				done = 1;
-			}
 
 			continue;
 out_continue:
#1 lore raw

PATCH 1/12 writeback: remove unused nonblocking and congestion checks

In reply to Wu Fengguang

- no one is calling wb_writeback and write_cache_pages with wbc.nonblocking=1 any more
- lumpy pageout will want to do nonblocking writeback without the congestion wait

So remove the congestion checks as suggested by Chris.

CC Chris Mason <[email protected]>
CC Jens Axboe <[email protected]>
---
 fs/fs-writeback.c |    9 ---------
 1 file changed, 9 deletions(-)
fs/fs-writeback.c +0 -9
@@ -640,14 +640,6 @@ static void writeback_inodes_wb(struct b
 			continue;
 		}
 
-		if (wbc->nonblocking && bdi_write_congested(wb->bdi)) {
-			wbc->encountered_congestion = 1;
-			if (!is_blkdev_sb)
-				break;		/* Skip a congested fs */
-			requeue_io(inode);
-			continue;		/* Skip a congested blockdev */
-		}
-
 		/*
 		 * Was this inode dirtied after sync_sb_inodes was called?
 		 * This keeps sync from extra jobs and livelock.
@@ -770,7 +762,6 @@ static long wb_writeback(struct bdi_writ
 			break;
 
 		wbc.more_io = 0;
-		wbc.encountered_congestion = 0;
 		wbc.nr_to_write = MAX_WRITEBACK_PAGES;
 		wbc.pages_skipped = 0;
 		writeback_inodes_wb(wb, &wbc);
#11 lore raw

PATCH 5/12 writeback: remove unused nonblocking and congestion checks (cifs)

In reply to Wu Fengguang

No one is calling wb_writeback and write_cache_pages with wbc.nonblocking=1 any more. And lumpy pageout will want to do nonblocking writeback without the congestion wait.

---
 fs/cifs/file.c |   10 ----------
 1 file changed, 10 deletions(-)
fs/cifs/file.c +0 -10
@@ -1379,16 +1379,6 @@ static int cifs_writepages(struct addres
 		return generic_writepages(mapping, wbc);
 
 
-	/*
-	 * BB: Is this meaningful for a non-block-device file system?
-	 * If it is, we should test it again after we do I/O
-	 */
-	if (wbc->nonblocking && bdi_write_congested(bdi)) {
-		wbc->encountered_congestion = 1;
-		kfree(iov);
-		return 0;
-	}
-
 	xid = GetXid();
 
 	pagevec_init(&pvec, 0);
#2 lore raw

PATCH 2/12 writeback: remove unused nonblocking and congestion checks (write_cache_pages)

In reply to Wu Fengguang

No one is calling wb_writeback and write_cache_pages with wbc.nonblocking=1 any more. And lumpy pageout will want to do nonblocking writeback without the congestion wait.

CC Chris Mason <[email protected]>
CC Jens Axboe <[email protected]>
---
 mm/page-writeback.c |   12 ------------
 1 file changed, 12 deletions(-)
mm/page-writeback.c +0 -12
@@ -828,7 +828,6 @@ int write_cache_pages(struct address_spa
 		      struct writeback_control *wbc, writepage_t writepage,
 		      void *data)
 {
-	struct backing_dev_info *bdi = mapping->backing_dev_info;
 	int ret = 0;
 	int done = 0;
 	struct pagevec pvec;
@@ -841,11 +840,6 @@ int write_cache_pages(struct address_spa
 	int range_whole = 0;
 	long nr_to_write = wbc->nr_to_write;
 
-	if (wbc->nonblocking && bdi_write_congested(bdi)) {
-		wbc->encountered_congestion = 1;
-		return 0;
-	}
-
 	pagevec_init(&pvec, 0);
 	if (wbc->range_cyclic) {
 		writeback_index = mapping->writeback_index; /* prev offset */
@@ -964,12 +958,6 @@ continue_unlock:
 					break;
 				}
 			}
-
-			if (wbc->nonblocking && bdi_write_congested(bdi)) {
-				wbc->encountered_congestion = 1;
-				done = 1;
-				break;
-			}
 		}
 		pagevec_release(&pvec);
 		cond_resched();
#20 lore raw

PATCH 8/12 ext4: remove encountered_congestion trace

In reply to Wu Fengguang

It is no longer set and scheduled to be removed.

---
 include/trace/events/ext4.h |    6 ++----
 1 file changed, 2 insertions(+), 4 deletions(-)
include/trace/events/ext4.h +2 -4
@@ -310,7 +310,6 @@ TRACE_EVENT(ext4_da_writepages_result,
 		__field(	int,	ret			)
 		__field(	int,	pages_written		)
 		__field(	long,	pages_skipped		)
-		__field(	char,	encountered_congestion	)
 		__field(	char,	more_io			)	
 		__field(	char,	no_nrwrite_index_update )
 		__field(       pgoff_t,	writeback_index		)
@@ -322,17 +321,16 @@ TRACE_EVENT(ext4_da_writepages_result,
 		__entry->ret		= ret;
 		__entry->pages_written	= pages_written;
 		__entry->pages_skipped	= wbc->pages_skipped;
-		__entry->encountered_congestion	= wbc->encountered_congestion;
 		__entry->more_io	= wbc->more_io;
 		__entry->no_nrwrite_index_update = wbc->no_nrwrite_index_update;
 		__entry->writeback_index = inode->i_mapping->writeback_index;
 	),
 
-	TP_printk("dev %s ino %lu ret %d pages_written %d pages_skipped %ld congestion %d more_io %d no_nrwrite_index_update %d writeback_index %lu",
+	TP_printk("dev %s ino %lu ret %d pages_written %d pages_skipped %ld more_io %d no_nrwrite_index_update %d writeback_index %lu",
 		  jbd2_dev_to_name(__entry->dev),
 		  (unsigned long) __entry->ino, __entry->ret,
 		  __entry->pages_written, __entry->pages_skipped,
-		  __entry->encountered_congestion, __entry->more_io,
+		  __entry->more_io,
 		  __entry->no_nrwrite_index_update,
 		  (unsigned long) __entry->writeback_index)
 );
#26 lore raw

PATCH 12/12 bdi: use bdi_stat_sum() for more accurate debugfs stats

In reply to Wu Fengguang
CC Peter Zijlstra <[email protected]>
---
 mm/backing-dev.c |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
mm/backing-dev.c +2 -2
@@ -104,8 +104,8 @@ static int bdi_debug_stats_show(struct s
 		   "wb_mask:          %8lx\n"
 		   "wb_list:          %8u\n"
 		   "wb_cnt:           %8u\n",
-		   (unsigned long) K(bdi_stat(bdi, BDI_WRITEBACK)),
-		   (unsigned long) K(bdi_stat(bdi, BDI_RECLAIMABLE)),
+		   (unsigned long) K(bdi_stat_sum(bdi, BDI_WRITEBACK)),
+		   (unsigned long) K(bdi_stat_sum(bdi, BDI_RECLAIMABLE)),
 		   K(bdi_thresh), K(dirty_thresh),
 		   K(background_thresh), nr_wb, nr_dirty, nr_io, nr_more_io,
 		   !list_empty(&bdi->bdi_list), bdi->state, bdi->wb_mask,
#8 lore raw

PATCH 4/12 writeback: remove unused nonblocking and congestion checks (afs)

In reply to Wu Fengguang

No one is calling wb_writeback and write_cache_pages with wbc.nonblocking=1 any more. And lumpy pageout will want to do nonblocking writeback without the congestion wait.

---
 fs/afs/write.c |   16 +---------------
 1 file changed, 1 insertion(+), 15 deletions(-)
fs/afs/write.c +1 -15
@@ -455,8 +455,6 @@ int afs_writepage(struct page *page, str
 	}
 
 	wbc->nr_to_write -= ret;
-	if (wbc->nonblocking && bdi_write_congested(bdi))
-		wbc->encountered_congestion = 1;
 
 	_leave(" = 0");
 	return 0;
@@ -529,11 +527,6 @@ static int afs_writepages_region(struct 
 
 		wbc->nr_to_write -= ret;
 
-		if (wbc->nonblocking && bdi_write_congested(bdi)) {
-			wbc->encountered_congestion = 1;
-			break;
-		}
-
 		cond_resched();
 	} while (index < end && wbc->nr_to_write > 0);
 
@@ -554,18 +547,11 @@ int afs_writepages(struct address_space 
 
 	_enter("");
 
-	if (wbc->nonblocking && bdi_write_congested(bdi)) {
-		wbc->encountered_congestion = 1;
-		_leave(" = 0 [congest]");
-		return 0;
-	}
-
 	if (wbc->range_cyclic) {
 		start = mapping->writeback_index;
 		end = -1;
 		ret = afs_writepages_region(mapping, wbc, start, end, &next);
-		if (start > 0 && wbc->nr_to_write > 0 && ret == 0 &&
-		    !(wbc->nonblocking && wbc->encountered_congestion))
+		if (start > 0 && wbc->nr_to_write > 0 && ret == 0)
 			ret = afs_writepages_region(mapping, wbc, 0, start,
 						    &next);
 		mapping->writeback_index = next;
#22 lore raw

PATCH 9/12 ext4: remove unused parameter wbc from __ext4_journalled_writepage()

In reply to Wu Fengguang
CC Theodore Ts'o <[email protected]>
CC Jan Kara <[email protected]>
---
 fs/ext4/inode.c |    3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
fs/ext4/inode.c +1 -2
@@ -2599,7 +2599,6 @@ static int bput_one(handle_t *handle, st
 }
 
 static int __ext4_journalled_writepage(struct page *page,
-				       struct writeback_control *wbc,
 				       unsigned int len)
 {
 	struct address_space *mapping = page->mapping;
@@ -2757,7 +2756,7 @@ static int ext4_writepage(struct page *p
 		 * doesn't seem much point in redirtying the page here.
 		 */
 		ClearPageChecked(page);
-		return __ext4_journalled_writepage(page, wbc, len);
+		return __ext4_journalled_writepage(page, len);
 	}
 
 	if (test_opt(inode->i_sb, NOBH) && ext4_should_writeback_data(inode))
#0 lore raw

cover trivial writeback cleanups/fixes

Hi,

Some trivial writeback cleanups and fixes for .33.

They are all independant ones (don't rely on each other), and can be pulled either by Andrew/Jens, or by individual filesystem maintainers.

Thanks,
Fengguang

#16 lore raw

PATCH 7/12 writeback: remove unused nonblocking and congestion checks (xfs)

No one is calling wb_writeback and write_cache_pages with wbc.nonblocking=1 any more. And lumpy pageout will want to do nonblocking writeback without the congestion wait.

CC Dave Chinner <[email protected]>
CC Christoph Hellwig <[email protected]>
---
 fs/xfs/linux-2.6/xfs_aops.c |    6 +-----
 1 file changed, 1 insertion(+), 5 deletions(-)
fs/xfs/linux-2.6/xfs_aops.c +1 -5
@@ -908,12 +908,8 @@ xfs_convert_page(
 
 			bdi = inode->i_mapping->backing_dev_info;
 			wbc->nr_to_write--;
-			if (bdi_write_congested(bdi)) {
-				wbc->encountered_congestion = 1;
+			if (wbc->nr_to_write <= 0)
 				done = 1;
-			} else if (wbc->nr_to_write <= 0) {
-				done = 1;
-			}
 		}
 		xfs_start_page_writeback(page, !page_dirty, count);
 	}
#24 lore raw

PATCH 10/12 writeback: remove the always false bdi_cap_writeback_dirty() test

In reply to Wu Fengguang

This is dead code because no bdi flush thread will be started for !bdi_cap_writeback_dirty bdis.

---
 fs/fs-writeback.c |   18 ------------------
 1 file changed, 18 deletions(-)
fs/fs-writeback.c +0 -18
@@ -597,7 +597,6 @@ static void writeback_inodes_wb(struct b
 				struct writeback_control *wbc)
 {
 	struct super_block *sb = wbc->sb, *pin_sb = NULL;
-	const int is_blkdev_sb = sb_is_blkdev_sb(sb);
 	const unsigned long start = jiffies;	/* livelock avoidance */
 
 	spin_lock(&inode_lock);
@@ -618,23 +617,6 @@ static void writeback_inodes_wb(struct b
 			continue;
 		}
 
-		if (!bdi_cap_writeback_dirty(wb->bdi)) {
-			redirty_tail(inode);
-			if (is_blkdev_sb) {
-				/*
-				 * Dirty memory-backed blockdev: the ramdisk
-				 * driver does this.  Skip just this inode
-				 */
-				continue;
-			}
-			/*
-			 * Dirty memory-backed inode against a filesystem other
-			 * than the kernel-internal bdev filesystem.  Skip the
-			 * entire superblock.
-			 */
-			break;
-		}
-
 		if (inode->i_state & (I_NEW | I_WILL_FREE)) {
 			requeue_io(inode);
 			continue;
Acked-by on v1 6/12
#13 lore raw

Re: [PATCH 06/12] writeback: remove unused nonblocking and congestion checks (gfs2)

In reply to Wu Fengguang

Hi,

That looks ok to me, modulo the warnings below. The only "non-obvious" thing in this area which the GFS2 writepage[s] code relies upon is the assumption that if we have a ->writepages() then ->writepage() will never be called from a context which requires the fs to actually do a write (i.e. the fs can refuse this if required). That is also only the case for journaled data files - normal writes don't have that requirement.

CC [M] fs/gfs2/aops.o
fs/gfs2/aops.c: In function ‘gfs2_write_jdata_pagevec’:
fs/gfs2/aops.c:272: warning: unused variable ‘bdi’ fs/gfs2/aops.c: In function ‘gfs2_write_cache_jdata’:
fs/gfs2/aops.c:336: warning: unused variable ‘bdi’

Once the warnings are fixed:

Do you want me to add this patch into my tree, or were you planning to submit via a different tree?

Steve.

Quoted 30 lines · Wu Fengguang

plain text document attachment (writeback-remove-congested-checks-linux_fs_gfs2_aops.patch) No one is calling wb_writeback and write_cache_pages with wbc.nonblocking=1 any more. And lumpy pageout will want to do nonblocking writeback without the congestion wait.

CC Steven Whitehouse <[email protected]>
---
 fs/gfs2/aops.c |   10 ----------
 1 file changed, 10 deletions(-)
fs/gfs2/aops.c +0 -5
@@ -313,11 +313,6 @@ static int gfs2_write_jdata_pagevec(stru
 
 		if (ret || (--(wbc->nr_to_write) <= 0))
 			ret = 1;
-		if (wbc->nonblocking && bdi_write_congested(bdi)) {
-			wbc->encountered_congestion = 1;
-			ret = 1;
-		}
-
 	}
 	gfs2_trans_end(sdp);
 	return ret;
@@ -348,11 +343,6 @@ static int gfs2_write_cache_jdata(struct
 	int scanned = 0;

fs/gfs2/aops.c static int gfs2_write_cache_jdata(struct · line 348

Quoted 10 lines
fs/gfs2/aops.c +0 -5
 	int range_whole = 0;
 
-	if (wbc->nonblocking && bdi_write_congested(bdi)) {
-		wbc->encountered_congestion = 1;
-		return 0;
-	}
-
 	pagevec_init(&pvec, 0);
 	if (wbc->range_cyclic) {
 		index = mapping->writeback_index; /* Start from prev offset */

--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

author Acked-by Steven Whitehouse
#14 lore raw

Re: [PATCH 06/12] writeback: remove unused nonblocking and congestion checks (gfs2)

Hi Steven,

Quoted 9 lines · Steven Whitehouse

Hi,

That looks ok to me, modulo the warnings below. The only "non-obvious" thing in this area which the GFS2 writepage[s] code relies upon is the assumption that if we have a ->writepages() then ->writepage() will never be called from a context which requires the fs to actually do a write (i.e. the fs can refuse this if required). That is also only the case for journaled data files - normal writes don't have that requirement.

Thank you for the tips. I don't think pageout() or migration writeout() has that hard expectation for ->writepage() :)

Quoted 5 lines

CC [M] fs/gfs2/aops.o
fs/gfs2/aops.c: In function ‘gfs2_write_jdata_pagevec’:
fs/gfs2/aops.c:272: warning: unused variable ‘bdi’ fs/gfs2/aops.c: In function ‘gfs2_write_cache_jdata’:
fs/gfs2/aops.c:336: warning: unused variable ‘bdi’

Ah sorry!

Quoted 6 lines

Once the warnings are fixed:

Do you want me to add this patch into my tree, or were you planning to submit via a different tree?

Thanks, can you pull the updated patch directly to your tree?

Thanks,
Fengguang
---
writeback: remove unused nonblocking and congestion checks (gfs2)

No one is calling wb_writeback and write_cache_pages with wbc.nonblocking=1 any more. And lumpy pageout will want to do nonblocking writeback without the congestion wait.

---
 fs/gfs2/aops.c |   12 ------------
 1 file changed, 12 deletions(-)
fs/gfs2/aops.c +0 -12
@@ -269,7 +269,6 @@ static int gfs2_write_jdata_pagevec(stru
 	pgoff_t end_index = i_size >> PAGE_CACHE_SHIFT;
 	unsigned offset = i_size & (PAGE_CACHE_SIZE-1);
 	unsigned nrblocks = nr_pages * (PAGE_CACHE_SIZE/inode->i_sb->s_blocksize);
-	struct backing_dev_info *bdi = mapping->backing_dev_info;
 	int i;
 	int ret;
 
@@ -313,11 +312,6 @@ static int gfs2_write_jdata_pagevec(stru
 
 		if (ret || (--(wbc->nr_to_write) <= 0))
 			ret = 1;
-		if (wbc->nonblocking && bdi_write_congested(bdi)) {
-			wbc->encountered_congestion = 1;
-			ret = 1;
-		}
-
 	}
 	gfs2_trans_end(sdp);
 	return ret;
@@ -338,7 +332,6 @@ static int gfs2_write_jdata_pagevec(stru
 static int gfs2_write_cache_jdata(struct address_space *mapping,
 				  struct writeback_control *wbc)
 {
-	struct backing_dev_info *bdi = mapping->backing_dev_info;
 	int ret = 0;
 	int done = 0;
 	struct pagevec pvec;
@@ -348,11 +341,6 @@ static int gfs2_write_cache_jdata(struct
 	int scanned = 0;
 	int range_whole = 0;
 
-	if (wbc->nonblocking && bdi_write_congested(bdi)) {
-		wbc->encountered_congestion = 1;
-		return 0;
-	}
-
 	pagevec_init(&pvec, 0);
 	if (wbc->range_cyclic) {
 		index = mapping->writeback_index; /* Start from prev offset */
#27 lore raw

Re: [PATCH 12/12] bdi: use bdi_stat_sum() for more accurate debugfs stats

In reply to Wu Fengguang
Quoted 11 lines · Wu Fengguang

plain text document attachment (bdi-debug-dump-sum.patch)

CC Peter Zijlstra <[email protected]>
---
 mm/backing-dev.c |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

mm/backing-dev.c static int bdi_debug_stats_show(struct s · line 111

Quoted 10 lines
mm/backing-dev.c +2 -2
@@ -104,8 +104,8 @@ static int bdi_debug_stats_show(struct s
 		   "wb_mask:          %8lx\n"
 		   "wb_list:          %8u\n"
 		   "wb_cnt:           %8u\n",
-		   (unsigned long) K(bdi_stat(bdi, BDI_WRITEBACK)),
-		   (unsigned long) K(bdi_stat(bdi, BDI_RECLAIMABLE)),
+		   (unsigned long) K(bdi_stat_sum(bdi, BDI_WRITEBACK)),
+		   (unsigned long) K(bdi_stat_sum(bdi, BDI_RECLAIMABLE)),
 		   K(bdi_thresh), K(dirty_thresh),
 		   K(background_thresh), nr_wb, nr_dirty, nr_io, nr_more_io,
 		   !list_empty(&bdi->bdi_list), bdi->state, bdi->wb_mask,

Is this really important? This patch is basically a local DoS for large machines.

Imagine someone doing:

while :; do cat /debug/bdi/*/stats; done

on a 512 cpu box.

#15 lore raw

Re: [PATCH 06/12] writeback: remove unused nonblocking and congestion checks (gfs2)

In reply to Wu Fengguang

Hi,

Quoted 24 lines · Wu Fengguang

Hi Steven,

Quoted 9 lines · Steven Whitehouse

Hi,

That looks ok to me, modulo the warnings below. The only "non-obvious" thing in this area which the GFS2 writepage[s] code relies upon is the assumption that if we have a ->writepages() then ->writepage() will never be called from a context which requires the fs to actually do a write (i.e. the fs can refuse this if required). That is also only the case for journaled data files - normal writes don't have that requirement.

Thank you for the tips. I don't think pageout() or migration writeout() has that hard expectation for ->writepage() :)

Quoted 5 lines

CC [M] fs/gfs2/aops.o
fs/gfs2/aops.c: In function ‘gfs2_write_jdata_pagevec’:
fs/gfs2/aops.c:272: warning: unused variable ‘bdi’ fs/gfs2/aops.c: In function ‘gfs2_write_cache_jdata’:
fs/gfs2/aops.c:336: warning: unused variable ‘bdi’

Ah sorry!

Quoted 9 lines
Quoted 6 lines

Once the warnings are fixed:

Do you want me to add this patch into my tree, or were you planning to submit via a different tree?

Thanks, can you pull the updated patch directly to your tree?

Yes, its in my -nmw tree now. Thanks,

Steve.

--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#17 lore raw

Re: [PATCH 07/12] writeback: remove unused nonblocking and congestion checks (xfs)

In reply to Wu Fengguang
Quoted 3 lines · Wu Fengguang

No one is calling wb_writeback and write_cache_pages with wbc.nonblocking=1 any more. And lumpy pageout will want to do nonblocking writeback without the congestion wait.

....

fs/xfs/linux-2.6/xfs_aops.c xfs_convert_page( · line 912

Quoted 8 lines
fs/xfs/linux-2.6/xfs_aops.c +0 -2
@@ -908,12 +908,8 @@ xfs_convert_page(
 
 			bdi = inode->i_mapping->backing_dev_info;
 			wbc->nr_to_write--;
-			if (bdi_write_congested(bdi)) {
-				wbc->encountered_congestion = 1;

bdi is unused now, so can be removed as well. Otherwise looks fine.

Cheers,

Dave.

Signature · 2 lines
Dave Chinner
[email protected]
#28 lore raw

Re: [PATCH 12/12] bdi: use bdi_stat_sum() for more accurate debugfs stats

In reply to Peter Zijlstra
Quoted 17 lines · Peter Zijlstra
Quoted 16 lines · Wu Fengguang

plain text document attachment (bdi-debug-dump-sum.patch)

CC Peter Zijlstra <[email protected]>
---
 mm/backing-dev.c |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
mm/backing-dev.c +1 -2
@@ -104,8 +104,8 @@ static int bdi_debug_stats_show(struct s
 		   "wb_mask:          %8lx\n"
 		   "wb_list:          %8u\n"
 		   "wb_cnt:           %8u\n",
-		   (unsigned long) K(bdi_stat(bdi, BDI_WRITEBACK)),
-		   (unsigned long) K(bdi_stat(bdi, BDI_RECLAIMABLE)),
+		   (unsigned long) K(bdi_stat_sum(bdi, BDI_WRITEBACK)),
Quoted 8 lines

mm/backing-dev.c static int bdi_debug_stats_show(struct s · line 111

Quoted 5 lines
mm/backing-dev.c +1 -0
+		   (unsigned long) K(bdi_stat_sum(bdi, BDI_RECLAIMABLE)),
 		   K(bdi_thresh), K(dirty_thresh),
 		   K(background_thresh), nr_wb, nr_dirty, nr_io, nr_more_io,
 		   !list_empty(&bdi->bdi_list), bdi->state, bdi->wb_mask,

Is this really important? This patch is basically a local DoS for large machines.

I did this patch after seeing inaccurate exported numbers, it may be confusing..

Quoted 5 lines

Imagine someone doing:

while :; do cat /debug/bdi/*/stats; done

on a 512 cpu box.

Yes there will be overheads. However it's always possible to create local DoS with some other kind of busy loop?

Thanks,
Fengguang

#18 lore raw

Re: [PATCH 07/12] writeback: remove unused nonblocking and congestion checks (xfs)

In reply to Dave Chinner
Quoted 11 lines · Dave Chinner
Quoted 3 lines · Wu Fengguang

No one is calling wb_writeback and write_cache_pages with wbc.nonblocking=1 any more. And lumpy pageout will want to do nonblocking writeback without the congestion wait.

....

Quoted 4 lines
fs/xfs/linux-2.6/xfs_aops.c +0 -0
@@ -908,12 +908,8 @@ xfs_convert_page(
 
Quoted 6 lines

fs/xfs/linux-2.6/xfs_aops.c xfs_convert_page( · line 912

Quoted 4 lines
fs/xfs/linux-2.6/xfs_aops.c +0 -2
 			bdi = inode->i_mapping->backing_dev_info;
 			wbc->nr_to_write--;
-			if (bdi_write_congested(bdi)) {
-				wbc->encountered_congestion = 1;

bdi is unused now, so can be removed as well. Otherwise looks fine.

Thanks, here is the updated patch.

---
writeback: remove unused nonblocking and congestion checks (xfs)

No one is calling wb_writeback and write_cache_pages with wbc.nonblocking=1 any more. And lumpy pageout will want to do nonblocking writeback without the congestion wait.

CC Dave Chinner <[email protected]>
CC Christoph Hellwig <[email protected]>
---
 fs/xfs/linux-2.6/xfs_aops.c |    9 +--------
 1 file changed, 1 insertion(+), 8 deletions(-)
fs/xfs/linux-2.6/xfs_aops.c +1 -8
@@ -904,16 +904,9 @@ xfs_convert_page(
 
 	if (startio) {
 		if (count) {
-			struct backing_dev_info *bdi;
-
-			bdi = inode->i_mapping->backing_dev_info;
 			wbc->nr_to_write--;
-			if (bdi_write_congested(bdi)) {
-				wbc->encountered_congestion = 1;
-				done = 1;
-			} else if (wbc->nr_to_write <= 0) {
+			if (wbc->nr_to_write <= 0)
 				done = 1;
-			}
 		}
 		xfs_start_page_writeback(page, !page_dirty, count);
 	}
#9 lore raw

Re: [PATCH 04/12] writeback: remove unused nonblocking and congestion checks (afs)

In reply to Wu Fengguang

Hi David,

Will you pick up this updated patch? It also removes the unused bdi.

Thanks,
Fengguang
---
writeback: remove unused nonblocking and congestion checks (afs)

No one is calling wb_writeback and write_cache_pages with wbc.nonblocking=1 any more. And lumpy pageout will want to do nonblocking writeback without the congestion wait.

---
 fs/afs/write.c |   19 +------------------
 1 file changed, 1 insertion(+), 18 deletions(-)
fs/afs/write.c +1 -18
@@ -438,7 +438,6 @@ no_more:
  */
 int afs_writepage(struct page *page, struct writeback_control *wbc)
 {
-	struct backing_dev_info *bdi = page->mapping->backing_dev_info;
 	struct afs_writeback *wb;
 	int ret;
 
@@ -455,8 +454,6 @@ int afs_writepage(struct page *page, str
 	}
 
 	wbc->nr_to_write -= ret;
-	if (wbc->nonblocking && bdi_write_congested(bdi))
-		wbc->encountered_congestion = 1;
 
 	_leave(" = 0");
 	return 0;
@@ -469,7 +466,6 @@ static int afs_writepages_region(struct 
 				 struct writeback_control *wbc,
 				 pgoff_t index, pgoff_t end, pgoff_t *_next)
 {
-	struct backing_dev_info *bdi = mapping->backing_dev_info;
 	struct afs_writeback *wb;
 	struct page *page;
 	int ret, n;
@@ -529,11 +525,6 @@ static int afs_writepages_region(struct 
 
 		wbc->nr_to_write -= ret;
 
-		if (wbc->nonblocking && bdi_write_congested(bdi)) {
-			wbc->encountered_congestion = 1;
-			break;
-		}
-
 		cond_resched();
 	} while (index < end && wbc->nr_to_write > 0);
 
@@ -548,24 +539,16 @@ static int afs_writepages_region(struct 
 int afs_writepages(struct address_space *mapping,
 		   struct writeback_control *wbc)
 {
-	struct backing_dev_info *bdi = mapping->backing_dev_info;
 	pgoff_t start, end, next;
 	int ret;
 
 	_enter("");
 
-	if (wbc->nonblocking && bdi_write_congested(bdi)) {
-		wbc->encountered_congestion = 1;
-		_leave(" = 0 [congest]");
-		return 0;
-	}
-
 	if (wbc->range_cyclic) {
 		start = mapping->writeback_index;
 		end = -1;
 		ret = afs_writepages_region(mapping, wbc, start, end, &next);
-		if (start > 0 && wbc->nr_to_write > 0 && ret == 0 &&
-		    !(wbc->nonblocking && wbc->encountered_congestion))
+		if (start > 0 && wbc->nr_to_write > 0 && ret == 0)
 			ret = afs_writepages_region(mapping, wbc, 0, start,
 						    &next);
 		mapping->writeback_index = next;
#4 lore raw

Re: [PATCH 03/12] writeback: remove unused nonblocking and congestion checks (pohmelfs)

In reply to Wu Fengguang

Hi Evgeniy,

Will you pick up this updated patch? It also removes the unused bdi.

Thanks,
Fengguang
---
writeback: remove unused nonblocking and congestion checks (pohmelfs)

No one is calling wb_writeback and write_cache_pages with wbc.nonblocking=1 any more. And lumpy pageout will want to do nonblocking writeback without the congestion wait.

CC Evgeniy Polyakov <[email protected]>
---
 drivers/staging/pohmelfs/inode.c |   10 ----------
 1 file changed, 10 deletions(-)
drivers/staging/pohmelfs/inode.c +0 -10
@@ -143,7 +143,6 @@ static int pohmelfs_writepages(struct ad
 	struct inode *inode = mapping->host;
 	struct pohmelfs_inode *pi = POHMELFS_I(inode);
 	struct pohmelfs_sb *psb = POHMELFS_SB(inode->i_sb);
-	struct backing_dev_info *bdi = mapping->backing_dev_info;
 	int err = 0;
 	int done = 0;
 	int nr_pages;
@@ -152,11 +151,6 @@ static int pohmelfs_writepages(struct ad
 	int scanned = 0;
 	int range_whole = 0;
 
-	if (wbc->nonblocking && bdi_write_congested(bdi)) {
-		wbc->encountered_congestion = 1;
-		return 0;
-	}
-
 	if (wbc->range_cyclic) {
 		index = mapping->writeback_index; /* Start from prev offset */
 		end = -1;
@@ -248,10 +242,6 @@ retry:
 
 			if (wbc->nr_to_write <= 0)
 				done = 1;
-			if (wbc->nonblocking && bdi_write_congested(bdi)) {
-				wbc->encountered_congestion = 1;
-				done = 1;
-			}
 
 			continue;
 out_continue:
#29 lore raw

Re: [PATCH 12/12] bdi: use bdi_stat_sum() for more accurate debugfs stats

In reply to Wu Fengguang
Quoted 30 lines · Wu Fengguang
Quoted 17 lines · Peter Zijlstra
Quoted 16 lines · Wu Fengguang

plain text document attachment (bdi-debug-dump-sum.patch)

CC Peter Zijlstra <[email protected]>
---
 mm/backing-dev.c |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
mm/backing-dev.c +1 -2
@@ -104,8 +104,8 @@ static int bdi_debug_stats_show(struct s
 		   "wb_mask:          %8lx\n"
 		   "wb_list:          %8u\n"
 		   "wb_cnt:           %8u\n",
-		   (unsigned long) K(bdi_stat(bdi, BDI_WRITEBACK)),
-		   (unsigned long) K(bdi_stat(bdi, BDI_RECLAIMABLE)),
+		   (unsigned long) K(bdi_stat_sum(bdi, BDI_WRITEBACK)),
Quoted 8 lines

mm/backing-dev.c static int bdi_debug_stats_show(struct s · line 111

Quoted 5 lines
mm/backing-dev.c +1 -0
+		   (unsigned long) K(bdi_stat_sum(bdi, BDI_RECLAIMABLE)),
 		   K(bdi_thresh), K(dirty_thresh),
 		   K(background_thresh), nr_wb, nr_dirty, nr_io, nr_more_io,
 		   !list_empty(&bdi->bdi_list), bdi->state, bdi->wb_mask,

Is this really important? This patch is basically a local DoS for large machines.

I did this patch after seeing inaccurate exported numbers, it may be confusing..

Quoted 8 lines
Quoted 5 lines

Imagine someone doing:

while :; do cat /debug/bdi/*/stats; done

on a 512 cpu box.

Yes there will be overheads. However it's always possible to create local DoS with some other kind of busy loop?

Preferably not of the kind that renders your box unusable, esp by unpriv users.

But yes there are probably a few local DoS' around, but I don't think that warrants adding another one.

#30 lore raw

Re: [PATCH 12/12] bdi: use bdi_stat_sum() for more accurate debugfs stats

Quoted 37 lines · Peter Zijlstra
Quoted 36 lines · Wu Fengguang
Quoted 17 lines · Peter Zijlstra
Quoted 16 lines · Wu Fengguang

plain text document attachment (bdi-debug-dump-sum.patch)

CC Peter Zijlstra <[email protected]>
---
 mm/backing-dev.c |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
mm/backing-dev.c +1 -2
@@ -104,8 +104,8 @@ static int bdi_debug_stats_show(struct s
 		   "wb_mask:          %8lx\n"
 		   "wb_list:          %8u\n"
 		   "wb_cnt:           %8u\n",
-		   (unsigned long) K(bdi_stat(bdi, BDI_WRITEBACK)),
-		   (unsigned long) K(bdi_stat(bdi, BDI_RECLAIMABLE)),
+		   (unsigned long) K(bdi_stat_sum(bdi, BDI_WRITEBACK)),
Quoted 8 lines

mm/backing-dev.c static int bdi_debug_stats_show(struct s · line 111

Quoted 5 lines
mm/backing-dev.c +1 -0
+		   (unsigned long) K(bdi_stat_sum(bdi, BDI_RECLAIMABLE)),
 		   K(bdi_thresh), K(dirty_thresh),
 		   K(background_thresh), nr_wb, nr_dirty, nr_io, nr_more_io,
 		   !list_empty(&bdi->bdi_list), bdi->state, bdi->wb_mask,

Is this really important? This patch is basically a local DoS for large machines.

I did this patch after seeing inaccurate exported numbers, it may be confusing..

Quoted 5 lines

Imagine someone doing:

while :; do cat /debug/bdi/*/stats; done

on a 512 cpu box.

Quoted 8 lines
Quoted 2 lines

Yes there will be overheads. However it's always possible to create local DoS with some other kind of busy loop?

Preferably not of the kind that renders your box unusable, esp by unpriv users.

But yes there are probably a few local DoS' around, but I don't think that warrants adding another one.

OK I'll drop this patch. I'm not feeling strong on either way.

Andrew/Jens, do you prefer for me to repost the non-fs-specific patches?

Thanks,
Fengguang

#5 lore raw

Re: [PATCH 03/12] writeback: remove unused nonblocking and congestion checks (pohmelfs)

In reply to Wu Fengguang

H1.

Quoted 1 line · Wu Fengguang

Will you pick up this updated patch? It also removes the unused bdi.

I'm ok with the changes, but I suppose it should be pushed as a single patchet into the tree, no?

Signature · 1 line
	Evgeniy Polyakov
maintainer
#10 lore raw

Re: [PATCH 04/12] writeback: remove unused nonblocking and congestion checks (afs)

In reply to Wu Fengguang
Quoted 1 line · Wu Fengguang

Will you pick up this updated patch? It also removes the unused bdi.

Sure - but it might be in a week and a bit. I'm scrambling to get stuff done before going on holiday next week.

David

#6 lore raw

Re: [PATCH 03/12] writeback: remove unused nonblocking and congestion checks (pohmelfs)

In reply to Evgeniy Polyakov

Hi Evgeniy,

Quoted 7 lines · Evgeniy Polyakov

H1.

Quoted 1 line · Wu Fengguang

Will you pick up this updated patch? It also removes the unused bdi.

I'm ok with the changes, but I suppose it should be pushed as a single patchet into the tree, no?

Either way works. I would submit a single patchset that includes the generic ones and the fs specific patches that have not been picked up into their fs tree :)

Thanks,
Fengguang

#19 lore raw

Re: [PATCH 07/12] writeback: remove unused nonblocking and congestion checks (xfs)

In reply to Wu Fengguang
Quoted 12 lines · Wu Fengguang
Quoted 11 lines · Dave Chinner
Quoted 3 lines · Wu Fengguang

No one is calling wb_writeback and write_cache_pages with wbc.nonblocking=1 any more. And lumpy pageout will want to do nonblocking writeback without the congestion wait.

....

Quoted 4 lines
fs/xfs/linux-2.6/xfs_aops.c +0 -0
@@ -908,12 +908,8 @@ xfs_convert_page(
 
Quoted 8 lines
Quoted 6 lines

fs/xfs/linux-2.6/xfs_aops.c xfs_convert_page( · line 912

Quoted 4 lines
fs/xfs/linux-2.6/xfs_aops.c +0 -2
 			bdi = inode->i_mapping->backing_dev_info;
 			wbc->nr_to_write--;
-			if (bdi_write_congested(bdi)) {
-				wbc->encountered_congestion = 1;

bdi is unused now, so can be removed as well. Otherwise looks fine.

Thanks, here is the updated patch.

Looks good.

Cheers,

Dave.

Signature · 2 lines
Dave Chinner
[email protected]
#7 lore raw

Re: [PATCH 03/12] writeback: remove unused nonblocking and congestion checks (pohmelfs)

In reply to Wu Fengguang

Hi.

Quoted 9 lines · Wu Fengguang
Quoted 5 lines · Wu Fengguang
Quoted 1 line

Will you pick up this updated patch? It also removes the unused bdi.

I'm ok with the changes, but I suppose it should be pushed as a single patchet into the tree, no?

Either way works. I would submit a single patchset that includes the generic ones and the fs specific patches that have not been picked up into their fs tree :)

Please push it within your patchset to eliminate merge dependencies if any. Thank you.

Signature · 1 line
	Evgeniy Polyakov