a few writeback fixes

GIT PULL Jens Axboe 4 participants 6 messages Sep 21, 2009

Subsystems (3)

Memory management NFS client Page cache

lore

Share & tools

lore

Jens Axboe

author

2009-09-21 12:56 UTC #0 lore raw

Hi Linus,

This fixes a problem with NFS killing its bdi without being initialized, and also contains a fix for a busy loop (where we forgot to set the task state before sleeping). Since these are a bit critical for -rc1, I'm pushing them out before leaving for Portland.

Jan, since we're not really sure on the inode wait yet, I dropped that patch. It's definitely something to look into, as are the congestion bits.

I plan on adding some writeback tracing code while on the plane for further analysis when I get back.

git://git.kernel.dk/linux-2.6-block.git writeback

Jens Axboe (4):
nfs: nfs_kill_super() should call bdi_unregister() after killing super
writeback: don't use schedule_timeout() without setting runstate
writeback: make balance_dirty_pages() gradually back more off
nfs: initialize the backing_dev_info when creating the server

 fs/nfs/client.c     |   10 ++++++----
 fs/nfs/super.c      |    2 +-
 mm/page-writeback.c |   11 ++++++++++-
 3 files changed, 17 insertions(+), 6 deletions(-)

fs/nfs/client.c +6 -4

index e350bd6..a7ce15d 100644
@@ -933,10 +933,6 @@ static int nfs_probe_fsinfo(struct nfs_server *server, struct nfs_fh *mntfh, str
 		goto out_error;
 
 	nfs_server_set_fsinfo(server, &fsinfo);
-	error = bdi_init(&server->backing_dev_info);
-	if (error)
-		goto out_error;
-
 
 	/* Get some general file system info */
 	if (server->namelen == 0) {

@@ -995,6 +991,12 @@ static struct nfs_server *nfs_alloc_server(void)
 		return NULL;
 	}
 
+	if (bdi_init(&server->backing_dev_info)) {
+		nfs_free_iostats(server->io_stats);
+		kfree(server);
+		return NULL;
+	}
+
 	return server;
 }

fs/nfs/super.c +1 -1

index de93569..f1cc058 100644
@@ -2190,8 +2190,8 @@ static void nfs_kill_super(struct super_block *s)
 {
 	struct nfs_server *server = NFS_SB(s);
 
-	bdi_unregister(&server->backing_dev_info);
 	kill_anon_super(s);
+	bdi_unregister(&server->backing_dev_info);
 	nfs_fscache_release_super_cookie(s);
 	nfs_free_server(server);
 }

mm/page-writeback.c +10 -1

index 1eea4fa..6bb510b 100644
@@ -485,6 +485,7 @@ static void balance_dirty_pages(struct address_space *mapping)
 	unsigned long bdi_thresh;
 	unsigned long pages_written = 0;
 	unsigned long write_chunk = sync_writeback_pages();
+	unsigned long pause = 1;
 
 	struct backing_dev_info *bdi = mapping->backing_dev_info;

@@ -561,7 +562,15 @@ static void balance_dirty_pages(struct address_space *mapping)
 		if (pages_written >= write_chunk)
 			break;		/* We've done our duty */
 
-		schedule_timeout(1);
+		schedule_timeout_interruptible(pause);
+
+		/*
+		 * Increase the delay for each loop, up to our previous
+		 * default of taking a 100ms nap.
+		 */
+		pause <<= 1;
+		if (pause < HZ / 10)
+			pause = HZ / 10;
 	}
 
 	if (bdi_nr_reclaimable + bdi_nr_writeback < bdi_thresh &&

Signature · 1 line

Jens Axboe

Wu Fengguang

2009-09-21 13:08 UTC #1 lore raw

Quoted 6 lines · Jens Axboe

Hi Linus,
This fixes a problem with NFS killing its bdi without being initialized, and also contains a fix for a busy loop (where we forgot to set the task state before sleeping). Since these are a bit critical for -rc1, I'm pushing them out before leaving for Portland.

Jens, I confirmed that it fixed the problem I encountered.

Thanks,
Fengguang

Jan Kara

maintainer

2009-09-21 13:11 UTC #2 lore raw

In reply to Jens Axboe

On Mon 21-09-09 14:56:20, Jens Axboe wrote:
Hi Jens,

Quoted 3 lines

Jan, since we're not really sure on the inode wait yet, I dropped that patch. It's definitely something to look into, as are the congestion bits.

OK.

Honza

Quoted 16 lines

mm/page-writeback.c +1 -0

index 1eea4fa..6bb510b 100644
@@ -485,6 +485,7 @@ static void balance_dirty_pages(struct address_space *mapping)
 	unsigned long bdi_thresh;
 	unsigned long pages_written = 0;
 	unsigned long write_chunk = sync_writeback_pages();
+	unsigned long pause = 1;
 
 	struct backing_dev_info *bdi = mapping->backing_dev_info;

@@ -561,7 +562,15 @@ static void balance_dirty_pages(struct address_space *mapping)
 		if (pages_written >= write_chunk)
 			break;		/* We've done our duty */

mm/page-writeback.c static void balance_dirty_pages(struct address_space *mapping) · line 573

Quoted 10 lines

mm/page-writeback.c +9 -1

-		schedule_timeout(1);
+		schedule_timeout_interruptible(pause);
+
+		/*
+		 * Increase the delay for each loop, up to our previous
+		 * default of taking a 100ms nap.
+		 */
+		pause <<= 1;
+		if (pause < HZ / 10)
+			pause = HZ / 10;

Isn't this check the other way around?

mm/page-writeback.c static void balance_dirty_pages(struct address_space *mapping) · line 576

Quoted 3 lines

mm/page-writeback.c +0 -0

 	}
 
 	if (bdi_nr_reclaimable + bdi_nr_writeback < bdi_thresh &&

Honza

Signature · 2 lines

Jan Kara <[email protected]>
SUSE Labs, CR

Jens Axboe

author

2009-09-21 13:41 UTC #3 lore raw

Quoted 26 lines · Jan Kara

On Mon 21-09-09 14:56:20, Jens Axboe wrote:
Hi Jens,
Quoted 3 lines
Jan, since we're not really sure on the inode wait yet, I dropped that patch. It's definitely something to look into, as are the congestion bits.
OK.
Honza
Quoted 16 lines
mm/page-writeback.c +1 -0
index 1eea4fa..6bb510b 100644
@@ -485,6 +485,7 @@ static void balance_dirty_pages(struct address_space *mapping)
 	unsigned long bdi_thresh;
 	unsigned long pages_written = 0;
 	unsigned long write_chunk = sync_writeback_pages();
+	unsigned long pause = 1;
 
 	struct backing_dev_info *bdi = mapping->backing_dev_info;
 
@@ -561,7 +562,15 @@ static void balance_dirty_pages(struct address_space *mapping)
 		if (pages_written >= write_chunk)
 			break;		/* We've done our duty */
 

Quoted 11 lines

mm/page-writeback.c static void balance_dirty_pages(struct address_space *mapping) · line 573
Quoted 10 lines
mm/page-writeback.c +9 -1
-		schedule_timeout(1);
+		schedule_timeout_interruptible(pause);
+
+		/*
+		 * Increase the delay for each loop, up to our previous
+		 * default of taking a 100ms nap.
+		 */
+		pause <<= 1;
+		if (pause < HZ / 10)
+			pause = HZ / 10;
Isn't this check the other way around?

Gah it is, how silly. I have fixed it up. Thanks, eagle eyes :-)

Signature · 1 line

Jens Axboe

Ingo Molnar

2009-09-21 16:42 UTC #4 lore raw

In reply to Jens Axboe

Quoted 22 lines · Jens Axboe

Hi Linus,
This fixes a problem with NFS killing its bdi without being initialized, and also contains a fix for a busy loop (where we forgot to set the task state before sleeping). Since these are a bit critical for -rc1, I'm pushing them out before leaving for Portland.
Jan, since we're not really sure on the inode wait yet, I dropped that patch. It's definitely something to look into, as are the congestion bits.
I plan on adding some writeback tracing code while on the plane for further analysis when I get back.
git://git.kernel.dk/linux-2.6-block.git writeback
Jens Axboe (4):
nfs: nfs_kill_super() should call bdi_unregister() after killing super
writeback: don't use schedule_timeout() without setting runstate
writeback: make balance_dirty_pages() gradually back more off
nfs: initialize the backing_dev_info when creating the server

Quoted 4 lines

 fs/nfs/client.c     |   10 ++++++----
 fs/nfs/super.c      |    2 +-
 mm/page-writeback.c |   11 ++++++++++-
 3 files changed, 17 insertions(+), 6 deletions(-)

just an update about the BDI slab corruption bugs i reported in this thread on lkml:

[origin tree SLAB corruption] BUG kmalloc-64: Poison overwritten,

        INFO: Allocated in bdi_alloc_work+0x2b/0x100 age=175 cpu=1 pid=3514

They have gone away entirely with your earlier BDI fixes/updates, on all -tip testsystems that triggered it. So that bug can be closed for good.

Thanks,

Ingo

Jens Axboe

author

2009-09-21 19:02 UTC #5 lore raw

Quoted 30 lines · Ingo Molnar

Quoted 22 lines · Jens Axboe
Hi Linus,
This fixes a problem with NFS killing its bdi without being initialized, and also contains a fix for a busy loop (where we forgot to set the task state before sleeping). Since these are a bit critical for -rc1, I'm pushing them out before leaving for Portland.
Jan, since we're not really sure on the inode wait yet, I dropped that patch. It's definitely something to look into, as are the congestion bits.
I plan on adding some writeback tracing code while on the plane for further analysis when I get back.
git://git.kernel.dk/linux-2.6-block.git writeback
Jens Axboe (4):
nfs: nfs_kill_super() should call bdi_unregister() after killing super
writeback: don't use schedule_timeout() without setting runstate
writeback: make balance_dirty_pages() gradually back more off
nfs: initialize the backing_dev_info when creating the server
Quoted 4 lines
 fs/nfs/client.c     |   10 ++++++----
 fs/nfs/super.c      |    2 +-
 mm/page-writeback.c |   11 ++++++++++-
 3 files changed, 17 insertions(+), 6 deletions(-)

Quoted 8 lines

just an update about the BDI slab corruption bugs i reported in this thread on lkml:
[origin tree SLAB corruption] BUG kmalloc-64: Poison overwritten,
        INFO: Allocated in bdi_alloc_work+0x2b/0x100 age=175 cpu=1 pid=3514
They have gone away entirely with your earlier BDI fixes/updates, on all -tip testsystems that triggered it. So that bug can be closed for good.

That's good, I was expecting Nicks fixes to resolve those. Thanks for confirming!

Signature · 1 line

Jens Axboe