Skip to contentSkip to conversation

writeback bug fixes and simplifications

PATCH Sascha Warner 5 participants 8 messages Dec 28, 2007 – Jan 9, 2008

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

Hi,

I applied your patches to 2.6.24-rc6-mm1, but now I am faced with one pdflush often using 100% CPU for a long time. There seem to be some rare pauses from its 100% usage, however.

On ~23 minutes uptime i have ~19 minutes pdflush runtime.

This is on E6600, x86_64, 2 Gig RAM, SATA HDD, running on gentoo ~x64_64

Let me know if you need more info.

Prost,
Sascha Warner

maintainer
#1 lore raw
Quoted 6 lines · On Thu, 27 Dec 2007 23:08:40 +0100 Sascha Warner <[email protected]> wrote:

Hi,

I applied your patches to 2.6.24-rc6-mm1, but now I am faced with one pdflush often using 100% CPU for a long time. There seem to be some rare pauses from its 100% usage, however.

Quoted 6 lines

On ~23 minutes uptime i have ~19 minutes pdflush runtime.

This is on E6600, x86_64, 2 Gig RAM, SATA HDD, running on gentoo ~x64_64

Let me know if you need more info.

(some) cc's restored. Please, always do reply-to-all.

#2 lore raw
Quoted 8 lines · Andrew Morton
Quoted 6 lines · On Thu, 27 Dec 2007 23:08:40 +0100 Sascha Warner <[email protected]> wrote:

Hi,

I applied your patches to 2.6.24-rc6-mm1, but now I am faced with one pdflush often using 100% CPU for a long time. There seem to be some rare pauses from its 100% usage, however.

Quoted 5 lines
Quoted 5 lines

On ~23 minutes uptime i have ~19 minutes pdflush runtime.

This is on E6600, x86_64, 2 Gig RAM, SATA HDD, running on gentoo ~x64_64

Let me know if you need more info.

Sascha, Thank you for the testing.

Replace the first patch with this one should help:

---
Subject: writeback: introduce s_more_io_wait for inodes to be re-synced after a while

Introduce super_block.s_more_io_wait and writeback_control.more_io_wait. They are for inodes that for some reason cannot be synced immediately.

These inodes will be moved to s_more_io_wait and retried after a while(waiting up to 0.1s). The normal lots-of-dirty-pages inodes will be moved to more_io and be synced after other inodes in s_io have been serviced(no sleep).

The data flow is made more simple:

s_dirty --> s_io --> s_more_io/s_more_io_wait --+

             ^                                  |
	     |                                  |
	     +----------------------------------+

- to fill s_io:

		s_more_io +
		s_dirty(expired) +
		s_more_io_wait
				---> s_io
- to drain s_io:
		s_io -+--> clean inodes goto inode_in_use/inode_unused
		      |
		      +--> s_more_io
		      |
		      +--> s_more_io_wait

Obviously there's no ordering or starvation problems in the queues:
- s_dirty is now a strict FIFO queue
- inode.dirtied_when now really means the first dirty time
- once exipired, the dirty inode will stay in s_*io* queues until made clean
- the dirty inodes in s_*io* queues will be revisted in order: no starvation

Cc Michael Rubin <[email protected]>
Cc Peter Zijlstra <[email protected]>
---
 fs/fs-writeback.c         |   26 ++++++++++++----
 fs/super.c                |    1 
 include/linux/fs.h        |    1 
 include/linux/writeback.h |    1 
 mm/page-writeback.c       |   56 ++++++++++++++++++++----------------
 5 files changed, 55 insertions(+), 30 deletions(-)
fs/fs-writeback.c +20 -6
@@ -172,6 +172,14 @@ static void requeue_io(struct inode *ino
 	list_move(&inode->i_list, &inode->i_sb->s_more_io);
 }
 
+/*
+ * The inode should be retried after _sleeping_ for a while.
+ */
+static void requeue_io_wait(struct inode *inode)
+{
+	list_move(&inode->i_list, &inode->i_sb->s_more_io_wait);
+}
+
 static void inode_sync_complete(struct inode *inode)
 {
 	/*
@@ -206,13 +214,15 @@ static void queue_io(struct super_block 
 {
 	list_splice_init(&sb->s_more_io, sb->s_io.prev);
 	move_expired_inodes(&sb->s_dirty, &sb->s_io, older_than_this);
+	list_splice_init(&sb->s_more_io_wait, sb->s_io.prev);
 }
 
 int sb_has_dirty_inodes(struct super_block *sb)
 {
-	return !list_empty(&sb->s_dirty) ||
-	       !list_empty(&sb->s_io) ||
-	       !list_empty(&sb->s_more_io);
+	return !list_empty(&sb->s_dirty)   ||
+	       !list_empty(&sb->s_io)      ||
+	       !list_empty(&sb->s_more_io) ||
+	       !list_empty(&sb->s_more_io_wait);
 }
 EXPORT_SYMBOL(sb_has_dirty_inodes);
 
@@ -472,11 +482,15 @@ int generic_sync_sb_inodes(struct super_
 		iput(inode);
 		cond_resched();
 		spin_lock(&inode_lock);
-		if (wbc->nr_to_write <= 0)
+		if (wbc->nr_to_write <= 0) {
+			wbc->more_io = 1;
 			break;
+		}
+		if (!list_empty(&sb->s_more_io))
+			wbc->more_io = 1;
+		if (!list_empty(&sb->s_more_io_wait))
+			wbc->more_io_wait = 1;
 	}
-	if (!list_empty(&sb->s_more_io))
-		wbc->more_io = 1;
 	spin_unlock(&inode_lock);
 	return ret;		/* Leave any unwritten inodes on s_io */
 }
mm/page-writeback.c +32 -24

87 lines not on this page · raw message

fs/super.c +1 -0
@@ -64,6 +64,7 @@ static struct super_block *alloc_super(s
 		INIT_LIST_HEAD(&s->s_dirty);
 		INIT_LIST_HEAD(&s->s_io);
 		INIT_LIST_HEAD(&s->s_more_io);
+		INIT_LIST_HEAD(&s->s_more_io_wait);
 		INIT_LIST_HEAD(&s->s_files);
 		INIT_LIST_HEAD(&s->s_instances);
 		INIT_HLIST_HEAD(&s->s_anon);
include/linux/fs.h +1 -0
@@ -1011,6 +1011,7 @@ struct super_block {
 	struct list_head	s_dirty;	/* dirty inodes */
 	struct list_head	s_io;		/* parked for writeback */
 	struct list_head	s_more_io;	/* parked for more writeback */
+	struct list_head	s_more_io_wait;	/* parked for sleep-then-retry */
 	struct hlist_head	s_anon;		/* anonymous dentries for (nfs) exporting */
 	struct list_head	s_files;
 
include/linux/writeback.h +1 -0
@@ -63,6 +63,7 @@ struct writeback_control {
 	unsigned for_writepages:1;	/* This is a writepages() call */
 	unsigned range_cyclic:1;	/* range_start is cyclic */
 	unsigned more_io:1;		/* more io to be dispatched */
+	unsigned more_io_wait:1;	/* more io to be dispatched after a while */
 };
 
 /*
#3 lore raw
In reply to Andrew Morton
Quoted 11 lines · Andrew Morton
Quoted 8 lines · On Thu, 27 Dec 2007 23:08:40 +0100 Sascha Warner <[email protected]> wrote:

Hi,

I applied your patches to 2.6.24-rc6-mm1, but now I am faced with one pdflush often using 100% CPU for a long time. There seem to be some rare pauses from its 100% usage, however.

On ~23 minutes uptime i have ~19 minutes pdflush runtime.

Quoted 8 lines
Quoted 5 lines

This is on E6600, x86_64, 2 Gig RAM, SATA HDD, running on gentoo ~x64_64

Let me know if you need more info.

(some) cc's restored. Please, always do reply-to-all.

Hi Wu,

Your latest patch does fix the seen pdflush 100% issue :)

P.S.: Andrew, sorry my subject pretended I was subscribed to lkml, I am not. I hoped for the script at lkml to pull the threads together because of the subject.

Thank you,
Sascha Warner

#4 lore raw
Quoted 4 lines

P.S.: Andrew, sorry my subject pretended I was subscribed to lkml, I am not. I hoped for the script at lkml to pull the threads together because of the subject.

Find the relevant mail at lkml.org and have it forwarded to you. Then you can do a proper reply-to-all without breaking the thread and keeping relevant cc's.

Sam

#5 lore raw
In reply to Sascha Warner
Quoted 18 lines · Sascha Warner
Quoted 17 lines · Andrew Morton
Quoted 13 lines · On Thu, 27 Dec 2007 23:08:40 +0100 Sascha Warner <[email protected]> wrote:

Hi,

I applied your patches to 2.6.24-rc6-mm1, but now I am faced with one pdflush often using 100% CPU for a long time. There seem to be some rare pauses from its 100% usage, however.

On ~23 minutes uptime i have ~19 minutes pdflush runtime.

This is on E6600, x86_64, 2 Gig RAM, SATA HDD, running on gentoo ~x64_64

Let me know if you need more info.

Quoted 5 lines
Quoted 2 lines

(some) cc's restored. Please, always do reply-to-all.

Hi Wu,

Your latest patch does fix the seen pdflush 100% issue :)

Thank you.

Andrew: I'll be off line for some days.
The patchset will be updated some time later.

Happy new year~

Fengguang

#6 lore raw
In reply to Sascha Warner

Am Freitag, 28. Dezember 2007 schrieb Sascha Warner:

Quoted 2 lines
Quoted 1 line · Andrew Morton

On Thu, 27 Dec 2007 23:08:40 +0100 Sascha Warner <[email protected]>

Quoted 8 lines · wrote:
Quoted 8 lines
Quoted 8 lines

Hi,

I applied your patches to 2.6.24-rc6-mm1, but now I am faced with one pdflush often using 100% CPU for a long time. There seem to be some rare pauses from its 100% usage, however.

On ~23 minutes uptime i have ~19 minutes pdflush runtime.

Quoted 8 lines
Quoted 6 lines
Quoted 4 lines

This is on E6600, x86_64, 2 Gig RAM, SATA HDD, running on gentoo ~x64_64

Let me know if you need more info.

(some) cc's restored. Please, always do reply-to-all.

Hi Wu,

Sascha, if you want to address Fengguang by his first name, note that chinese and bavarians (and some others I forgot now, too) typically use the order:

lastname firstname

when they spell their names. Another evidence is, that the name Wu is a pretty common chinese family name.

Fengguang, if it's the other way around, correct me please (and I'm going to wear a big brown paper bag for the rest of the day..).

Quoted 5 lines

Your latest patch does fix the seen pdflush 100% issue :)

P.S.: Andrew, sorry my subject pretended I was subscribed to lkml, I am not. I hoped for the script at lkml to pull the threads together because of the subject.

Such a script _cannot_ exist for a mailing list. Sane mailing list software mangles messages in the least intrusive way and highly depend on users doing the right thing. In this case, that's what Sam suggested below!

A-happy-new-year-for-all-kernel-hacking-heros-ly-yrs,

Pete

#7 lore raw
Quoted 25 lines · Hans-Peter Jansen

Am Freitag, 28. Dezember 2007 schrieb Sascha Warner:

Quoted 2 lines
Quoted 1 line · Andrew Morton

On Thu, 27 Dec 2007 23:08:40 +0100 Sascha Warner <[email protected]>

Quoted 8 lines · wrote:
Quoted 8 lines
Quoted 8 lines

Hi,

I applied your patches to 2.6.24-rc6-mm1, but now I am faced with one pdflush often using 100% CPU for a long time. There seem to be some rare pauses from its 100% usage, however.

On ~23 minutes uptime i have ~19 minutes pdflush runtime.

Quoted 8 lines
Quoted 6 lines
Quoted 4 lines

This is on E6600, x86_64, 2 Gig RAM, SATA HDD, running on gentoo ~x64_64

Let me know if you need more info.

(some) cc's restored. Please, always do reply-to-all.

Hi Wu,

Sascha, if you want to address Fengguang by his first name, note that chinese and bavarians (and some others I forgot now, too) typically use the order:

Quoted 7 lines

lastname firstname

when they spell their names. Another evidence is, that the name Wu is a pretty common chinese family name.

Fengguang, if it's the other way around, correct me please (and I'm going to wear a big brown paper bag for the rest of the day..).

You are right. We normally do "Fengguang" or "Mr. Wu" :-) For LKML the first name is less ambiguous.

Thanks,
Fengguang