Skip to contentSkip to conversation

mm: write_cache_pages be more sequential

RFC PATCH In review Adrian Hunter 4 participants 7 messages Sep 7, 2009 – Sep 14, 2009

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

RFC PATCH 1/1 mm: write_cache_pages be more sequential

Quoted 1 line

From 6f3bb7c26936c45d810048f59c369e8d5a5623fc Mon Sep 17 00:00:00 2001

From: Adrian Hunter <[email protected]>
Date: Mon, 7 Sep 2009 10:49:11 +0300
Subject: [PATCH] mm: write_cache_pages be more sequential

If a file is written to sequentially, then writeback should write the pages sequentially also. However, that does not always happen. For example:

1) user writes pages 0, 1 and 2 but 2 is incomplete
2) write_cache_pages writes pages 0, 1 and 2 and sets writeback_index to 3
3) user finishes writing page 2 and writes pages 3 and 4
4) write_cache_pages writes pages 3 and 4, and then cycles back and writes page 2 again.

So the pages are written out in the order 0, 1, 2, 3 ,4 ,2 instead of 0, 1, 2, 2, 3, 4.

This situation was noticed on UBIFS because it writes directly from writepage. Hence if there is an unexpected power-loss, a file will end up with a hole even though the file was written sequentially by the user.

---
mm/page-writeback.c | 2 ++

1 files changed, 2 insertions(+), 0 deletions(-)
mm/page-writeback.c +0 -0
index 81627eb..7410b7a 100644
@@ -960,6 +960,8 @@ int write_cache_pages(struct address_space *mapping,
	pagevec_init(&pvec, 0);
	if (wbc->range_cyclic) {
		writeback_index = mapping->writeback_index; /* prev offset */
+		if (writeback_index)
+			writeback_index -= 1;
		index = writeback_index;
		if (index == 0)
			cycled = 1;
Signature · 1 line
1.5.6.3
#1 lore raw
Quoted 16 lines · Adrian Hunter
Quoted 1 line

From 6f3bb7c26936c45d810048f59c369e8d5a5623fc Mon Sep 17 00:00:00 2001

From: Adrian Hunter <[email protected]>
Date: Mon, 7 Sep 2009 10:49:11 +0300
Subject: [PATCH] mm: write_cache_pages be more sequential

If a file is written to sequentially, then writeback should write the pages sequentially also. However, that does not always happen. For example:

1) user writes pages 0, 1 and 2 but 2 is incomplete
2) write_cache_pages writes pages 0, 1 and 2 and sets writeback_index to 3
3) user finishes writing page 2 and writes pages 3 and 4
4) write_cache_pages writes pages 3 and 4, and then cycles back and writes page 2 again.

Quoted 2 lines

So the pages are written out in the order 0, 1, 2, 3 ,4 ,2 instead of 0, 1, 2, 2, 3, 4.

Why does page 2 get set dirty if the write was incomplete?

Quoted 15 lines

This situation was noticed on UBIFS because it writes directly from writepage. Hence if there is an unexpected power-loss, a file will end up with a hole even though the file was written sequentially by the user.

---
mm/page-writeback.c | 2 ++

1 files changed, 2 insertions(+), 0 deletions(-)
mm/page-writeback.c +0 -0
index 81627eb..7410b7a 100644

mm/page-writeback.c int write_cache_pages(struct address_space *mapping, · line 967

Quoted 8 lines
mm/page-writeback.c +2 -0
@@ -960,6 +960,8 @@ int write_cache_pages(struct address_space *mapping,
 	pagevec_init(&pvec, 0);
 	if (wbc->range_cyclic) {
 		writeback_index = mapping->writeback_index; /* prev offset */
+		if (writeback_index)
+			writeback_index -= 1;
 		index = writeback_index;
 		if (index == 0)
 			cycled = 1;

Doesn't this just break range_cyclic? range_cyclic is supposed to work across calls to write_cache_pages, and it's there I guess so background writeout will be able to eventually get around to writing all pages relatively fairly in the presence of redirtying operations.

#2 lore raw
Quoted 17 lines · Nick Piggin
Quoted 16 lines · Adrian Hunter
Quoted 1 line

From 6f3bb7c26936c45d810048f59c369e8d5a5623fc Mon Sep 17 00:00:00 2001

From: Adrian Hunter <[email protected]>
Date: Mon, 7 Sep 2009 10:49:11 +0300
Subject: [PATCH] mm: write_cache_pages be more sequential

If a file is written to sequentially, then writeback should write the pages sequentially also. However, that does not always happen. For example:

1) user writes pages 0, 1 and 2 but 2 is incomplete
2) write_cache_pages writes pages 0, 1 and 2 and sets writeback_index to 3
3) user finishes writing page 2 and writes pages 3 and 4
4) write_cache_pages writes pages 3 and 4, and then cycles back and writes page 2 again.

Quoted 4 lines
Quoted 2 lines

So the pages are written out in the order 0, 1, 2, 3 ,4 ,2 instead of 0, 1, 2, 2, 3, 4.

Why does page 2 get set dirty if the write was incomplete?

I meant that only part of the page was written. e.g. write 10240 bytes, wait for writeback, then write another 10240 bytes. The pages will be written out in the order 0, 1, 2, 3, 4, 2

Quoted 20 lines
Quoted 20 lines

This situation was noticed on UBIFS because it writes directly from writepage. Hence if there is an unexpected power-loss, a file will end up with a hole even though the file was written sequentially by the user.

---
mm/page-writeback.c | 2 ++

1 files changed, 2 insertions(+), 0 deletions(-)
mm/page-writeback.c +2 -0
index 81627eb..7410b7a 100644
@@ -960,6 +960,8 @@ int write_cache_pages(struct address_space *mapping,
 	pagevec_init(&pvec, 0);
 	if (wbc->range_cyclic) {
 		writeback_index = mapping->writeback_index; /* prev offset */
+		if (writeback_index)
+			writeback_index -= 1;
Quoted 8 lines

mm/page-writeback.c int write_cache_pages(struct address_space *mapping, · line 967

Quoted 3 lines
mm/page-writeback.c +0 -0
 		index = writeback_index;
 		if (index == 0)
 			cycled = 1;

Doesn't this just break range_cyclic? range_cyclic is supposed to work across calls to write_cache_pages, and it's there I guess so background writeout will be able to eventually get around to writing all pages relatively fairly in the presence of redirtying operations.

I do not immediately see how it breaks range_cyclic. Can you give an example?

#3 lore raw
Quoted 21 lines · Adrian Hunter
Quoted 20 lines · Nick Piggin
Quoted 18 lines · Adrian Hunter
Quoted 1 line

From 6f3bb7c26936c45d810048f59c369e8d5a5623fc Mon Sep 17 00:00:00 2001

From: Adrian Hunter <[email protected]>
Date: Mon, 7 Sep 2009 10:49:11 +0300
Subject: [PATCH] mm: write_cache_pages be more sequential

If a file is written to sequentially, then writeback should write the pages sequentially also. However, that does not always happen. For example:

1) user writes pages 0, 1 and 2 but 2 is incomplete
2) write_cache_pages writes pages 0, 1 and 2 and sets writeback_index to 3
3) user finishes writing page 2 and writes pages 3 and 4
4) write_cache_pages writes pages 3 and 4, and then cycles back and writes page 2 again.

So the pages are written out in the order 0, 1, 2, 3 ,4 ,2 instead of 0, 1, 2, 2, 3, 4.

Quoted 6 lines
Quoted 1 line

Why does page 2 get set dirty if the write was incomplete?

I meant that only part of the page was written. e.g. write 10240 bytes, wait for writeback, then write another 10240 bytes. The pages will be written out in the order 0, 1, 2, 3, 4, 2

OK...

Quoted 29 lines
Quoted 20 lines
Quoted 20 lines

This situation was noticed on UBIFS because it writes directly from writepage. Hence if there is an unexpected power-loss, a file will end up with a hole even though the file was written sequentially by the user.

---
mm/page-writeback.c | 2 ++

1 files changed, 2 insertions(+), 0 deletions(-)
mm/page-writeback.c +2 -0
index 81627eb..7410b7a 100644
@@ -960,6 +960,8 @@ int write_cache_pages(struct address_space *mapping,
 	pagevec_init(&pvec, 0);
 	if (wbc->range_cyclic) {
 		writeback_index = mapping->writeback_index; /* prev offset */
+		if (writeback_index)
+			writeback_index -= 1;
Quoted 8 lines

mm/page-writeback.c int write_cache_pages(struct address_space *mapping, · line 967

Quoted 3 lines
mm/page-writeback.c +0 -0
 		index = writeback_index;
 		if (index == 0)
 			cycled = 1;

Doesn't this just break range_cyclic? range_cyclic is supposed to work across calls to write_cache_pages, and it's there I guess so background writeout will be able to eventually get around to writing all pages relatively fairly in the presence of redirtying operations.

Quoted 2 lines

I do not immediately see how it breaks range_cyclic. Can you give an example?

Oh, I must be dyslexic, I read it as writeback_index = -1; :P But I think it can still cause some subtle problems with error cases.

I guess you could just make the done_index assignment more logical and make it page->index. Then add a comment when assigning to writeback_index that you want to start up again at the previously written page to help this case.

Also, check to ensure the error cases are going to still work correctly. Eg. you might want to increment done_index in the case of error.

I guess it is a reasonable workaround for the problem. It is a bit unsatisfying to special case on a page basis like this, but anyway I don't think there should be a realistic downside in practice.

#4 lore raw
Quoted 10 lines · Nick Piggin
Quoted 4 lines

I meant that only part of the page was written. e.g. write 10240 bytes, wait for writeback, then write another 10240 bytes. The pages will be written out in the order 0, 1, 2, 3, 4, 2

OK...

I guess it is a reasonable workaround for the problem. It is a bit unsatisfying to special case on a page basis like this, but anyway I don't think there should be a realistic downside in practice.

*NOTE* we also see this in NFS (I think) where a file is written to sequentially yet readers will see holes in the file.

This is allowed behavior for NFS but annoying. So a work-around would be great!

					=a=
Signature · 3 lines
===================
Aaron Straus
[email protected]
#5 lore raw
In reply to Nick Piggin
Quoted 60 lines · ext Nick Piggin
Quoted 18 lines · Adrian Hunter
Quoted 17 lines · Nick Piggin
Quoted 16 lines · Adrian Hunter
Quoted 1 line

From 6f3bb7c26936c45d810048f59c369e8d5a5623fc Mon Sep 17 00:00:00 2001

From: Adrian Hunter <[email protected]>
Date: Mon, 7 Sep 2009 10:49:11 +0300
Subject: [PATCH] mm: write_cache_pages be more sequential

If a file is written to sequentially, then writeback should write the pages sequentially also. However, that does not always happen. For example:

1) user writes pages 0, 1 and 2 but 2 is incomplete
2) write_cache_pages writes pages 0, 1 and 2 and sets writeback_index to 3
3) user finishes writing page 2 and writes pages 3 and 4
4) write_cache_pages writes pages 3 and 4, and then cycles back and writes page 2 again.

Quoted 7 lines
Quoted 3 lines
Quoted 2 lines

So the pages are written out in the order 0, 1, 2, 3 ,4 ,2 instead of 0, 1, 2, 2, 3, 4.

Why does page 2 get set dirty if the write was incomplete?

I meant that only part of the page was written. e.g. write 10240 bytes, wait for writeback, then write another 10240 bytes. The pages will be written out in the order 0, 1, 2, 3, 4, 2

OK...

Quoted 27 lines
Quoted 19 lines
Quoted 19 lines

This situation was noticed on UBIFS because it writes directly from writepage. Hence if there is an unexpected power-loss, a file will end up with a hole even though the file was written sequentially by the user.

---
mm/page-writeback.c | 2 ++

1 files changed, 2 insertions(+), 0 deletions(-)
mm/page-writeback.c +1 -0
index 81627eb..7410b7a 100644
@@ -960,6 +960,8 @@ int write_cache_pages(struct address_space *mapping,
 	pagevec_init(&pvec, 0);
 	if (wbc->range_cyclic) {
 		writeback_index = mapping->writeback_index; /* prev offset */
+		if (writeback_index)
Quoted 8 lines

mm/page-writeback.c int write_cache_pages(struct address_space *mapping, · line 967

Quoted 4 lines
mm/page-writeback.c +1 -0
+			writeback_index -= 1;
 		index = writeback_index;
 		if (index == 0)
 			cycled = 1;

Doesn't this just break range_cyclic? range_cyclic is supposed to work across calls to write_cache_pages, and it's there I guess so background writeout will be able to eventually get around to writing all pages relatively fairly in the presence of redirtying operations.

Quoted 2 lines

I do not immediately see how it breaks range_cyclic. Can you give an example?

Quoted 8 lines

Oh, I must be dyslexic, I read it as writeback_index = -1; :P But I think it can still cause some subtle problems with error cases.

I guess you could just make the done_index assignment more logical and make it page->index. Then add a comment when assigning to writeback_index that you want to start up again at the previously written page to help this case.

That means changing slightly the meaning of writeback_index which will mean more analysis to avoid unexpected side-effects.

Speaking of unexpected side-effects, I glanced at ext4_da_writepages() which contains the line:

		wbc->nr_to_write -= mpd.pages_written;

which should probably be:

		if (mpd.pages_written >= wbc->nr_to_write)
			wbc->nr_to_write = 0;
		else
			wbc->nr_to_write -= mpd.pages_written;

now that write_cache_pages() can write more than wbc->nr_to_write pages. What do you think?

Quoted 2 lines

Also, check to ensure the error cases are going to still work correctly. Eg. you might want to increment done_index in the case of error.

Sure.

Quoted 3 lines

I guess it is a reasonable workaround for the problem. It is a bit unsatisfying to special case on a page basis like this, but anyway I don't think there should be a realistic downside in practice.

#6 lore raw
In reply to Adrian Hunter
Quoted 19 lines · Adrian Hunter

From 6f3bb7c26936c45d810048f59c369e8d5a5623fc Mon Sep 17 00:00:00 2001
From: Adrian Hunter <[email protected]>
Date: Mon, 7 Sep 2009 10:49:11 +0300
Subject: [PATCH] mm: write_cache_pages be more sequential

If a file is written to sequentially, then writeback should write the pages sequentially also. However, that does not always happen. For example:

1) user writes pages 0, 1 and 2 but 2 is incomplete
2) write_cache_pages writes pages 0, 1 and 2 and sets writeback_index to 3
3) user finishes writing page 2 and writes pages 3 and 4
4) write_cache_pages writes pages 3 and 4, and then cycles back and writes page 2 again.

So the pages are written out in the order 0, 1, 2, 3 ,4 ,2 instead of 0, 1, 2, 2, 3, 4.

Quoted 6 lines

This situation was noticed on UBIFS because it writes directly from writepage. Hence if there is an unexpected power-loss, a file will end up with a hole even though the file was written sequentially by the user.

I wonder, who would merge this patch?

Signature · 2 lines
Best Regards,
Artem Bityutskiy (Артём Битюцкий)