write-back: fix nr_to_write counter
Subsystems (3)
PATCH 1/1 write-back: fix nr_to_write counter
Hi,
commit 05fe478dd04e02fa230c305ab9b5616669821dd3
Author: Nick Piggin <[email protected]> Date: Tue Jan 6 14:39:08 2009 -0800
mm: write_cache_pages integrity fix
broke wbc->nr_to_write handling. Here is the fix.
I'm not 100% sure I got things right, because I am far not expert in the area. Please, review it. The patch fixes my UBIFS issues, which are caused by the fact that wbc->nr_to_write is not updated.
======================================================================
From: Artem Bityutskiy <[email protected]>
Date: Mon, 2 Feb 2009 18:33:49 +0200
Subject: [PATCH] write-back: fix nr_to_write counter
Commit 05fe478dd04e02fa230c305ab9b5616669821dd3 broke @wbc->nr_to_write. 'write_cache_pages()' changes it in the loop, but restores the original value from @nr_to_write at the end, because of this code:
if (!wbc->no_nrwrite_index_update) {
if (wbc->range_cyclic || (range_whole && nr_to_write > 0))
mapping->writeback_index = done_index;
wbc->nr_to_write = nr_to_write;
}Well, in case of @wbc->no_nrwrite_index_update != 0, we do change wbc->nr_to_write, while we should not. This patch fixes this behavior.
Also, I think wbc->nr_to_write should be changed in all cases, not only when wbc->sync_mode == WB_SYNC_NONE.
Also, I add a comment explaining why we do not stop writing back.
--- mm/page-writeback.c | 21 +++++++++++++++------ 1 files changed, 15 insertions(+), 6 deletions(-)
mm/page-writeback.c +15 -6
@@ -1051,13 +1051,22 @@ continue_unlock: } } -if (wbc->sync_mode ==WB_SYNC_NONE ) { - wbc->nr_to_write--; -if (wbc->nr_to_write <=0 ) { - done =1 ; -break ; - } +if (nr_to_write >0 ) + nr_to_write--; +else if (wbc->sync_mode ==WB_SYNC_NONE ) { +/* +* We stop writing back only if we are not +* doing integrity sync. In case of integrity +* sync we have to keep going because someone +* may be concurrently dirtying pages, and we +* might have synced a lot of newly appeared +* dirty pages, bud have not synced all of the +* old dirty pages. +*/ + done =1 ; +break ; } +if (wbc->nonblocking &&bdi_write_congested (bdi)) { wbc->encountered_congestion =1 ; done =1 ;
Signature · 5 lines
1.6.0.6 -- Best regards, Artem Bityutskiy (Битюцкий Артём)
Quoted 23 lines · On Tuesday 03 February 2009 19:42:22 Artem Bityutskiy wrote:
Hi,
commit
05fe478dd04e02fa230c305ab9b5616669821dd3Author: Nick Piggin <[email protected]> Date: Tue Jan 6 14:39:08 2009 -0800mm:
write_cache_pagesintegrity fixbroke
wbc->nr_to_writehandling. Here is the fix.I'm not 100% sure I got things right, because I am far not expert in the area. Please, review it. The patch fixes my UBIFS issues, which are caused by the fact that
wbc->nr_to_writeis not updated.
======================================================================From: Artem Bityutskiy <[email protected]>
Date: Mon, 2 Feb 2009 18:33:49 +0200
Subject: [PATCH] write-back: fixnr_to_writecounterCommit
05fe478dd04e02fa230c305ab9b5616669821dd3broke @wbc->nr_to_write. 'write_cache_pages()' changes it in the loop, but restores the original value from @nr_to_write at the end, because of this code:
Quoted 5 lines
if (!wbc->no_nrwrite_index_update) {
if (wbc->range_cyclic || (range_whole && nr_to_write > 0))
mapping->writeback_index = done_index;
wbc->nr_to_write = nr_to_write;
}The commit you quote only moves nr_to_write to not take effect for WB_SYNC_ALL (ie. data integrity) writeout. And makes no other change to write_cache_pages.
I thought your problem might have been that you were calling this with WB_SYNC_ALL and expecting it to heed nr_to_write, however...
Quoted 2 lines
Also, I think
wbc->nr_to_writeshould be changed in all cases, not only whenwbc->sync_mode==WB_SYNC_NONE.
... you mention this here like it is an *additional* issue on top of your problem. So I fail to see how my commit could have caused this problem?
Quoted 2 lines
Well, in case of @wbc->no_nrwrite_index_update != 0, we do change
wbc->nr_to_write, while we should not. This patch fixes this behavior.
And I don't know what you mean by this because the patch doesn't fix any problem there AFAIKS.
Anyway, I did probably not pay enough attention to ubifs when making this change, and if it wants wbc->nr_to_write updated even for data integrity syncs, I don't see the harm in that. So I don't have any objection to your patch. Thanks.
Can you cc [email protected] when a final version gets merged upstream please?
Quoted 43 lines
Also, I add a comment explaining why we do not stop writing back.
Signed-off-by Artem Bityutskiy <[email protected]>--- mm/page-writeback.c | 21 +++++++++++++++------ 1 files changed, 15 insertions(+), 6 deletions(-)mm/page-writeback.c +15
-7@@ -1051,13 +1051,22 @@ continue_unlock: } } -if (wbc->sync_mode ==WB_SYNC_NONE ) { - wbc->nr_to_write--; -if (wbc->nr_to_write <=0 ) { - done =1 ; -break ; - } +if (nr_to_write >0 ) + nr_to_write--; +else if (wbc->sync_mode ==WB_SYNC_NONE ) { +/* +* We stop writing back only if we are not +* doing integrity sync. In case of integrity +* sync we have to keep going because someone +* may be concurrently dirtying pages, and we +* might have synced a lot of newly appeared +* dirty pages, bud have not synced all of the +* old dirty pages. +*/ + done =1 ; +break ; } +if (wbc->nonblocking &&bdi_write_congested (bdi)) { wbc->encountered_congestion =1 ; done =1 ; --1.6.0.6
Quoted 50 lines · On Tuesday 03 February 2009 20:13:56 Nick Piggin wrote:
Quoted 23 lines · On Tuesday 03 February 2009 19:42:22 Artem Bityutskiy wrote:
Hi,
commit
05fe478dd04e02fa230c305ab9b5616669821dd3Author: Nick Piggin <[email protected]> Date: Tue Jan 6 14:39:08 2009 -0800mm:
write_cache_pagesintegrity fixbroke
wbc->nr_to_writehandling. Here is the fix.I'm not 100% sure I got things right, because I am far not expert in the area. Please, review it. The patch fixes my UBIFS issues, which are caused by the fact that
wbc->nr_to_writeis not updated.
======================================================================From: Artem Bityutskiy <[email protected]>
Date: Mon, 2 Feb 2009 18:33:49 +0200
Subject: [PATCH] write-back: fixnr_to_writecounterCommit
05fe478dd04e02fa230c305ab9b5616669821dd3broke @wbc->nr_to_write. 'write_cache_pages()' changes it in the loop, but restores the original value from @nr_to_write at the end, because of this code:Quoted 5 lines
if (!wbc->no_nrwrite_index_update) { if (wbc->range_cyclic || (range_whole && nr_to_write > 0)) mapping->writeback_index = done_index; wbc->nr_to_write = nr_to_write; }The commit you quote only moves
nr_to_writeto not take effect forWB_SYNC_ALL(ie. data integrity) writeout. And makes no other change towrite_cache_pages.I thought your problem might have been that you were calling this with
WB_SYNC_ALLand expecting it to heednr_to_write, however...Quoted 2 lines
Also, I think
wbc->nr_to_writeshould be changed in all cases, not only whenwbc->sync_mode==WB_SYNC_NONE.... you mention this here like it is an *additional* issue on top of your problem. So I fail to see how my commit could have caused this problem?
Quoted 2 lines
Well, in case of @wbc->no_nrwrite_index_update != 0, we do change
wbc->nr_to_write, while we should not. This patch fixes this behavior.And I don't know what you mean by this because the patch doesn't fix any problem there AFAIKS.
Quoted 7 lines
Anyway, I did probably not pay enough attention to ubifs when making this change, and if it wants
wbc->nr_to_writeupdated even for data integrity syncs, I don't see the harm in that. So I don't have any objection to your patch. Thanks.Can you cc [email protected] when a final version gets merged upstream please?
What I mean to say is that while I think the patch is not a problem, and actually should be merged if it is required for ubifs, I would like to just see the changelog clarified...
Also if you add a nice comment there, maybe mention that nr_to_write must reflect the number of pages written even for the WB_SYNC_ALL case (arguably the data integrity issue which you comment on is more obvious in comparison to the nr_to_write requirement of callers).
Quoted 30 lines · Nick Piggin
Quoted 23 lines · On Tuesday 03 February 2009 19:42:22 Artem Bityutskiy wrote:
Hi,
commit
05fe478dd04e02fa230c305ab9b5616669821dd3Author: Nick Piggin <[email protected]> Date: Tue Jan 6 14:39:08 2009 -0800mm:
write_cache_pagesintegrity fixbroke
wbc->nr_to_writehandling. Here is the fix.I'm not 100% sure I got things right, because I am far not expert in the area. Please, review it. The patch fixes my UBIFS issues, which are caused by the fact that
wbc->nr_to_writeis not updated.
======================================================================From: Artem Bityutskiy <[email protected]>
Date: Mon, 2 Feb 2009 18:33:49 +0200
Subject: [PATCH] write-back: fixnr_to_writecounterCommit
05fe478dd04e02fa230c305ab9b5616669821dd3broke @wbc->nr_to_write. 'write_cache_pages()' changes it in the loop, but restores the original value from @nr_to_write at the end, because of this code:Quoted 5 lines
if (!wbc->no_nrwrite_index_update) { if (wbc->range_cyclic || (range_whole && nr_to_write > 0)) mapping->writeback_index = done_index; wbc->nr_to_write = nr_to_write; }
Quoted 3 lines
The commit you quote only moves
nr_to_writeto not take effect forWB_SYNC_ALL(ie. data integrity) writeout. And makes no other change towrite_cache_pages.
Nick, I'm sorry if my e-mail looked like I'm blame you, I referred the commit because git-bisect pointed to it and I though it for me :-)
And I apologize if I write stupid things.
Here is the commit 05fe478dd04e02fa230c305ab9b5616669821dd3:
mm/filemap.c +1 -1
@@ -210,7 +210,7 @@ int __filemap_fdatawrite_range(struct address_space *mapping, loff_t start,int ret;struct writeback_control wbc = { .sync_mode = sync_mode, - .nr_to_write = mapping->nrpages *2 , + .nr_to_write =LONG_MAX , .range_start = start, .range_end = end, };
mm/page-writeback.c +4 -2
@@ -963,8 +963,10 @@ retry: } } -if (--nr_to_write <=0 ) - done =1 ; +if (wbc->sync_mode ==WB_SYNC_NONE ) { +if (--wbc->nr_to_write <=0 ) + done =1 ; + }if (wbc->nonblocking &&bdi_write_congested (bdi)) { wbc->encountered_congestion =1 ; done =1 ;
It makes the following changes:
1. Decrement wbc->nr_to_write instead of nr_to_write
2. Decrement wbc->nr_to_write _only_ if wbc->sync_mode == WB_SYNC_NONE
3. If synced nr_to_write pages, stop only if if wbc->sync_mode == WB_SYNC_NONE, otherwise keep going.
The commit message talks only about item 3, at leaset as I understand it. I do not quote the commit message because it is large. Thus, I thought changes 1 and 2 were not intentional.
In my patch I try to
1. Undo changes 1 and 2
2. Add a comment explaining change 3 (it very useful to have comments in _code_, not only in the commit)
Quoted 2 lines
I thought your problem might have been that you were calling this with
WB_SYNC_ALLand expecting it to heednr_to_write, however...
Err, my problem is that wbc->nr_to_write is not updated.
Quoted 6 lines
Quoted 2 lines
Also, I think
wbc->nr_to_writeshould be changed in all cases, not only whenwbc->sync_mode==WB_SYNC_NONE.... you mention this here like it is an *additional* issue on top of your problem. So I fail to see how my commit could have caused this problem?
This is the change number 2 in your commit.
Quoted 5 lines
Quoted 2 lines
Well, in case of @wbc->no_nrwrite_index_update != 0, we do change
wbc->nr_to_write, while we should not. This patch fixes this behavior.And I don't know what you mean by this because the patch doesn't fix any problem there AFAIKS.
This is about change number 1. In the "for" loop you change wbc->nr_to_write, instead of nr_to_write. But before the function returns, it writes nr_to_write back to wbc->nr_to_write, so the result is that the caller sees wbc->nr_to_write unchanged.
Quoted 4 lines
Anyway, I did probably not pay enough attention to ubifs when making this change, and if it wants
wbc->nr_to_writeupdated even for data integrity syncs, I don't see the harm in that. So I don't have any objection to your patch. Thanks.
It is just how things were _before_ your patch.
Quoted 2 lines
Can you cc [email protected] when a final version gets merged upstream please?
Sure, I just want to get blessing of one of MM/FS gurus. E.g. you Acked-by would be enough.
Thanks.
Signature · 7 lines
Best regards, Artem Bityutskiy (Битюцкий Артём) -- To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 32 lines · On Tuesday 03 February 2009 20:32:49 Artem Bityutskiy wrote:
Quoted 31 lines · Nick Piggin
Quoted 29 lines · On Tuesday 03 February 2009 19:42:22 Artem Bityutskiy wrote:
Hi,
commit
05fe478dd04e02fa230c305ab9b5616669821dd3Author: Nick Piggin <[email protected]> Date: Tue Jan 6 14:39:08 2009 -0800mm:
write_cache_pagesintegrity fixbroke
wbc->nr_to_writehandling. Here is the fix.I'm not 100% sure I got things right, because I am far not expert in the area. Please, review it. The patch fixes my UBIFS issues, which are caused by the fact that
wbc->nr_to_writeis not updated.
======================================================================From: Artem Bityutskiy <[email protected]>
Date: Mon, 2 Feb 2009 18:33:49 +0200
Subject: [PATCH] write-back: fixnr_to_writecounterCommit
05fe478dd04e02fa230c305ab9b5616669821dd3broke @wbc->nr_to_write. 'write_cache_pages()' changes it in the loop, but restores the original value from @nr_to_write at the end, because of this code:if (!wbc->no_nrwrite_index_update) { if (wbc->range_cyclic || (range_whole && nr_to_write > 0)) mapping->writeback_index = done_index; wbc->nr_to_write = nr_to_write; }
Quoted 8 lines
Quoted 3 lines
The commit you quote only moves
nr_to_writeto not take effect forWB_SYNC_ALL(ie. data integrity) writeout. And makes no other change towrite_cache_pages.Nick, I'm sorry if my e-mail looked like I'm blame you, I referred the commit because git-bisect pointed to it and I though it for me :-)
And I apologize if I write stupid things.
No, you are probably right to blame me ;) but I just am not quite sure if I read your description correctly.
Quoted 33 lines
Here is the commit
05fe478dd04e02fa230c305ab9b5616669821dd3:mm/filemap.c +1
-1@@ -210,7 +210,7 @@ int __filemap_fdatawrite_range(struct address_space *mapping, loff_t start, int ret;struct writeback_control wbc = { .sync_mode = sync_mode, - .nr_to_write = mapping->nrpages *2 , + .nr_to_write =LONG_MAX , .range_start = start, .range_end = end, };mm/page-writeback.c +4
-2@@ -963,8 +963,10 @@ retry: } } -if (--nr_to_write <=0 ) - done =1 ; +if (wbc->sync_mode ==WB_SYNC_NONE ) { +if (--wbc->nr_to_write <=0 ) + done =1 ; + }if (wbc->nonblocking &&bdi_write_congested (bdi)) { wbc->encountered_congestion =1 ; done =1 ;
Quoted 2 lines
It makes the following changes:
1. Decrementwbc->nr_to_writeinstead ofnr_to_write
Ah, ok drat yes I missed this. Indeed this is a stupid bug :(
Quoted 7 lines
2. Decrement
wbc->nr_to_write_only_ ifwbc->sync_mode==WB_SYNC_NONE
3. If syncednr_to_writepages, stop only if ifwbc->sync_mode==WB_SYNC_NONE, otherwise keep going.The commit message talks only about item 3, at leaset as I understand it. I do not quote the commit message because it is large. Thus, I thought changes 1 and 2 were not intentional.
2 was more or less intentional but I glossed over ubifs. At any rate I'm happy if you restore the old behaviour.
Quoted 10 lines
In my patch I try to
1. Undo changes 1 and 2
2. Add a comment explaining change 3 (it very useful to have comments in _code_, not only in the commit)Quoted 2 lines
I thought your problem might have been that you were calling this with
WB_SYNC_ALLand expecting it to heednr_to_write, however...Err, my problem is that
wbc->nr_to_writeis not updated.
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines
Also, I think
wbc->nr_to_writeshould be changed in all cases, not only whenwbc->sync_mode==WB_SYNC_NONE.... you mention this here like it is an *additional* issue on top of your problem. So I fail to see how my commit could have caused this problem?
This is the change number 2 in your commit.
OK, it just wasn't clear to me from the changelog (I missed noticing bug #1).
Quoted 16 lines
Quoted 5 lines
Quoted 2 lines
Well, in case of @wbc->no_nrwrite_index_update != 0, we do change
wbc->nr_to_write, while we should not. This patch fixes this behavior.And I don't know what you mean by this because the patch doesn't fix any problem there AFAIKS.
This is about change number 1. In the "for" loop you change
wbc->nr_to_write, instead ofnr_to_write. But before the function returns, it writesnr_to_writeback towbc->nr_to_write, so the result is that the caller seeswbc->nr_to_writeunchanged.Quoted 4 lines
Anyway, I did probably not pay enough attention to ubifs when making this change, and if it wants
wbc->nr_to_writeupdated even for data integrity syncs, I don't see the harm in that. So I don't have any objection to your patch. Thanks.
Quoted 7 lines
It is just how things were _before_ your patch.
Quoted 2 lines
Can you cc [email protected] when a final version gets merged upstream please?
Sure, I just want to get blessing of one of MM/FS gurus. E.g. you Acked-by would be enough.
Yes please put an Acked-by: Nick Piggin <[email protected]> on there. But I found your comments in this mail to be easier to understand. Perhaps incorporate them into the changelog?
Quoted 9 lines · Nick Piggin
Quoted 5 lines
Quoted 2 lines
Can you cc [email protected] when a final version gets merged upstream please?
Sure, I just want to get blessing of one of MM/FS gurus. E.g. you Acked-by would be enough.
Yes please put an Acked-by: Nick Piggin <[email protected]> on there. But I found your comments in this mail to be easier to understand. Perhaps incorporate them into the changelog?
Will do and re-send it.
Signature · 7 lines
Best regards, Artem Bityutskiy (Битюцкий Артём) -- To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html