bdi writeback fixes
cover bdi writeback fixes
Hi,
Ran into a few crashes with the bdi writeback stuff. Things are a bit more stable now. Although I've still got a seemingly unrelated GFP in __d_lookup that I've seen twice now :(
Thanks,
Nick
PATCH 5/5 fs: fix bdi writeback use after free 2
work can be gone after wb_clear_pending, so load the sb from it first.
--- fs/fs-writeback.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
fs/fs-writeback.c +4 -1
@@ -783,6 +783,7 @@ long wb_do_writeback(struct bdi_writebacwhile ((work =get_next_work_item (bdi, wb)) !=NULL ) {enum writeback_sync_modes sync_mode; +struct super_block *sb; nr_pages = work->nr_pages;
@@ -794,6 +795,8 @@ long wb_do_writeback(struct bdi_writebacelse sync_mode = work->sync_mode; + sb = work->sb; +/* * If this isn't a data integrity operation, just notify * that we have seen this work and we are now starting it.
@@ -801,7 +804,7 @@ long wb_do_writeback(struct bdi_writebacif (sync_mode ==WB_SYNC_NONE )wb_clear_pending (wb, work); - wrote +=wb_writeback (wb, nr_pages, work->sb, sync_mode,0 ); + wrote +=wb_writeback (wb, nr_pages, sb, sync_mode,0 );/* * This is a data integrity writeback, so only do the
PATCH 3/5 fs: fix bdi writeback use after free 1
In reply to [email protected]By the time bdi_work_on_stack gets evaluated again in bdi_work_free, it cna already have been deallocated and used for something else in the !on stack case, giving a false positive in this test and causing corruption.
--- fs/fs-writeback.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
fs/fs-writeback.c +3 -2
@@ -114,6 +114,7 @@ static void bdi_work_free(struct rcu_heastatic void wb_work_complete (struct bdi_work *work) {const enum writeback_sync_modes sync_mode = work->sync_mode; +int onstack =bdi_work_on_stack (work);/* * For allocated work, we can clear the done/seen bit right here.
@@ -121,9 +122,9 @@ static void wb_work_complete(struct bdi_* to after the RCU grace period, since the stack could be invalidated * as soon as bdi_work_clear() has done the wakeup. */ -if (!bdi_work_on_stack (work)) +if (!onstack)bdi_work_clear (work); -if (sync_mode ==WB_SYNC_NONE ||bdi_work_on_stack (work)) +if (sync_mode ==WB_SYNC_NONE || onstack)call_rcu (&work->rcu_head, bdi_work_free); }
PATCH 1/5 fs: make use of rcu helpers
In reply to [email protected]list_add_tail_rcu contains required barriers.
--- fs/fs-writeback.c | 5 ----- 1 file changed, 5 deletions(-)
fs/fs-writeback.c +0 -5
@@ -152,11 +152,6 @@ static void bdi_queue_work(struct backinatomic_set (&work->pending, bdi->wb_cnt);BUG_ON (!bdi->wb_cnt); -/* -* Make sure stores are seen before it appears on the list -*/ -smp_mb (); -spin_lock (&bdi->wb_lock);list_add_tail_rcu (&work->list, &bdi->work_list);spin_unlock (&bdi->wb_lock);
PATCH 2/5 fs: improve scalability of bdi writeback work queues
In reply to [email protected]If you're going to do an atomic RMW on each list entry, there's not much point in all the RCU complexities of the list walking. This is only going to help the multi-thread case I guess, but it doesn't hurt to do now.
--- fs/fs-writeback.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
fs/fs-writeback.c +2 -1
@@ -736,8 +736,9 @@ static struct bdi_work *get_next_work_itrcu_read_lock ();list_for_each_entry_rcu (work, &bdi->work_list, list) { -if (!test_and_clear_bit (wb->nr, &work->seen)) +if (!test_bit (wb->nr, &work->seen))continue ; +clear_bit (wb->nr, &work->seen); ret = work;break ;
PATCH 4/5 fs: fix possible bdi writeback refcounting problem
In reply to [email protected]wb_clear_pending AFAIKS should not be called after the item has been put on the list, except by the worker threads. It could lead to the situation where the refcount is decremented below 0 and cause lots of problems.
Presumably the !wb_has_dirty_io case is not a common one, so it can be discovered when the thread wakes up to check?
Also add a comment in bdi_work_clear.
--- fs/fs-writeback.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-)
fs/fs-writeback.c +6 -4
@@ -98,6 +98,11 @@ static void bdi_work_clear(struct bdi_wo {clear_bit (WS_USED_B , &work->state);smp_mb__after_clear_bit (); +/* +* work can have disappeared at this point. bit waitq functions +* should be able to tolerate this, provided bdi_sched_wait does +* not dereference it's pointer argument. +*/ wake_up_bit (&work->state,WS_USED_B ); }
@@ -172,10 +177,7 @@ static void bdi_queue_work(struct backin* thread always. As a safety precaution, it'll flush out * everything */ -if (!wb_has_dirty_io (wb)) { -if (work) -wb_clear_pending (wb, work); - }else if (wb->task) +if (wb->task)wake_up_process (wb->task); } }
Re: [patch 5/5] fs: fix bdi writeback use after free 2
In reply to [email protected]Quoted 1 line · On Wed, Sep 16 2009, [email protected] wrote:
work can be gone after
wb_clear_pending, so load the sb from it first.
This one is already moot, I fixed that in an earlier patch.
Quoted 38 lines
Signed-off-by Nick Piggin <[email protected]>--- fs/fs-writeback.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)Index:
linux-2.6/fs/fs-writeback.c
===================================================================fs/fs-writeback.c +4
-1@@ -783,6 +783,7 @@ long wb_do_writeback(struct bdi_writebacwhile ((work =get_next_work_item (bdi, wb)) !=NULL ) {enum writeback_sync_modes sync_mode; +struct super_block *sb; nr_pages = work->nr_pages;@@ -794,6 +795,8 @@ long wb_do_writeback(struct bdi_writebacelse sync_mode = work->sync_mode; + sb = work->sb; +/* * If this isn't a data integrity operation, just notify * that we have seen this work and we are now starting it. @@ -801,7 +804,7 @@ long wb_do_writeback(struct bdi_writebacif (sync_mode ==WB_SYNC_NONE )wb_clear_pending (wb, work); - wrote +=wb_writeback (wb, nr_pages, work->sb, sync_mode,0 ); + wrote +=wb_writeback (wb, nr_pages, sb, sync_mode,0 );/* * This is a data integrity writeback, so only do the
Signature · 1 line
Jens Axboe
Re: [patch 4/5] fs: fix possible bdi writeback refcounting problem
In reply to [email protected]Quoted 4 lines · On Wed, Sep 16 2009, [email protected] wrote:
wb_clear_pendingAFAIKS should not be called after the item has been put on the list, except by the worker threads. It could lead to the situation where the refcount is decremented below 0 and cause lots of problems.
Good point!
Quoted 2 lines
Presumably the !wb_has_dirty_io case is not a common one, so it can be discovered when the thread wakes up to check?
It's checked earlier as well, so I see no problem in killing the check there.
Quoted 38 lines
Also add a comment in
bdi_work_clear.Signed-off-by Nick Piggin <[email protected]>--- fs/fs-writeback.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-)Index:
linux-2.6/fs/fs-writeback.c
===================================================================fs/fs-writeback.c +6
-4@@ -98,6 +98,11 @@ static void bdi_work_clear(struct bdi_wo {clear_bit (WS_USED_B , &work->state);smp_mb__after_clear_bit (); +/* +* work can have disappeared at this point. bit waitq functions +* should be able to tolerate this, provided bdi_sched_wait does +* not dereference it's pointer argument. +*/ wake_up_bit (&work->state,WS_USED_B ); }@@ -172,10 +177,7 @@ static void bdi_queue_work(struct backin* thread always. As a safety precaution, it'll flush out * everything */ -if (!wb_has_dirty_io (wb)) { -if (work) -wb_clear_pending (wb, work); - }else if (wb->task) +if (wb->task)wake_up_process (wb->task); } }
Signature · 1 line
Jens Axboe
Re: [patch 2/5] fs: improve scalability of bdi writeback work queues
In reply to [email protected]Quoted 3 lines · On Wed, Sep 16 2009, [email protected] wrote:
If you're going to do an atomic RMW on each list entry, there's not much point in all the RCU complexities of the list walking. This is only going to help the multi-thread case I guess, but it doesn't hurt to do now.
Thanks Nick, good optimization.
Signature · 1 line
Jens Axboe
Jens Axboe
AppliedRe: [patch 1/5] fs: make use of rcu helpers
In reply to [email protected]Quoted 1 line · On Wed, Sep 16 2009, [email protected] wrote:
list_add_tail_rcucontains required barriers.
Thanks, applied as well.
Quoted 24 lines
Signed-off-by Nick Piggin <[email protected]>--- fs/fs-writeback.c | 5 ----- 1 file changed, 5 deletions(-)Index:
linux-2.6/fs/fs-writeback.c
===================================================================fs/fs-writeback.c +0
-5@@ -152,11 +152,6 @@ static void bdi_queue_work(struct backinatomic_set (&work->pending, bdi->wb_cnt);BUG_ON (!bdi->wb_cnt); -/* -* Make sure stores are seen before it appears on the list -*/ -smp_mb (); -spin_lock (&bdi->wb_lock);list_add_tail_rcu (&work->list, &bdi->work_list);spin_unlock (&bdi->wb_lock);
Signature · 1 line
Jens Axboe
Re: [patch 3/5] fs: fix bdi writeback use after free 1
In reply to [email protected]Quoted 3 lines · On Wed, Sep 16 2009, [email protected] wrote:
By the time
bdi_work_on_stackgets evaluated again inbdi_work_free, it cna already have been deallocated and used for something else in the !on stack case, giving a false positive in this test and causing corruption.
You are right, that is also buggy... Applied.
Quoted 32 lines
Signed-off-by Nick Piggin <[email protected]>--- fs/fs-writeback.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)Index:
linux-2.6/fs/fs-writeback.c
===================================================================fs/fs-writeback.c +3
-2@@ -114,6 +114,7 @@ static void bdi_work_free(struct rcu_heastatic void wb_work_complete (struct bdi_work *work) {const enum writeback_sync_modes sync_mode = work->sync_mode; +int onstack =bdi_work_on_stack (work);/* * For allocated work, we can clear the done/seen bit right here. @@ -121,9 +122,9 @@ static void wb_work_complete(struct bdi_* to after the RCU grace period, since the stack could be invalidated * as soon as bdi_work_clear() has done the wakeup. */ -if (!bdi_work_on_stack (work)) +if (!onstack)bdi_work_clear (work); -if (sync_mode ==WB_SYNC_NONE ||bdi_work_on_stack (work)) +if (sync_mode ==WB_SYNC_NONE || onstack)call_rcu (&work->rcu_head, bdi_work_free); }
Signature · 1 line
Jens Axboe