Skip to contentSkip to conversation

bio refcount problem

DISCUSSION Philip R Auld 2 participants 4 messages Feb 18, 2005

lore
Share & tools
lore
#0 lore raw

Hi,
I think there are some potential issues with the reference
counting of bios as used in 2.6.10. The __make_request function which is the default block device routine accesses the bio structure after issuing the call to add_request. This means that the bio could have completed before __make_request uses it.

The submit_bh path takes an extra reference with an explicit bio_get/put pair around the submit_bio, but many other users of submit_bio do not. Given that most of the end_io routines remove a reference and hence could free the bio this can lead at the least to __make_request mis-reading the sync flag. In more extreme cases it can cause an oops when run with CONFIG_DEBUG_PAGEALLOC.

The question is what is the preferred fix? I think it may be to simply have submit_bio take its own reference (and remove the extra one from submit_bh).

Alternatively __make_request could be adjusted so that it does not access the bio after calling add_request. All it is doing is checking the bi_rw field for the sync bit.

Or make all users of submit_bio take and release and extra reference like submit_bh.

Thoughts?

Cheers,

Phil

Signature · 3 lines
Philip R. Auld, Ph.D.  	        	       Egenera, Inc.    
Software Architect                            165 Forest St.
(508) 858-2628                            Marlboro, MA 01752
#1 lore raw
Quoted 18 lines · Philip R Auld

Hi,
I think there are some potential issues with the reference
counting of bios as used in 2.6.10. The __make_request function which is the default block device routine accesses the bio structure after issuing the call to add_request. This means that the bio could have completed before __make_request uses it.

The submit_bh path takes an extra reference with an explicit bio_get/put pair around the submit_bio, but many other users of submit_bio do not. Given that most of the end_io routines remove a reference and hence could free the bio this can lead at the least to __make_request mis-reading the sync flag. In more extreme cases it can cause an oops when run with CONFIG_DEBUG_PAGEALLOC.

The question is what is the preferred fix? I think it may be to simply have submit_bio take its own reference (and remove the extra one from submit_bh).

Quoted 6 lines

Alternatively __make_request could be adjusted so that it does not access the bio after calling add_request. All it is doing is checking the bi_rw field for the sync bit.

Or make all users of submit_bio take and release and extra reference like submit_bh.

The queue lock is still held at that point, so the driver hasn't had a chance to process the request yet.

Signature · 1 line
Jens Axboe
#2 lore raw

Hi,

Quoted 1 line · Rumor has it that on Fri, Feb 18, 2005 at 02:59:32PM +0100 Jens Axboe said:

On Fri, Feb 18 2005, Philip R Auld wrote:

...

Quoted 5 lines
Quoted 2 lines

Or make all users of submit_bio take and release and extra reference like submit_bh.

The queue lock is still held at that point, so the driver hasn't had a chance to process the request yet.

Interesting. This is not a theoretical problem though. I've got traces of the oops showing the bio getting freed before the bio_sync(bio) test. When you say driver here what level do you mean? scsi_request_fn at least drops the queue lock.

What if it's merged instead of added directly? That could also get to the same place.

The end_io callback _is_ getting called before __make_request does its "if(bio_sync(bio))" test.

Cheers,

Phil

Quoted 3 lines
Signature · 2 lines
-- 
Jens Axboe
Signature · 3 lines
Philip R. Auld, Ph.D.  	        	       Egenera, Inc.    
Software Architect                            165 Forest St.
(508) 858-2628                            Marlboro, MA 01752
#3 lore raw
Quoted 9 lines · Philip R Auld

Hi,

Quoted 1 line · Rumor has it that on Fri, Feb 18, 2005 at 02:59:32PM +0100 Jens Axboe said:

On Fri, Feb 18 2005, Philip R Auld wrote:

...

Quoted 3 lines
Quoted 2 lines

Or make all users of submit_bio take and release and extra reference like submit_bh.

Quoted 7 lines
Quoted 2 lines

The queue lock is still held at that point, so the driver hasn't had a chance to process the request yet.

Interesting. This is not a theoretical problem though. I've got traces of the oops showing the bio getting freed before the bio_sync(bio) test. When you say driver here what level do you mean? scsi_request_fn at least drops the queue lock.

But it must be holding the lock to retrieve the request in question, so there should be no opening for a completion race there.

Quoted 2 lines

What if it's merged instead of added directly? That could also get to the same place.

Same deal - if the request is already seen by the driver, merging is not allowed. If not, then the same rules apply.

Quoted 2 lines

The end_io callback _is_ getting called before __make_request does its "if(bio_sync(bio))" test.

Sounds strange, it sounds like a driver issue. If you have time, please do poke some more at this. I'll try to be responsive, but I'm busy with other things atm. Are you using a vanilla kernel?

Signature · 1 line
Jens Axboe