bio refcount problem
Hi,
I think there are some potential issues with the reference
counting of bios as used in 2.6.10. The __make_request function which is the default block device routine accesses the bio structure after issuing the call to add_request. This means that the bio could have completed before __make_request uses it.
The submit_bh path takes an extra reference with an explicit bio_get/put pair around the submit_bio, but many other users of submit_bio do not. Given that most of the end_io routines remove a reference and hence could free the bio this can lead at the least to __make_request mis-reading the sync flag. In more extreme cases it can cause an oops when run with CONFIG_DEBUG_PAGEALLOC.
The question is what is the preferred fix? I think it may be to simply have submit_bio take its own reference (and remove the extra one from submit_bh).
Alternatively __make_request could be adjusted so that it does not access the bio after calling add_request. All it is doing is checking the bi_rw field for the sync bit.
Or make all users of submit_bio take and release and extra reference like submit_bh.
Thoughts?
Cheers,
Phil
Signature · 3 lines
Philip R. Auld, Ph.D. Egenera, Inc. Software Architect 165 Forest St. (508) 858-2628 Marlboro, MA 01752
Quoted 18 lines · Philip R Auld
Hi,
I think there are some potential issues with the reference
counting of bios as used in 2.6.10. The__make_requestfunction which is the default block device routine accesses the bio structure after issuing the call toadd_request. This means that the bio could have completed before__make_requestuses it.The
submit_bhpath takes an extra reference with an explicit bio_get/put pair around thesubmit_bio, but many other users ofsubmit_biodo not. Given that most of theend_ioroutines remove a reference and hence could free the bio this can lead at the least to__make_requestmis-reading the sync flag. In more extreme cases it can cause an oops when run withCONFIG_DEBUG_PAGEALLOC.The question is what is the preferred fix? I think it may be to simply have
submit_biotake its own reference (and remove the extra one fromsubmit_bh).
Quoted 6 lines
Alternatively
__make_requestcould be adjusted so that it does not access the bio after callingadd_request. All it is doing is checking thebi_rwfield for the sync bit.Or make all users of
submit_biotake and release and extra reference likesubmit_bh.
The queue lock is still held at that point, so the driver hasn't had a chance to process the request yet.
Signature · 1 line
Jens Axboe
Hi,
Quoted 1 line · Rumor has it that on Fri, Feb 18, 2005 at 02:59:32PM +0100 Jens Axboe said:
On Fri, Feb 18 2005, Philip R Auld wrote:
...
Quoted 5 lines
Quoted 2 lines
Or make all users of
submit_biotake and release and extra reference likesubmit_bh.The queue lock is still held at that point, so the driver hasn't had a chance to process the request yet.
Interesting. This is not a theoretical problem though. I've got traces of the oops showing the bio getting freed before the bio_sync(bio) test. When you say driver here what level do you mean? scsi_request_fn at least drops the queue lock.
What if it's merged instead of added directly? That could also get to the same place.
The end_io callback _is_ getting called before __make_request does its "if(bio_sync(bio))" test.
Cheers,
Phil
Quoted 3 lines
Signature · 2 lines
-- Jens Axboe
Signature · 3 lines
Philip R. Auld, Ph.D. Egenera, Inc. Software Architect 165 Forest St. (508) 858-2628 Marlboro, MA 01752
Quoted 9 lines · Philip R Auld
Hi,
Quoted 1 line · Rumor has it that on Fri, Feb 18, 2005 at 02:59:32PM +0100 Jens Axboe said:
On Fri, Feb 18 2005, Philip R Auld wrote:
...
Quoted 3 lines
Quoted 2 lines
Or make all users of
submit_biotake and release and extra reference likesubmit_bh.
Quoted 7 lines
Quoted 2 lines
The queue lock is still held at that point, so the driver hasn't had a chance to process the request yet.
Interesting. This is not a theoretical problem though. I've got traces of the oops showing the bio getting freed before the
bio_sync(bio)test. When you say driver here what level do you mean?scsi_request_fnat least drops the queue lock.
But it must be holding the lock to retrieve the request in question, so there should be no opening for a completion race there.
Quoted 2 lines
What if it's merged instead of added directly? That could also get to the same place.
Same deal - if the request is already seen by the driver, merging is not allowed. If not, then the same rules apply.
Quoted 2 lines
The
end_iocallback _is_ getting called before__make_requestdoes its "if(bio_sync(bio))" test.
Sounds strange, it sounds like a driver issue. If you have time, please do poke some more at this. I'll try to be responsive, but I'm busy with other things atm. Are you using a vanilla kernel?
Signature · 1 line
Jens Axboe