fix bio_add_page for non trivial merge_bvec_fn case
PATCH 1/1 fix bio_add_page for non trivial merge_bvec_fn case
We have to properly decrease all related bio's counters, especially bi_size in order to merge_bvec_fn return right result. Usually this result in false merge rejects for two absolutely valid bio_vecs. This may cause significant performance penalty for example Itanium: page_size == 16k, fs_block_size == 1k and block device is raid with small chunk_size.
--- fs/bio.c | 16 ++++++++++++---- 1 files changed, 12 insertions(+), 4 deletions(-)
fs/bio.c +11 -4
@@ -332,14 +332,21 @@ static int __bio_add_page(struct request_queue *q, struct bio *bio, struct pageif (page == prev->bv_page && offset == prev->bv_offset + prev->bv_len) { +/* Temprory detacth last bio_vec. */ + bio->bi_size -= prev->bv_len; + bio->bi_vcnt--; + bio->bi_phys_segments--; + bio->bi_hw_segments--; + prev->bv_len += len;if (q->merge_bvec_fn && q->merge_bvec_fn (q, bio, prev) < len) { prev->bv_len -= len; -return 0 ; + len =0 ; } - goto done; + bio->bi_size += prev->bv_len; + goto out_add_bvec; } }
@@ -394,11 +401,12 @@ static int __bio_add_page(struct request_queue *q, struct bio *bio, struct page BIOVEC_VIRT_MERGEABLE(bvec-1, bvec))) bio->bi_flags &= ~(1 <<BIO_SEG_VALID ); + bio->bi_size += len; +out_add_bvec: bio->bi_vcnt++; bio->bi_phys_segments++; bio->bi_hw_segments++; - done: - bio->bi_size += len;return len; }
Signature · 1 line
1.5.4.rc4
Quoted 6 lines · On Sun, 08 Jun 2008 19:45:08 +0400 Dmitri Monakhov <[email protected]> wrote:
We have to properly decrease all related bio's counters, especially
bi_sizein order tomerge_bvec_fnreturn right result. Usually this result in false merge rejects for two absolutely validbio_vecs. This may cause significant performance penalty for example Itanium:page_size== 16k,fs_block_size== 1k and block device is raid with smallchunk_size.
Please cc Jens on BIO changes.
fs/bio.c static int __bio_add_page(struct request_queue *q, struct bio *bio, struct page · line 335
Quoted 13 lines
--- fs/bio.c | 16 ++++++++++++---- 1 files changed, 12 insertions(+), 4 deletions(-)fs/bio.c +1
-0@@ -332,14 +332,21 @@ static int __bio_add_page(struct request_queue *q, struct bio *bio, struct pageif (page == prev->bv_page && offset == prev->bv_offset + prev->bv_len) { +/* Temprory detacth last bio_vec. */
whoa, drunken speling.
Quoted 22 lines
fs/bio.c +8
-2+ bio->bi_size -= prev->bv_len; + bio->bi_vcnt--; + bio->bi_phys_segments--; + bio->bi_hw_segments--; + prev->bv_len += len;if (q->merge_bvec_fn && q->merge_bvec_fn (q, bio, prev) < len) { prev->bv_len -= len; -return 0 ; + len =0 ; } - goto done; + bio->bi_size += prev->bv_len; + goto out_add_bvec; } }@@ -394,11 +401,12 @@ static int __bio_add_page(struct request_queue *q, struct bio *bio, struct page BIOVEC_VIRT_MERGEABLE(bvec-1, bvec))) bio->bi_flags &= ~(1 <<BIO_SEG_VALID );
fs/bio.c static int __bio_add_page(struct request_queue *q, struct bio *bio, struct page · line 410
Quoted 10 lines
fs/bio.c +2
-2+ bio->bi_size += len; +out_add_bvec: bio->bi_vcnt++; bio->bi_phys_segments++; bio->bi_hw_segments++; - done: - bio->bi_size += len;return len; }
For some reason patch(1) claims this hunk was corrupted. I typed it in by hand.
Quoted 28 lines · Andrew Morton
Quoted 6 lines · On Sun, 08 Jun 2008 19:45:08 +0400 Dmitri Monakhov <[email protected]> wrote:
We have to properly decrease all related bio's counters, especially
bi_sizein order tomerge_bvec_fnreturn right result. Usually this result in false merge rejects for two absolutely validbio_vecs. This may cause significant performance penalty for example Itanium:page_size== 16k,fs_block_size== 1k and block device is raid with smallchunk_size.Please cc Jens on BIO changes.
fs/bio.c static int __bio_add_page(struct request_queue *q, struct bio *bio, struct page · line 335
Quoted 13 lines
--- fs/bio.c | 16 ++++++++++++---- 1 files changed, 12 insertions(+), 4 deletions(-)fs/bio.c +1
-0@@ -332,14 +332,21 @@ static int __bio_add_page(struct request_queue *q, struct bio *bio, struct pageif (page == prev->bv_page && offset == prev->bv_offset + prev->bv_len) { +/* Temprory detacth last bio_vec. */ whoa, drunken speling.
Quoted 5 lines
fs/bio.c static int __bio_add_page(struct request_queue *q, struct bio *bio, struct page · line 340
Quoted 5 lines
fs/bio.c +5
-0+ bio->bi_size -= prev->bv_len; + bio->bi_vcnt--; + bio->bi_phys_segments--; + bio->bi_hw_segments--; +
This logic isn't quite right, the rules for what constitutes a new hw or phys segment is not a 1:1 mapping with number of pages in the bio. How about just dropping the segment decrement? The merge_bvec fn should not care, and we'll retry and coalesce segment count if we get to the limit anyway.
Signature · 1 line
Jens Axboe