md: improve io stats accounting
PATCH v3 1/1 md: improve io stats accounting
Use generic io accounting functions to manage io stats. There was an attempt to do this earlier in commit 18c0b223cf99 ("md: use generic io stats accounting functions to simplify io stat accounting"), but it did not include a call to generic_end_io_acct() and caused issues with tracking in-flight IOs, so it was later removed in commit 74672d069b29 ("md: fix md io stats accounting broken").
This patch attempts to fix this by using both bio_start_io_acct() and bio_end_io_acct(). To make it possible, a struct md_io is allocated for every new md bio, which includes the io start_time. A new mempool is introduced for this purpose. We override bio->bi_end_io with our own callback and call bio_start_io_acct() before passing the bio to md_handle_request(). When it completes, we call bio_end_io_acct() and the original bi_end_io callback.
This adds correct statistics about in-flight IOs and IO processing time, interpreted e.g. in iostat as await, svctm, aqu-sz and %util.
It also fixes a situation where too many IOs where reported if a bio was re-submitted to the mddev, because io accounting is now performed only on newly arriving bios.
---
v3:
- Use bio_start_io_acct() return value for md_io->start_time (thanks Guoqing!)
v2:
- Just override the bi_end_io without having to clone the original bio.
- Rebased onto latest md-next.
drivers/md/md.c | 55 ++++++++++++++++++++++++++++++++++++++----------- drivers/md/md.h | 1 + 2 files changed, 44 insertions(+), 12 deletions(-)
drivers/md/md.c +43 -12
@@ -463,12 +463,33 @@ void md_handle_request(struct mddev *mddev, struct bio *bio) }EXPORT_SYMBOL (md_handle_request); +struct md_io { +struct mddev *mddev; +bio_end_io_t *orig_bi_end_io; +void *orig_bi_private; +unsigned long start_time; +}; + +static void md_end_io (struct bio *bio) +{ +struct md_io *md_io = bio->bi_private; +struct mddev *mddev = md_io->mddev; + +bio_end_io_acct (bio, md_io->start_time); + + bio->bi_end_io = md_io->orig_bi_end_io; + bio->bi_private = md_io->orig_bi_private; + +mempool_free (md_io, &mddev->md_io_pool); + +if (bio->bi_end_io) + bio->bi_end_io (bio); +} +static blk_qc_t md_submit_bio (struct bio *bio) {const int rw =bio_data_dir (bio); -const int sgrp =op_stat_group (bio_op (bio));struct mddev *mddev = bio->bi_disk->private_data; -unsigned int sectors;if (unlikely (test_bit (MD_BROKEN , &mddev->flags)) && (rw ==WRITE )) {bio_io_error (bio);
@@ -488,21 +509,25 @@ static blk_qc_t md_submit_bio(struct bio *bio)return BLK_QC_T_NONE ; } -/* -* save the sectors now since our bio can -* go away inside make_request -*/ - sectors =bio_sectors (bio); +if (bio->bi_end_io != md_end_io) { +struct md_io *md_io; + + md_io =mempool_alloc (&mddev->md_io_pool,GFP_NOIO ); + md_io->mddev = mddev; + md_io->orig_bi_end_io = bio->bi_end_io; + md_io->orig_bi_private = bio->bi_private; + + bio->bi_end_io = md_end_io; + bio->bi_private = md_io; + + md_io->start_time =bio_start_io_acct (bio); + } +/* bio could be mergeable after passing to underlayer */ bio->bi_opf &= ~REQ_NOMERGE ;md_handle_request (mddev, bio); -part_stat_lock (); -part_stat_inc (&mddev->gendisk->part0, ios[sgrp]); -part_stat_add (&mddev->gendisk->part0, sectors[sgrp], sectors); -part_stat_unlock (); -return BLK_QC_T_NONE ; }
@@ -5545,6 +5570,7 @@ static void md_free(struct kobject *ko)bioset_exit (&mddev->bio_set);bioset_exit (&mddev->sync_set); +mempool_exit (&mddev->md_io_pool);kfree (mddev); }
@@ -5640,6 +5666,11 @@ static int md_alloc(dev_t dev, char *name)*/ mddev->hold_active =UNTIL_STOP ; + error =mempool_init_kmalloc_pool (&mddev->md_io_pool,BIO_POOL_SIZE , +sizeof (struct md_io )); +if (error) + goto abort; + error = -ENOMEM ; mddev->queue =blk_alloc_queue (NUMA_NO_NODE );if (!mddev->queue)
drivers/md/md.h +1 -0
@@ -481,6 +481,7 @@ struct mddev {struct bio_set sync_set;/* for sync operations like * metadata and bitmap writes */ +mempool_t md_io_pool;/* Generic flush handling. * The last to finish preflush schedules a worker to submit
Signature · 1 line
2.26.0
Song Liu
maintainerQuoted 16 lines · Artur Paszkiewicz
Use generic io accounting functions to manage io stats. There was an attempt to do this earlier in commit
18c0b223cf99("md: use generic io stats accounting functions to simplify io stat accounting"), but it did not include a call togeneric_end_io_acct()and caused issues with tracking in-flight IOs, so it was later removed in commit74672d069b29("md: fix md io stats accounting broken").This patch attempts to fix this by using both
bio_start_io_acct()andbio_end_io_acct(). To make it possible, a structmd_iois allocated for every new md bio, which includes the iostart_time. A new mempool is introduced for this purpose. We overridebio->bi_end_iowith our own callback and callbio_start_io_acct()before passing the bio tomd_handle_request(). When it completes, we callbio_end_io_acct()and the originalbi_end_iocallback.
Quoted 8 lines
This adds correct statistics about in-flight IOs and IO processing time, interpreted e.g. in iostat as await, svctm, aqu-sz and %util.
It also fixes a situation where too many IOs where reported if a bio was re-submitted to the mddev, because io accounting is now performed only on newly arriving bios.
Signed-off-by Artur Paszkiewicz <[email protected]>
Thanks Artur and Guoqing!
I run quick test with this. Seems it only adds proper statistics to raid5 array, but
not to raid0 array. Is this expected?
Song
[...]
Quoted 24 lines · Song Liu
Quoted 22 lines · Artur Paszkiewicz
Use generic io accounting functions to manage io stats. There was an attempt to do this earlier in commit
18c0b223cf99("md: use generic io stats accounting functions to simplify io stat accounting"), but it did not include a call togeneric_end_io_acct()and caused issues with tracking in-flight IOs, so it was later removed in commit74672d069b29("md: fix md io stats accounting broken").This patch attempts to fix this by using both
bio_start_io_acct()andbio_end_io_acct(). To make it possible, a structmd_iois allocated for every new md bio, which includes the iostart_time. A new mempool is introduced for this purpose. We overridebio->bi_end_iowith our own callback and callbio_start_io_acct()before passing the bio tomd_handle_request(). When it completes, we callbio_end_io_acct()and the originalbi_end_iocallback.This adds correct statistics about in-flight IOs and IO processing time, interpreted e.g. in iostat as await, svctm, aqu-sz and %util.
It also fixes a situation where too many IOs where reported if a bio was re-submitted to the mddev, because io accounting is now performed only on newly arriving bios.
Quoted 6 lines
Quoted 1 line
Signed-off-by Artur Paszkiewicz <[email protected]>Thanks Artur and Guoqing!
I run quick test with this. Seems it only adds proper statistics to raid5 array, but
not to raid0 array. Is this expected?
Because bio_endio is not called, and it is same for linear and faulty. I think we have to clone bio for them ..., then it is better to do the job in the personality layer.
1. For md-multipath, raid1 and raid10, track start time by change the multipath_bh, r1bio and r10bio.
2. For raid5, override the bi_end_io like this.
3. Then other personalities have to clone bio ...
Thanks,
Guoqing
Quoted 4 lines · Guoqing Jiang
Quoted 3 lines · Song Liu
I run quick test with this. Seems it only adds proper statistics to raid5 array, but
not to raid0 array. Is this expected?
Oh, sorry about that. Of course it should work.
Quoted 3 lines
Because
bio_endiois not called, and it is same for linear and faulty. I think we have to clone bio for them ..., then it is better to do the job in the personality layer.
It's not that bad actually. The issue is simply because those personalities change the original bio's bi_disk and then bio_end_io_acct() uses a different gendisk. So I think we can either use disk_{start,end}_io_acct() instead of bio_{start,end}_io_acct(), or change bio->bi_disk back to mddev->gendisk before calling bio_end_io_acct(). I prefer the first option. What do you think?
Thanks,
Artur
Quoted 10 lines · Artur Paszkiewicz
Quoted 4 lines · Guoqing Jiang
Quoted 3 lines · Song Liu
I run quick test with this. Seems it only adds proper statistics to raid5 array, but
not to raid0 array. Is this expected?Oh, sorry about that. Of course it should work.
Quoted 3 lines
Because
bio_endiois not called, and it is same for linear and faulty. I think we have to clone bio for them ..., then it is better to do the job in the personality layer.
Quoted 6 lines
It's not that bad actually. The issue is simply because those personalities change the original bio's
bi_diskand thenbio_end_io_acct()uses a different gendisk. So I think we can either use disk_{start,end}_io_acct() instead of bio_{start,end}_io_acct(), or changebio->bi_diskback tomddev->gendiskbefore callingbio_end_io_acct(). I prefer the first option. What do you think?
I am fine with either of them, though currently the only caller of disk_start_io_acct is bio_start_io_acct :-)
Thanks,
Guoqing