block/dm: fix bio-based DM IO accounting

PATCH Superseded Mike Snitzer 2 participants 7 messages Jan 28, 2022

Subsystems (2)

Block layer Device mapper

lore

Share & tools

lore

Mike Snitzer

author

2022-01-28 04:17 UTC #0 lore raw

v3 cover block/dm: fix bio-based DM IO accounting

Hi Jens,

Just over 3 years ago, with commit a1e1cb72d9649 ("dm: fix redundant IO accounting for bios that need splitting") I focused too narrowly on fixing the reported potential for redundant accounting for IO totals. Which, at least mentally for me, papered over how inaccurate all other bio-based DM's IO accounting is for bios that get split.

This set fixes things up properly by allowing DM to start IO accounting _after_ IO is submitted and a split may have occurred. The proper start_time is still established (prior to submission), it is passed in to a new bio_start_io_acct_time(). This eliminates the need for any DM hack to rewind block core's accounting that was started before any potential bio split.

All said: If you'd provide your Acked-by(s) I'm happy to send this set to Linus for v5.17-rc (and shepherd the changes into stable@ kernels).

Or you're welcome to pickup this set to send along (I'd obviously still do any stable@ backports). NOTE: the 3rd patch references the linux-dm.git commit id for the 1st patch.. so that'll require tweaking no matter who sends the changes to Linus.

Please advise, thanks.
Mike

v3: fix patch 3 to call bio_start_io_acct_time
v2: made block changes suggested by Christoph

Mike Snitzer (3):
block: add bio_start_io_acct_time() to control start_time
dm: revert partial fix for redundant bio-based IO accounting
dm: properly fix redundant bio-based IO accounting

 block/blk-core.c       | 25 +++++++++++++++++++------
 drivers/md/dm.c        | 20 +++-----------------
 include/linux/blkdev.h |  1 +
 3 files changed, 23 insertions(+), 23 deletions(-)

Signature · 1 line

2.15.0

Mike Snitzer

author

2022-01-28 04:17 UTC #1 lore raw

PATCH v3 1/3 block: add bio_start_io_acct_time() to control start_time

bio_start_io_acct_time() interface is like bio_start_io_acct() that allows start_time to be passed in. This gives drivers the ability to defer starting accounting until after IO is issued (but possibily not entirely due to bio splitting).

Signed-off-by Mike Snitzer <[email protected]>

---
 block/blk-core.c       | 25 +++++++++++++++++++------
 include/linux/blkdev.h |  1 +
 2 files changed, 20 insertions(+), 6 deletions(-)

block/blk-core.c +19 -6

index 97f8bc8d3a79..d93e3bb9a769 100644
@@ -1061,20 +1061,32 @@ void update_io_ticks(struct block_device *part, unsigned long now, bool end)
 }
 
 static unsigned long __part_start_io_acct(struct block_device *part,
-					  unsigned int sectors, unsigned int op)
+					  unsigned int sectors, unsigned int op,
+					  unsigned long start_time)
 {
 	const int sgrp = op_stat_group(op);
-	unsigned long now = READ_ONCE(jiffies);
 
 	part_stat_lock();
-	update_io_ticks(part, now, false);
+	update_io_ticks(part, start_time, false);
 	part_stat_inc(part, ios[sgrp]);
 	part_stat_add(part, sectors[sgrp], sectors);
 	part_stat_local_inc(part, in_flight[op_is_write(op)]);
 	part_stat_unlock();
 
-	return now;
+	return start_time;
+}
+
+/**
+ * bio_start_io_acct_time - start I/O accounting for bio based drivers
+ * @bio:	bio to start account for
+ * @start_time:	start time that should be passed back to bio_end_io_acct().
+ */
+void bio_start_io_acct_time(struct bio *bio, unsigned long start_time)
+{
+	__part_start_io_acct(bio->bi_bdev, bio_sectors(bio),
+			     bio_op(bio), start_time);
 }
+EXPORT_SYMBOL_GPL(bio_start_io_acct_time);
 
 /**
  * bio_start_io_acct - start I/O accounting for bio based drivers

@@ -1084,14 +1096,15 @@ static unsigned long __part_start_io_acct(struct block_device *part,
  */
 unsigned long bio_start_io_acct(struct bio *bio)
 {
-	return __part_start_io_acct(bio->bi_bdev, bio_sectors(bio), bio_op(bio));
+	return __part_start_io_acct(bio->bi_bdev, bio_sectors(bio),
+				    bio_op(bio), jiffies);
 }
 EXPORT_SYMBOL_GPL(bio_start_io_acct);
 
 unsigned long disk_start_io_acct(struct gendisk *disk, unsigned int sectors,
 				 unsigned int op)
 {
-	return __part_start_io_acct(disk->part0, sectors, op);
+	return __part_start_io_acct(disk->part0, sectors, op, jiffies);
 }
 EXPORT_SYMBOL(disk_start_io_acct);

include/linux/blkdev.h +1 -0

index 9c95df26fc26..f35aea98bc35 100644
@@ -1258,6 +1258,7 @@ unsigned long disk_start_io_acct(struct gendisk *disk, unsigned int sectors,
 void disk_end_io_acct(struct gendisk *disk, unsigned int op,
 		unsigned long start_time);
 
+void bio_start_io_acct_time(struct bio *bio, unsigned long start_time);
 unsigned long bio_start_io_acct(struct bio *bio);
 void bio_end_io_acct_remapped(struct bio *bio, unsigned long start_time,
 		struct block_device *orig_bdev);

Signature · 1 line

2.15.0

Christoph Hellwig

Reviewed-by on v3 1/3

2022-01-28 06:13 UTC #2 lore raw

Re: [PATCH v3 1/3] block: add bio_start_io_acct_time() to control start_time

Quoted 8 lines · Mike Snitzer

block/blk-core.c +3 -0

+	__part_start_io_acct(bio->bi_bdev, bio_sectors(bio),
+			     bio_op(bio), start_time);
 }
+EXPORT_SYMBOL_GPL(bio_start_io_acct_time);
 
 /**
  * bio_start_io_acct - start I/O accounting for bio based drivers

block/blk-core.c static unsigned long __part_start_io_acct(struct block_device *part, · line 1100

Quoted 6 lines

block/blk-core.c +2 -1

@@ -1084,14 +1096,15 @@ static unsigned long __part_start_io_acct(struct block_device *part,
  */
 unsigned long bio_start_io_acct(struct bio *bio)
 {
-	return __part_start_io_acct(bio->bi_bdev, bio_sectors(bio), bio_op(bio));
+	return __part_start_io_acct(bio->bi_bdev, bio_sectors(bio),
+				    bio_op(bio), jiffies);

Is droppingthe READ_ONCE safe here? This is a honest question as these helpers still confuse me.

The rest looks good:

Reviewed-by Christoph Hellwig <[email protected]>

Mike Snitzer

author

2022-01-28 15:16 UTC #3 lore raw

Re: [PATCH v3 1/3] block: add bio_start_io_acct_time() to control start_time

Quoted 6 lines · Christoph Hellwig

Quoted 5 lines · Mike Snitzer

block/blk-core.c +3 -0

+	__part_start_io_acct(bio->bi_bdev, bio_sectors(bio),
+			     bio_op(bio), start_time);
 }
+EXPORT_SYMBOL_GPL(bio_start_io_acct_time);

Quoted 12 lines

block/blk-core.c static unsigned long __part_start_io_acct(struct block_device *part, · line 1100
Quoted 9 lines
/**
* bio_start_io_acct - start I/O accounting for bio based drivers
block/blk-core.c +2 -1
@@ -1084,14 +1096,15 @@ static unsigned long __part_start_io_acct(struct block_device *part,
  */
 unsigned long bio_start_io_acct(struct bio *bio)
 {
-	return __part_start_io_acct(bio->bi_bdev, bio_sectors(bio), bio_op(bio));
+	return __part_start_io_acct(bio->bi_bdev, bio_sectors(bio),
+				    bio_op(bio), jiffies);
Is droppingthe READ_ONCE safe here? This is a honest question as these helpers still confuse me.

I'm not sure either, commit 956d510ee78ca ("block: add disk/bio-based accounting helpers") doesn't offer any insight on the need.

Very little kernel code uses READ_ONCE(jiffies).

git diff 24d69293d9a561645e0b4d78c2fb179827e35f53^..e722fff238bbfe6308d7778a8c2163c181bf998a shows that at the time the outgoing generic_{start,end}_io_acct didn't
use READ_ONCE (nor did any of the drivers that were updated to use the
new helpers).

ACCESS_ONCE() was used prior to READ_ONCE() -- see commit cbbce82209490 and then 316c1608d15c7.

Anyway, looks to be cargo cult at this point right?
(if so, implies the use of READ_ONCE() in patch 3 isn't needed either)

Quoted 3 lines

The rest looks good:
Reviewed-by Christoph Hellwig <[email protected]>

Thanks.

Mike Snitzer

author

2022-01-28 04:18 UTC #4 lore raw

PATCH v3 2/3 dm: revert partial fix for redundant bio-based IO accounting

In reply to Mike Snitzer

Reverts a1e1cb72d9649 ("dm: fix redundant IO accounting for bios that need splitting") because it was too narrow in scope (only addressed redundant 'sectors[]' accounting and not ios, nsecs[], etc).

Cc [email protected]

Signed-off-by Mike Snitzer <[email protected]>

---
 drivers/md/dm.c | 15 ---------------
 1 file changed, 15 deletions(-)

drivers/md/dm.c +0 ~~-15~~

index c0ae8087c602..9849114b3c08 100644
@@ -1442,9 +1442,6 @@ static void init_clone_info(struct clone_info *ci, struct mapped_device *md,
 	ci->sector = bio->bi_iter.bi_sector;
 }
 
-#define __dm_part_stat_sub(part, field, subnd)	\
-	(part_stat_get(part, field) -= (subnd))
-
 /*
  * Entry point to split a bio into clones and submit them to the targets.
  */

@@ -1480,18 +1477,6 @@ static void __split_and_process_bio(struct mapped_device *md,
 						  GFP_NOIO, &md->queue->bio_split);
 			ci.io->orig_bio = b;
 
-			/*
-			 * Adjust IO stats for each split, otherwise upon queue
-			 * reentry there will be redundant IO accounting.
-			 * NOTE: this is a stop-gap fix, a proper fix involves
-			 * significant refactoring of DM core's bio splitting
-			 * (by eliminating DM's splitting and just using bio_split)
-			 */
-			part_stat_lock();
-			__dm_part_stat_sub(dm_disk(md)->part0,
-					   sectors[op_stat_group(bio_op(bio))], ci.sector_count);
-			part_stat_unlock();
-
 			bio_chain(b, bio);
 			trace_block_split(b, bio->bi_iter.bi_sector);
 			submit_bio_noacct(bio);

Signature · 1 line

2.15.0

Mike Snitzer

author

2022-01-28 04:18 UTC #5 lore raw

PATCH v3 3/3 dm: properly fix redundant bio-based IO accounting

In reply to Mike Snitzer

Record the start_time for a bio but defer the starting block core's IO accounting until after IO is submitted using bio_start_io_acct_time().

This approach avoids the need to mess around with any of the individual IO stats in response to a bio_split() that follows bio submission.

Reported-by Bud Brown <[email protected]>

Cc [email protected]

Depends-on: f9893e1da2e3 ("block: add bio_start_io_acct_time() to control start_time")

Signed-off-by Mike Snitzer <[email protected]>

---
 drivers/md/dm.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

drivers/md/dm.c +3 -2

index 9849114b3c08..144436301a57 100644
@@ -489,7 +489,7 @@ static void start_io_acct(struct dm_io *io)
 	struct mapped_device *md = io->md;
 	struct bio *bio = io->orig_bio;
 
-	io->start_time = bio_start_io_acct(bio);
+	bio_start_io_acct_time(bio, io->start_time);
 	if (unlikely(dm_stats_used(&md->stats)))
 		dm_stats_account_io(&md->stats, bio_data_dir(bio),
 				    bio->bi_iter.bi_sector, bio_sectors(bio),

@@ -535,7 +535,7 @@ static struct dm_io *alloc_io(struct mapped_device *md, struct bio *bio)
 	io->md = md;
 	spin_lock_init(&io->endio_lock);
 
-	start_io_acct(io);
+	io->start_time = READ_ONCE(jiffies);
 
 	return io;
 }

@@ -1482,6 +1482,7 @@ static void __split_and_process_bio(struct mapped_device *md,
 			submit_bio_noacct(bio);
 		}
 	}
+	start_io_acct(ci.io);
 
 	/* drop the extra reference count */
 	dm_io_dec_pending(ci.io, errno_to_blk_status(error));

Signature · 1 line

2.15.0

Christoph Hellwig

Reviewed-by on v3 3/3

2022-01-28 06:15 UTC #6 lore raw

Re: [PATCH v3 3/3] dm: properly fix redundant bio-based IO accounting

Quoted 6 lines · Mike Snitzer

Record the start_time for a bio but defer the starting block core's IO accounting until after IO is submitted using bio_start_io_acct_time().
This approach avoids the need to mess around with any of the individual IO stats in response to a bio_split() that follows bio submission.

Looks good:

Reviewed-by Christoph Hellwig <[email protected]>