Skip to contentSkip to conversation

blkcg: add blk-iotrack

RFC PATCH In review Weiping Zhang 1 participant 1 message Mar 27, 2020

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

RFC PATCH v2 3/3 blkcg: add blk-iotrack

blk-iotrack was designed to track various io statistic of block cgroup, it is based on rq_qos framework. It only tracks io and does not do any throttlling.

Compare to blk-iolatency, it provides 8 configurable latency buckets, /sys/fs/cgroup/io.iotrack.lat_thresh, blk-iotrack will account the number of IOs whose latency less than corresponding threshold. In this way we can get the cgroup's latency distribution. The default latency bucket is 50us, 100us, 200us, 400us, 1ms, 2ms, 4ms, 8ms.

Compare to io.stat.{rbytes,wbytes,rios,wios,dbytes,dios}, it account IOs when IO completed, instead of submited. If IO was throttled by io scheduler or other throttle policy, then there is a gap, these IOs have not been completed yet.

The previous patch has record the timestamp for each bio, when it was issued to the disk driver. Then we can get the disk latency in rq_qos_done_bio, this is also be called D2C time. In rq_qos_done_bio, blk-iotrack also record total latency(now - bio_issue_time), actually it can be treated as the Q2C time. In this way, we can get the percentile %d2c=D2C/Q2C for each cgroup. It's very useful for detect the main latency is from disk or software e.g. io scheduler or other block cgroup throttle policy.

The user space tool, wihch called iotrack, used to collect these basic io statistics and then generate more valuable metrics at cgroup level. From iotrack, you can get a cgroup's percentile for io, bytes, total_time and disk_time of the whole disk. It can easily to evaluate the real weight of the weight based policy(bfq, blk-iocost). For more details, please visit: https://github.com/dublio/iotrack.

Change-Id I17b12b309709eb3eca3b3ff75a1f636981c70ce5
---
 block/Kconfig       |   6 +
 block/Makefile      |   1 +
 block/blk-cgroup.c  |   4 +
 block/blk-iotrack.c | 436 ++++++++++++++++++++++++++++++++++++++++++++
 block/blk-rq-qos.h  |   3 +
 block/blk.h         |   7 +
 6 files changed, 457 insertions(+)
 create mode 100644 block/blk-iotrack.c
block/Kconfig +6 -0
index 3bc76bb113a0..d3073e4b048f 100644
@@ -132,6 +132,12 @@ config BLK_WBT
 	dynamically on an algorithm loosely based on CoDel, factoring in
 	the realtime performance of the disk.
 
+config BLK_CGROUP_IOTRACK
+       bool "Enable support for track io latency bucket"
+       depends on BLK_CGROUP=y
+       ---help---
+       count the io latency in several bucket.
+
 config BLK_CGROUP_IOLATENCY
 	bool "Enable support for latency based cgroup IO protection"
 	depends on BLK_CGROUP=y
block/Makefile +1 -0
index 1a43750f4b01..b0fc4d6f3cda 100644
@@ -18,6 +18,7 @@ obj-$(CONFIG_BLK_DEV_BSGLIB)	+= bsg-lib.o
 obj-$(CONFIG_BLK_CGROUP)	+= blk-cgroup.o
 obj-$(CONFIG_BLK_CGROUP_RWSTAT)	+= blk-cgroup-rwstat.o
 obj-$(CONFIG_BLK_DEV_THROTTLING)	+= blk-throttle.o
+obj-$(CONFIG_BLK_CGROUP_IOTRACK)	+= blk-iotrack.o
 obj-$(CONFIG_BLK_CGROUP_IOLATENCY)	+= blk-iolatency.o
 obj-$(CONFIG_BLK_CGROUP_IOCOST)	+= blk-iocost.o
 obj-$(CONFIG_MQ_IOSCHED_DEADLINE)	+= mq-deadline.o
block/blk-cgroup.c +4 -0
index a229b94d5390..85825f663a53 100644
@@ -1045,6 +1045,10 @@ int blkcg_init_queue(struct request_queue *q)
 	if (ret)
 		goto err_destroy_all;
 
+	ret = blk_iotrack_init(q);
+	if (ret)
+		goto err_destroy_all;
+
 	ret = blk_throtl_init(q);
 	if (ret)
 		goto err_destroy_all;
block/blk-iotrack.c new file +436 -0

436 lines not on this page · raw message

block/blk-rq-qos.h +3 -0
index 2bc43e94f4c4..3066d3afe77d 100644
@@ -16,6 +16,7 @@ enum rq_qos_id {
 	RQ_QOS_WBT,
 	RQ_QOS_LATENCY,
 	RQ_QOS_COST,
+	RQ_QOS_IOTRACK,
 };
 
 struct rq_wait {
@@ -87,6 +88,8 @@ static inline const char *rq_qos_id_to_name(enum rq_qos_id id)
 		return "latency";
 	case RQ_QOS_COST:
 		return "cost";
+	case RQ_QOS_IOTRACK:
+		return "iotrack";
 	}
 	return "unknown";
 }
block/blk.h +7 -0
index 670337b7cfa0..3520b5ab7971 100644
@@ -346,6 +346,13 @@ extern int blk_iolatency_init(struct request_queue *q);
 static inline int blk_iolatency_init(struct request_queue *q) { return 0; }
 #endif
 
+#ifdef CONFIG_BLK_CGROUP_IOTRACK
+extern int blk_iotrack_init(struct request_queue *q);
+#else
+static inline int blk_iotrack_init(struct request_queue *q) { return 0; }
+#endif
+
+
 struct bio *blk_next_bio(struct bio *bio, unsigned int nr_pages, gfp_t gfp);
 
 #ifdef CONFIG_BLK_DEV_ZONED
Signature · 1 line
2.18.1