Skip to contentSkip to conversation

block: fix the problem of io_ticks becoming smaller

PATCH In review brookxu 4 participants 5 messages Jul 5, 2021 – Jul 7, 2021 Block layer

lore
Share & tools
lore

brookxu

author
#0 lore raw

PATCH v3 1/1 block: fix the problem of io_ticks becoming smaller

From: Chunguang Xu <[email protected]>

On the IO submission path, blk_account_io_start() may interrupt the system interruption. When the interruption returns, the value of part->stamp may have been updated by other cores, so the time value collected before the interruption may be less than part-> stamp. So when this happens, we should do nothing to make io_ticks more accurate? For kernels less than 5.0, this may cause io_ticks to become smaller, which in turn may cause abnormal ioutil values.

v3: update the commit log
v2: sorry, fix compile error due to the missed ')'

---
 block/blk-core.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
block/blk-core.c +1 -1
index 514838c..bbf56ae 100644
@@ -1243,7 +1243,7 @@ static void update_io_ticks(struct block_device *part, unsigned long now,
 	unsigned long stamp;
 again:
 	stamp = READ_ONCE(part->bd_stamp);
-	if (unlikely(stamp != now)) {
+	if (unlikely(time_after(now, stamp))) {
 		if (likely(cmpxchg(&part->bd_stamp, stamp, now) == stamp))
 			__part_stat_add(part, io_ticks, end ? now - stamp : 1);
 	}
Signature · 1 line
1.8.3.1
Reviewed-by on v3 1/1
#1 lore raw
Quoted 10 lines · brookxu

From: Chunguang Xu <[email protected]>

On the IO submission path, blk_account_io_start() may interrupt the system interruption. When the interruption returns, the value of part->stamp may have been updated by other cores, so the time value collected before the interruption may be less than part-> stamp. So when this happens, we should do nothing to make io_ticks more accurate? For kernels less than 5.0, this may cause io_ticks to become smaller, which in turn may cause abnormal ioutil values.

Quoted 4 lines

v3: update the commit log
v2: sorry, fix compile error due to the missed ')'

The change looks good:

Although I still have trouble understanding the commit log, especially the last sentence.

author
#2 lore raw

Christoph Hellwig wrote on 2021/7/6 13:25:

Quoted 14 lines · brookxu
Quoted 13 lines

From: Chunguang Xu <[email protected]>

On the IO submission path, blk_account_io_start() may interrupt the system interruption. When the interruption returns, the value of part->stamp may have been updated by other cores, so the time value collected before the interruption may be less than part-> stamp. So when this happens, we should do nothing to make io_ticks more accurate? For kernels less than 5.0, this may cause io_ticks to become smaller, which in turn may cause abnormal ioutil values.

v3: update the commit log
v2: sorry, fix compile error due to the missed ')'

Quoted 8 lines
Quoted 1 line

The change looks good:

Although I still have trouble understanding the commit log, especially the last sentence.

Thanks for your time,5b18b5a73760 ("block: delete part_round_stats and switch to less precise counting") Merged in 5.0-rc1. Before that, we relied on in_flight to count the disk's io_ticks,as follows:

static void part_round_stats_single(struct request_queue *q,
struct hd_struct *part, unsigned long now, unsigned int inflight)

{
	if (inflight) {
		__part_stat_add(part, time_in_queue,
				inflight * (now - part->stamp));
		__part_stat_add(part, io_ticks, (now - part->stamp));
	}
	part->stamp = now;
}

The value of io_ticks should increase monotonically before the count wraps around. However, due to the reasons mentioned above, the interrupt on the IO submission path may cause the time obtained before the interrupt to be less than part->stamp after the interrupt returns, resulting in the value of io_ticks becoming smaller than the previous value. If the user task periodically samples /proc/diskstats and calculates ioutil, since io_ticks increases non-monotonously, the difference between adjacent times may be negative, which in turn makes ioutil abnormal.Fortunately, this problem can be easily circumvented.

#3 lore raw
In reply to brookxu
Quoted 9 lines · brookxu

From: Chunguang Xu <[email protected]>

On the IO submission path, blk_account_io_start() may interrupt the system interruption. When the interruption returns, the value of part->stamp may have been updated by other cores, so the time value collected before the interruption may be less than part-> stamp. So when this happens, we should do nothing to make io_ticks more accurate? For kernels less than 5.0, this may cause io_ticks to become smaller, which in turn may cause abnormal ioutil values.

Just be curious, is there any user visible difference with this change?

It can only be one issue if the stamp update crosses two jiffies, but this event should be very unlikely, right?

Thanks,
Ming

maintainer
#4 lore raw
In reply to brookxu
Quoted 10 lines · brookxu

From: Chunguang Xu <[email protected]>

On the IO submission path, blk_account_io_start() may interrupt the system interruption. When the interruption returns, the value of part->stamp may have been updated by other cores, so the time value collected before the interruption may be less than part-> stamp. So when this happens, we should do nothing to make io_ticks more accurate? For kernels less than 5.0, this may cause io_ticks to become smaller, which in turn may cause abnormal ioutil values.

Quoted 2 lines

v3: update the commit log
v2: sorry, fix compile error due to the missed ')'

The version log goes below the '---' in the email, not in the commit message. I fixed that up. Applied for 5.14, thanks.

Signature · 1 line
Jens Axboe