Skip to contentSkip to conversation

wrong in_flight diskstat in 2.6.16.1

DISCUSSION Martin Devera 2 participants 4 messages May 23, 2006 Block layer

lore
Share & tools
lore
#0 lore raw

Hello,

I see weird output from /sys/block/sd{a,b}/stat on our AMD64-X2 smp machine with HT1000 (Broadcom) SATA with 2 WD 250GB HDDs in MD raid1. AS scheduler was used, change to noop didn't change anything. It is vanilla 2.6.16.1 and here are absolute values in hex and one second differences below:

132CDF 56D62 61753C0 6966BC 165A8EB 5FF5B6C 3B32D6C0 1110594C FFCA89A4 FEE85878 49FF74E4 132CDF 56D62 61753C0 6966BC 165A8EF 5FF5B6C 3B32D6E0 111059D0 FFCA89A1 FEE85C60 79291244
0: 0
1: 0
2: 0
3: 0
4: 4
5: 0
6: 32
7: 132
8: -3
9: 1000
10: 791256416

As you can see in_flight is constantly negative and it is DECREASING slowly all the time. I can't find any reason for it :-\

Martin Devera

#1 lore raw
Quoted 24 lines · Martin Devera

Hello,

I see weird output from /sys/block/sd{a,b}/stat on our AMD64-X2 smp machine with HT1000 (Broadcom) SATA with 2 WD 250GB HDDs in MD raid1. AS scheduler was used, change to noop didn't change anything. It is vanilla 2.6.16.1 and here are absolute values in hex and one second differences below:

132CDF 56D62 61753C0 6966BC 165A8EB 5FF5B6C 3B32D6C0 1110594C FFCA89A4 FEE85878 49FF74E4
132CDF 56D62 61753C0 6966BC 165A8EF 5FF5B6C 3B32D6E0 111059D0 FFCA89A1 FEE85C60 79291244
0: 0
1: 0
2: 0
3: 0
4: 4
5: 0
6: 32
7: 132
8: -3
9: 1000
10: 791256416

Quoted 3 lines

As you can see in_flight is constantly negative and it is DECREASING slowly all the time.
I can't find any reason for it :-\

Are you using io barriers?

[PATCH] blk: fix gendisk->in_flight accounting during barrier sequence

While executing barrrier sequence, the bar_rq which carries actual write was accounted as normal IO on completion, while it wasn't on queueing. This caused gendisk->in_flight to be decremented by 1 after each barrier thus messed up statistics.

This patch makes bar_rq not accounted as normal IO. As the containing barrier request as a whole is accounted, part of it shouldn't be.

block/ll_rw_blk.c +6 -1
index eac48be..7eb36c5 100644
@@ -3452,7 +3452,12 @@ void end_that_request_last(struct reques
 	if (unlikely(laptop_mode) && blk_fs_request(req))
 		laptop_io_completion();
 
-	if (disk && blk_fs_request(req)) {
+	/*
+	 * Account IO completion.  bar_rq isn't accounted as a normal
+	 * IO on queueing nor completion.  Accounting the containing
+	 * request is enough.
+	 */
+	if (disk && blk_fs_request(req) && req != &req->q->bar_rq) {
 		unsigned long duration = jiffies - req->start_time;
 		const int rw = rq_data_dir(req);
 
Signature · 1 line
Jens Axboe
#2 lore raw
Quoted 19 lines · Jens Axboe
Quoted 18 lines · Martin Devera

Hello,

I see weird output from /sys/block/sd{a,b}/stat on our AMD64-X2 smp machine with HT1000 (Broadcom) SATA with 2 WD 250GB HDDs in MD raid1. AS scheduler was used, change to noop didn't change anything. It is vanilla 2.6.16.1 and here are absolute values in hex and one second differences below:

132CDF 56D62 61753C0 6966BC 165A8EB 5FF5B6C 3B32D6C0 1110594C FFCA89A4 FEE85878 49FF74E4 132CDF 56D62 61753C0 6966BC 165A8EF 5FF5B6C 3B32D6E0 111059D0 FFCA89A1 FEE85C60 79291244
0: 0
1: 0
2: 0
3: 0
4: 4
5: 0
6: 32
7: 132
8: -3
9: 1000
10: 791256416

Quoted 6 lines
Quoted 2 lines

As you can see in_flight is constantly negative and it is DECREASING slowly all the time. I can't find any reason for it :-\

Are you using io barriers?

[PATCH] blk: fix gendisk->in_flight accounting during barrier sequence

I don't think so, I just used
mount / -o remount,barrier=0
to make it sure and it keeps decrementing. I'll however apply the patch (and others up to 2.6.16.18) at night (I'm not allowed to restart the machine just now). thanks, Martin

#3 lore raw
Quoted 32 lines · Martin Devera
Quoted 31 lines · Jens Axboe
Quoted 24 lines · Martin Devera

Hello,

I see weird output from /sys/block/sd{a,b}/stat on our AMD64-X2 smp machine with HT1000 (Broadcom) SATA with 2 WD 250GB HDDs in MD raid1. AS scheduler was used, change to noop didn't change anything. It is vanilla 2.6.16.1 and here are absolute values in hex and one second differences below:

132CDF 56D62 61753C0 6966BC 165A8EB 5FF5B6C 3B32D6C0 1110594C FFCA89A4 FEE85878 49FF74E4
132CDF 56D62 61753C0 6966BC 165A8EF 5FF5B6C 3B32D6E0 111059D0 FFCA89A1 FEE85C60 79291244
0: 0
1: 0
2: 0
3: 0
4: 4
5: 0
6: 32
7: 132
8: -3
9: 1000
10: 791256416

Quoted 3 lines

As you can see in_flight is constantly negative and it is DECREASING slowly all the time.
I can't find any reason for it :-\

Are you using io barriers?

Quoted 7 lines
Quoted 1 line

[PATCH] blk: fix gendisk->in_flight accounting during barrier sequence

I don't think so, I just used
mount / -o remount,barrier=0
to make it sure and it keeps decrementing. I'll however apply the patch (and others up to 2.6.16.18) at night (I'm not allowed to restart the machine just now).

Ah, but you are using md, which uses barriers for superblock updates. So the patch will likely help you.

Signature · 1 line
Jens Axboe