Skip to contentSkip to conversation

High read Latency test (Anticipatory I/O scheduler)

DISCUSSION John Chatelle 4 participants 4 messages Feb 20, 2004 – Feb 21, 2004 Block layer

lore
Share & tools
lore
#0 lore raw

I haven't seen much duplicated results regarding the Robert Love article
in the February 2004 Linux Journal article, also reachable in the hyperlink:

          http://www.linuxjournal.com/article.php?sid=6931

Although the 1st simple test: "Write starved reads" gets results comparable
to the results reported in the Article, Our results for the 2nd test: "High Read latency" delivers results opposite our expectations...

Kernel 2.6.2 Results: (Anticipatory I/O scheduler).

real    43m41.138s       Nearly 44 minutes! 2nd run similar.
user    0m4.715s
sys     0m11.179s

Kernel 2.4.20-28.7p21gsmp Results:

real    0m41.535s        Only 42 seconds!   2nd run similar.
user    0m4.720s
sys     0m15.470s

Our dmesg shows wer're running the Anticipatory scheduler when testing under the 2.6.2 kernel.

The 2 shell scripts, StreamingRead.sh and WHR.sh (bash, actually), implement the test:

#StreamingRead.sh --simple 4 line shell script:
while true
do
cat ../data/oneGBfile >/dev/null
done

#WHR.sh -- simple 2 (or 3) line shell script.
StreamingRead.sh &

   time find /usr/src/linux-2.4.20-18.7  -type f -exec cat \
     '{}' ';' > /dev/null

I'm reading a 1G binary garbage file repeatedly while timing
the transversal and reading of the 2.4 Kernel source tree, just as the test 2 example shows in the article. I would think I/O anticipating the 1G garbage file would be likely, where the I/O anticipation of the reading of the source tree under the 'find' command would be far choppier and more difficult.The 'time' command, however measures the less anticipatory and choppier reads of the 'find' command! I therefore see the results of table 1 for test 2 to be very counter intuitive!

Has anyone else seen such divergent results compared to those reported in
the article? Does anyone else see the same results with the anticipatory I/O scheduler?

John Chatelle
[email protected]
Community Computer Service
15 Hulbert Street - P.O. Box 980
Auburn, New York 13021

Phone:  (315)-255-1751 
Fax:    (315)-255-3539

--
This message and any attachments may contain information that is protected by law as privileged and confidential, and is transmitted for the sole use of the intended recipient(s). If you are not the intended recipient, you are hereby notified that any use, dissemination, copying or retention of this e-mail or the information contained herein is strictly prohibited. If you have received this e-mail in error, please immediately notify the sender by e-mail, and permanently delete this e-mail.

Signature · 2 lines
This message has been scanned for viruses and
dangerous content by MailScanner.
#1 lore raw

I was able to repeat Robert Love's results on a machine with a simple IDE controller, two-processer system, with 500 meg memory, ext2 file system, mounted with the noatime option. I think Robert's article also indicated he used an IDE controller for disk.

The results can be influenced a lot by your disk/controller configuration. RAID controllers in particular, or disks with large TCQ may not perform as well with anticipatory scheduler.

Also, there were some changes to the read-ahead policy that came between 2.6.2 and 2.6.3, I think. That has been shown to make a measureable difference on systems here.

Try again with 2.6.3.

Also, try setting the antic_expire scheduler parameter for the queue for your disk, and see what results you get. That should hopefully be close to 2.4's behavior. If it isn't, then there's probably something more going on than just the IO scheduler.

Quoted 83 lines · John Chatelle

I haven't seen much duplicated results regarding the Robert Love article
in the February 2004 Linux Journal article, also reachable in the hyperlink:

          http://www.linuxjournal.com/article.php?sid=6931

Although the 1st simple test: "Write starved reads" gets results comparable
to the results reported in the Article, Our results for the 2nd test: "High Read latency" delivers results opposite our expectations...

Kernel 2.6.2 Results: (Anticipatory I/O scheduler).

real    43m41.138s       Nearly 44 minutes! 2nd run similar.
user    0m4.715s
sys     0m11.179s

Kernel 2.4.20-28.7p21gsmp Results:

real    0m41.535s        Only 42 seconds!   2nd run similar.
user    0m4.720s
sys     0m15.470s

Our dmesg shows wer're running the Anticipatory scheduler when testing under the 2.6.2 kernel.

The 2 shell scripts, StreamingRead.sh and WHR.sh (bash, actually), implement the test:

#StreamingRead.sh --simple 4 line shell script:
while true
do
cat ../data/oneGBfile >/dev/null
done

#WHR.sh -- simple 2 (or 3) line shell script.
StreamingRead.sh &

   time find /usr/src/linux-2.4.20-18.7  -type f -exec cat \
     '{}' ';' > /dev/null

I'm reading a 1G binary garbage file repeatedly while timing
the transversal and reading of the 2.4 Kernel source tree, just as the test 2 example shows in the article. I would think I/O anticipating the 1G garbage file would be likely, where the I/O anticipation of the reading of the source tree under the 'find' command would be far choppier and more difficult.The 'time' command, however measures the less anticipatory and choppier reads of the 'find' command! I therefore see the results of table 1 for test 2 to be very counter intuitive!

Has anyone else seen such divergent results compared to those reported in
the article? Does anyone else see the same results with the anticipatory I/O scheduler?

John Chatelle
[email protected]
Community Computer Service
15 Hulbert Street - P.O. Box 980
Auburn, New York 13021

Phone:  (315)-255-1751 
Fax:    (315)-255-3539

--
This message and any attachments may contain information that is protected by law as privileged and confidential, and is transmitted for the sole use of the intended recipient(s). If you are not the intended recipient, you are hereby notified that any use, dissemination, copying or retention of this e-mail or the information contained herein is strictly prohibited. If you have received this e-mail in error, please immediately notify the sender by e-mail, and permanently delete this e-mail.

Signature · 9 lines
This message has been scanned for viruses and
dangerous content by MailScanner.


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/
#2 lore raw
In reply to John Chatelle
Quoted 8 lines · John Chatelle

I haven't seen much duplicated results regarding the Robert Love article
in the February 2004 Linux Journal article, also reachable in the hyperlink:

         http://www.linuxjournal.com/article.php?sid=6931

Although the 1st simple test: "Write starved reads" gets results comparable
to the results reported in the Article, Our results for the 2nd test: "High Read latency" delivers results opposite our expectations...

Hi John,
Can you try the following patch please? If that doesn't help, can you show me what /sys/block/hda/queue/iosched/est_time says after your test has been running for a couple of minutes.

Thanks
Nick

 linux-2.6-npiggin/drivers/block/as-iosched.c |    2 ++
 1 files changed, 2 insertions(+)
drivers/block/as-iosched.c renamed from drivers/block/as-iosched.c~as-exit-prob +2 -0
diff -puN drivers/block/as-iosched.c~as-exit-prob drivers/block/as-iosched.c
@@ -734,8 +734,10 @@ static int as_can_break_anticipation(str
 	if (aic->ttime_samples == 0) {
 		if (ad->new_ttime_mean > ad->antic_expire)
 			return 1;
+#if 0
 		if (ad->exit_prob > 128)
 			return 1;
+#endif
 	} else if (aic->ttime_mean > ad->antic_expire) {
 		/* the process thinks too much between requests */
 		return 1;

_

#3 lore raw
In reply to John Chatelle
Quoted 11 lines · John Chatelle

#StreamingRead.sh --simple 4 line shell script:
while true
do
cat ../data/oneGBfile >/dev/null
done

#WHR.sh -- simple 2 (or 3) line shell script.
StreamingRead.sh &

   time find /usr/src/linux-2.4.20-18.7  -type f -exec cat \
     '{}' ';' > /dev/null

Note that the latter test runs `cat' once per file: over ten thousand times. You should also test:

	time  (find /usr/src/linux-2.4.20-18.7 -type f | xargs cat > /dev/null )

which will run `cat' only some tens of times.

For the anticipatory scheduler, this makes a significant difference - the run-cat-once-per-file problem has specific fixes and perhaps they're not working right at present.

And yes, please describe your disk system, tell us the tag depth (if it is scsi) and try Nick's patch, thanks.