Skip to contentSkip to conversation

block IO read-ahead during sequential *write*?

noob q. on block layer Frantisek Rysanek 2 participants 3 messages Jan 7, 2008

lore
Share & tools
lore
#0 lore raw

Dear Everyone,

let me start with a simple example. The following commands:

cp /dev/zero /dev/hda
dd if=/dev/zero of=/dev/hda [bs=512]

both have one common side-effect: apart from the disk being properly overwritten with zeroes, the kernel seems to keep reading sectors ahead of the current seek position of the sequential write.

This is kernel 2.6.22.6 and around that - previously 2.6.16.* and 2.6.18.* . I believe Linux 2.4 kernels didn't do that.

I observe that behavior using iostat 1 (from the Sysstat package). After I spotted the behavior, I first upgraded to Sysstat 8.0.3 (latest at the time) to avoid any possible misinterpretation of /proc/diskstats or what the relevant /proc node is. The upgrade brought no difference.

I'm using commands along those lines to erase hard drives, to check their sequential write speed, and as an additional test when checking for bad sectors / flawed behavior that goes unreported by SMART. The read-ahead skews my view of sequential write performance :-)

Note that if I specify count=<some fixed number> to 'dd', the read-ahead doesn't kick in.

Makes me wonder if perhaps "cp" and "dd without count=" perform their writes un-aligned to sector boundaries, so that the Linux block layer feels obliged to prefetch the trailing sector that gets only partially written within a given transaction. It's weird though. At least bs=512 should be aligned to sector boundaries, even if uncounted. And, it doesn't seem like every N'th sector - the read and write rates are about equal.

I've already discovered "hdparm -a", and some follow-ups called ioctl(BLKRASET) and 'bdi->ra_pages' in $KERNEL_SRC/block/* . That's where I got stuck - trying to understand the whole read-ahead logic from the source code would take me too long to be worth it. That's when I resorted to posting this to the LKML...

Is "hdparm -a" my only chance? Is there any kernel command line argument or /proc entry or ioctl() to modify this behavior selectively for writes? Should I write my own test prog, that will take care to strictly walk the sector boundaries?

Any ideas are welcome :-)

Frank Rysanek

#1 lore raw
Quoted 9 lines · Frantisek Rysanek

let me start with a simple example. The following commands:

cp /dev/zero /dev/hda
dd if=/dev/zero of=/dev/hda [bs=512]

both have one common side-effect: apart from the disk being properly overwritten with zeroes, the kernel seems to keep reading sectors ahead of the current seek position of the sequential write.

Block devices are cached in the page cache. If you write less than a full page, any remainder has to be read from the device.

If you retry the dd with bs=4096 (or whatever your architecture's page size happens to be), does this still occur?

Jörn

Signature · 2 lines
Chance favors only the prepared mind.
-- Louis Pasteur
#2 lore raw
Quoted 6 lines · Jörn Engel

If you retry the dd with bs=4096 (or whatever your architecture's page size happens to be), does this still occur?

JĂśrn

wow, thanks, it works, this is so obvious.
I tend to use values like bs=512000 or suchlike, but probably never stroke this particular value when I was looking at iostat. Must've missed that modulo 4096 somehow.

Seems like cp is using 512B per transaction, judging by how slow the transfer goes. If I use bs=500000 or even bs=1024, it goes faster.

Have a nice day :-}

Frank Rysanek