benchmarks of O_STREAMING in 2.5

I gave the O_STREAMING in Andrew's 2.5-mm tree the treatment..

Short summary: It works.

The streaming read test in the following benchmarks is simply a read() in 64KB byte chunks of an 800MB file.

First test. Show the cache effects are indeed as we intend and the overhead is negligible. Here, mem=2G.

	O_STREAMING?	Wall time	Cache Delta
	Yes		21.827s		0
	No		21.734s		+800MB

Second test. Same deal, but mem=8M. There was slight swapping, so I suspect the reduced VM pressure is why the O_STREAMING run is faster.

	O_STREAMING?	Wall time	Cache Delta
	Yes		22.303s		0
	No		28.812s		+1MB

Third and final test. Kernel compile (make -j2) with a couple streaming reads in the background. Again, mem=2G. This shows that actually saving the pagecache from the horrid waste is useful.

	O_STREAMING	Wall time to complete Kernel compile
	Yes		5m30.494s
	No		4m59.661s

So, uh, Andrew's 2.5 code works ;-)

Someone buy me a dual Xeon,

Robert Love

?? O_STREAMING makes it slower or is it a typo ?

Bye.

Yes, typo, those should be switched.

Robert Love

All you now need to do is make it work with an API thats usable by the other 99% of real world apps, is extensible and sensible ways and therefore can be used.

I'm confused. Isn't this just an O_STREAM flag on open/fcntl ? How could it be simpler? I suppose the VM could do better than it currently does automatically but there is no harm in the app giving a hint like this thus allowing stuff to be dropped from the cache more aggresively?

Pádraig.

Which bit are you streaming, what kind of things are you doing with the data ?

Take a database table are you - accessing it sequentially with caching, accessing it randomly without readahead or streaming it and discarding data. All three are valid answers, for the same data, and potentially on a per table basis in the same file.

See the fadvise proposal. O_STREAMING is basically a subset of what is needed