Skip to contentSkip to conversation

Adaptive read-ahead

DISCUSSION Wu Fengguang 1 participant 1 message May 26, 2006 8 messages not in the archive

lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): [email protected]

#8 lore raw

Don Capps,

Quoted 35 lines · Iozone
Quoted 14 lines · Iozone
Quoted 13 lines

Wu Fengguang,

I noticed someone asking for real-world examples where a novel read-ahead would be helpful.

Ok...

Examples:

Structural analysis codes:
Writes large files, forward sequential.
then reads them backwards.

Quoted 7 lines
Quoted 5 lines

Weather codes:
Input from vertical scanning radar is written to a file, then the file is processed by strata. (Written in column order, read in row order..
thus a strided reader)

Precious data! Thanks very much.

The next time you get in your car, or on an airplane, remember that the structural design was done using the structures codes. Without these codes, people , to put it simply, would die. Also these codes permit molecular modeling that increases material strength of the design while reducing weight and cost. GM, Ford, AirBus, Boing, and many others run these codes every hour of every day. Yes, they are important, really :-)

Without the weather codes, folks would not have received early notification of the tornados, again it's pretty important to those programs :-)

Quoted 6 lines

Humorus note: Golf club manufacturers are also using structure codes to design better clubs that help bad golfers hit the ball further... Not exactly a noble goal, but yet another example of the critical competitive nature of the business world and their depencency on structural analysis codes :-)

Wow...

Quoted 12 lines
Quoted 11 lines
Quoted 8 lines

It was the study of these codes that lead to the inclusion of Iozone's backwards reader, and strided reader test cases :-)

If your system does not have a novel read-ahead algorithm then these types of applications will blind side the traditional read-ahead algorithm and the performance will be unacceptable.

Till now I have not implemented the strided readahead feature. Maybe we should tamp the existing features first :)

Quoted 16 lines
       Or, you could implement the strided feature, as it's
       pretty easy to detect.  Just need to trigger on two
       equa-distance accesses: A -> B and B -> C  where
       C-B = B-A then  the prediction D is C + distance between 
       A->B or B->C.
       The math turns out to be  easy to optimize:
           D= C+(B-A) 
       for both forward and backward strided readers. 

       Example:
       A = 1, B = 3  C=5

       Forward strided reader reading 1, 3, 5 -> predict 7
       3-1=2  and 5-3 = 2 and therefore  predict 5+2 = 7
       or going backwards,  reading 7,5, 3 -> predict 1
       5-7 = -2 and 3-5 = -2 and  3 + (-2) = 1 ... Cool huh :-)

Maybe it's useful for HPC applications, too. I'll consider it on time of convenient. Though it can still be tricky for implementing. So I'd concentrate on the top-priority thing first: to stabilize the existing features, so that the majority can enjoy them earlier.

Quoted 3 lines

Hmmm, I still remember what I wrote 8 years ago... Now that's amazing. I can't remember what I had for breakfast today, but this stuff seems fresh in memory banks :-) (weird)

It's a lucky thing for you to do it 8 years ago ;-)

Thanks,
Wu