Skip to contentSkip to conversation

Ingo's RAID patch for 2.2.18 final?

DISCUSSION Sasi Peter 5 participants 7 messages Jan 24, 2001 – Feb 17, 2001

lore
Share & tools
lore

Sasi Peter

author
#0 lore raw
Quoted 2 lines
Quoted 1 line

(30+ high speed streams from 4 disks does really need some caching).

This isn't obvious. Your working may not fit in cache and so the

kernel

Quoted 1 line

understand it's worthless to swapout stuff to make space to a

polluted cache.

But your understanding agrees on that the larger chunks for each stream we read into cache, the more efficient for this kind of RAID disk structure the read is, thus basically the more cache we have, the more bandwidth we can serve. (disks give more data in the same time with fewer long reads than with several shorter ones)

So might it have been an accidental behaviour of the previous kernels to swap out pages in favor of caching under high I/O pressure, but it was certainly a benefical behaviour.

Quoted 1 line
Quoted 1 line

Can't say, of these many daemons nothing can be swapped out (and

should

Quoted 2 lines
Quoted 1 line

under I/O load)!

If you run cp /dev/zero . on a smart VM nothing must be swapped out

even if

Quoted 1 line

it generated nearly the maximal I/O flood possible. It's worthless to

let a

Quoted 1 line

polluted cache to grow. It won't help anyways and you'll run slower

the next

Quoted 2 lines

time you'll have to pagein from swap.
It _enterely_ depends on the I/O load pattern if it worth to swapout

or not

Quoted 1 line

to make space for filesystem cache.

Ok (possibily incorrect, but simple) definition of I/O pressure of mine is when the _real_ _physical_ disks are working all the time, pushing data out of the box (in this case through the network).

"cp /dev/zero ." is a somewhat different from my case:
- mine is an IRL case (don't know how often a pattern like the "cp" case show up IRL)
- Mine is about _reading_ from disks

Quoted 2 lines
Quoted 1 line

Be this, if this is the price for stability.

As said we can add bits of page aging (that can't destabilize

anything and it

Quoted 1 line

will only affect performance behaviour), but I'd prefer to be sure

you really

Quoted 1 line

get a slowdown due the new VM behaviour (because more aging if done

without

Quoted 1 line

multiqueue O(1) approch can introduces waste of CPU and cachelines in

kernel

Quoted 1 line

space), so could you try to kill notes and squid and the other unused

stuff and

Quoted 1 line

to see if you return to deliver performance as with the older

kernels? I still

It might and it should, but actually I gotta have these started in case someone drops in for using them. As I understand the only thing this is worth trying out for is that maybe even with more cache I will have less performance, than before, because in this case to or not to swap out dows not really matter. Is this correct? Beacuse I will have to have these running anyways...

Quoted 1 line

miss this important information (last thing you said me was that with

100mbyte

Quoted 1 line

in cache it swapouts, and without knowing the details of the I/O

pattern it

like when decreasing constantly, at reaching that only 100MB cache we have left do we start swapping to have more cache, or at least have the 100MB not less.

Quoted 1 line

looked sane). After that I'd also like to know what happens with

2.4.0 that

Quoted 1 line

uses multiqueue and that is also able to detect pollution and to

avoid swapping

Quoted 1 line

out in such case.

What should I test with? (2.4.0/1pre?)

-- SaPE / Sasi Péter / mailto: [email protected] / http://sape.iq.rulez.org/
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/

#1 lore raw
Quoted 9 lines · Sasi Peter
Quoted 3 lines

This isn't obvious. Your working may not fit in cache and so the kernel understand it's worthless to swapout stuff to make space to a polluted cache.

But your understanding agrees on that the larger chunks for each stream we read into cache, the more efficient for this kind of RAID disk structure the read is, thus basically the more cache we have, the more bandwidth we can serve. (disks give more data in the same time with fewer long reads than with several shorter ones)

The size of the I/O requests doesn't grow linearly with with the size of the cache, as far as you have some mbyte of cache you will also be able to sumbit full sized requests to disk (512K per req on 2.4). In your workload you just had enough memory for the readahead.

In general if your working set doesn't fit in cache, the size of the cache is unrelated to the bandwith you get out of your RAID, infact if your working set doesn't fit in cache you should not pass through the cache at all to get the best performance and to save CPU cycles and L1 dcache and L2 cache (O_DIRECT).

Quoted 3 lines

So might it have been an accidental behaviour of the previous kernels to swap out pages in favor of caching under high I/O pressure, but it was certainly a benefical behaviour.

I trust you this is the case for your workload, but make sure to not assume that because there's less cache and no swap you're running slower, you may be running _much_ faster instead ;).

Quoted 1 line

What should I test with? (2.4.0/1pre?)

latest pre patch is ok.

Andrea

#2 lore raw
In reply to Sasi Peter
Quoted 17 lines · Andrea Arcangeli
Quoted 9 lines · Sasi Peter
Quoted 3 lines

This isn't obvious. Your working may not fit in cache and so the kernel understand it's worthless to swapout stuff to make space to a polluted cache.

But your understanding agrees on that the larger chunks for each stream we read into cache, the more efficient for this kind of RAID disk structure the read is, thus basically the more cache we have, the more bandwidth we can serve. (disks give more data in the same time with fewer long reads than with several shorter ones)

The size of the I/O requests doesn't grow linearly with with the size of the cache, as far as you have some mbyte of cache you will also be able to sumbit full sized requests to disk (512K per req on 2.4). In your workload you just had enough memory for the readahead.

Quoted 5 lines

In general if your working set doesn't fit in cache, the size of the cache is unrelated to the bandwith you get out of your RAID, infact if your working set doesn't fit in cache you should not pass through the cache at all to get the best performance and to save CPU cycles and L1 dcache and L2 cache (O_DIRECT).

This may be a bit OT, but when you say O_DIRECT, that implies that you can pass that flag to open(2) and it will bypass the page cache, and read directly into user-space buffers (zero-copy IO)? Does this also bypass the read-ahead mechanisms in the kernel? Does it imply O_SYNC?

Lots of questions... no answers. Sigh.

David

Signature · 3 lines
David Mansfield                                           (718) 963-2020
[email protected]
Ultramaster Group, LLC                               www.ultramaster.com
#3 lore raw

Can 2.2 Linux boot from an IDE ZIP?

Can 2.2.x linux made to boot from an ide zip drive? If so, what is required?

DB

#4 lore raw

Re: Can 2.2 Linux boot from an IDE ZIP?

Quoted 1 line · Dennis

Can 2.2.x linux made to boot from an ide zip drive? If so, what is required?

There is a (rather old, but sufficient) mini-HOWTO about this. I use it to create a Linux-From-Scratch with RPMs. The url is:
http://www.linuxdoc.org/HOWTO/mini/Zip-Install.html

Signature · 1 line
 Sven Vermeulen
#6 lore raw
In reply to Sasi Peter
Quoted 2 lines · David Mansfield

This may be a bit OT, but when you say O_DIRECT, that implies that you can pass that flag to open(2) and it will bypass the page cache, and

yes.

Quoted 1 line

read directly into user-space buffers (zero-copy IO)? Does this also

yes.

Quoted 1 line

bypass the read-ahead mechanisms in the kernel? Does it imply O_SYNC?

yes.

It's rawio through the fs. It's not included into 2.4-latest yet.

Andrea