Ingo's RAID patch for 2.2.18 final?
Quoted 2 lines
Quoted 1 line
(30+ high speed streams from 4 disks does really need some caching).
This isn't obvious. Your working may not fit in cache and so the
kernel
Quoted 1 line
understand it's worthless to swapout stuff to make space to a
polluted cache.
But your understanding agrees on that the larger chunks for each stream we read into cache, the more efficient for this kind of RAID disk structure the read is, thus basically the more cache we have, the more bandwidth we can serve. (disks give more data in the same time with fewer long reads than with several shorter ones)
So might it have been an accidental behaviour of the previous kernels to swap out pages in favor of caching under high I/O pressure, but it was certainly a benefical behaviour.
Quoted 1 line
Quoted 1 line
Can't say, of these many daemons nothing can be swapped out (and
should
Quoted 2 lines
Quoted 1 line
under I/O load)!
If you run
cp /dev/zero .on a smart VM nothing must be swapped out
even if
Quoted 1 line
it generated nearly the maximal I/O flood possible. It's worthless to
let a
Quoted 1 line
polluted cache to grow. It won't help anyways and you'll run slower
the next
Quoted 2 lines
time you'll have to pagein from swap.
It _enterely_ depends on the I/O load pattern if it worth to swapout
or not
Quoted 1 line
to make space for filesystem cache.
Ok (possibily incorrect, but simple) definition of I/O pressure of mine is when the _real_ _physical_ disks are working all the time, pushing data out of the box (in this case through the network).
"cp /dev/zero ." is a somewhat different from my case:
- mine is an IRL case (don't know how often a pattern like the "cp" case show up IRL)
- Mine is about _reading_ from disks
Quoted 2 lines
Quoted 1 line
Be this, if this is the price for stability.
As said we can add bits of page aging (that can't destabilize
anything and it
Quoted 1 line
will only affect performance behaviour), but I'd prefer to be sure
you really
Quoted 1 line
get a slowdown due the new VM behaviour (because more aging if done
without
Quoted 1 line
multiqueue O(1) approch can introduces waste of CPU and cachelines in
kernel
Quoted 1 line
space), so could you try to kill notes and squid and the other unused
stuff and
Quoted 1 line
to see if you return to deliver performance as with the older
kernels? I still
It might and it should, but actually I gotta have these started in case someone drops in for using them. As I understand the only thing this is worth trying out for is that maybe even with more cache I will have less performance, than before, because in this case to or not to swap out dows not really matter. Is this correct? Beacuse I will have to have these running anyways...
Quoted 1 line
miss this important information (last thing you said me was that with
100mbyte
Quoted 1 line
in cache it swapouts, and without knowing the details of the I/O
pattern it
like when decreasing constantly, at reaching that only 100MB cache we have left do we start swapping to have more cache, or at least have the 100MB not less.
Quoted 1 line
looked sane). After that I'd also like to know what happens with
2.4.0 that
Quoted 1 line
uses multiqueue and that is also able to detect pollution and to
avoid swapping
Quoted 1 line
out in such case.
What should I test with? (2.4.0/1pre?)
-- SaPE / Sasi Péter / mailto: [email protected] / http://sape.iq.rulez.org/
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/
Quoted 9 lines · Sasi Peter
Quoted 3 lines
This isn't obvious. Your working may not fit in cache and so the kernel understand it's worthless to swapout stuff to make space to a polluted cache.
But your understanding agrees on that the larger chunks for each stream we read into cache, the more efficient for this kind of RAID disk structure the read is, thus basically the more cache we have, the more bandwidth we can serve. (disks give more data in the same time with fewer long reads than with several shorter ones)
The size of the I/O requests doesn't grow linearly with with the size of the cache, as far as you have some mbyte of cache you will also be able to sumbit full sized requests to disk (512K per req on 2.4). In your workload you just had enough memory for the readahead.
In general if your working set doesn't fit in cache, the size of the cache is unrelated to the bandwith you get out of your RAID, infact if your working set doesn't fit in cache you should not pass through the cache at all to get the best performance and to save CPU cycles and L1 dcache and L2 cache (O_DIRECT).
Quoted 3 lines
So might it have been an accidental behaviour of the previous kernels to swap out pages in favor of caching under high I/O pressure, but it was certainly a benefical behaviour.
I trust you this is the case for your workload, but make sure to not assume that because there's less cache and no swap you're running slower, you may be running _much_ faster instead ;).
Quoted 1 line
What should I test with? (2.4.0/1pre?)
latest pre patch is ok.
Andrea
Quoted 17 lines · Andrea Arcangeli
Quoted 9 lines · Sasi Peter
Quoted 3 lines
This isn't obvious. Your working may not fit in cache and so the kernel understand it's worthless to swapout stuff to make space to a polluted cache.
But your understanding agrees on that the larger chunks for each stream we read into cache, the more efficient for this kind of RAID disk structure the read is, thus basically the more cache we have, the more bandwidth we can serve. (disks give more data in the same time with fewer long reads than with several shorter ones)
The size of the I/O requests doesn't grow linearly with with the size of the cache, as far as you have some mbyte of cache you will also be able to sumbit full sized requests to disk (512K per req on 2.4). In your workload you just had enough memory for the readahead.
Quoted 5 lines
In general if your working set doesn't fit in cache, the size of the cache is unrelated to the bandwith you get out of your RAID, infact if your working set doesn't fit in cache you should not pass through the cache at all to get the best performance and to save CPU cycles and L1 dcache and L2 cache (
O_DIRECT).
This may be a bit OT, but when you say O_DIRECT, that implies that you can pass that flag to open(2) and it will bypass the page cache, and read directly into user-space buffers (zero-copy IO)? Does this also bypass the read-ahead mechanisms in the kernel? Does it imply O_SYNC?
Lots of questions... no answers. Sigh.
David
Signature · 3 lines
David Mansfield (718) 963-2020 [email protected] Ultramaster Group, LLC www.ultramaster.com
Can 2.2 Linux boot from an IDE ZIP?
Can 2.2.x linux made to boot from an ide zip drive? If so, what is required?
DB
Re: Can 2.2 Linux boot from an IDE ZIP?
Quoted 1 line · Dennis
Can 2.2.x linux made to boot from an ide zip drive? If so, what is required?
There is a (rather old, but sufficient) mini-HOWTO about this. I use it to create a Linux-From-Scratch with RPMs. The url is:
http://www.linuxdoc.org/HOWTO/mini/Zip-Install.html
Signature · 1 line
Sven Vermeulen
Re: Can 2.2 Linux boot from an IDE ZIP?
In reply to DennisQuoted 1 line · Dennis
Can 2.2.x linux made to boot from an ide zip drive? If so, what is required?
Oops, sorry, wrong URL (damned):
http://www.linuxdoc.org/HOWTO/mini/Install-From-ZIP.html
Found it with google ("zip install linux mini"). Just to make sure that, if it's the wrong URL again (but I doubt it), it's easy to find yourself.
Signature · 1 line
Sven Vermeulen
Quoted 2 lines · David Mansfield
This may be a bit OT, but when you say
O_DIRECT, that implies that you can pass that flag to open(2) and it will bypass the page cache, and
yes.
Quoted 1 line
read directly into user-space buffers (zero-copy IO)? Does this also
yes.
Quoted 1 line
bypass the read-ahead mechanisms in the kernel? Does it imply
O_SYNC?
yes.
It's rawio through the fs. It's not included into 2.4-latest yet.
Andrea