More on O_STREAMING (goodby read pauses)
Hi all...
I have discovered one other (good) side effect of O_STREAMING.
I tested it on my duap PII@400, ~900Mb RAM.
Timings:
Without O_STREAMING:
0.00user 10.13system 0:41.16elapsed 24%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (81major+2574minor)pagefaults 0swaps
With O_STREAMING:
0.00user 11.03system 0:40.06elapsed 27%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (81major+2574minor)pagefaults 0swaps
Memory:
Before
total: used: free: shared: buffers: cached:
Mem: 926797824 228376576 698421248 0 20729856 95637504 Swap: 353644544 114688 353529856 MemTotal: 905076 kB MemFree: 682052 kB MemShared: 0 kB Buffers: 20244 kB Cached: 93284 kB SwapCached: 112 kB Active: 77284 kB Inactive: 97160 kB HighTotal: 0 kB HighFree: 0 kB LowTotal: 905076 kB LowFree: 682052 kB SwapTotal: 345356 kB SwapFree: 345244 kB BigFree: 0 kB
After test without O_STREAMING:
total: used: free: shared: buffers: cached:
Mem: 926797824 910376960 16420864 0 13369344 788623360 Swap: 353644544 1077248 352567296 MemTotal: 905076 kB MemFree: 16036 kB MemShared: 0 kB Buffers: 13056 kB Cached: 769088 kB SwapCached: 1052 kB Active: 83672 kB Inactive: 759388 kB HighTotal: 0 kB HighFree: 0 kB LowTotal: 905076 kB LowFree: 16036 kB SwapTotal: 345356 kB SwapFree: 344304 kB BigFree: 0 kB
After test with O_STREAMING:
total: used: free: shared: buffers: cached:
Mem: 926797824 228241408 698556416 0 20647936 95596544 Swap: 353644544 114688 353529856 MemTotal: 905076 kB MemFree: 682184 kB MemShared: 0 kB Buffers: 20164 kB Cached: 93244 kB SwapCached: 112 kB Active: 77200 kB Inactive: 97112 kB HighTotal: 0 kB HighFree: 0 kB LowTotal: 905076 kB LowFree: 682184 kB SwapTotal: 345356 kB SwapFree: 345244 kB BigFree: 0 kB
But I did the test with an addition: read a 1Gb file and print an '*' after every 10M. Without O_STREAMING, when memory fills, the 'progress bar' stalls for a few seconds while pages are sent to disk. So the patch also favours a constant sustained rate of read from the disk. Very interesting for things like video edition and so on. I like it ;).
Thanks.
Signature · 4 lines
J.A. Magallon <[email protected]> \ Software is like sex: werewolf.able.es \ It's better when it's free Mandrake Linux release 9.1 (Cooker) for i586 Linux 2.4.20-pre10-jam1 (gcc 3.2 (Mandrake Linux 9.0 3.2-2mdk))
Quoted 6 lines · J.A. Magallon
But I did the test with an addition: read a 1Gb file and print an '*' after every 10M. Without
O_STREAMING, when memory fills, the 'progress bar' stalls for a few seconds while pages are sent to disk. So the patch also favours a constant sustained rate of read from the disk. Very interesting for things like video edition and so on. I like it ;).
This is 100% the point of the patch and hopefully the point I proved when I first posted it.
Robert Love
Quoted 9 lines · Robert Love
Quoted 6 lines · J.A. Magallon
But I did the test with an addition: read a 1Gb file and print an '*' after every 10M. Without
O_STREAMING, when memory fills, the 'progress bar' stalls for a few seconds while pages are sent to disk. So the patch also favours a constant sustained rate of read from the disk. Very interesting for things like video edition and so on. I like it ;).This is 100% the point of the patch and hopefully the point I proved when I first posted it.
I assume the stall is not 'while pages are sent to disk', but rather until kswapd gets around to freeing enough pages to allow memory to fill again. The stall is due to the pages being fully analyzed to determine which ones should go, and which ones shouldn't. O_STREAMING removes the pages ahead of time, so no analysis is ever required.
mark
Signature · 9 lines
[email protected]/[email protected]/[email protected] __________________________ . . _ ._ . . .__ . . ._. .__ . . . .__ | Neighbourhood Coder |\/| |_| |_| |/ |_ |\/| | |_ | |/ |_ | | | | | | \ | \ |__ . | | .|. |__ |__ | \ |__ | Ottawa, Ontario, Canada One ring to rule them all, one ring to find them, one ring to bring them all and in the darkness bind them... http://mark.mielke.cc/
Quoted 8 lines · On 2002.10.10 Mark Mielke wrote:
Quoted 7 lines · Robert Love
Quoted 6 lines · J.A. Magallon
But I did the test with an addition: read a 1Gb file and print an '*' after every 10M. Without
O_STREAMING, when memory fills, the 'progress bar' stalls for a few seconds while pages are sent to disk. So the patch also favours a constant sustained rate of read from the disk. Very interesting for things like video edition and so on. I like it ;).
Quoted 9 lines
Quoted 2 lines
This is 100% the point of the patch and hopefully the point I proved when I first posted it.
I assume the stall is not 'while pages are sent to disk', but rather until kswapd gets around to freeing enough pages to allow memory to fill again. The stall is due to the pages being fully analyzed to determine which ones should go, and which ones shouldn't.
O_STREAMINGremoves the pages ahead of time, so no analysis is ever required.
I can _hear_ the disk activity when the stall happens, so selecting what to drop is fast, but then you have to write it...
Signature · 4 lines
J.A. Magallon <[email protected]> \ Software is like sex: werewolf.able.es \ It's better when it's free Mandrake Linux release 9.1 (Cooker) for i586 Linux 2.4.20-pre10-jam1 (gcc 3.2 (Mandrake Linux 9.0 3.2-2mdk))
Quoted 8 lines · J.A. Magallon
Quoted 5 lines · On 2002.10.10 Mark Mielke wrote:
I assume the stall is not 'while pages are sent to disk', but rather until kswapd gets around to freeing enough pages to allow memory to fill again. The stall is due to the pages being fully analyzed to determine which ones should go, and which ones shouldn't.
O_STREAMINGremoves the pages ahead of time, so no analysis is ever required.I can _hear_ the disk activity when the stall happens, so selecting what to drop is fast, but then you have to write it...
I don't think this is right. Prove me wrong by explaining how kswapd works, but if a page isn't dirty, there is no need to write it out to disk.
My (perhaps incorrect) assumption is that kswapd prefers to swap on clean pages over dirty pages. If your pages are mostly clean, there is nothing to write to disk the clear majority of the time.
Clean read-only pages should *never* be written to swap. They can be re-read from their source.
I _think_ what you are seeing is that kswapd is not cleaning pages out fast enough, which means that *other* tasks executing need to have their *swapped out* pages *read* from disk. I.e. the churning you hear is probably mostly reads - not writes.
mark
Signature · 9 lines
[email protected]/[email protected]/[email protected] __________________________ . . _ ._ . . .__ . . ._. .__ . . . .__ | Neighbourhood Coder |\/| |_| |_| |/ |_ |\/| | |_ | |/ |_ | | | | | | \ | \ |__ . | | .|. |__ |__ | \ |__ | Ottawa, Ontario, Canada One ring to rule them all, one ring to find them, one ring to bring them all and in the darkness bind them... http://mark.mielke.cc/
Quoted 7 lines · On 2002.10.10 Mark Mielke wrote:
Quoted 6 lines · J.A. Magallon
Quoted 5 lines · On 2002.10.10 Mark Mielke wrote:
I assume the stall is not 'while pages are sent to disk', but rather until kswapd gets around to freeing enough pages to allow memory to fill again. The stall is due to the pages being fully analyzed to determine which ones should go, and which ones shouldn't.
O_STREAMINGremoves the pages ahead of time, so no analysis is ever required.
Quoted 6 lines
Quoted 2 lines
I can _hear_ the disk activity when the stall happens, so selecting what to drop is fast, but then you have to write it...
I don't think this is right. Prove me wrong by explaining how kswapd works, but if a page isn't dirty, there is no need to write it out to disk.
You suppose it is a page of code ? What about data that programs malloc() ??? You can also send data memory to swap. If you do not write it on disk, how do you recover it ???
Quoted 6 lines
My (perhaps incorrect) assumption is that kswapd prefers to swap on clean pages over dirty pages. If your pages are mostly clean, there is nothing to write to disk the clear majority of the time.
Clean read-only pages should *never* be written to swap. They can be re-read from their source.
That is your fault, <read-only>. Pages maped read-only are those from binary executables or shared libraries, but, again, what about data ?
Quoted 6 lines
I _think_ what you are seeing is that kswapd is not cleaning pages out fast enough, which means that *other* tasks executing need to have their *swapped out* pages *read* from disk. I.e. the churning you hear is probably mostly reads - not writes.
I look at gnome system monitor graph for mem. I start with a tiny amount of used memory. Start the 1Gb read without O_STREAM, the blue area in monitor starts to grow linearly in time, stars (*) from the reader appear at a given rate, and as soon as it touches the top limit the stars stop, the disk begins to thrash, and swap space used grows. After a 2-4 seconds, the stars go again with the same rate. Tell me what is this but swapper writing pages, and reading the new pages for my giga.
With O_STREAM, the 'blue bar' does not move from its place, and star rate (ie, read rate from disk), stays uniform.
Signature · 4 lines
J.A. Magallon <[email protected]> \ Software is like sex: werewolf.able.es \ It's better when it's free Mandrake Linux release 9.1 (Cooker) for i586 Linux 2.4.20-pre10-jam1 (gcc 3.2 (Mandrake Linux 9.0 3.2-2mdk))
Quoted 10 lines · J.A. Magallon
...
I look at gnome system monitor graph for mem. I start with a tiny amount of used memory. Start the 1Gb read withoutO_STREAM, the blue area in monitor starts to grow linearly in time, stars (*) from the reader appear at a given rate, and as soon as it touches the top limit the stars stop, the disk begins to thrash, and swap space used grows. After a 2-4 seconds, the stars go again with the same rate. Tell me what is this but swapper writing pages, and reading the new pages for my giga.
That's fairly rude behaviour for a 2.4 kernel. Sounds like 2.5 ;)
What kernel is that?
Quoted 6 lines · J.A. Magallon
Quoted 2 lines · On 2002.10.10 Mark Mielke wrote:
I don't think this is right. Prove me wrong by explaining how kswapd works, but if a page isn't dirty, there is no need to write it out to disk.
You suppose it is a page of code ? What about data that programs
malloc()??? You can also send data memory to swap. If you do not write it on disk, how do you recover it ???
Unless you are telling me that you have 1Gb of dirty pages in your swap, it doesn't matter. The majority of the pages are not dirty, and they are only in use because the 1Gb+ file was read from front to back.
Quoted 4 lines
Quoted 2 lines
Clean read-only pages should *never* be written to swap. They can be re-read from their source.
That is your fault, <read-only>. Pages maped read-only are those from binary executables or shared libraries, but, again, what about data ?
I shouldn't have said read only. The real qualifier is the 'clean' adjective.
It doesn't matter if a page is read-write or read-only, if the page came directly from disk, and no modifications have been made to the page, the page can safely be turfed and should _never_ be written to swap.
Quoted 4 lines
Quoted 4 lines
I _think_ what you are seeing is that kswapd is not cleaning pages out fast enough, which means that *other* tasks executing need to have their *swapped out* pages *read* from disk. I.e. the churning you hear is probably mostly reads - not writes.
Quoted 9 lines
I look at gnome system monitor graph for mem. I start with a tiny amount of used memory. Start the 1Gb read without
O_STREAM, the blue area in monitor starts to grow linearly in time, stars (*) from the reader appear at a given rate, and as soon as it touches the top limit the stars stop, the disk begins to thrash, and swap space used grows. After a 2-4 seconds, the stars go again with the same rate. Tell me what is this but swapper writing pages, and reading the new pages for my giga.
WithO_STREAM, the 'blue bar' does not move from its place, and star rate (ie, read rate from disk), stays uniform.
It sounds like your kernel is broken.
mark
Signature · 9 lines
[email protected]/[email protected]/[email protected] __________________________ . . _ ._ . . .__ . . ._. .__ . . . .__ | Neighbourhood Coder |\/| |_| |_| |/ |_ |\/| | |_ | |/ |_ | | | | | | \ | \ |__ . | | .|. |__ |__ | \ |__ | Ottawa, Ontario, Canada One ring to rule them all, one ring to find them, one ring to bring them all and in the darkness bind them... http://mark.mielke.cc/