EXT3 Testing w/ rmap14a with contest results
2.4.19-rmap14a - on Athlon MP 2000+ w/ 512MB RAM EXT3 Filesystem used
with hdparm tweaks
noload 4:57.93 99%
cpuload 6:06.85 79%
memload 6:47.27 76%
ioloadhalf 10:36.93 47%
ioloadfull 8:06.46 62%
without hdparm tweaks
noload 4:57.53 99%
cpuload 6:06.68 79%
memload 6:44.22 77%
ioloadhalf 10:02.30 50%
ioloadfull 8:14.96 61%
-------------------------------------------------
on Athlon MP 2000+ w/ 512MB RAM EXT3 Filesystem used 2.4.20-pre7-rmap14a-xfs-uml-shawn12d - with hdparm tweaks
noload 4:56.86 99%
cpuload 6:06.35 79%
memload 6:27.43 79%
ioloadhalf 9:00.57 55%
ioloadfull 8:11.82 61%
on Athlon MP 2000+ w/ 512MB RAM EXT3 Filesystem used 2.4.20-pre7-rmap14a-xfs-uml-shawn12d - without hdparm tweaks
noload 5:02.87 97%
cpuload 6:07.29 79%
memload 6:24.29 80%
ioloadhalf 8:05.11 61%
ioloadfull 7:55.27 63%
For comparsion from my older machine
--------------------------------------------------------------
2.4.20-pre7-rmap14a-xfs-uml-shawn12d - Pentium 200Mhz 64MB Ram EXT3 Filesystem
noload 52:46.43 99%
cpuload 1:05:14 79%
memload 1:11:12 79%
ioloadhalf 1:10:14 78%
ioloadfull 1:10:31 78%
--------------------------------------------------------------
on Athlon MP 2000+ w/ 512MB RAM 2.4.20-pre7-rmap14a-xfs-uml-shawn12d - without hdparm tweaks ___WITH EXT3___
noload Time: 265.82 CPU: 96% Major Faults: 755922 Minor Faults: 1150416 process_load Time: 314.80 CPU: 80% Major Faults: 727159 Minor Faults:
1146210io_halfmem Time: 444.25 CPU: 58% Major Faults: 726724 Minor Faults: 1146174 Was writing number 33 of a 257Mb sized io_load file after 449 seconds Was writing number 9 of a 514Mb sized io_load file after 262 seconds
----------------------------------------------------------------------------------------------------
on Athlon MP 2000+ w/ 512MB RAM 2.4.20-pre7-rmap14a-xfs-uml-shawn12d - without hdparm tweaks __WITHOUT EXT3___
noload Time: 259.41 CPU: 99% Major Faults: 770937 Minor Faults: 1173712 process_load Time: 318.63 CPU: 80% Major Faults: 742261 Minor Faults:
1169518io_halfmem Time: 305.75 CPU: 87% Major Faults: 742000 Minor Faults: 1169497 Was writing number 32 of a 257Mb sized io_load file after 306 seconds io_fullmem Time: 321.77 CPU: 83% Major Faults: 742000 Minor Faults: 1169497 Was writing number 16 of a 514Mb sized io_load file after 324 seconds mem_load Time: 345.10 CPU: 78% Major Faults: 743807 Minor Faults: 1170248
As you can see, there's something DEFINATELY wrong here =)
* NOTES:
The old test at the top with the old format are from contest 0.22, the new results seen below are 0.34.
<conman_work> yes, but results from different versions of contest are
incompatible
<ShawnCONSOLE> riel uses EXT3
<riel> my cpu is slower
<ShawnCONSOLE> but you have fast disks?
<riel> so it doesn't fall idle as quickly as yours, when waiting on the disk <riel> not very fast ;)
<riel> old 8 GB IDE disk
<ShawnCONSOLE> so having a fast disk and a fast CPU causes the cpu to wait longer cause the disk finishes its tasks much faster then the cpu expects? <ShawnCONSOLE> mem_load final test = 78%
<ShawnCONSOLE> so final numbers:
<ShawnCONSOLE> 99, 80%, 87%, 83%, 75%
<riel> yes, a very fast CPU falls idle more quickly
<riel> but it's very curious that ext3 is _that_ much worse than ext2 <ShawnCONSOLE> thats much better.
<riel> definately worth pointing out to the ext3 maintainers
Quoted 2 lines · Shawn Starr
As you can see, there's something DEFINATELY wrong here =)
Well I can't see what it is.
Please, try not to dump tons of raw numbers onto dazed onlookers.
There is great value in you performing the initial processing; distilling useful information and the "means to reproduce" out of the raw data. The old human touch.
Ideally: just two numbers, and the barest, simplest way of demonstrating the problem.
I'm reformatting the email with useless stuff dropped sorry =)
Quoted 13 lines · Andrew Morton
Quoted 1 line · Shawn Starr
As you can see, there's something DEFINATELY wrong here =)
Well I can't see what it is.
Please, try not to dump tons of raw numbers onto dazed onlookers.
There is great value in you performing the initial processing; distilling useful information and the "means to reproduce" out of the raw data. The old human touch.
Ideally: just two numbers, and the barest, simplest way of demonstrating the problem.
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): [email protected]
[BENCHMARK] EXT3 vs EXT2 results with rmap14a and testing with contest 0.34
Sorry about the confusing email before. This should make more sense =)
These results compare EXT3 against EXT2 with rmap using the contest tool you can get it at: http://contest.kolivas.net
These tests are from a Athlon MP 2000+ w/ 512MB RAM
noload:
Kernel Time CPU
2.4.20-pre7-rmap14a-xfs-uml-shawn12d 259.47 99%
2.4.20-pre7-rmap14a-xfs-uml-shawn12d 267.66 97%
process_load:
Kernel Time CPU
2.4.20-pre7-rmap14a-xfs-uml-shawn12d 318.91 80% 2.4.20-pre7-rmap14a-xfs-uml-shawn12d 324.44 79%
io_halfmem:
Kernel Time CPU
2.4.20-pre7-rmap14a-xfs-uml-shawn12d 306.82 87% 2.4.20-pre7-rmap14a-xfs-uml-shawn12d 461.74 57%
io full mem:
Kernel Time CPU
2.4.20-pre7-rmap14a-xfs-uml-shawn12d 325.39 82% 2.4.20-pre7-rmap14a-xfs-uml-shawn12d 411.47 64%
full logs of the tests are:
WITH EXT2
------------
noload Time: 259.47 CPU: 99% Major Faults: 770937 Minor Faults: 1173705 process_load Time: 318.91 CPU: 80% Major Faults: 742261 Minor Faults:
1169516io_halfmem Time: 306.82 CPU: 87% Major Faults: 742000 Minor Faults: 1169497 Was writing number 33 of a 257Mb sized io_load file after 307 seconds io_fullmem Time: 325.39 CPU: 82% Major Faults: 742000 Minor Faults: 1169494 Was writing number 16 of a 514Mb sized io_load file after 337 seconds mem_load Time: 340.32 CPU: 79% Major Faults: 743307 Minor Faults: 1170011
WITH EXT3
-----------
noload Time: 267.66 CPU: 97% Major Faults: 771111 Minor Faults: 1173722 process_load Time: 324.44 CPU: 79% Major Faults: 742261 Minor Faults:
1169518io_halfmem Time: 461.74 CPU: 57% Major Faults: 742000 Minor Faults: 1169496 Was writing number 34 of a 257Mb sized io_load file after 465 seconds io_fullmem Time: 411.47 CPU: 64% Major Faults: 742000 Minor Faults: 1169494 Was writing number 15 of a 514Mb sized io_load file after 425 seconds mem_load Time: 333.99 CPU: 81% Major Faults: 743320 Minor Faults: 1170021
NOTES:
====
As you can see, there's something DEFINATELY wrong here. EXT3 is much slower then EXT2. I converted the EXT3 disk back to EXT2 to do the second test.
Also, I specified no mount options for EXT3 (which means it uses ordered mode). The journal was created with tune2fs -j /dev/hda#
Quoted 1 line
From #Kernelnewbies (snip)
==============
<ShawnCONSOLE> riel uses EXT3
<riel> my cpu is slower
<ShawnCONSOLE> but you have fast disks?
<riel> so it doesn't fall idle as quickly as yours, when waiting on the disk <riel> not very fast ;)
<riel> old 8 GB IDE disk
<ShawnCONSOLE> so having a fast disk and a fast CPU causes the cpu to wait longer cause the disk finishes its tasks much faster then the cpu expects? <ShawnCONSOLE> mem load final test = 78%
<ShawnCONSOLE> so final numbers:
<ShawnCONSOLE> 99, 80%, 87%, 83%, 75%
<riel> yes, a very fast CPU falls idle more quickly
<riel> but it's very curious that ext3 is that much worse than ext2 <ShawnCONSOLE> thats much better.
<riel> definately worth pointing out to the ext3 maintainers.
Re: [BENCHMARK] EXT3 vs EXT2 results with rmap14a and testing with contest 0.34
Legend:
Kernel Time CPU
2.4.20-pre7-rmap14a-xfs-uml-shawn12d(EXT2)
2.4.20-pre7-rmap14a-xfs-uml-shawn12d(EXT3)
Quoted 87 lines · Shawn Starr
Sorry about the confusing email before. This should make more sense =)
These results compare EXT3 against EXT2 with rmap using the contest tool you can get it at: http://contest.kolivas.net
These tests are from a Athlon MP 2000+ w/ 512MB RAM
noload:
Kernel Time CPU
2.4.20-pre7-rmap14a-xfs-uml-shawn12d 259.47 99%
2.4.20-pre7-rmap14a-xfs-uml-shawn12d 267.66 97%process load:
Kernel Time CPU
2.4.20-pre7-rmap14a-xfs-uml-shawn12d 318.91 80% 2.4.20-pre7-rmap14a-xfs-uml-shawn12d 324.44 79%io halfmem:
Kernel Time CPU
2.4.20-pre7-rmap14a-xfs-uml-shawn12d 306.82 87% 2.4.20-pre7-rmap14a-xfs-uml-shawn12d 461.74 57%io full mem:
Kernel Time CPU
2.4.20-pre7-rmap14a-xfs-uml-shawn12d 325.39 82% 2.4.20-pre7-rmap14a-xfs-uml-shawn12d 411.47 64%full logs of the tests are:
WITH EXT2
------------
noload Time: 259.47 CPU: 99% Major Faults: 770937 Minor Faults: 1173705 process load Time: 318.91 CPU: 80% Major Faults: 742261 Minor Faults:
1169516
io halfmem Time: 306.82 CPU: 87% Major Faults: 742000 Minor Faults:
1169497 Was writing number 33 of a 257Mb sized io load file after 307 seconds io fullmem Time: 325.39 CPU: 82% Major Faults: 742000 Minor
Faults: 1169494 Was writing number 16 of a 514Mb sized io load file after 337 seconds mem load Time: 340.32 CPU: 79% Major Faults: 743307 Minor
Faults: 1170011WITH EXT3
-----------noload Time: 267.66 CPU: 97% Major Faults: 771111 Minor Faults: 1173722 process load Time: 324.44 CPU: 79% Major Faults: 742261 Minor Faults:
1169518
io halfmem Time: 461.74 CPU: 57% Major Faults: 742000 Minor Faults:
1169496 Was writing number 34 of a 257Mb sized io load file after 465 seconds io fullmem Time: 411.47 CPU: 64% Major Faults: 742000 Minor
Faults: 1169494 Was writing number 15 of a 514Mb sized io load file after 425 seconds mem load Time: 333.99 CPU: 81% Major Faults: 743320 Minor
Faults: 1170021NOTES:
====As you can see, there's something DEFINATELY wrong here. EXT3 is much slower then EXT2. I converted the EXT3 disk back to EXT2 to do the second test.
Also, I specified no mount options for EXT3 (which means it uses ordered mode). The journal was created with tune2fs -j /dev/hda#
From #Kernelnewbies (snip)
==============
<ShawnCONSOLE> riel uses EXT3
<riel> my cpu is slower
<ShawnCONSOLE> but you have fast disks?
<riel> so it doesn't fall idle as quickly as yours, when waiting on the disk <riel> not very fast ;)
<riel> old 8 GB IDE disk
<ShawnCONSOLE> so having a fast disk and a fast CPU causes the cpu to wait longer cause the disk finishes its tasks much faster then the cpu expects? <ShawnCONSOLE> mem load final test = 78%
<ShawnCONSOLE> so final numbers:
<ShawnCONSOLE> 99, 80%, 87%, 83%, 75%
<riel> yes, a very fast CPU falls idle more quickly
<riel> but it's very curious that ext3 is that much worse than ext2 <ShawnCONSOLE> thats much better.
<riel> definately worth pointing out to the ext3 maintainers.
Re: [BENCHMARK] EXT3 vs EXT2 results with rmap14a and testing with contest 0.34
In reply to Shawn StarrQuoted 20 lines · Shawn Starr
These results compare EXT3 against EXT2 with rmap using the contest tool you can get it at: http://contest.kolivas.net
These tests are from a Athlon MP 2000+ w/ 512MB RAM
noload:
Kernel Time CPU
2.4.20-pre7-rmap14a-xfs-uml-shawn12d 259.47 99%
2.4.20-pre7-rmap14a-xfs-uml-shawn12d 267.66 97%
process_load:Kernel Time CPU
2.4.20-pre7-rmap14a-xfs-uml-shawn12d 318.91 80% 2.4.20-pre7-rmap14a-xfs-uml-shawn12d 324.44 79%
io_halfmem:Kernel Time CPU
Quoted 8 lines
2.4.20-pre7-rmap14a-xfs-uml-shawn12d 306.82 87% 2.4.20-pre7-rmap14a-xfs-uml-shawn12d 461.74 57%io full mem:
Kernel Time CPU
2.4.20-pre7-rmap14a-xfs-uml-shawn12d 325.39 82% 2.4.20-pre7-rmap14a-xfs-uml-shawn12d 411.47 64%
I don't see this as hugely surprising. ext3 uses more CPU than ext2. If you are using up the CPU doing other things, then naturally ext3 will take a longer wall-clock time to complete the same tasks as ext2.
I know that Andrew has been doing a bunch of work to reduce ext3 CPU usage/locking/etc., but I think that is all in 2.5 kernels.
Cheers, Andreas
--
Andreas Dilger
http://www-mddsp.enel.ucalgary.ca/People/adilger/
http://sourceforge.net/projects/ext2resize/
Re: [BENCHMARK] EXT3 vs EXT2 results with rmap14a and testing with contest 0.34
Quoted 9 lines · Andreas Dilger
...
Quoted 3 lines
Kernel Time CPU 2.4.20-pre7-rmap14a-xfs-uml-shawn12d 325.39 82% 2.4.20-pre7-rmap14a-xfs-uml-shawn12d 411.47 64%I don't see this as hugely surprising. ext3 uses more CPU than ext2. If you are using up the CPU doing other things, then naturally ext3 will take a longer wall-clock time to complete the same tasks as ext2.
Yup. But here the CPU load is less; obviously some more seeking was done. That's fairly normal for ext3 - it has to write the journal as well as the filesystem....
Quoted 3 lines
I know that Andrew has been doing a bunch of work to reduce ext3 CPU usage/locking/etc., but I think that is all in 2.5 kernels.
I had a little patch. Stephen is working on the big fix.
Re: [BENCHMARK] EXT3 vs EXT2 results with rmap14a and testing with contest 0.34
Hi,
Quoted 1 line · Andrew Morton
I had a little patch. Stephen is working on the big fix.
It passed an overnight Cerberus at the end of last week. :-) Checking into CVS shortly, then I need to set up a pile of recovery tests to make sure it's still writing everything it needs to in time.
--Stephen
Re: [BENCHMARK] EXT3 vs EXT2 results with rmap14a and testing with contest 0.34
Which branch of the kernel is this going into? an -ac branch or 2.5 bk?
Quoted 10 lines · Stephen C. Tweedie
Hi,
Quoted 1 line · Andrew Morton
I had a little patch. Stephen is working on the big fix.
It passed an overnight Cerberus at the end of last week. :-) Checking into CVS shortly, then I need to set up a pile of recovery tests to make sure it's still writing everything it needs to in time.
--Stephen
Re: [BENCHMARK] EXT3 vs EXT2 results with rmap14a and testing with contest 0.34
Quoted 5 lines · Shawn Starr
Quoted 3 lines · Stephen C. Tweedie
Hi,
On Wed, Sep 18, 2002 at 11:32:19PM -0700, Andrew Morton wrote:
Quoted 9 lines
Quoted 6 lines
Quoted 1 line
I had a little patch. Stephen is working on the big fix.
It passed an overnight Cerberus at the end of last week. :-) Checking into CVS shortly, then I need to set up a pile of recovery tests to make sure it's still writing everything it needs to in time.
Which branch of the kernel is this going into? an -ac branch or 2.5 bk?
The way it usually goes is that Stephen checks it into ext3 CVS (based off current Marcelo kernel) and I port it up to 2.5.
Re: [BENCHMARK] EXT3 vs EXT2 results with rmap14a and testing with contest 0.34
In reply to Shawn StarrHi,
Quoted 1 line · Shawn Starr
Which branch of the kernel is this going into? an -ac branch or 2.5 bk?
Ext3 CVS, but I'll also make a clean, broken-down patchset for 2.4.19 bk.
In fact, the only reason it's not in ext3 cvs already is because it's such a pain keeping discrete broken-down patches separate using a form of SCM like bk or cvs; having decent scripts to track multiple patches in a working source tree just works a lot better for one developer when it comes to merging stuff upstream.
Cheers,
Stephen
Re: [BENCHMARK] EXT3 vs EXT2 results with rmap14a and testing with contest 0.34
In reply to Andreas DilgerWell, all the tests were done in single user mode =)
Quoted 42 lines · Andreas Dilger
Quoted 20 lines · Shawn Starr
These results compare EXT3 against EXT2 with rmap using the contest tool you can get it at: http://contest.kolivas.net
These tests are from a Athlon MP 2000+ w/ 512MB RAM
noload:
Kernel Time CPU
2.4.20-pre7-rmap14a-xfs-uml-shawn12d 259.47 99%
2.4.20-pre7-rmap14a-xfs-uml-shawn12d 267.66 97%
process_load:Kernel Time CPU
2.4.20-pre7-rmap14a-xfs-uml-shawn12d 318.91 80% 2.4.20-pre7-rmap14a-xfs-uml-shawn12d 324.44 79%
io_halfmem:Kernel Time CPU
Quoted 8 lines
2.4.20-pre7-rmap14a-xfs-uml-shawn12d 306.82 87% 2.4.20-pre7-rmap14a-xfs-uml-shawn12d 461.74 57%io full mem:
Kernel Time CPU
2.4.20-pre7-rmap14a-xfs-uml-shawn12d 325.39 82% 2.4.20-pre7-rmap14a-xfs-uml-shawn12d 411.47 64%I don't see this as hugely surprising. ext3 uses more CPU than ext2. If you are using up the CPU doing other things, then naturally ext3 will take a longer wall-clock time to complete the same tasks as ext2.
I know that Andrew has been doing a bunch of work to reduce ext3 CPU usage/locking/etc., but I think that is all in 2.5 kernels.
Cheers, Andreas
--
Andreas Dilger
http://www-mddsp.enel.ucalgary.ca/People/adilger/
http://sourceforge.net/projects/ext2resize/