Skip to contentSkip to conversation

2.5.39 with contest 0.41

BENCHMARK Con Kolivas 3 participants 6 messages Sep 28, 2002

lore
Share & tools
lore
#0 lore raw

Here follow the latest benchmarks with contest (http://contest.kolivas.net)

noload:

Kernel                  Time            CPU             Ratio
2.4.19                  67.71           98%             1.00*
2.5.38                  72.38           94%             1.07
2.5.38-mm3              73.00           93%             1.08
2.5.39                  73.17           93%             1.08

process_load:

Kernel                  Time            CPU             Ratio
2.4.19                  110.75          57%             1.64*
2.5.38                  85.71           79%             1.27
2.5.38-mm3              96.32           72%             1.42*
2.5.39                  88.18           77%             1.30

io_load:

Kernel                  Time            CPU             Ratio
2.4.19                  216.05          33%             3.19
2.5.38                  887.76          8%              13.11*
2.5.38-mm3              105.17          70%             1.55*
2.5.39                  216.81          37%             3.20

mem_load:

Kernel                  Time            CPU             Ratio
2.4.19                  105.40          70%             1.56
2.5.38                  107.89          73%             1.59
2.5.38-mm3              117.09          63%             1.73*
2.5.39                  103.72          72%             1.53

Things to note:
Despite the new deadline scheduler, performance under IO load is worse than 2.5.38-mm3 (something else?)

Asterisks are placed where the difference was statistically significant from 2.5.39

Hardware: 1133Mhz P3, 224Mb Ram, IDE ATA100 5400rpm drive, IO load on same drive as compile.

*NOTE* New version of contest means results are not compatible with results of 0.3x. Process Load now more of a process load (and less of a cpu load), and changed priming (memory flushing) prior to each test results in far greater resolution of results - hence the notable change in even noload results.

Comments?
Con.

#1 lore raw
Quoted 10 lines · Con Kolivas

Here follow the latest benchmarks with contest (http://contest.kolivas.net)

noload:

Kernel                  Time            CPU             Ratio
2.4.19                  67.71           98%             1.00*
2.5.38                  72.38           94%             1.07
2.5.38-mm3              73.00           93%             1.08
2.5.39                  73.17           93%             1.08
Quoted 6 lines

process_load:

Kernel                  Time            CPU             Ratio
2.4.19                  110.75          57%             1.64*
2.5.38                  85.71           79%             1.27
2.5.38-mm3              96.32           72%             1.42*
2.5.39                  88.18           77%             1.30

well that's funny.

Quoted 6 lines

io_load:

Kernel                  Time            CPU             Ratio
2.4.19                  216.05          33%             3.19
2.5.38                  887.76          8%              13.11*
2.5.38-mm3              105.17          70%             1.55*
2.5.39                  216.81          37%             3.20

-mm3 has fifo_batch=16. 2.5.39 has fifo_batch=32.

Quoted 6 lines

mem_load:

Kernel                  Time            CPU             Ratio
2.4.19                  105.40          70%             1.56
2.5.38                  107.89          73%             1.59
2.5.38-mm3              117.09          63%             1.73*
2.5.39                  103.72          72%             1.53

2.5's swapout is still fairly synchronously sucky. So low-latency writeout could be advantageous there. This difference is probably also the fifo_batch thing. Or maybe statistical?

I did some testing with your latest. 4xPIII, mem=512m, SCSI, tag depth = 0, 2.5.39-mm1 candidate:

fifo_batch=32:

	noload          2:34.53         291%
	cpuload         2:36.20         286%
	memload         2:19.44         333%
	ioloadhalf      2:34.81         303%
	ioloadfull      3:15.47         238%

(err. memload sped it up!)

fifo_batch=16:

	noload          2:00.03         380%
	cpuload         2:27.62         304%
	memload         2:22.59         326%
	ioloadhalf      2:33.75         306%
	ioloadfull      2:59.18         259%

- Something went horridly wrong in the first `noload' test.
- fifo_batch=16 is better than 32.
- you see a 4x hit from io_load. I see a 1.5x hit.

This is all pretty wild. I'll go profile process_load a bit.

BTW, please change all the

	#define dprintf(...) printf(__VA_ARGS__)

to

	#define dprintf(x...) printf(x)

so people who use crufty old compilers can build it.

#2 lore raw

Quoting Andrew Morton <[email protected]>:

Quoted 73 lines
Quoted 2 lines · Con Kolivas

Here follow the latest benchmarks with contest

(http://contest.kolivas.net)

Quoted 8 lines

noload:

Kernel                  Time            CPU             Ratio
2.4.19                  67.71           98%             1.00*
2.5.38                  72.38           94%             1.07
2.5.38-mm3              73.00           93%             1.08
2.5.39                  73.17           93%             1.08
Quoted 6 lines

process_load:

Kernel                  Time            CPU             Ratio
2.4.19                  110.75          57%             1.64*
2.5.38                  85.71           79%             1.27
2.5.38-mm3              96.32           72%             1.42*
2.5.39                  88.18           77%             1.30

well that's funny.

Quoted 6 lines

io_load:

Kernel                  Time            CPU             Ratio
2.4.19                  216.05          33%             3.19
2.5.38                  887.76          8%              13.11*
2.5.38-mm3              105.17          70%             1.55*
2.5.39                  216.81          37%             3.20

-mm3 has fifo_batch=16. 2.5.39 has fifo_batch=32.

Quoted 6 lines

mem_load:

Kernel                  Time            CPU             Ratio
2.4.19                  105.40          70%             1.56
2.5.38                  107.89          73%             1.59
2.5.38-mm3              117.09          63%             1.73*
2.5.39                  103.72          72%             1.53

2.5's swapout is still fairly synchronously sucky. So low-latency writeout could be advantageous there. This difference is probably also the fifo_batch thing. Or maybe statistical?

I did some testing with your latest. 4xPIII, mem=512m, SCSI, tag depth = 0, 2.5.39-mm1 candidate:

fifo_batch=32:

	noload          2:34.53         291%
	cpuload         2:36.20         286%
	memload         2:19.44         333%
	ioloadhalf      2:34.81         303%
	ioloadfull      3:15.47         238%

(err. memload sped it up!)

fifo_batch=16:

	noload          2:00.03         380%
	cpuload         2:27.62         304%
	memload         2:22.59         326%
	ioloadhalf      2:33.75         306%
	ioloadfull      2:59.18         259%

- Something went horridly wrong in the first `noload' test.
- fifo_batch=16 is better than 32.
- you see a 4x hit from io_load. I see a 1.5x hit.

This is all pretty wild. I'll go profile process_load a bit.

BTW, please change all the

Quoted 8 lines
	#define dprintf(...) printf(__VA_ARGS__)

to

	#define dprintf(x...) printf(x)

so people who use crufty old compilers can build it.

Ok will fix. But please Andrew use version 0.41 of contest (posted only 2 hours ago). The results from that are far more meaningful and reproducible.

Con

#3 lore raw
Quoted 5 lines · Con Kolivas

...
Ok will fix. But please Andrew use version 0.41 of contest (posted only 2 hours ago). The results from that are far more meaningful and reproducible.

I am doing. I just tested for the "proces load" variation which
you saw (I assume that's what the test calls "CPU load"). With and without the -mm patches:

cpuload         2:31.55         297%
cpuload         2:31.75         295%

So... Maybe some extra samples would be needed there.

#4 lore raw
In reply to Con Kolivas
Quoted 8 lines · Con Kolivas
Quoted 8 lines
Quoted 6 lines

io_load:

Kernel                  Time            CPU             Ratio
2.4.19                  216.05          33%             3.19
2.5.38                  887.76          8%              13.11*
2.5.38-mm3              105.17          70%             1.55*
2.5.39                  216.81          37%             3.20

-mm3 has fifo_batch=16. 2.5.39 has fifo_batch=32.

That's not the only difference, btw.

Quoted 33 lines
Quoted 33 lines
Quoted 6 lines

mem_load:

Kernel                  Time            CPU             Ratio
2.4.19                  105.40          70%             1.56
2.5.38                  107.89          73%             1.59
2.5.38-mm3              117.09          63%             1.73*
2.5.39                  103.72          72%             1.53

2.5's swapout is still fairly synchronously sucky. So low-latency writeout could be advantageous there. This difference is probably also the fifo_batch thing. Or maybe statistical?

I did some testing with your latest. 4xPIII, mem=512m, SCSI, tag depth = 0, 2.5.39-mm1 candidate:

fifo_batch=32:

	noload          2:34.53         291%
	cpuload         2:36.20         286%
	memload         2:19.44         333%
	ioloadhalf      2:34.81         303%
	ioloadfull      3:15.47         238%

(err. memload sped it up!)

fifo_batch=16:

	noload          2:00.03         380%
	cpuload         2:27.62         304%
	memload         2:22.59         326%
	ioloadhalf      2:33.75         306%
	ioloadfull      2:59.18         259%
Quoted 3 lines
Quoted 3 lines

- Something went horridly wrong in the first `noload' test.
- fifo_batch=16 is better than 32.
- you see a 4x hit from io_load. I see a 1.5x hit.

So far fifo_batch=16 looks pretty good. Doesn't quite make sense to me. Need to bench/test some more :-)

Signature · 1 line
Jens Axboe
#5 lore raw

Quoting Jens Axboe <[email protected]>:

Quoted 45 lines · Con Kolivas
Quoted 8 lines
Quoted 8 lines
Quoted 6 lines

io_load:

Kernel                  Time            CPU             Ratio
2.4.19                  216.05          33%             3.19
2.5.38                  887.76          8%              13.11*
2.5.38-mm3              105.17          70%             1.55*
2.5.39                  216.81          37%             3.20

-mm3 has fifo_batch=16. 2.5.39 has fifo_batch=32.

That's not the only difference, btw.

Quoted 33 lines
Quoted 33 lines
Quoted 6 lines

mem_load:

Kernel                  Time            CPU             Ratio
2.4.19                  105.40          70%             1.56
2.5.38                  107.89          73%             1.59
2.5.38-mm3              117.09          63%             1.73*
2.5.39                  103.72          72%             1.53

2.5's swapout is still fairly synchronously sucky. So low-latency writeout could be advantageous there. This difference is probably also the fifo_batch thing. Or maybe statistical?

I did some testing with your latest. 4xPIII, mem=512m, SCSI, tag depth = 0, 2.5.39-mm1 candidate:

fifo_batch=32:

	noload          2:34.53         291%
	cpuload         2:36.20         286%
	memload         2:19.44         333%
	ioloadhalf      2:34.81         303%
	ioloadfull      3:15.47         238%

(err. memload sped it up!)

fifo_batch=16:

	noload          2:00.03         380%
	cpuload         2:27.62         304%
	memload         2:22.59         326%
	ioloadhalf      2:33.75         306%
	ioloadfull      2:59.18         259%
Quoted 6 lines
Quoted 3 lines
Quoted 3 lines

- Something went horridly wrong in the first `noload' test.
- fifo_batch=16 is better than 32.
- you see a 4x hit from io_load. I see a 1.5x hit.

So far fifo_batch=16 looks pretty good. Doesn't quite make sense to me. Need to bench/test some more :-)

Andrew was using an older version of contest which may have been misrepresenting things as there were serious limitations in the older versions. I've directed him to the new version which has worked around (most) of the limitations. SMP on the older version was particularly bad.

Con.