Skip to contentSkip to conversation

WimMark I report for 2.5.65-mm2 (now with deadline)

DISCUSSION Joel Becker 3 participants 5 messages Mar 20, 2003 – Mar 21, 2003

lore
Share & tools
lore
#0 lore raw

WimMark I report for 2.5.65-mm2

Runs (antic):  1374.22 1487.19 1437.26
Runs (deadline):  1238.58 1537.36 1513.04

WimMark I is a rough benchmark we have been running
here at Oracle against various kernels. Each run tests an OLTP workload on the Oracle database with somewhat restrictive memory conditions. This reduces in-memory buffering of data, allowing for more I/O. The I/O is read and sync write, random and seek-laden.
The benchmark is called "WimMark I" because it has no
official standing and is only a relative benchmark useful for comparing kernel changes. The benchmark is normalized an arbitrary kernel, which scores 1000.0. All other numbers are relative to this.
The machine in question is a 4 way 700 MHz Xeon machine with 2GB
of RAM. CONFIG_HIGHMEM4GB is selected. The disk accessed for data is a 10K RPM U2W SCSI of similar vintage. The data files are living on an ext3 filesystem. Unless mentioned, all runs are
on this machine (variation in hardware would indeed change the benchmark).

Signature · 10 lines
"Always give your best, never get discouraged, never be petty; always
 remember, others may hate you.  Those who hate you don't win unless
 you hate them.  And then you destroy yourself."
	- Richard M. Nixon

Joel Becker
Senior Member of Technical Staff
Oracle Corporation
E-mail: [email protected]
Phone: (650) 506-8127
#1 lore raw
Quoted 6 lines · Joel Becker

WimMark I report for 2.5.65-mm2

Runs (antic):  1374.22 1487.19 1437.26
Runs (deadline):  1238.58 1537.36 1513.04

The averages of these are equal. Can we safely conclude that this is fixed up now?

#2 lore raw
Quoted 7 lines · Andrew Morton
Quoted 4 lines

WimMark I report for 2.5.65-mm2

Runs (antic):  1374.22 1487.19 1437.26
Runs (deadline):  1238.58 1537.36 1513.04

The averages of these are equal. Can we safely conclude that this is fixed up now?

Not really, I think, because the 1238 seems a slightly fluke
run. I see these from time to time, but almost all of the deadline runs are in the 1500-1600 range. In fact, -mm2 is lower than some other kernels by about 50 for deadline.
I wouldn't disagree with you if I didn't see that consistency.
antic has never really jumped above 1500, and deadline almost never goes below.
Certainly they are much closer than they were.

Joel

Signature · 9 lines
Life's Little Instruction Book #109

	"Know how to drive a stick shift."

Joel Becker
Senior Member of Technical Staff
Oracle Corporation
E-mail: [email protected]
Phone: (650) 506-8127
#3 lore raw
Quoted 11 lines · Joel Becker
Quoted 8 lines · Andrew Morton
Quoted 5 lines

WimMark I report for 2.5.65-mm2

Runs (antic):  1374.22 1487.19 1437.26
Runs (deadline):  1238.58 1537.36 1513.04

The averages of these are equal. Can we safely conclude that this is fixed up now?

Quoted 9 lines

Not really, I think, because the 1238 seems a slightly fluke
run. I see these from time to time, but almost all of the deadline runs are in the 1500-1600 range. In fact, -mm2 is lower than some other kernels by about 50 for deadline.
I wouldn't disagree with you if I didn't see that consistency.
antic has never really jumped above 1500, and deadline almost never goes below.
Certainly they are much closer than they were.

The smaller first runs are not due to the benchmark running for the first

Quoted 5 lines · time, are they? In your mm1 tests you wrote:

WimMark I report for 2.5.65-mm1

Runs (antic):  1559.32 1025.38 1579.98
Runs (deadline):  1554.48 1589.89 1350.37

So it does seem to be quite varied, but yes I'll keep working on it. BTW. how do these results compare with 2.4 and other operating systems on the same hardware, out of interest?

#4 lore raw
Quoted 2 lines · Nick Piggin

The smaller first runs are not due to the benchmark running for the first time, are they? In your mm1 tests you wrote:

The benchmark has a ramp-up period built-in to populate the
database caches. The first run is not always the slowest.
The thing is, the runs can be sensitive to some things (someone
logs in and runs something, Linux decides to flush some cache, etc). The runs take long enough without extending them to flatten this some.

Quoted 2 lines
Quoted 2 lines
Runs (antic):  1559.32 1025.38 1579.98
Runs (deadline):  1554.48 1589.89 1350.37

If you notice, the variance always is of a fluke sort. This is
why I do multiple runs. The non-fluke runs are very consistent. See runs 1 and 3 for antic and 1 and 2 for deadline in the quote.

Quoted 3 lines

So it does seem to be quite varied, but yes I'll keep working on it. BTW. how do these results compare with 2.4 and other operating systems on the same hardware, out of interest?

I've not run other operating systems, as I don't have the
software or OSes installed. I've not run a vanilla 2.4 recently, and probably should do that.

Joel

Signature · 9 lines
 "I'm living so far beyond my income that we may almost be said
 to be living apart."
         - e e cummings

Joel Becker
Senior Member of Technical Staff
Oracle Corporation
E-mail: [email protected]
Phone: (650) 506-8127