Skip to contentSkip to conversation

Statistical representation of IO load results with contest

BENCHMARK Con Kolivas 3 participants 3 messages Sep 23, 2002 – Sep 24, 2002

lore
Share & tools
lore
#0 lore raw

Thank you all who responded with suggestions on how to get useful data out of the IO load module from contest. These are _new_ results with a sync,swapoff,swapon before conducting just the IO load. I have digested all your suggestions and come up with the following:

n=5 for number of samples

Kernel          Mean    CI(95%)
2.5.38          411     344-477
2.5.39-gcc32    371     224-519
2.5.38-mm2      95      84-105

The mean is a simple average of the results, and the CI(95%) are the 95% confidence intervals the mean lies between those numbers. These numbers seem to be the most useful for comparison.

Comparing 2.5.38(gcc2.95.3) with 2.5.38(gcc3.2) there is NO significant difference (p 0.56)

Comparing 2.5.38 with 2.5.38-mm2 there is a significant diffence (p<0.001)

After playing with all these it appears I should do the following to contest:

Add sync,swapoff,swapon before each load
Perform noload and process_load twice to ensure no abnormal results Perform mem_load 3 times
Perform IO_fullmem 5 times (and rename it just IO_load) Drop IO_halfmem (adds no more useful information and just adds time). Do a statistical analysis like the above when posting information.

Comments?

Con

#1 lore raw
Quoted 1 line · Con Kolivas

Do a statistical analysis like the above when posting information.

That's really the most important piece, imo.

Signature · 2 lines
Ryan Anderson
  sometimes Pug Majere
#2 lore raw
In reply to Con Kolivas
Quoted 11 lines · Con Kolivas

Thank you all who responded with suggestions on how to get useful data out of the IO load module from contest. These are _new_ results with a sync,swapoff,swapon before conducting just the IO load. I have digested all your suggestions and come up with the following:

n=5 for number of samples

Kernel          Mean    CI(95%)
2.5.38          411     344-477
2.5.39-gcc32    371     224-519
2.5.38-mm2      95      84-105

/me wonders what other benchmarks numbers will look like. (db stuff should rock?)

-Mike