Skip to contentSkip to conversation

Interbench v0.21

ANNOUNCE Con Kolivas 3 participants 5 messages Jul 15, 2005 – Jul 17, 2005

lore
Share & tools
lore
#0 lore raw

Interbech is a an application is designed to benchmark interactivity in Linux.

Version 0.21 update

http://ck.kolivas.org/apps/interbench/interbench-0.21.tar.bz2

Changes:

Changed the design to run the benchmarked and background loads as separate processes that spawn their own threads instead of everything running as a thread of the same process. This was suggested to me originally by Ingo Molnar who noticed significant slowdown due to conflict over ->mm->mmap_sem, invalidating the benchmark results when run in real time mode. This makes a large difference to the latencies measured under mem_load particularly when running real time benchmarks on a RT-PREEMPT kernel.

Accounting changes to max_latency to only measure the largest latency of a single scheduling frame - this makes max_latency much smaller (and probably more realistic). Often you may see max latency exactly one frame wide now (consistent with one dropped frame) such as 16.7ms on video.

Minor cleanups.

Cheers,
Con

#1 lore raw

Con Kolivas wrote: {
Version 0.21 update
Changed the design to run the benchmarked and background loads as separate processes that spawn their own threads instead of everything running as a thread of the same process.
}

Great!

#2 lore raw
In reply to Con Kolivas
Quoted 2 lines · Con Kolivas

his makes a large difference to the latencies measured under mem_load particularly when running real time benchmarks on a RT-PREEMPT kernel

Here are some results from my 600MHz C3. In realtime mode, the PREEMPT_RT kernel performs as expected, max latencies are around 55 usecs with a very tight distribution.

Based on what we already know about the RT kernel I think this validates the benchmark. So the numbers Con posted showing an interactivity regression from HZ=250 should be taken seriously.

Lee

Realtime mode:

rlrevell@mindpipe:~/kernel-source/interbench-0.21$ ./interbench -r -t 5
84648 loops_per_ms read from file interbench.loops_per_ms

Using 84648 loops per ms, running every load for 10 seconds Benchmarking kernel 2.6.12-RT-V0.7.51-28 at datestamp 200507160215

--- Benchmarking Audio real time in the presence of loads ---
Latency +/- SD (us) Max Latency % Desired CPU % Deadlines Met

None         35 +/- 3.85          41             100            100
Video        36 +/- 4.15          51             100            100
X            37 +/- 4.39          49             100            100
Burn         34 +/- 4.19          50             100            100
Write        40 +/- 4.9           52             100            100
Read         38 +/- 3.36          46             100            100
Compile      40 +/- 4.11          54             100            100
Memload      41 +/- 4.24          51             100            100

--- Benchmarking Video real time in the presence of loads ---
Latency +/- SD (us) Max Latency % Desired CPU % Deadlines Met

None         29 +/- 4.15          42             100            100
X            28 +/- 3.52          46             100            100
Burn         28 +/- 3.37          41             100            100
Write        41 +/- 3.25          59             100            100
Read         37 +/- 3.07          43             100            100
Compile      36 +/- 4.99          54             100            100
Memload      38 +/- 3.39          48             100            100

Non realtime mode:

rlrevell@mindpipe:~/kernel-source/interbench-0.21$ ./interbench -t 5
84648 loops_per_ms read from file interbench.loops_per_ms

Using 84648 loops per ms, running every load for 10 seconds Benchmarking kernel 2.6.12-RT-V0.7.51-28 at datestamp 200507160237

--- Benchmarking Audio in the presence of loads ---
Latency +/- SD (ms) Max Latency % Desired CPU % Deadlines Met

None      0.042 +/- 0.0234     0.176             100            100
Video     0.121 +/- 0.926         13             100            100
X          1.35 +/- 2.93        19.4             100            100
Burn      0.067 +/- 0.215       3.02             100            100
Write     0.763 +/- 2.18        16.9             100            100
Read      0.263 +/- 1.12        9.01             100            100
Compile   0.216 +/- 1.06         9.2             100            100
Memload   0.541 +/- 1.86        13.1             100            100

--- Benchmarking Video in the presence of loads ---
Latency +/- SD (ms) Max Latency % Desired CPU % Deadlines Met

None      0.428 +/- 1.07          17             100           97.2
X           4.6 +/- 4.23          50             100           68.3
Burn      0.394 +/- 1.18        16.8             100           97.8
Write      1.31 +/- 2.05        39.4             100             93
Read      0.462 +/- 0.809       18.2             100           96.8
Compile   0.991 +/- 1.67        42.3             100           94.2
Memload   0.558 +/- 0.949       18.9             100           97.2

--- Benchmarking X in the presence of loads ---
Latency +/- SD (ms) Max Latency % Desired CPU % Deadlines Met

None      0.599 +/- 0.599         24             100             95
Video      13.3 +/- 13.4          93             100             64
Burn      0.519 +/- 0.52          15             100             94
Write      1.91 +/- 1.91          77             100             91
Read      0.449 +/- 0.45          20             100             95
Compile    1.01 +/- 1.04          30             100             93
Memload    1.26 +/- 1.26          30             100             88
#3 lore raw
In reply to Con Kolivas
Quoted 6 lines · Con Kolivas

Interbech is a an application is designed to benchmark interactivity in Linux.

Version 0.21 update

http://ck.kolivas.org/apps/interbench/interbench-0.21.tar.bz2

I would suggest using microseconds for both the RT and non RT tests. It would allow easier comparison of results. I have a pretty slow machine and the max result would only be ~44000 usecs.

Also, if it's run with -r and sched_setscheduler fails, rather than saying "you must be root for SCHED_FIFO" the error message should encourage the user to try a 2.6.12+ kernel and add themselves to the "audio" or "realtime" group, and to file a feature request if their distro does not support the new realtime rlimit feature.

We should encourage more applications to take advantage of, and distros to support, the non-root RT scheduling available in 2.6.12+. I really think the kernel is good enough at this point that we could achieve OSX-like multimedia performance on the desktop if more apps like xmms, xine, and mplayer were to adopt a multithreaded model with the time-critical rendering threads running RT. XMMS recently adopted such a model, but I don't think the audio thread runs SCHED_FIFO yet. These benchmarks imply that it would be a massive improvement.

Lee

#4 lore raw
Quoted 11 lines · Lee Revell
Quoted 6 lines · Con Kolivas

Interbech is a an application is designed to benchmark interactivity in Linux.

Version 0.21 update

http://ck.kolivas.org/apps/interbench/interbench-0.21.tar.bz2

I would suggest using microseconds for both the RT and non RT tests. It would allow easier comparison of results. I have a pretty slow machine and the max result would only be ~44000 usecs.

I think the significance of usec values from the non-rt tests makes this an inappropriate thing to do. The variation in results will be greater than usec resolution.

Quoted 6 lines

Also, if it's run with -r and sched_setscheduler fails, rather than saying "you must be root for SCHED_FIFO" the error message should encourage the user to try a 2.6.12+ kernel and add themselves to the "audio" or "realtime" group, and to file a feature request if their distro does not support the new realtime rlimit feature.

Quoted 8 lines

We should encourage more applications to take advantage of, and distros to support, the non-root RT scheduling available in 2.6.12+. I really think the kernel is good enough at this point that we could achieve OSX-like multimedia performance on the desktop if more apps like xmms, xine, and mplayer were to adopt a multithreaded model with the time-critical rendering threads running RT. XMMS recently adopted such a model, but I don't think the audio thread runs SCHED_FIFO yet. These benchmarks imply that it would be a massive improvement.

While I agree with you in principal on getting the rlimit feature working and supported, this benchmark is meant to be run in single user mode for most reproducible and valid results. However, clearly there will be people using it cautiously as a normal user first. I originally did not include the information that you need to be root in v.20 and said in the documentation "need rt privileges" but within about 5 minutes of posting it I had someone not understanding what "unable to get SCHED_FIFO" meant. I guess a more verbose message will be required explaining non-root RT as well.

Cheers,
Con