Skip to contentSkip to conversation

Latency issues with x86.git

DISCUSSION Kevin Winchester 2 participants 2 messages Feb 14, 2008

lore
Share & tools
lore
#0 lore raw

Hi Ingo,

I have encountered (a handful of times in the past few months) some real interactivity problems on my system. Moving the mouse or typing a key on the keyboard takes around a second to show any response. Once I perform a reboot, the problem is gone again. I am currently running x86.git mm branch, but I switch between that branch, mainline git, and mm kernels, so I cannot guarantee on which trees I have or have not seen the problem.

It seems to have started around the time that CFS came into being, but it might be something completely unrelated.

When it happened this evening, I ran the cfs-debug-info.sh script to which you had referred me in another thread. I'm not sure if this helps to debug the problem, but it was all I could think to try. I have LatencyTOP in my kernel, so I guess I could try the userspace tool as well the next time.

The output of the script is at:

http://personal.nbnet.nb.ca/kwin/cfs-debug-info-2008.02.13-20.56.38

Unfortunately, I cannot reproduce the problem intentionally - it only happens once a month or so.

If there is any other info you need, please let me know, or any suggestions for what to try the next time.

Thanks,

Signature · 1 line
Kevin Winchester
#1 lore raw
Quoted 9 lines · Kevin Winchester

Hi Ingo,

I have encountered (a handful of times in the past few months) some real interactivity problems on my system. Moving the mouse or typing a key on the keyboard takes around a second to show any response. Once I perform a reboot, the problem is gone again. I am currently running x86.git mm branch, but I switch between that branch, mainline git, and mm kernels, so I cannot guarantee on which trees I have or have not seen the problem.

please try sched-devel.git, which has both the latest scheduler fixes, and also the new "ftrace" latency tracing framework that can be used to trace various latency problems. You can pick up sched-devel.git via:

http://people.redhat.com/mingo/sched-devel.git/README

firstly, there's a chance that sched-devel.git solves the problem - in that case please report it.

if it doesnt, then you can trace various latencies via these:

CONFIG_FTRACE=y
CONFIG_IRQSOFF_TRACER=y
CONFIG_SCHED_TRACER=y
CONFIG_CONTEXT_SWITCH_TRACER=y
CONFIG_DYNAMIC_FTRACE=y

just enable them, boot into the new kernel and mount debugfs:

mount -t debugfs nodev /debug

and then you can select one of the tracers (that have been enabled in the .config) via /debug/tracing/* files. The usage of these files should be self-explaining - if any of them wasnt then please let us know and we'll make the "first quick glance experience" better :)

the one interesting to you would be the "wakeup" tracer. Enable it, and if you echo 0 into /debug/tracing/tracing_max_latency it starts tracking the worst-case delay experienced on your system and should start reporting them to the syslog.

if you encounter any problems during these steps then please let us know
- this code is quite fresh so expect some rough edges.

Ingo