Skip to contentSkip to conversation

Unusually high system CPU usage with recent kernels

DISCUSSION Tibor Billes 2 participants 2 messages Aug 21, 2013

lore
Share & tools
lore
#0 lore raw
Quoted 22 lines

From: Paul E. McKenney Sent: 08/20/13 11:43 PM

Quoted 20 lines · Tibor Billes
Quoted 19 lines

From: Paul E. McKenney Sent: 08/20/13 04:53 PM

Quoted 10 lines · Tibor Billes

Hi,

I was using the 3.9.7 stable release and tried to upgrade to the 3.10.x series. The 3.10.x series was showing unusually high (>75%) system CPU usage in some situations, making things really slow. The latest stable I tried is 3.10.7. I also tried 3.11-rc5, they both show this behaviour. This behaviour doesn't show up when the system is idling, only when doing some CPU intensive work, like compiling with multiple threads. Compiling with only one thread seems not to trigger this behaviour.

Quoted 2 lines

To be more precise I did a perf record -a while compiling a large C++ program with scons using 4 threads, the result is appended at the end of this email.

New one on me! You are running a mainstream system (x86_64), so I am surprised no one else noticed.

Could you please send along your .config file?

Quoted 8 lines
Quoted 1 line

Here it is

Interesting. I don't see RCU stuff all that high on the list, but the items I do see lead me to suspect RCU_FAST_NO_HZ, which has some relevance to the otherwise inexplicable group of commits you located with your bisection. Could you please rerun with CONFIG_RCU_FAST_NO_HZ=n?

If that helps, there are some things I could try.

It did help. I didn't notice anything unusual when running with CONFIG_RCU_FAST_NO_HZ=n.

Tibor

#1 lore raw
Quoted 24 lines · Tibor Billes
Quoted 24 lines

From: Paul E. McKenney Sent: 08/20/13 11:43 PM

Quoted 21 lines · Tibor Billes
Quoted 12 lines

From: Paul E. McKenney Sent: 08/20/13 04:53 PM

Quoted 10 lines · Tibor Billes

Hi,

I was using the 3.9.7 stable release and tried to upgrade to the 3.10.x series. The 3.10.x series was showing unusually high (>75%) system CPU usage in some situations, making things really slow. The latest stable I tried is 3.10.7. I also tried 3.11-rc5, they both show this behaviour. This behaviour doesn't show up when the system is idling, only when doing some CPU intensive work, like compiling with multiple threads. Compiling with only one thread seems not to trigger this behaviour.

Quoted 7 lines
Quoted 2 lines

To be more precise I did a perf record -a while compiling a large C++ program with scons using 4 threads, the result is appended at the end of this email.

New one on me! You are running a mainstream system (x86_64), so I am surprised no one else noticed.

Could you please send along your .config file?

Here it is

Quoted 8 lines
Quoted 6 lines

Interesting. I don't see RCU stuff all that high on the list, but the items I do see lead me to suspect RCU_FAST_NO_HZ, which has some relevance to the otherwise inexplicable group of commits you located with your bisection. Could you please rerun with CONFIG_RCU_FAST_NO_HZ=n?

If that helps, there are some things I could try.

It did help. I didn't notice anything unusual when running with CONFIG_RCU_FAST_NO_HZ=n.

Interesting. Thank you for trying this -- and we at least have a short-term workaround for this problem. I will put a patch together for further investigation.

In the meantime, could you please tell me how you were measuring performance for your kernel builds? Wall-clock time required to complete one build? Number of builds completed per unit time? Something else?

Thanx, Paul