Unusually high system CPU usage with recent kernels
Quoted 53 lines
From: Paul E. McKenney Sent: 08/24/13 02:18 AM
Quoted 51 lines · Tibor Billes
Quoted 50 lines
From: Paul E. McKenney Sent: 08/22/13 12:09 AM
Quoted 33 lines · Tibor Billes
Quoted 33 lines
From: Paul E. McKenney Sent: 08/21/13 09:12 PM
Quoted 31 lines · Tibor Billes
Quoted 30 lines
From: Paul E. McKenney Sent: 08/20/13 11:43 PM
Quoted 15 lines · Tibor Billes
Quoted 15 lines
From: Paul E. McKenney Sent: 08/20/13 04:53 PM
Quoted 12 lines · Tibor Billes
Hi,
I was using the 3.9.7 stable release and tried to upgrade to the 3.10.x series. The 3.10.x series was showing unusually high (>75%) system CPU usage in some situations, making things really slow. The latest stable I tried is 3.10.7. I also tried 3.11-rc5, they both show this behaviour. This behaviour doesn't show up when the system is idling, only when doing some CPU intensive work, like compiling with multiple threads. Compiling with only one thread seems not to trigger this behaviour.
To be more precise I did a
perf record -awhile compiling a large C++ program with scons using 4 threads, the result is appended at the end of this email.Quoted 6 lines
Quoted 4 lines
New one on me! You are running a mainstream system (
x86_64), so I am surprised no one else noticed.Could you please send along your .config file?
Here it is
Interesting. I don't see RCU stuff all that high on the list, but the items I do see lead me to suspect
RCU_FAST_NO_HZ, which has some relevance to the otherwise inexplicable group of commits you located with your bisection. Could you please rerun withCONFIG_RCU_FAST_NO_HZ=n?If that helps, there are some things I could try.
Quoted 8 lines
Quoted 5 lines
Quoted 1 line
It did help. I didn't notice anything unusual when running with
CONFIG_RCU_FAST_NO_HZ=n.Interesting. Thank you for trying this -- and we at least have a short-term workaround for this problem. I will put a patch together for further investigation.
I don't specifically need this config option so I'm fine without it in the long term, but I guess it's not supposed to behave like that.
OK, good, we have a long-term workload for your specific case, even better. ;-)
But yes, there are situations where
RCU_FAST_NO_HZneeds to work a bit better. I hope you will bear with me with a bit more testing...
Quoted 8 lines
Quoted 3 lines
Don't worry, I will :) Unfortunately I didn't have time yesterday and I won't have time today either. But I'll do what you asked tomorrow and I'll send you the results.
Not a problem! I did find one issue that -might- help, please see the patch below. (Run with
CONFIG_RCU_FAST_NO_HZ=y.)Please let me know how it goes!
I applied the patch on top of 3.11-rc6, but it did not help, I'm still hitting this issue.
Let me know if you have anything else that I can test!
Tibor