Large slowdown with 'x86: Avoid invoking RCU when CPU is idle'
Hi Paul,
Over in Fedora land, I applied your patch from this thread:
https://lkml.org/lkml/2012/1/24/441
to our 3.3-rc3/rc4 based rawhide kernels. The intention was to solve an RCU issue that was very similar to what Eric originally reported, and the RCU splat did indeed go away[1].
However, we then got a few reports of kernels containing that patch being extremely slow. When the patch was dropped, the slowness goes away according to one reporter. The details can be found in this bug:
https://bugzilla.redhat.com/show_bug.cgi?id=795050
The slowness doesn't seem to hit everyone, and in my local testing things seem to be working just fine. The reporters have widely varying hardware as well, so it doesn't seem machine specific.
Perhaps I misdiagnosed the original issue, or perhaps I missed something else that needs to be applied prior to this but I thought I would point this out in case you had any ideas.
josh
Quoted 16 lines · Josh Boyer
Hi Paul,
Over in Fedora land, I applied your patch from this thread:
https://lkml.org/lkml/2012/1/24/441
to our 3.3-rc3/rc4 based rawhide kernels. The intention was to solve an RCU issue that was very similar to what Eric originally reported, and the RCU splat did indeed go away[1].
However, we then got a few reports of kernels containing that patch being extremely slow. When the patch was dropped, the slowness goes away according to one reporter. The details can be found in this bug:
Quoted 7 lines
The slowness doesn't seem to hit everyone, and in my local testing things seem to be working just fine. The reporters have widely varying hardware as well, so it doesn't seem machine specific.
Perhaps I misdiagnosed the original issue, or perhaps I missed something else that needs to be applied prior to this but I thought I would point this out in case you had any ideas.
This patch has been obsoleted by patches #45-47 in this series:
https://lkml.org/lkml/2012/2/3/459
And patch #47 in that series has been obsoleted by another series from Steven Rostedt:
https://lkml.org/lkml/2012/2/7/231
Hopefully these fix both splats and slowness.
Thanx, Paul
Quoted 4 lines
josh
Quoted 17 lines · Paul E. McKenney
Quoted 16 lines · Josh Boyer
Hi Paul,
Over in Fedora land, I applied your patch from this thread:
https://lkml.org/lkml/2012/1/24/441
to our 3.3-rc3/rc4 based rawhide kernels. The intention was to solve an RCU issue that was very similar to what Eric originally reported, and the RCU splat did indeed go away[1].
However, we then got a few reports of kernels containing that patch being extremely slow. When the patch was dropped, the slowness goes away according to one reporter. The details can be found in this bug:
Quoted 7 lines
Quoted 7 lines
The slowness doesn't seem to hit everyone, and in my local testing things seem to be working just fine. The reporters have widely varying hardware as well, so it doesn't seem machine specific.
Perhaps I misdiagnosed the original issue, or perhaps I missed something else that needs to be applied prior to this but I thought I would point this out in case you had any ideas.
First off, thanks for the quick reply!
Quoted 3 lines
This patch has been obsoleted by patches #45-47 in this series:
https://lkml.org/lkml/2012/2/3/459
Holy lots of patches...
Quoted 4 lines
And patch #47 in that series has been obsoleted by another series from Steven Rostedt:
https://lkml.org/lkml/2012/2/7/231
Ok.
Quoted 1 line
Hopefully these fix both splats and slowness.
So again, I'm slightly confused on how RCU patches flow. Eric originally reported the bug for which you created the patch I applied against 3.3. The giant patch series above seems queued for 3.4.
I don't see stable CC'd on 45-47, nor any of Steven's patches. I doubt I'd want to go applying the 47-patch series on 3.3 at the moment, and given you have these marked for 3.4 I don't think you do either. However, is there some kind of fix for the original bug report against 3.3?
josh
Quoted 13 lines · Josh Boyer
Quoted 4 lines
And patch #47 in that series has been obsoleted by another series from Steven Rostedt:
https://lkml.org/lkml/2012/2/7/231Ok.
Quoted 1 line
Hopefully these fix both splats and slowness.
So again, I'm slightly confused on how RCU patches flow. Eric originally reported the bug for which you created the patch I applied against 3.3. The giant patch series above seems queued for 3.4.
Quoted 5 lines
I don't see stable CC'd on 45-47, nor any of Steven's patches. I doubt I'd want to go applying the 47-patch series on 3.3 at the moment, and given you have these marked for 3.4 I don't think you do either. However, is there some kind of fix for the original bug report against 3.3?
I was being sincere when I asked the above questions. Could you describe how you handle RCU patches across releases and if there is a fix for the 3.3-rcX issue Eric reported that is going into 3.3?
I know you're quite busy, but I'd like to understand your thinking so I know what to expect going forward.
josh
Quoted 20 lines · Josh Boyer
Quoted 13 lines · Josh Boyer
Quoted 4 lines
And patch #47 in that series has been obsoleted by another series from Steven Rostedt:
https://lkml.org/lkml/2012/2/7/231Ok.
Quoted 1 line
Hopefully these fix both splats and slowness.
So again, I'm slightly confused on how RCU patches flow. Eric originally reported the bug for which you created the patch I applied against 3.3. The giant patch series above seems queued for 3.4.
Quoted 5 lines
I don't see stable CC'd on 45-47, nor any of Steven's patches. I doubt I'd want to go applying the 47-patch series on 3.3 at the moment, and given you have these marked for 3.4 I don't think you do either. However, is there some kind of fix for the original bug report against 3.3?
Quoted 6 lines
I was being sincere when I asked the above questions. Could you describe how you handle RCU patches across releases and if there is a fix for the 3.3-rcX issue Eric reported that is going into 3.3?
I know you're quite busy, but I'd like to understand your thinking so I know what to expect going forward.
Apologies for being slow, but could you please point me at the original bug report that the old patch was designed to fix? My email filing seems to have failed me in this case.
My guess is that the best short-term fix for Fedora is to disable the warning, but I do need to see the original bug to work out if that really is a prudent course of action.
Thanx, Paul
Quoted 25 lines · Paul E. McKenney
Quoted 24 lines · Josh Boyer
Quoted 13 lines · Josh Boyer
Quoted 4 lines
And patch #47 in that series has been obsoleted by another series from Steven Rostedt:
https://lkml.org/lkml/2012/2/7/231Ok.
Quoted 1 line
Hopefully these fix both splats and slowness.
So again, I'm slightly confused on how RCU patches flow. Eric originally reported the bug for which you created the patch I applied against 3.3. The giant patch series above seems queued for 3.4.
Quoted 5 lines
I don't see stable CC'd on 45-47, nor any of Steven's patches. I doubt I'd want to go applying the 47-patch series on 3.3 at the moment, and given you have these marked for 3.4 I don't think you do either. However, is there some kind of fix for the original bug report against 3.3?
I was being sincere when I asked the above questions. Could you describe how you handle RCU patches across releases and if there is a fix for the 3.3-rcX issue Eric reported that is going into 3.3?
Quoted 6 lines
Quoted 2 lines
I know you're quite busy, but I'd like to understand your thinking so I know what to expect going forward.
Apologies for being slow, but could you please point me at the original bug report that the old patch was designed to fix? My email filing seems to have failed me in this case.
Same thread I linked in my original email:
https://lkml.org/lkml/2012/1/24/203
Quoted 3 lines
My guess is that the best short-term fix for Fedora is to disable the warning, but I do need to see the original bug to work out if that really is a prudent course of action.
Honestly, I don't care from a Fedora perspective. I can do what I need to do there without too much trouble. I'm asking because afaik, upstream still has this problem. The thread gets a bit curvy but from what I can tell it resulted in the patch I highlighted as having issues. Maybe I overlooked something else that fixed Eric's problem?
josh
Quoted 29 lines · Josh Boyer
Quoted 28 lines · Paul E. McKenney
Quoted 26 lines · Josh Boyer
Quoted 13 lines · Josh Boyer
Quoted 4 lines
And patch #47 in that series has been obsoleted by another series from Steven Rostedt:
https://lkml.org/lkml/2012/2/7/231Ok.
Quoted 1 line
Hopefully these fix both splats and slowness.
So again, I'm slightly confused on how RCU patches flow. Eric originally reported the bug for which you created the patch I applied against 3.3. The giant patch series above seems queued for 3.4.
Quoted 5 lines
I don't see stable CC'd on 45-47, nor any of Steven's patches. I doubt I'd want to go applying the 47-patch series on 3.3 at the moment, and given you have these marked for 3.4 I don't think you do either. However, is there some kind of fix for the original bug report against 3.3?
I was being sincere when I asked the above questions. Could you describe how you handle RCU patches across releases and if there is a fix for the 3.3-rcX issue Eric reported that is going into 3.3?
I know you're quite busy, but I'd like to understand your thinking so I know what to expect going forward.
Quoted 7 lines
Quoted 3 lines
Apologies for being slow, but could you please point me at the original bug report that the old patch was designed to fix? My email filing seems to have failed me in this case.
Same thread I linked in my original email:
Thank you!
Quoted 9 lines
Quoted 3 lines
My guess is that the best short-term fix for Fedora is to disable the warning, but I do need to see the original bug to work out if that really is a prudent course of action.
Honestly, I don't care from a Fedora perspective. I can do what I need to do there without too much trouble. I'm asking because afaik, upstream still has this problem. The thread gets a bit curvy but from what I can tell it resulted in the patch I highlighted as having issues. Maybe I overlooked something else that fixed Eric's problem?
"A bit curvy" is right -- which is why the fixes ended up at the end of my large patch series for 3.4.
But after looking this over, Steven Rostedt's three-patch set should suffice:
https://lkml.org/lkml/2012/2/7/231
The reason that mine are not needed is that the problematic code is called -only- from idle, not from process context, and also that the problematic code is tracing. My patch #45 is required for code that is called from both process context and from idle. My patch #46 is required for non-tracing uses of RCU from within the idle loop -- along with TBD patches to wrap those uses of RCU in the RCU_NONIDLE() macro.
So again, in your particular case of x86's power-tracing features, Steven Rostedt's three-patch series called out above should be all that you need.
I have CCed Steven in case there is some prerequisite to his patch set.
Thanx, Paul
Quoted 12 lines · Paul E. McKenney
But after looking this over, Steven Rostedt's three-patch set should suffice:
https://lkml.org/lkml/2012/2/7/231The reason that mine are not needed is that the problematic code is called -only- from idle, not from process context, and also that the problematic code is tracing. My patch #45 is required for code that is called from both process context and from idle. My patch #46 is required for non-tracing uses of RCU from within the idle loop -- along with TBD patches to wrap those uses of RCU in the
RCU_NONIDLE()macro.
Quoted 5 lines
So again, in your particular case of x86's power-tracing features, Steven Rostedt's three-patch series called out above should be all that you need.
I have CCed Steven in case there is some prerequisite to his patch set.
The above link is the RFC, it probably still suffices, but the patches that are going into mainline are here:
https://lkml.org/lkml/2012/2/13/530
https://lkml.org/lkml/2012/2/13/525
https://lkml.org/lkml/2012/2/13/524
The above is the order to apply them, (patch 3, 4, and 5) even though they arrived to LKML out of order.
-- Steve