Skip to contentSkip to conversation

Large slowdown with 'x86: Avoid invoking RCU when CPU is idle'

DISCUSSION Josh Boyer 3 participants 8 messages Feb 22, 2012 – Feb 24, 2012

lore
Share & tools
lore

Josh Boyer

author
#0 lore raw

Hi Paul,

Over in Fedora land, I applied your patch from this thread:

https://lkml.org/lkml/2012/1/24/441

to our 3.3-rc3/rc4 based rawhide kernels. The intention was to solve an RCU issue that was very similar to what Eric originally reported, and the RCU splat did indeed go away[1].

However, we then got a few reports of kernels containing that patch being extremely slow. When the patch was dropped, the slowness goes away according to one reporter. The details can be found in this bug:

https://bugzilla.redhat.com/show_bug.cgi?id=795050

The slowness doesn't seem to hit everyone, and in my local testing things seem to be working just fine. The reporters have widely varying hardware as well, so it doesn't seem machine specific.

Perhaps I misdiagnosed the original issue, or perhaps I missed something else that needs to be applied prior to this but I thought I would point this out in case you had any ideas.

josh

[1] https://bugzilla.redhat.com/show_bug.cgi?id=789641

#1 lore raw
Quoted 16 lines · Josh Boyer

Hi Paul,

Over in Fedora land, I applied your patch from this thread:

https://lkml.org/lkml/2012/1/24/441

to our 3.3-rc3/rc4 based rawhide kernels. The intention was to solve an RCU issue that was very similar to what Eric originally reported, and the RCU splat did indeed go away[1].

However, we then got a few reports of kernels containing that patch being extremely slow. When the patch was dropped, the slowness goes away according to one reporter. The details can be found in this bug:

https://bugzilla.redhat.com/show_bug.cgi?id=795050

Quoted 7 lines

The slowness doesn't seem to hit everyone, and in my local testing things seem to be working just fine. The reporters have widely varying hardware as well, so it doesn't seem machine specific.

Perhaps I misdiagnosed the original issue, or perhaps I missed something else that needs to be applied prior to this but I thought I would point this out in case you had any ideas.

This patch has been obsoleted by patches #45-47 in this series:

	https://lkml.org/lkml/2012/2/3/459

And patch #47 in that series has been obsoleted by another series from Steven Rostedt:

	https://lkml.org/lkml/2012/2/7/231

Hopefully these fix both splats and slowness.

Thanx, Paul

Quoted 4 lines

josh

[1] https://bugzilla.redhat.com/show_bug.cgi?id=789641

author
#2 lore raw
Quoted 17 lines · Paul E. McKenney
Quoted 16 lines · Josh Boyer

Hi Paul,

Over in Fedora land, I applied your patch from this thread:

https://lkml.org/lkml/2012/1/24/441

to our 3.3-rc3/rc4 based rawhide kernels. The intention was to solve an RCU issue that was very similar to what Eric originally reported, and the RCU splat did indeed go away[1].

However, we then got a few reports of kernels containing that patch being extremely slow. When the patch was dropped, the slowness goes away according to one reporter. The details can be found in this bug:

https://bugzilla.redhat.com/show_bug.cgi?id=795050

Quoted 7 lines
Quoted 7 lines

The slowness doesn't seem to hit everyone, and in my local testing things seem to be working just fine. The reporters have widely varying hardware as well, so it doesn't seem machine specific.

Perhaps I misdiagnosed the original issue, or perhaps I missed something else that needs to be applied prior to this but I thought I would point this out in case you had any ideas.

First off, thanks for the quick reply!

Quoted 3 lines

This patch has been obsoleted by patches #45-47 in this series:

	https://lkml.org/lkml/2012/2/3/459

Holy lots of patches...

Quoted 4 lines

And patch #47 in that series has been obsoleted by another series from Steven Rostedt:

	https://lkml.org/lkml/2012/2/7/231

Ok.

Quoted 1 line

Hopefully these fix both splats and slowness.

So again, I'm slightly confused on how RCU patches flow. Eric originally reported the bug for which you created the patch I applied against 3.3. The giant patch series above seems queued for 3.4.

I don't see stable CC'd on 45-47, nor any of Steven's patches. I doubt I'd want to go applying the 47-patch series on 3.3 at the moment, and given you have these marked for 3.4 I don't think you do either. However, is there some kind of fix for the original bug report against 3.3?

josh

author
#3 lore raw
Quoted 13 lines · Josh Boyer
Quoted 4 lines

And patch #47 in that series has been obsoleted by another series from Steven Rostedt:

	https://lkml.org/lkml/2012/2/7/231

Ok.

Quoted 1 line

Hopefully these fix both splats and slowness.

So again, I'm slightly confused on how RCU patches flow. Eric originally reported the bug for which you created the patch I applied against 3.3. The giant patch series above seems queued for 3.4.

Quoted 5 lines

I don't see stable CC'd on 45-47, nor any of Steven's patches. I doubt I'd want to go applying the 47-patch series on 3.3 at the moment, and given you have these marked for 3.4 I don't think you do either. However, is there some kind of fix for the original bug report against 3.3?

I was being sincere when I asked the above questions. Could you describe how you handle RCU patches across releases and if there is a fix for the 3.3-rcX issue Eric reported that is going into 3.3?

I know you're quite busy, but I'd like to understand your thinking so I know what to expect going forward.

josh

#4 lore raw
Quoted 20 lines · Josh Boyer
Quoted 13 lines · Josh Boyer
Quoted 4 lines

And patch #47 in that series has been obsoleted by another series from Steven Rostedt:

	https://lkml.org/lkml/2012/2/7/231

Ok.

Quoted 1 line

Hopefully these fix both splats and slowness.

So again, I'm slightly confused on how RCU patches flow. Eric originally reported the bug for which you created the patch I applied against 3.3. The giant patch series above seems queued for 3.4.

Quoted 5 lines

I don't see stable CC'd on 45-47, nor any of Steven's patches. I doubt I'd want to go applying the 47-patch series on 3.3 at the moment, and given you have these marked for 3.4 I don't think you do either. However, is there some kind of fix for the original bug report against 3.3?

Quoted 6 lines

I was being sincere when I asked the above questions. Could you describe how you handle RCU patches across releases and if there is a fix for the 3.3-rcX issue Eric reported that is going into 3.3?

I know you're quite busy, but I'd like to understand your thinking so I know what to expect going forward.

Apologies for being slow, but could you please point me at the original bug report that the old patch was designed to fix? My email filing seems to have failed me in this case.

My guess is that the best short-term fix for Fedora is to disable the warning, but I do need to see the original bug to work out if that really is a prudent course of action.

Thanx, Paul

author
#5 lore raw
Quoted 25 lines · Paul E. McKenney
Quoted 24 lines · Josh Boyer
Quoted 13 lines · Josh Boyer
Quoted 4 lines

And patch #47 in that series has been obsoleted by another series from Steven Rostedt:

	https://lkml.org/lkml/2012/2/7/231

Ok.

Quoted 1 line

Hopefully these fix both splats and slowness.

So again, I'm slightly confused on how RCU patches flow. Eric originally reported the bug for which you created the patch I applied against 3.3. The giant patch series above seems queued for 3.4.

Quoted 5 lines

I don't see stable CC'd on 45-47, nor any of Steven's patches. I doubt I'd want to go applying the 47-patch series on 3.3 at the moment, and given you have these marked for 3.4 I don't think you do either. However, is there some kind of fix for the original bug report against 3.3?

I was being sincere when I asked the above questions. Could you describe how you handle RCU patches across releases and if there is a fix for the 3.3-rcX issue Eric reported that is going into 3.3?

Quoted 6 lines
Quoted 2 lines

I know you're quite busy, but I'd like to understand your thinking so I know what to expect going forward.

Apologies for being slow, but could you please point me at the original bug report that the old patch was designed to fix? My email filing seems to have failed me in this case.

Same thread I linked in my original email:

https://lkml.org/lkml/2012/1/24/203

Quoted 3 lines

My guess is that the best short-term fix for Fedora is to disable the warning, but I do need to see the original bug to work out if that really is a prudent course of action.

Honestly, I don't care from a Fedora perspective. I can do what I need to do there without too much trouble. I'm asking because afaik, upstream still has this problem. The thread gets a bit curvy but from what I can tell it resulted in the patch I highlighted as having issues. Maybe I overlooked something else that fixed Eric's problem?

josh

#6 lore raw
Quoted 29 lines · Josh Boyer
Quoted 28 lines · Paul E. McKenney
Quoted 26 lines · Josh Boyer
Quoted 13 lines · Josh Boyer
Quoted 4 lines

And patch #47 in that series has been obsoleted by another series from Steven Rostedt:

	https://lkml.org/lkml/2012/2/7/231

Ok.

Quoted 1 line

Hopefully these fix both splats and slowness.

So again, I'm slightly confused on how RCU patches flow. Eric originally reported the bug for which you created the patch I applied against 3.3. The giant patch series above seems queued for 3.4.

Quoted 5 lines

I don't see stable CC'd on 45-47, nor any of Steven's patches. I doubt I'd want to go applying the 47-patch series on 3.3 at the moment, and given you have these marked for 3.4 I don't think you do either. However, is there some kind of fix for the original bug report against 3.3?

I was being sincere when I asked the above questions. Could you describe how you handle RCU patches across releases and if there is a fix for the 3.3-rcX issue Eric reported that is going into 3.3?

I know you're quite busy, but I'd like to understand your thinking so I know what to expect going forward.

Quoted 7 lines
Quoted 3 lines

Apologies for being slow, but could you please point me at the original bug report that the old patch was designed to fix? My email filing seems to have failed me in this case.

Same thread I linked in my original email:

https://lkml.org/lkml/2012/1/24/203

Thank you!

Quoted 9 lines
Quoted 3 lines

My guess is that the best short-term fix for Fedora is to disable the warning, but I do need to see the original bug to work out if that really is a prudent course of action.

Honestly, I don't care from a Fedora perspective. I can do what I need to do there without too much trouble. I'm asking because afaik, upstream still has this problem. The thread gets a bit curvy but from what I can tell it resulted in the patch I highlighted as having issues. Maybe I overlooked something else that fixed Eric's problem?

"A bit curvy" is right -- which is why the fixes ended up at the end of my large patch series for 3.4.

But after looking this over, Steven Rostedt's three-patch set should suffice:

	https://lkml.org/lkml/2012/2/7/231

The reason that mine are not needed is that the problematic code is called -only- from idle, not from process context, and also that the problematic code is tracing. My patch #45 is required for code that is called from both process context and from idle. My patch #46 is required for non-tracing uses of RCU from within the idle loop -- along with TBD patches to wrap those uses of RCU in the RCU_NONIDLE() macro.

So again, in your particular case of x86's power-tracing features, Steven Rostedt's three-patch series called out above should be all that you need.

I have CCed Steven in case there is some prerequisite to his patch set.

Thanx, Paul

#7 lore raw
Quoted 12 lines · Paul E. McKenney

But after looking this over, Steven Rostedt's three-patch set should suffice:

	https://lkml.org/lkml/2012/2/7/231

The reason that mine are not needed is that the problematic code is called -only- from idle, not from process context, and also that the problematic code is tracing. My patch #45 is required for code that is called from both process context and from idle. My patch #46 is required for non-tracing uses of RCU from within the idle loop -- along with TBD patches to wrap those uses of RCU in the RCU_NONIDLE() macro.

Quoted 5 lines

So again, in your particular case of x86's power-tracing features, Steven Rostedt's three-patch series called out above should be all that you need.

I have CCed Steven in case there is some prerequisite to his patch set.

The above link is the RFC, it probably still suffices, but the patches that are going into mainline are here:

https://lkml.org/lkml/2012/2/13/530
https://lkml.org/lkml/2012/2/13/525
https://lkml.org/lkml/2012/2/13/524

The above is the order to apply them, (patch 3, 4, and 5) even though they arrived to LKML out of order.

-- Steve