Skip to contentSkip to conversation

Question on __ARCH_WANT_UNLOCKED_CTXSW

ia64 Peter Zijlstra 4 participants 5 messages Aug 24, 2011 – Sep 16, 2011

lore
Share & tools
lore
#0 lore raw

Ken, Tony,

happen to remember what the perceived benefit of using __ARCH_WANT_UNLOCKED_CTXSW was about?

---
commit f8efa27662532ad5adb2790bfc3f4c78e019cfad

Author: Chen, Kenneth W <[email protected]>
Date:   Thu Jan 26 18:24:59 2006 -0800

[IA64] remove staled comments in asm/system.h

With the recent optimization made to wrap_mmu_context function, we don't hold tasklist_lock anymore when wrapping context id. The comments in asm/system.h must fall through the crack earlier. Remove staled comments.

I believe it is still beneficial to unlock the runqueue lock across context switch. So leave __ARCH_WANT_UNLOCKED_CTXSW on.

include/asm-ia64/system.h +0 -25
index 80c5a23..0625387 100644
@@ -249,32 +249,7 @@ extern void ia64_load_extra (struct task_struct *task);
 # define switch_to(prev,next,last)	__switch_to(prev, next, last)
 #endif
 
-/*
- * On IA-64, we don't want to hold the runqueue's lock during the low-level context-switch,
- * because that could cause a deadlock.  Here is an example by Erich Focht:
- *
- * Example:
- * CPU#0:
- * schedule()
- *    -> spin_lock_irq(&rq->lock)
- *    -> context_switch()
- *       -> wrap_mmu_context()
- *          -> read_lock(&tasklist_lock)
- *
- * CPU#1:
- * sys_wait4() or release_task() or forget_original_parent()
- *    -> write_lock(&tasklist_lock)
- *    -> do_notify_parent()
- *       -> wake_up_parent()
- *          -> try_to_wake_up()
- *             -> spin_lock_irq(&parent_rq->lock)
- *
- * If the parent's rq happens to be on CPU#0, we'll wait for the rq->lock
- * of that CPU which will not be released, because there we wait for the
- * tasklist_lock to become available.
- */
 #define __ARCH_WANT_UNLOCKED_CTXSW
-
 #define ARCH_HAS_PREFETCH_SWITCH_STACK
 #define ia64_platform_is(x) (strcmp(x, platform_name) == 0)
 
#1 lore raw
Quoted 4 lines · Peter Zijlstra

Ken, Tony,

happen to remember what the perceived benefit of using __ARCH_WANT_UNLOCKED_CTXSW was about?

Boy, this was so long ago and my memory is fading. Low level ia64 arch has lengthy context switch path in assembly code due to large CPU register file, IIRC __ARCH_WANT_UNLOCKED_CTXSW gave lower runqueue lock contention for workloads that was heavy in wakeup and ctx switch events (I remember it was particularly useful on large smp machine that has hundreds of CPUs). I don't know whether the situation still applies today when scheduler code evolved quite a bit over past 5 years. I think the needs for ia64 require a fresh re-evaluation.

- Ken

#2 lore raw
In reply to Peter Zijlstra
Quoted 2 lines

happen to remember what the perceived benefit of using __ARCH_WANT_UNLOCKED_CTXSW was about?

No - digging around the code hasn't rung any bells for me either.

Perhaps just general goodness for not holding a lock for longer than we need to? But that would imply some case where someone else could do something useful by being able to grab the lock when we drop it. About the only thing I can think of is that it would allow tasks to be re-balanced just a teeny bit earlier --- but re-balancing ought to be somewhat rare, yes?

-Tony ���{.n�+�������+%�����ݶ��w��{.n�+����{��G�����{ay�ʇڙ�,j��f���h��������z_�(�階�ݢj"���m������G������?����&���~��iO��z��v�^�m���� ������?�I�

#3 lore raw
Quoted 5 lines · Tony
Quoted 2 lines

happen to remember what the perceived benefit of using __ARCH_WANT_UNLOCKED_CTXSW was about?

No - digging around the code hasn't rung any bells for me either.

Quoted 7 lines

Perhaps just general goodness for not holding a lock for longer than we need to? But that would imply some case where someone else could do something useful by being able to grab the lock when we drop it. About the only thing I can think of is that it would allow tasks to be re-balanced just a teeny bit earlier --- but re-balancing ought to be somewhat rare, yes?

Mostly yes, except remote wakeups, however that got a complete overhaul in 3.0. Instead of taking the remote rq->lock we now enqueue the task on a list and IPI the thing, then let the IPI do the remote enqueue and trigger the reschedule.

So it might make sense to re-evaluate this on ia64 like Ken suggested.. then again, who has a large ia64 box and is still willing to put time in?

#4 lore raw
Quoted 14 lines · Peter Zijlstra
Quoted 5 lines · Tony
Quoted 2 lines

happen to remember what the perceived benefit of using __ARCH_WANT_UNLOCKED_CTXSW was about?

No - digging around the code hasn't rung any bells for me either.

Quoted 7 lines

Perhaps just general goodness for not holding a lock for longer than we need to? But that would imply some case where someone else could do something useful by being able to grab the lock when we drop it. About the only thing I can think of is that it would allow tasks to be re-balanced just a teeny bit earlier --- but re-balancing ought to be somewhat rare, yes?

Quoted 8 lines

Mostly yes, except remote wakeups, however that got a complete overhaul in 3.0. Instead of taking the remote rq->lock we now enqueue the task on a list and IPI the thing, then let the IPI do the remote enqueue and trigger the reschedule.

So it might make sense to re-evaluate this on ia64 like Ken suggested.. then again, who has a large ia64 box and is still willing to put time in?

Hum -- perhaps they'll come out of the woodwork if we just rip it out (if they exist)

- Paul