time: dynamic irq affinity
Subsystems (4)
v2 cover time: dynamic irq affinity
When a cpu goes to a deep idle state where its local timer is shutdown, it notifies the time framework to use the broadcast timer instead.
Unfortunately, the broadcast device could wake up any CPU, including an idle one which is not concerned by the wake up at all.
This implies, in the worst case, an idle CPU will wake up to send an IPI to another idle cpu.
This patch solves this by setting the irq affinity to the cpu concerned by the nearest timer event, by this way, the CPU which is wake up is guarantee to be the one concerned by the next event and we are safe with unnecessary wakeup for another idle CPU.
As the irq affinity is not supported by all the archs, a flag is needed to specify which clocksource can handle it : CLOCK_EVT_FEAT_DYNIRQ.
Tested on a u8500 board with a test program doing indefinitely usleep 10000 wired on each CPU.
With dynamic irq affinity:
Log is 10.042298 secs long with 4190 events
cpu0/state0, 24 hits, total 2718.00us, avg 113.25us, min 0.00us, max 854.00us cpu0/state1, 994 hits, total 9874827.00us, avg 9934.43us, min 30.00us, max 10346.00us cpu1/state0, 73 hits, total 17001.00us, avg 232.89us, min 0.00us, max 10040.00us cpu1/state1, 1002 hits, total 9883840.00us, avg 9864.11us, min 0.00us, max 10742.00us cluster/state0, 0 hits, total 0.00us, avg 0.00us, min 0.00us, max 0.00us cluster/state1, 1931 hits, total 9762328.00us, avg 5055.58us, min 30.00us, max 9308.00us
Without dynamic irq affinity:
Log is 10.036834 secs long with 6574 events
cpu0/state0, 114 hits, total 20107.00us, avg 176.38us, min 0.00us, max 7233.00us cpu0/state1, 1951 hits, total 9833836.00us, avg 5040.41us, min 0.00us, max 9217.00us cpu1/state0, 223 hits, total 21140.00us, avg 94.80us, min 0.00us, max 2960.00us cpu1/state1, 997 hits, total 9879748.00us, avg 9909.48us, min 0.00us, max 10346.00us cluster/state0, 5 hits, total 5462.00us, avg 1092.40us, min 580.00us, max 2899.00us cluster/state1, 2298 hits, total 9740988.00us, avg 4238.90us, min 30.00us, max 9217.00us
Results for the specific test case 'usleep 10000'
* reduced by 40% the number of wake up on the system
* reduced by 49% the number of wake up for CPU0
* increased by factor two idle time for CPU0
* increase by 16% package idle hits + 16% average package idle time
Changelog:
V2 :
* mentioned CLOCK_EVT_FEAT_DYNIRQ flag name in patch description
* added comments for CLOCK_EVT_FEAT_DYNIRQ
* replaced tick_broadcast_set_affinity parameter to use a cpumask
V1 : initial post
Daniel Lezcano (3):
time : pass broadcast parameter
time : set broadcast irq affinity
ARM: nomadik: add dynamic irq flag to the timer
Viresh Kumar (1):
ARM: timer-sp: Set dynamic irq affinity
arch/arm/common/timer-sp.c | 3 ++- drivers/clocksource/nomadik-mtu.c | 3 ++- include/linux/clockchips.h | 5 +++++ kernel/time/tick-broadcast.c | 41 +++++++++++++++++++++++++++++-------- 4 files changed, 42 insertions(+), 10 deletions(-)
Signature · 1 line
1.7.9.5
PATCH v2 1/4 time : pass broadcast parameter
The broadcast timer could be passed as parameter to the function instead of using again tick_broadcast_device.evtdev which was previously used in the caller function.
--- kernel/time/tick-broadcast.c | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-)
kernel/time/tick-broadcast.c +5 -6
@@ -406,10 +406,9 @@ struct cpumask *tick_get_broadcast_oneshot_mask(void)return to_cpumask (tick_broadcast_oneshot_mask); } -static int tick_broadcast_set_event (ktime_t expires,int force) +static int tick_broadcast_set_event (struct clock_event_device *bc, +ktime_t expires,int force) { -struct clock_event_device *bc = tick_broadcast_device.evtdev; -if (bc->mode !=CLOCK_EVT_MODE_ONESHOT )clockevents_set_mode (bc,CLOCK_EVT_MODE_ONESHOT );
@@ -479,7 +478,7 @@ again:* Rearm the broadcast device. If event expired, * repeat the above */ -if (tick_broadcast_set_event (next_event,0 )) +if (tick_broadcast_set_event (dev, next_event,0 )) goto again; }raw_spin_unlock (&tick_broadcast_lock);
@@ -522,7 +521,7 @@ void tick_broadcast_oneshot_control(unsigned long reason)cpumask_set_cpu (cpu,tick_get_broadcast_oneshot_mask ());clockevents_set_mode (dev,CLOCK_EVT_MODE_SHUTDOWN );if (dev->next_event.tv64 < bc->next_event.tv64) -tick_broadcast_set_event (dev->next_event,1 ); +tick_broadcast_set_event (bc, dev->next_event,1 ); } }else {if (cpumask_test_cpu (cpu,tick_get_broadcast_oneshot_mask ())) {
@@ -591,7 +590,7 @@ void tick_broadcast_setup_oneshot(struct clock_event_device *bc)clockevents_set_mode (bc,CLOCK_EVT_MODE_ONESHOT );tick_broadcast_init_next_event (to_cpumask (tmpmask), tick_next_period); -tick_broadcast_set_event (tick_next_period,1 ); +tick_broadcast_set_event (bc, tick_next_period,1 ); }else bc->next_event.tv64 =KTIME_MAX ; }else {
Signature · 1 line
1.7.9.5
tip-bot for Daniel Lezcano
merge bot[tip:timers/core] tick: Pass broadcast device to tick_broadcast_set_event()
tick: Pass broadcast device to tick_broadcast_set_event()
Commit-ID: f9ae39d04ccdec8d8ecf532191b7056c279a22c0 Gitweb: http://git.kernel.org/tip/f9ae39d04ccdec8d8ecf532191b7056c279a22c0 Author: Daniel Lezcano <[email protected]> AuthorDate: Sat, 2 Mar 2013 11:10:10 +0100 Committer: Thomas Gleixner <[email protected]> CommitDate: Thu, 7 Mar 2013 16:13:26 +0100
tick: Pass broadcast device to tick_broadcast_set_event()
Pass the broadcast timer to tick_broadcast_set_event() instead of reevaluating tick_broadcast_device.evtdev.
[ tglx: Massaged changelog ]
--- kernel/time/tick-broadcast.c | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-)
kernel/time/tick-broadcast.c +5 -6
@@ -401,10 +401,9 @@ struct cpumask *tick_get_broadcast_oneshot_mask(void)return tick_broadcast_oneshot_mask; } -static int tick_broadcast_set_event (ktime_t expires,int force) +static int tick_broadcast_set_event (struct clock_event_device *bc, +ktime_t expires,int force) { -struct clock_event_device *bc = tick_broadcast_device.evtdev; -if (bc->mode !=CLOCK_EVT_MODE_ONESHOT )clockevents_set_mode (bc,CLOCK_EVT_MODE_ONESHOT );
@@ -474,7 +473,7 @@ again:* Rearm the broadcast device. If event expired, * repeat the above */ -if (tick_broadcast_set_event (next_event,0 )) +if (tick_broadcast_set_event (dev, next_event,0 )) goto again; }raw_spin_unlock (&tick_broadcast_lock);
@@ -516,7 +515,7 @@ void tick_broadcast_oneshot_control(unsigned long reason)if (!cpumask_test_and_set_cpu (cpu, tick_broadcast_oneshot_mask)) {clockevents_set_mode (dev,CLOCK_EVT_MODE_SHUTDOWN );if (dev->next_event.tv64 < bc->next_event.tv64) -tick_broadcast_set_event (dev->next_event,1 ); +tick_broadcast_set_event (bc, dev->next_event,1 ); } }else {if (cpumask_test_and_clear_cpu (cpu, tick_broadcast_oneshot_mask)) {
@@ -582,7 +581,7 @@ void tick_broadcast_setup_oneshot(struct clock_event_device *bc)clockevents_set_mode (bc,CLOCK_EVT_MODE_ONESHOT );tick_broadcast_init_next_event (tmpmask, tick_next_period); -tick_broadcast_set_event (tick_next_period,1 ); +tick_broadcast_set_event (bc, tick_next_period,1 ); }else bc->next_event.tv64 =KTIME_MAX ; }else {
PATCH v2 2/4 time : set broadcast irq affinity
In reply to Daniel LezcanoWhen a cpu goes to a deep idle state where its local timer is shutdown, it notifies the time frame work to use the broadcast timer instead.
Unfortunately, the broadcast device could wake up any CPU, including an idle one which is not concerned by the wake up at all.
This implies, in the worst case, an idle CPU will wake up to send an IPI to another idle cpu.
This patch solves this by setting the irq affinity to the cpu concerned by the nearest timer event, by this way, the CPU which is wake up is guarantee to be the one concerned by the next event and we are safe with unnecessary wakeup for another idle CPU.
As the irq affinity is not supported by all the archs, a flag is needed to specify which clocksource can handle it : CLOCK_EVT_FEAT_DYNIRQ
--- include/linux/clockchips.h | 5 +++++ kernel/time/tick-broadcast.c | 40 +++++++++++++++++++++++++++++++++------- 2 files changed, 38 insertions(+), 7 deletions(-)
include/linux/clockchips.h +5 -0
@@ -55,6 +55,11 @@ enum clock_event_nofitiers {#define CLOCK_EVT_FEAT_C3STOP 0x000008#define CLOCK_EVT_FEAT_DUMMY 0x000010 +/* +* Clock event device can set its irq affinity dynamically +*/ +#define CLOCK_EVT_FEAT_DYNIRQ 0x000020 +/** * struct clock_event_device - clock event device descriptor * @event_handler: Assigned by the framework to be called by the low
kernel/time/tick-broadcast.c +33 -7
@@ -406,13 +406,37 @@ struct cpumask *tick_get_broadcast_oneshot_mask(void)return to_cpumask (tick_broadcast_oneshot_mask); } -static int tick_broadcast_set_event (struct clock_event_device *bc, +/* +* Set broadcast interrupt affinity +*/ +static void tick_broadcast_set_affinity (struct clock_event_device *bc, +const struct cpumask *cpumask) +{ +if (!(bc->features &CLOCK_EVT_FEAT_DYNIRQ )) +return ; + +if (cpumask_equal (bc->cpumask, cpumask)) +return ; + + bc->cpumask = cpumask; +irq_set_affinity (bc->irq, bc->cpumask); +} + +static int tick_broadcast_set_event (struct clock_event_device *bc,int cpu,ktime_t expires,int force) { +int ret; +if (bc->mode !=CLOCK_EVT_MODE_ONESHOT )clockevents_set_mode (bc,CLOCK_EVT_MODE_ONESHOT ); -return clockevents_program_event (bc, expires, force); + ret =clockevents_program_event (bc, expires, force); +if (ret) +return ret; + +tick_broadcast_set_affinity (bc,cpumask_of (cpu)); + +return 0 ; }int tick_resume_broadcast_oneshot (struct clock_event_device *bc)
@@ -441,7 +465,7 @@ static void tick_handle_oneshot_broadcast(struct clock_event_device *dev) {struct tick_device *td;ktime_t now, next_event; -int cpu; +int cpu, next_cpu;raw_spin_lock (&tick_broadcast_lock); again:
@@ -454,8 +478,10 @@ again: td = &per_cpu (tick_cpu_device, cpu);if (td->evtdev->next_event.tv64 <= now.tv64)cpumask_set_cpu (cpu,to_cpumask (tmpmask)); -else if (td->evtdev->next_event.tv64 < next_event.tv64) +else if (td->evtdev->next_event.tv64 < next_event.tv64) { next_event.tv64 = td->evtdev->next_event.tv64; + next_cpu = cpu; + } }/*
@@ -478,7 +504,7 @@ again:* Rearm the broadcast device. If event expired, * repeat the above */ -if (tick_broadcast_set_event (dev, next_event,0 )) +if (tick_broadcast_set_event (dev, next_cpu, next_event,0 )) goto again; }raw_spin_unlock (&tick_broadcast_lock);
@@ -521,7 +547,7 @@ void tick_broadcast_oneshot_control(unsigned long reason)cpumask_set_cpu (cpu,tick_get_broadcast_oneshot_mask ());clockevents_set_mode (dev,CLOCK_EVT_MODE_SHUTDOWN );if (dev->next_event.tv64 < bc->next_event.tv64) -tick_broadcast_set_event (bc, dev->next_event,1 ); +tick_broadcast_set_event (bc, cpu, dev->next_event,1 ); } }else {if (cpumask_test_cpu (cpu,tick_get_broadcast_oneshot_mask ())) {
@@ -590,7 +616,7 @@ void tick_broadcast_setup_oneshot(struct clock_event_device *bc)clockevents_set_mode (bc,CLOCK_EVT_MODE_ONESHOT );tick_broadcast_init_next_event (to_cpumask (tmpmask), tick_next_period); -tick_broadcast_set_event (bc, tick_next_period,1 ); +tick_broadcast_set_event (bc, cpu, tick_next_period,1 ); }else bc->next_event.tv64 =KTIME_MAX ; }else {
Signature · 1 line
1.7.9.5
Re: [PATCH 2/4][V2] time : set broadcast irq affinity
Quoted 51 lines · Daniel Lezcano
When a cpu goes to a deep idle state where its local timer is shutdown, it notifies the time frame work to use the broadcast timer instead.
Unfortunately, the broadcast device could wake up any CPU, including an idle one which is not concerned by the wake up at all.
This implies, in the worst case, an idle CPU will wake up to send an IPI to another idle cpu.
This patch solves this by setting the irq affinity to the cpu concerned by the nearest timer event, by this way, the CPU which is wake up is guarantee to be the one concerned by the next event and we are safe with unnecessary wakeup for another idle CPU.
As the irq affinity is not supported by all the archs, a flag is needed to specify which clocksource can handle it :
CLOCK_EVT_FEAT_DYNIRQSigned-off-by Daniel Lezcano <[email protected]>--- include/linux/clockchips.h | 5 +++++ kernel/time/tick-broadcast.c | 40 +++++++++++++++++++++++++++++++++------- 2 files changed, 38 insertions(+), 7 deletions(-)include/linux/clockchips.h +5
-0@@ -55,6 +55,11 @@ enum clock_event_nofitiers {#define CLOCK_EVT_FEAT_C3STOP 0x000008#define CLOCK_EVT_FEAT_DUMMY 0x000010 +/* +* Clock event device can set its irq affinity dynamically +*/ +#define CLOCK_EVT_FEAT_DYNIRQ 0x000020 +/** * struct clock_event_device - clock event device descriptor * @event_handler: Assigned by the framework to be called by the low kernel/time/tick-broadcast.c +3
-1@@ -406,13 +406,37 @@ struct cpumask *tick_get_broadcast_oneshot_mask(void)return to_cpumask (tick_broadcast_oneshot_mask); } -static int tick_broadcast_set_event(struct clock_event_device *bc, +/* +* Set broadcast interrupt affinity +*/
kernel/time/tick-broadcast.c struct cpumask *tick_get_broadcast_oneshot_mask(void) · line 421
Quoted 10 lines
kernel/time/tick-broadcast.c +10
-0+static void tick_broadcast_set_affinity (struct clock_event_device *bc, +const struct cpumask *cpumask) +{ +if (!(bc->features &CLOCK_EVT_FEAT_DYNIRQ )) +return ; + +if (cpumask_equal (bc->cpumask, cpumask)) +return ; + + bc->cpumask = cpumask;
This breaks with CONFIG_CPUMASK_OFFSTACK=y. cpumask_copy() is your friend!
Thanks,
tglx
Re: [PATCH 2/4][V2] time : set broadcast irq affinity
Quoted 48 lines · Thomas Gleixner
Quoted 47 lines · Daniel Lezcano
When a cpu goes to a deep idle state where its local timer is shutdown, it notifies the time frame work to use the broadcast timer instead.
Unfortunately, the broadcast device could wake up any CPU, including an idle one which is not concerned by the wake up at all.
This implies, in the worst case, an idle CPU will wake up to send an IPI to another idle cpu.
This patch solves this by setting the irq affinity to the cpu concerned by the nearest timer event, by this way, the CPU which is wake up is guarantee to be the one concerned by the next event and we are safe with unnecessary wakeup for another idle CPU.
As the irq affinity is not supported by all the archs, a flag is needed to specify which clocksource can handle it :
CLOCK_EVT_FEAT_DYNIRQSigned-off-by Daniel Lezcano <[email protected]>--- include/linux/clockchips.h | 5 +++++ kernel/time/tick-broadcast.c | 40 +++++++++++++++++++++++++++++++++------- 2 files changed, 38 insertions(+), 7 deletions(-)include/linux/clockchips.h +5
-0@@ -55,6 +55,11 @@ enum clock_event_nofitiers {#define CLOCK_EVT_FEAT_C3STOP 0x000008#define CLOCK_EVT_FEAT_DUMMY 0x000010 +/* +* Clock event device can set its irq affinity dynamically +*/ +#define CLOCK_EVT_FEAT_DYNIRQ 0x000020 +/** * struct clock_event_device - clock event device descriptor * @event_handler: Assigned by the framework to be called by the low kernel/time/tick-broadcast.c +0
-0@@ -406,13 +406,37 @@ struct cpumask *tick_get_broadcast_oneshot_mask(void)return to_cpumask (tick_broadcast_oneshot_mask); }
Quoted 16 lines
Quoted 4 lines
kernel/time/tick-broadcast.c +3
-1-static int tick_broadcast_set_event(struct clock_event_device *bc, +/* +* Set broadcast interrupt affinity +*/ kernel/time/tick-broadcast.c struct cpumask *tick_get_broadcast_oneshot_mask(void) · line 421
Quoted 10 lines
kernel/time/tick-broadcast.c +10
-0+static void tick_broadcast_set_affinity (struct clock_event_device *bc, +const struct cpumask *cpumask) +{ +if (!(bc->features &CLOCK_EVT_FEAT_DYNIRQ )) +return ; + +if (cpumask_equal (bc->cpumask, cpumask)) +return ; + + bc->cpumask = cpumask;This breaks with
CONFIG_CPUMASK_OFFSTACK=y.cpumask_copy()is your friend!
This instruction copies the pointer, not the cpumask content.
bc->cpumask is defined as a const struct cpumask * and is used to copy a cpumask pointer not the content.
The cpumask parameter is a pointer to a global cpumask provided by the cpumask_of macro.
But to be in the safe side, I compiled tested with CONFIG_CPUMASK_OFFSTACK=y without problem.
Did I missed something ?
Thanks
-- Daniel
Quoted 5 lines
--
To unsubscribe from this list: send the line "unsubscribe linux-pm" in the body of a message to majordomo at vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Signature · 5 lines
<http://www.linaro.org/> Linaro.org ? Open source software for ARM SoCs Follow Linaro: <http://www.facebook.com/pages/Linaro> Facebook | <http://twitter.com/#!/linaroorg> Twitter | <http://www.linaro.org/linaro-blog/> Blog
Re: [PATCH 2/4][V2] time : set broadcast irq affinity
Quoted 71 lines · Daniel Lezcano
Quoted 48 lines · Thomas Gleixner
Quoted 47 lines · Daniel Lezcano
When a cpu goes to a deep idle state where its local timer is shutdown, it notifies the time frame work to use the broadcast timer instead.
Unfortunately, the broadcast device could wake up any CPU, including an idle one which is not concerned by the wake up at all.
This implies, in the worst case, an idle CPU will wake up to send an IPI to another idle cpu.
This patch solves this by setting the irq affinity to the cpu concerned by the nearest timer event, by this way, the CPU which is wake up is guarantee to be the one concerned by the next event and we are safe with unnecessary wakeup for another idle CPU.
As the irq affinity is not supported by all the archs, a flag is needed to specify which clocksource can handle it :
CLOCK_EVT_FEAT_DYNIRQSigned-off-by Daniel Lezcano <[email protected]>--- include/linux/clockchips.h | 5 +++++ kernel/time/tick-broadcast.c | 40 +++++++++++++++++++++++++++++++++------- 2 files changed, 38 insertions(+), 7 deletions(-)include/linux/clockchips.h +5
-0@@ -55,6 +55,11 @@ enum clock_event_nofitiers {#define CLOCK_EVT_FEAT_C3STOP 0x000008#define CLOCK_EVT_FEAT_DUMMY 0x000010 +/* +* Clock event device can set its irq affinity dynamically +*/ +#define CLOCK_EVT_FEAT_DYNIRQ 0x000020 +/** * struct clock_event_device - clock event device descriptor * @event_handler: Assigned by the framework to be called by the low kernel/time/tick-broadcast.c +0
-0@@ -406,13 +406,37 @@ struct cpumask *tick_get_broadcast_oneshot_mask(void)return to_cpumask (tick_broadcast_oneshot_mask); }Quoted 16 lines
Quoted 4 lines
kernel/time/tick-broadcast.c +3
-1-static int tick_broadcast_set_event(struct clock_event_device *bc, +/* +* Set broadcast interrupt affinity +*/ kernel/time/tick-broadcast.c struct cpumask *tick_get_broadcast_oneshot_mask(void) · line 421
Quoted 10 lines
kernel/time/tick-broadcast.c +10
-0+static void tick_broadcast_set_affinity (struct clock_event_device *bc, +const struct cpumask *cpumask) +{ +if (!(bc->features &CLOCK_EVT_FEAT_DYNIRQ )) +return ; + +if (cpumask_equal (bc->cpumask, cpumask)) +return ; + + bc->cpumask = cpumask;This breaks with
CONFIG_CPUMASK_OFFSTACK=y.cpumask_copy()is your friend!This instruction copies the pointer, not the cpumask content.
bc->cpumaskis defined as a const struct cpumask * and is used to copy a cpumask pointer not the content.
Quoted 7 lines
The cpumask parameter is a pointer to a global cpumask provided by the
cpumask_ofmacro.But to be in the safe side, I compiled tested with
CONFIG_CPUMASK_OFFSTACK=ywithout problem.Did I missed something ?
No, I misinterpreted the patch. Assigning a pointer is safe.
Thanks,
tglx
Re: [PATCH 2/4][V2] time : set broadcast irq affinity
Quoted 75 lines · Thomas Gleixner
Quoted 74 lines · Daniel Lezcano
Quoted 48 lines · Thomas Gleixner
Quoted 47 lines · Daniel Lezcano
When a cpu goes to a deep idle state where its local timer is shutdown, it notifies the time frame work to use the broadcast timer instead.
Unfortunately, the broadcast device could wake up any CPU, including an idle one which is not concerned by the wake up at all.
This implies, in the worst case, an idle CPU will wake up to send an IPI to another idle cpu.
This patch solves this by setting the irq affinity to the cpu concerned by the nearest timer event, by this way, the CPU which is wake up is guarantee to be the one concerned by the next event and we are safe with unnecessary wakeup for another idle CPU.
As the irq affinity is not supported by all the archs, a flag is needed to specify which clocksource can handle it :
CLOCK_EVT_FEAT_DYNIRQSigned-off-by Daniel Lezcano <[email protected]>--- include/linux/clockchips.h | 5 +++++ kernel/time/tick-broadcast.c | 40 +++++++++++++++++++++++++++++++++------- 2 files changed, 38 insertions(+), 7 deletions(-)include/linux/clockchips.h +5
-0@@ -55,6 +55,11 @@ enum clock_event_nofitiers {#define CLOCK_EVT_FEAT_C3STOP 0x000008#define CLOCK_EVT_FEAT_DUMMY 0x000010 +/* +* Clock event device can set its irq affinity dynamically +*/ +#define CLOCK_EVT_FEAT_DYNIRQ 0x000020 +/** * struct clock_event_device - clock event device descriptor * @event_handler: Assigned by the framework to be called by the low kernel/time/tick-broadcast.c +0
-0@@ -406,13 +406,37 @@ struct cpumask *tick_get_broadcast_oneshot_mask(void)return to_cpumask (tick_broadcast_oneshot_mask); }Quoted 16 lines
Quoted 4 lines
kernel/time/tick-broadcast.c +3
-1-static int tick_broadcast_set_event(struct clock_event_device *bc, +/* +* Set broadcast interrupt affinity +*/ kernel/time/tick-broadcast.c struct cpumask *tick_get_broadcast_oneshot_mask(void) · line 421
Quoted 10 lines
kernel/time/tick-broadcast.c +10
-0+static void tick_broadcast_set_affinity (struct clock_event_device *bc, +const struct cpumask *cpumask) +{ +if (!(bc->features &CLOCK_EVT_FEAT_DYNIRQ )) +return ; + +if (cpumask_equal (bc->cpumask, cpumask)) +return ; + + bc->cpumask = cpumask;This breaks with
CONFIG_CPUMASK_OFFSTACK=y.cpumask_copy()is your friend!This instruction copies the pointer, not the cpumask content.
bc->cpumaskis defined as a const struct cpumask * and is used to copy a cpumask pointer not the content.The cpumask parameter is a pointer to a global cpumask provided by the
cpumask_ofmacro.
Quoted 6 lines
Quoted 4 lines
But to be in the safe side, I compiled tested with
CONFIG_CPUMASK_OFFSTACK=ywithout problem.Did I missed something ?
No, I misinterpreted the patch. Assigning a pointer is safe.
Ok, thanks anyway for reviewing the patch.
Do you think it is acceptable for upstreaming ?
-- Daniel
<http://www.linaro.org/> Linaro.org ? Open source software for ARM SoCs
Follow Linaro: <http://www.facebook.com/pages/Linaro> Facebook |
<http://twitter.com/#!/linaroorg> Twitter |
<http://www.linaro.org/linaro-blog/> Blog
Re: [PATCH 2/4][V2] time : set broadcast irq affinity
In reply to Thomas GleixnerQuoted 69 lines · Thomas Gleixner
Quoted 68 lines · Daniel Lezcano
Quoted 48 lines · Thomas Gleixner
Quoted 47 lines · Daniel Lezcano
When a cpu goes to a deep idle state where its local timer is shutdown, it notifies the time frame work to use the broadcast timer instead.
Unfortunately, the broadcast device could wake up any CPU, including an idle one which is not concerned by the wake up at all.
This implies, in the worst case, an idle CPU will wake up to send an IPI to another idle cpu.
This patch solves this by setting the irq affinity to the cpu concerned by the nearest timer event, by this way, the CPU which is wake up is guarantee to be the one concerned by the next event and we are safe with unnecessary wakeup for another idle CPU.
As the irq affinity is not supported by all the archs, a flag is needed to specify which clocksource can handle it :
CLOCK_EVT_FEAT_DYNIRQSigned-off-by Daniel Lezcano <[email protected]>--- include/linux/clockchips.h | 5 +++++ kernel/time/tick-broadcast.c | 40 +++++++++++++++++++++++++++++++++------- 2 files changed, 38 insertions(+), 7 deletions(-)include/linux/clockchips.h +5
-0@@ -55,6 +55,11 @@ enum clock_event_nofitiers {#define CLOCK_EVT_FEAT_C3STOP 0x000008#define CLOCK_EVT_FEAT_DUMMY 0x000010 +/* +* Clock event device can set its irq affinity dynamically +*/ +#define CLOCK_EVT_FEAT_DYNIRQ 0x000020 +/** * struct clock_event_device - clock event device descriptor * @event_handler: Assigned by the framework to be called by the low kernel/time/tick-broadcast.c +0
-0@@ -406,13 +406,37 @@ struct cpumask *tick_get_broadcast_oneshot_mask(void)return to_cpumask (tick_broadcast_oneshot_mask); }Quoted 16 lines
Quoted 4 lines
kernel/time/tick-broadcast.c +3
-1-static int tick_broadcast_set_event(struct clock_event_device *bc, +/* +* Set broadcast interrupt affinity +*/ kernel/time/tick-broadcast.c struct cpumask *tick_get_broadcast_oneshot_mask(void) · line 421
Quoted 10 lines
kernel/time/tick-broadcast.c +10
-0+static void tick_broadcast_set_affinity (struct clock_event_device *bc, +const struct cpumask *cpumask) +{ +if (!(bc->features &CLOCK_EVT_FEAT_DYNIRQ )) +return ; + +if (cpumask_equal (bc->cpumask, cpumask)) +return ; + + bc->cpumask = cpumask;This breaks with
CONFIG_CPUMASK_OFFSTACK=y.cpumask_copy()is your friend!This instruction copies the pointer, not the cpumask content.
Quoted 8 lines
Quoted 8 lines
bc->cpumaskis defined as a const struct cpumask * and is used to copy a cpumask pointer not the content.The cpumask parameter is a pointer to a global cpumask provided by the
cpumask_ofmacro.But to be in the safe side, I compiled tested with
CONFIG_CPUMASK_OFFSTACK=ywithout problem.
Hi Thomas,
thanks for merging the patch 1 and 2.
I was wondering if it would be possible to take the 3/4 and 4/4 otherwise the flag dependency will prevent to send those to the maintainer's tree until they gain visibility on it.
Signature · 5 lines
<http://www.linaro.org/> Linaro.org ? Open source software for ARM SoCs Follow Linaro: <http://www.facebook.com/pages/Linaro> Facebook | <http://twitter.com/#!/linaroorg> Twitter | <http://www.linaro.org/linaro-blog/> Blog
Re: [PATCH 2/4][V2] time : set broadcast irq affinity
Quoted 4 lines · Daniel Lezcano
On 03/06/2013 10:48 AM, Thomas Gleixner wrote:
I was wondering if it would be possible to take the 3/4 and 4/4 otherwise the flag dependency will prevent to send those to the maintainer's tree until they gain visibility on it.
I can take them with the ack of arm soc folks.
Thanks,
tglx
Arnd Bergmann
Acked-byRe: [PATCH 2/4][V2] time : set broadcast irq affinity
Quoted 7 lines · Thomas Gleixner
Quoted 4 lines · Daniel Lezcano
On 03/06/2013 10:48 AM, Thomas Gleixner wrote:
I was wondering if it would be possible to take the 3/4 and 4/4 otherwise the flag dependency will prevent to send those to the maintainer's tree until they gain visibility on it.I can take them with the ack of arm soc folks.
Sounds good,
tip-bot for Daniel Lezcano
merge bot[tip:timers/core] tick: Dynamically set broadcast irq affinity
In reply to Daniel Lezcanotick: Dynamically set broadcast irq affinity
Commit-ID: d2348fb6fdc6d671ad45b62db237f76c8c115603 Gitweb: http://git.kernel.org/tip/d2348fb6fdc6d671ad45b62db237f76c8c115603 Author: Daniel Lezcano <[email protected]> AuthorDate: Sat, 2 Mar 2013 11:10:11 +0100 Committer: Thomas Gleixner <[email protected]> CommitDate: Thu, 7 Mar 2013 16:13:26 +0100
tick: Dynamically set broadcast irq affinity
When a cpu goes to a deep idle state where its local timer is shutdown, it notifies the time frame work to use the broadcast timer instead. Unfortunately, the broadcast device could wake up any CPU, including an idle one which is not concerned by the wake up at all. So in the worst case an idle CPU will wake up to send an IPI to the CPU whose timer expired.
Provide an opt-in feature CLOCK_EVT_FEAT_DYNIRQ which tells the core that is should set the interrupt affinity of the broadcast interrupt to the cpu which has the earliest expiry time. This avoids unnecessary spurious wakeups and IPIs.
[ tglx: Adopted to cpumask rework, silenced an uninitialized warning,
massaged changelog ]
--- include/linux/clockchips.h | 5 +++++ kernel/time/tick-broadcast.c | 39 +++++++++++++++++++++++++++++++-------- 2 files changed, 36 insertions(+), 8 deletions(-)
include/linux/clockchips.h +5 -0
@@ -55,6 +55,11 @@ enum clock_event_nofitiers {#define CLOCK_EVT_FEAT_C3STOP 0x000008#define CLOCK_EVT_FEAT_DUMMY 0x000010 +/* +* Core shall set the interrupt affinity dynamically in broadcast mode +*/ +#define CLOCK_EVT_FEAT_DYNIRQ 0x000020 +/** * struct clock_event_device - clock event device descriptor * @event_handler: Assigned by the framework to be called by the low
kernel/time/tick-broadcast.c +31 -8
@@ -401,13 +401,34 @@ struct cpumask *tick_get_broadcast_oneshot_mask(void)return tick_broadcast_oneshot_mask; } -static int tick_broadcast_set_event (struct clock_event_device *bc, +/* +* Set broadcast interrupt affinity +*/ +static void tick_broadcast_set_affinity (struct clock_event_device *bc, +const struct cpumask *cpumask) +{ +if (!(bc->features &CLOCK_EVT_FEAT_DYNIRQ )) +return ; + +if (cpumask_equal (bc->cpumask, cpumask)) +return ; + + bc->cpumask = cpumask; +irq_set_affinity (bc->irq, bc->cpumask); +} + +static int tick_broadcast_set_event (struct clock_event_device *bc,int cpu,ktime_t expires,int force) { +int ret; +if (bc->mode !=CLOCK_EVT_MODE_ONESHOT )clockevents_set_mode (bc,CLOCK_EVT_MODE_ONESHOT ); -return clockevents_program_event (bc, expires, force); + ret =clockevents_program_event (bc, expires, force); +if (!ret) +tick_broadcast_set_affinity (bc,cpumask_of (cpu)); +return ret; }int tick_resume_broadcast_oneshot (struct clock_event_device *bc)
@@ -436,7 +457,7 @@ static void tick_handle_oneshot_broadcast(struct clock_event_device *dev) {struct tick_device *td;ktime_t now, next_event; -int cpu; +int cpu, next_cpu =0 ;raw_spin_lock (&tick_broadcast_lock); again:
@@ -447,10 +468,12 @@ again:/* Find all expired events */ for_each_cpu (cpu, tick_broadcast_oneshot_mask) { td = &per_cpu (tick_cpu_device, cpu); -if (td->evtdev->next_event.tv64 <= now.tv64) +if (td->evtdev->next_event.tv64 <= now.tv64) {cpumask_set_cpu (cpu, tmpmask); -else if (td->evtdev->next_event.tv64 < next_event.tv64) + }else if (td->evtdev->next_event.tv64 < next_event.tv64) { next_event.tv64 = td->evtdev->next_event.tv64; + next_cpu = cpu; + } }/*
@@ -473,7 +496,7 @@ again:* Rearm the broadcast device. If event expired, * repeat the above */ -if (tick_broadcast_set_event (dev, next_event,0 )) +if (tick_broadcast_set_event (dev, next_cpu, next_event,0 )) goto again; }raw_spin_unlock (&tick_broadcast_lock);
@@ -515,7 +538,7 @@ void tick_broadcast_oneshot_control(unsigned long reason)if (!cpumask_test_and_set_cpu (cpu, tick_broadcast_oneshot_mask)) {clockevents_set_mode (dev,CLOCK_EVT_MODE_SHUTDOWN );if (dev->next_event.tv64 < bc->next_event.tv64) -tick_broadcast_set_event (bc, dev->next_event,1 ); +tick_broadcast_set_event (bc, cpu, dev->next_event,1 ); } }else {if (cpumask_test_and_clear_cpu (cpu, tick_broadcast_oneshot_mask)) {
@@ -581,7 +604,7 @@ void tick_broadcast_setup_oneshot(struct clock_event_device *bc)clockevents_set_mode (bc,CLOCK_EVT_MODE_ONESHOT );tick_broadcast_init_next_event (tmpmask, tick_next_period); -tick_broadcast_set_event (bc, tick_next_period,1 ); +tick_broadcast_set_event (bc, cpu, tick_next_period,1 ); }else bc->next_event.tv64 =KTIME_MAX ; }else {
PATCH v2 3/4 ARM: nomadik: add dynamic irq flag to the timer
In reply to Daniel LezcanoAdd the dynamic irq affinity feature to the timer clock device.
--- drivers/clocksource/nomadik-mtu.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/clocksource/nomadik-mtu.c +2 -1
@@ -136,7 +136,8 @@ static void nmdk_clkevt_mode(enum clock_event_mode mode,static struct clock_event_device nmdk_clkevt = { .name ="mtu_1" , - .features =CLOCK_EVT_FEAT_ONESHOT |CLOCK_EVT_FEAT_PERIODIC , + .features =CLOCK_EVT_FEAT_ONESHOT |CLOCK_EVT_FEAT_PERIODIC | +CLOCK_EVT_FEAT_DYNIRQ , .rating =200 , .set_mode = nmdk_clkevt_mode, .set_next_event = nmdk_clkevt_next,
Signature · 1 line
1.7.9.5
PATCH v2 3/4 ARM: nomadik: add dynamic irq flag to the timer
In reply to Daniel LezcanoQuoted 6 lines
Add the dynamic irq affinity feature to the timer clock device.
Signed-off-by Daniel Lezcano <[email protected]>Reviewed-by Vincent Guittot <[email protected]>---
Hi Linus,
AFAIU, you are ok with this patch, Rickard acked it and Vincent reviewed it.
Thomas agreed to merge it into its tree as I requested because we will need to wait for your tree to gain visibility to this new flag before submitting this patch, which can take awhile.
Could you please ack it if it is ok for you ?
Thanks
-- Daniel
Quoted 20 lines
drivers/clocksource/nomadik-mtu.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)drivers/clocksource/nomadik-mtu.c +2
-1@@ -136,7 +136,8 @@ static void nmdk_clkevt_mode(enum clock_event_mode mode,static struct clock_event_device nmdk_clkevt = { .name ="mtu_1" , - .features =CLOCK_EVT_FEAT_ONESHOT |CLOCK_EVT_FEAT_PERIODIC , + .features =CLOCK_EVT_FEAT_ONESHOT |CLOCK_EVT_FEAT_PERIODIC | +CLOCK_EVT_FEAT_DYNIRQ , .rating =200 , .set_mode = nmdk_clkevt_mode, .set_next_event = nmdk_clkevt_next,Signature · 1 line
1.7.9.5
Re: [PATCH 3/4][V2] ARM: nomadik: add dynamic irq flag to the timer
Quoted 9 lines · Daniel Lezcano
Quoted 6 lines
Add the dynamic irq affinity feature to the timer clock device.
Signed-off-by Daniel Lezcano <[email protected]>Reviewed-by Vincent Guittot <[email protected]>---
Hi Linus,
Quoted 7 lines
AFAIU, you are ok with this patch, Rickard acked it and Vincent reviewed it.
Thomas agreed to merge it into its tree as I requested because we will need to wait for your tree to gain visibility to this new flag before submitting this patch, which can take awhile.
Could you please ack it if it is ok for you ?
I can provide a for-arm tree to pull as well. Either way fine with me.
Thanks,
tglx
Re: [PATCH 3/4][V2] ARM: nomadik: add dynamic irq flag to the timer
Quoted 13 lines · Thomas Gleixner
Quoted 11 lines · Daniel Lezcano
Quoted 6 lines
Add the dynamic irq affinity feature to the timer clock device.
Signed-off-by Daniel Lezcano <[email protected]>Reviewed-by Vincent Guittot <[email protected]>---
Hi Linus,
AFAIU, you are ok with this patch, Rickard acked it and Vincent reviewed it.
Quoted 7 lines
Quoted 5 lines
Thomas agreed to merge it into its tree as I requested because we will need to wait for your tree to gain visibility to this new flag before submitting this patch, which can take awhile.
Could you please ack it if it is ok for you ?
I can provide a for-arm tree to pull as well. Either way fine with me.
Hi Thomas,
the patch 3/4 and 4/4 have been acked-by the different maintainers.
I saw you created a 'for-arm' branch (thanks), but the patches 3/4 and 4/4 aren't there. Did I missed something ?
Thanks
-- Daniel
Signature · 5 lines
<http://www.linaro.org/> Linaro.org ? Open source software for ARM SoCs Follow Linaro: <http://www.facebook.com/pages/Linaro> Facebook | <http://twitter.com/#!/linaroorg> Twitter | <http://www.linaro.org/linaro-blog/> Blog
Re: [PATCH 3/4][V2] ARM: nomadik: add dynamic irq flag to the timer
In reply to Daniel LezcanoQuoted 9 lines · Daniel Lezcano
Quoted 6 lines
Add the dynamic irq affinity feature to the timer clock device.
Signed-off-by Daniel Lezcano <[email protected]>Reviewed-by Vincent Guittot <[email protected]>---
Hi Linus,
Quoted 7 lines
AFAIU, you are ok with this patch, Rickard acked it and Vincent reviewed it.
Thomas agreed to merge it into its tree as I requested because we will need to wait for your tree to gain visibility to this new flag before submitting this patch, which can take awhile.
Could you please ack it if it is ok for you ?
Oh of course!
Yours,
Linus Walleij
PATCH v2 4/4 ARM: timer-sp: Set dynamic irq affinity
In reply to Daniel LezcanoFrom: Viresh Kumar <[email protected]>
When a cpu goes to a deep idle state where its local timer is shutdown, it notifies the time frame work to use the broadcast timer instead.
Unfortunately, the broadcast device could wake up any CPU, including an idle one which is not concerned by the wake up at all.
This implies, in the worst case, an idle CPU will wake up to send an IPI to another idle cpu.
This patch fixes this for ARM platforms using timer-sp, by setting CLOCK_EVT_FEAT_DYNIRQ feature.
--- arch/arm/common/timer-sp.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
arch/arm/common/timer-sp.c +2 -1
@@ -158,7 +158,8 @@ static int sp804_set_next_event(unsigned long next, }static struct clock_event_device sp804_clockevent = { - .features =CLOCK_EVT_FEAT_PERIODIC |CLOCK_EVT_FEAT_ONESHOT , + .features =CLOCK_EVT_FEAT_PERIODIC |CLOCK_EVT_FEAT_ONESHOT | +CLOCK_EVT_FEAT_DYNIRQ , .set_mode = sp804_set_mode, .set_next_event = sp804_set_next_event, .rating =300 ,
Signature · 1 line
1.7.9.5
Re: [PATCH 4/4][V2] ARM: timer-sp: Set dynamic irq affinity
Quoted 11 lines · Daniel Lezcano
From: Viresh Kumar <[email protected]>
When a cpu goes to a deep idle state where its local timer is shutdown, it notifies the time frame work to use the broadcast timer instead.
Unfortunately, the broadcast device could wake up any CPU, including an idle one which is not concerned by the wake up at all.
This implies, in the worst case, an idle CPU will wake up to send an IPI to another idle cpu.
Quoted 6 lines
This patch fixes this for ARM platforms using timer-sp, by setting
CLOCK_EVT_FEAT_DYNIRQfeature.Signed-off-by Viresh Kumar <[email protected]>Signed-off-by Daniel Lezcano <[email protected]>---
Hi Russell,
I asked Thomas if he could take this patch in its tree but it falls under your umbrella. Thomas agreed to take it if you ack this patch.
Is this patch acceptable for you ?
Thanks in advance
-- Daniel
Quoted 18 lines
arch/arm/common/timer-sp.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)arch/arm/common/timer-sp.c +2
-1@@ -158,7 +158,8 @@ static int sp804_set_next_event(unsigned long next, }static struct clock_event_device sp804_clockevent = { - .features =CLOCK_EVT_FEAT_PERIODIC |CLOCK_EVT_FEAT_ONESHOT , + .features =CLOCK_EVT_FEAT_PERIODIC |CLOCK_EVT_FEAT_ONESHOT | +CLOCK_EVT_FEAT_DYNIRQ , .set_mode = sp804_set_mode, .set_next_event = sp804_set_next_event, .rating =300 ,
Signature · 5 lines
<http://www.linaro.org/> Linaro.org ? Open source software for ARM SoCs Follow Linaro: <http://www.facebook.com/pages/Linaro> Facebook | <http://twitter.com/#!/linaroorg> Twitter | <http://www.linaro.org/linaro-blog/> Blog
Russell King - ARM Linux
Acked-by Russell KingRe: [PATCH 4/4][V2] ARM: timer-sp: Set dynamic irq affinity
Quoted 19 lines · Daniel Lezcano
Quoted 11 lines · Daniel Lezcano
From: Viresh Kumar <[email protected]>
When a cpu goes to a deep idle state where its local timer is shutdown, it notifies the time frame work to use the broadcast timer instead.
Unfortunately, the broadcast device could wake up any CPU, including an idle one which is not concerned by the wake up at all.
This implies, in the worst case, an idle CPU will wake up to send an IPI to another idle cpu.
Quoted 6 lines
This patch fixes this for ARM platforms using timer-sp, by setting
CLOCK_EVT_FEAT_DYNIRQfeature.Signed-off-by Viresh Kumar <[email protected]>Signed-off-by Daniel Lezcano <[email protected]>---
Quoted 6 lines
Hi Russell,
I asked Thomas if he could take this patch in its tree but it falls under your umbrella. Thomas agreed to take it if you ack this patch.
Is this patch acceptable for you ?
It's simple enough.