metag: smp/hotplug fixes
cover metag: smp/hotplug fixes
Kirill Tkhai kindly pointed out a bug in the metag SMP code, prompting me to give it a bashing and weed out the hotplug bugs that have crept in (or been around for a while). This patchset fixes various issues noticed that I intend to apply for v3.11.
They're all short patches so please do give them a glance over in case I've misunderstood anything.
James Hogan (5):
metag: use clear_tasks_mm_cpumask()
metag: smp: enable irqs after set_cpu_online
metag: smp: don't spin waiting for CPU to start
metag: kick: prevent nested kick handlers
metag: cpu hotplug: route_irq: preserve irq mask
arch/metag/kernel/irq.c | 5 +++-- arch/metag/kernel/kick.c | 9 +++++++++ arch/metag/kernel/smp.c | 35 +++++++++++++---------------------- 3 files changed, 25 insertions(+), 24 deletions(-)
Signature · 1 line
1.8.1.2
PATCH 1/5 metag: use clear_tasks_mm_cpumask()
Checking for process->mm is not enough because process' main thread may exit or detach its mm via use_mm(), but other threads may still have a valid mm.
To fix this we would need to use find_lock_task_mm(), which would walk up all threads and returns an appropriate task (with task lock held).
clear_tasks_mm_cpumask() was introduced in v3.5-rc1 to fix this issue, so let's use it for metag too.
--- arch/metag/kernel/smp.c | 8 +------- 1 file changed, 1 insertion(+), 7 deletions(-)
arch/metag/kernel/smp.c +1 -7
@@ -276,7 +276,6 @@ static DECLARE_COMPLETION(cpu_killed); int __cpuexit __cpu_disable(void) {unsigned int cpu =smp_processor_id (); -struct task_struct *p;/* * Take this CPU offline. Once we clear this, we can't return,
@@ -296,12 +295,7 @@ int __cpuexit __cpu_disable(void)flush_cache_all ();local_flush_tlb_all (); -read_lock (&tasklist_lock); -for_each_process (p) { -if (p->mm) -cpumask_clear_cpu (cpu,mm_cpumask (p->mm)); - } -read_unlock (&tasklist_lock); +clear_tasks_mm_cpumask (cpu);return 0 ; }
Signature · 1 line
1.8.1.2
PATCH 2/5 metag: smp: enable irqs after set_cpu_online
In reply to James HoganIn secondary_start_kernel() interrupts should be enabled with local_irq_enable() after the cpu is marked as online with set_cpu_online(). Otherwise it's possible for a timer interrupt to trigger a softirq, which if the cpu is marked as offline may have it's affinity altered.
--- arch/metag/kernel/smp.c | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-)
arch/metag/kernel/smp.c +6 -5
@@ -379,12 +379,7 @@ asmlinkage void secondary_start_kernel(void)setup_priv (); -/* -* Enable local interrupts. -*/ -tbi_startup_interrupt (TBID_SIGNUM_TRT );notify_cpu_starting (cpu); -local_irq_enable ();pr_info ("CPU%u (thread %u): Booted secondary processor\n" , cpu, cpu_2_hwthread_id[cpu]);
@@ -398,6 +393,12 @@ asmlinkage void secondary_start_kernel(void)set_cpu_online (cpu, true);/* +* Enable local interrupts. +*/ +tbi_startup_interrupt (TBID_SIGNUM_TRT ); +local_irq_enable (); + +/* * OK, it's off to the idle thread for us */ cpu_startup_entry (CPUHP_ONLINE );
Signature · 1 line
1.8.1.2
Re: [PATCH 2/5] metag: smp: enable irqs after set_cpu_online
Quoted 10 lines · James Hogan
In
secondary_start_kernel()interrupts should be enabled withlocal_irq_enable()after the cpu is marked as online withset_cpu_online(). Otherwise it's possible for a timer interrupt to trigger a softirq, which if the cpu is marked as offline may have it's affinity altered.Reported-by Kirill Tkhai <[email protected]>Signed-off-by James Hogan <[email protected]>Cc Kirill Tkhai <[email protected]>Cc "Srivatsa S. Bhat" <[email protected]>
Re: [PATCH 2/5] metag: smp: enable irqs after set_cpu_online
In reply to James HoganQuoted 6 lines · James Hogan
In
secondary_start_kernel()interrupts should be enabled withlocal_irq_enable()after the cpu is marked as online withset_cpu_online(). Otherwise it's possible for a timer interrupt to trigger a softirq, which if the cpu is marked as offline may have it's affinity altered.
Quoted 6 lines
Reported-by Kirill Tkhai <[email protected]>Signed-off-by James Hogan <[email protected]>Cc Kirill Tkhai <[email protected]>Cc "Srivatsa S. Bhat" <[email protected]>Cc Thomas Gleixner <[email protected]>---
Regards,
Srivatsa S. Bhat
Quoted 34 lines
arch/metag/kernel/smp.c | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-)arch/metag/kernel/smp.c +6
-5@@ -379,12 +379,7 @@ asmlinkage void secondary_start_kernel(void)setup_priv (); -/* -* Enable local interrupts. -*/ -tbi_startup_interrupt (TBID_SIGNUM_TRT );notify_cpu_starting (cpu); -local_irq_enable ();pr_info ("CPU%u (thread %u): Booted secondary processor\n" , cpu, cpu_2_hwthread_id[cpu]);@@ -398,6 +393,12 @@ asmlinkage void secondary_start_kernel(void)set_cpu_online (cpu, true);/* +* Enable local interrupts. +*/ +tbi_startup_interrupt (TBID_SIGNUM_TRT ); +local_irq_enable (); + +/* * OK, it's off to the idle thread for us */ cpu_startup_entry (CPUHP_ONLINE );
PATCH 3/5 metag: smp: don't spin waiting for CPU to start
In reply to James HoganUse a completion to block until a secondary CPU has started up, like ARM do, instead of a loop of udelays.
On Meta, SMP is really SMT, with each "CPU" being a different hardware thread on the same Meta processor core, so as well as being more efficient and latency friendly, using a completion prevents the bogomips of the secondary CPU from being drastically skewed every time by the execution of the tight in-cache udelay loop on the other CPU.
--- arch/metag/kernel/smp.c | 16 ++++++---------- 1 file changed, 6 insertions(+), 10 deletions(-)
arch/metag/kernel/smp.c +6 -10
@@ -8,6 +8,7 @@* published by the Free Software Foundation. */ #include <linux/atomic.h> +#include <linux/completion.h> #include <linux/delay.h> #include <linux/init.h> #include <linux/spinlock.h>
@@ -62,6 +63,8 @@ static DEFINE_PER_CPU(struct ipi_data, ipi_data) = {static DEFINE_SPINLOCK (boot_lock); +static DECLARE_COMPLETION (cpu_running); +/* * "thread" is assumed to be a valid Meta hardware thread ID. */
@@ -235,20 +238,12 @@ int __cpuinit __cpu_up(unsigned int cpu, struct task_struct *idle)*/ ret =boot_secondary (thread, idle);if (ret ==0 ) { -unsigned long timeout; -/* * CPU was successfully started, wait for it * to come online or time out. */ - timeout = jiffies +HZ ; -while (time_before (jiffies, timeout)) { -if (cpu_online (cpu)) -break ; - -udelay (10 ); -barrier (); - } +wait_for_completion_timeout (&cpu_running, +msecs_to_jiffies (1000 ));if (!cpu_online (cpu)) ret = -EIO ;
@@ -391,6 +386,7 @@ asmlinkage void secondary_start_kernel(void)* OK, now it's safe to let the boot CPU continue */ set_cpu_online (cpu, true); +complete (&cpu_running);/* * Enable local interrupts.
Signature · 1 line
1.8.1.2
Re: [PATCH 3/5] metag: smp: don't spin waiting for CPU to start
Quoted 8 lines · James Hogan
Use a completion to block until a secondary CPU has started up, like ARM do, instead of a loop of udelays.
On Meta, SMP is really SMT, with each "CPU" being a different hardware thread on the same Meta processor core, so as well as being more efficient and latency friendly, using a completion prevents the bogomips of the secondary CPU from being drastically skewed every time by the execution of the tight in-cache udelay loop on the other CPU.
The least of our worries is the correctness of bogomips. But if that patch makes bogomips less bogus while fixing a real issue, please add my
Re: [PATCH 3/5] metag: smp: don't spin waiting for CPU to start
Quoted 10 lines · Thomas Gleixner
Quoted 8 lines · James Hogan
Use a completion to block until a secondary CPU has started up, like ARM do, instead of a loop of udelays.
On Meta, SMP is really SMT, with each "CPU" being a different hardware thread on the same Meta processor core, so as well as being more efficient and latency friendly, using a completion prevents the bogomips of the secondary CPU from being drastically skewed every time by the execution of the tight in-cache udelay loop on the other CPU.
Quoted 5 lines
The least of our worries is the correctness of bogomips. But if that patch makes bogomips less bogus while fixing a real issue, please add my
Acked-by Thomas Gleixner <[email protected]>
Yes, it's not so much the absolute value, but that secondary cpus will have a bogomips around 25% higher than the boot cpu, even though they should be about equal (ignoring the effects of other cpus). If I understand correctly, all else being equal this would make udelay block for a different amount of time on different cpus (although I haven't tried measuring it).
James
Re: [PATCH 3/5] metag: smp: don't spin waiting for CPU to start
In reply to James HoganQuoted 9 lines · James Hogan
Use a completion to block until a secondary CPU has started up, like ARM do, instead of a loop of udelays.
On Meta, SMP is really SMT, with each "CPU" being a different hardware thread on the same Meta processor core, so as well as being more efficient and latency friendly, using a completion prevents the bogomips of the secondary CPU from being drastically skewed every time by the execution of the tight in-cache udelay loop on the other CPU.
Quoted 4 lines
Signed-off-by James Hogan <[email protected]>Cc "Srivatsa S. Bhat" <[email protected]>Cc Thomas Gleixner <[email protected]>---
Regards,
Srivatsa S. Bhat
Quoted 56 lines
arch/metag/kernel/smp.c | 16 ++++++---------- 1 file changed, 6 insertions(+), 10 deletions(-)arch/metag/kernel/smp.c +6
-10@@ -8,6 +8,7 @@* published by the Free Software Foundation. */ #include <linux/atomic.h> +#include <linux/completion.h> #include <linux/delay.h> #include <linux/init.h> #include <linux/spinlock.h> @@ -62,6 +63,8 @@ static DEFINE_PER_CPU(struct ipi_data, ipi_data) = {static DEFINE_SPINLOCK (boot_lock); +static DECLARE_COMPLETION (cpu_running); +/* * "thread" is assumed to be a valid Meta hardware thread ID. */ @@ -235,20 +238,12 @@ int __cpuinit __cpu_up(unsigned int cpu, struct task_struct *idle)*/ ret =boot_secondary (thread, idle);if (ret ==0 ) { -unsigned long timeout; -/* * CPU was successfully started, wait for it * to come online or time out. */ - timeout = jiffies +HZ ; -while (time_before (jiffies, timeout)) { -if (cpu_online (cpu)) -break ; - -udelay (10 ); -barrier (); - } +wait_for_completion_timeout (&cpu_running, +msecs_to_jiffies (1000 ));if (!cpu_online (cpu)) ret = -EIO ;@@ -391,6 +386,7 @@ asmlinkage void secondary_start_kernel(void)* OK, now it's safe to let the boot CPU continue */ set_cpu_online (cpu, true); +complete (&cpu_running);/* * Enable local interrupts.
PATCH 4/5 metag: kick: prevent nested kick handlers
In reply to James HoganThe main kick trigger handler iterates a list of kick handlers and calls each one. This is done with the kick_handlers_lock spin lock held, but this causes a problem on SMP where IPIs are implemented with kicks. A reschedule IPI calls scheduler_ipi() which uses irq_enter() and irq_exit(). This results in the scheduler being invoked with kick_handlers_lock held which can result in a nested kick trigger attempting to acquire the lock, resulting in deadlock.
irq_enter() and irq_exit() can nest, so call them from the main kick interrupt handler so that softirqs are only handled after kick_handlers_lock is released.
--- arch/metag/kernel/kick.c | 9 +++++++++ 1 file changed, 9 insertions(+)
arch/metag/kernel/kick.c +9 -0
@@ -26,6 +26,8 @@* pass it as an argument. */ #include <linux/export.h> +#include <linux/hardirq.h> +#include <linux/irq.h> #include <linux/kernel.h> #include <linux/mm.h> #include <linux/types.h>
@@ -66,6 +68,7 @@ EXPORT_SYMBOL(kick_unregister_func);TBIRES kick_handler (TBIRES State,int SigNum,int Triggers,int Inst,PTBI pTBI) { +struct pt_regs *old_regs;struct kick_irq_handler *kh;struct list_head *lh;int handled =0 ;
@@ -79,6 +82,9 @@ kick_handler(TBIRES State, int SigNum, int Triggers, int Inst, PTBI pTBI)trace_hardirqs_off (); + old_regs =set_irq_regs ((struct pt_regs *)State.Sig.pCtx); +irq_enter (); +/* * There is no need to disable interrupts here because we * can't nest KICK interrupts in a KICK interrupt handler.
@@ -97,5 +103,8 @@ kick_handler(TBIRES State, int SigNum, int Triggers, int Inst, PTBI pTBI)WARN_ON (!handled); +irq_exit (); +set_irq_regs (old_regs); +return tail_end (ret); }
Signature · 1 line
1.8.1.2
Re: [PATCH 4/5] metag: kick: prevent nested kick handlers
Quoted 11 lines · James Hogan
The main kick trigger handler iterates a list of kick handlers and calls each one. This is done with the
kick_handlers_lockspin lock held, but this causes a problem on SMP where IPIs are implemented with kicks. A reschedule IPI callsscheduler_ipi()which usesirq_enter()andirq_exit(). This results in the scheduler being invoked withkick_handlers_lockheld which can result in a nested kick trigger attempting to acquire the lock, resulting in deadlock.
irq_enter()andirq_exit()can nest, so call them from the main kick interrupt handler so that softirqs are only handled afterkick_handlers_lockis released.
This changelog is confusing. What I decode from the patch is, that you are adding a missing irq_enter/exit pair to the kick_handler, right ?
Thanks,
tglx
Re: [PATCH 4/5] metag: kick: prevent nested kick handlers
Quoted 10 lines · Thomas Gleixner
Quoted 8 lines · James Hogan
The main kick trigger handler iterates a list of kick handlers and calls each one. This is done with the
kick_handlers_lockspin lock held, but this causes a problem on SMP where IPIs are implemented with kicks. A reschedule IPI callsscheduler_ipi()which usesirq_enter()andirq_exit(). This results in the scheduler being invoked withkick_handlers_lockheld which can result in a nested kick trigger attempting to acquire the lock, resulting in deadlock.
Quoted 6 lines
Quoted 3 lines
irq_enter()andirq_exit()can nest, so call them from the main kick interrupt handler so that softirqs are only handled afterkick_handlers_lockis released.This changelog is confusing. What I decode from the patch is, that you are adding a missing irq_enter/exit pair to the
kick_handler, right ?
Yes. Previously the outermost pair of irq_enter/exit was inside the spin lock critical section (inside scheduler_ipi). so soft-irqs (apparently including the scheduler) would run from irq_exit with the spinlock still held. Now it waits until the new outermost irq_exit(), after the spin lock is released.
I should probably have increased the number of lines of diff context.
http://lxr.linux.no/#linux+v3.10/arch/metag/kernel/kick.c#L66
Cheers
James
Re: [PATCH 4/5] metag: kick: prevent nested kick handlers
Quoted 18 lines · James Hogan
Quoted 10 lines · Thomas Gleixner
Quoted 8 lines · James Hogan
The main kick trigger handler iterates a list of kick handlers and calls each one. This is done with the
kick_handlers_lockspin lock held, but this causes a problem on SMP where IPIs are implemented with kicks. A reschedule IPI callsscheduler_ipi()which usesirq_enter()andirq_exit(). This results in the scheduler being invoked withkick_handlers_lockheld which can result in a nested kick trigger attempting to acquire the lock, resulting in deadlock.Quoted 6 lines
Quoted 3 lines
irq_enter()andirq_exit()can nest, so call them from the main kick interrupt handler so that softirqs are only handled afterkick_handlers_lockis released.This changelog is confusing. What I decode from the patch is, that you are adding a missing irq_enter/exit pair to the
kick_handler, right ?
Quoted 8 lines
Yes. Previously the outermost pair of irq_enter/exit was inside the spin lock critical section (inside
scheduler_ipi). so soft-irqs (apparently including the scheduler) would run fromirq_exitwith the spinlock still held. Now it waits until the new outermostirq_exit(), after the spin lock is released.I should probably have increased the number of lines of diff context.
http://lxr.linux.no/#linux+v3.10/arch/metag/kernel/kick.c#L66
No, you should have written the changelog less confusing. I can see the context by just looking at the current and the patched code.
The explanation you provided right now that the outermost pair of irq_enter/exit was inside the spin locked section matches the patch and makes sense.
So the real issue is, that all of your various interrupts come from the same place, i.e. kick_handler. The kick_handler locks a global lock and calls the handlers which have registered. These handlers (partially calling generic code) might invoke irq_enter/exit pairs, which leads to the underlying problem.
If irq_exit() results in a nest count of hard interrupts = 0, then it invokes eventually pending soft interrupts. The softirq callbacks can issue an IPI of some sorts and because this IPI will end up in the kick handler, the systems livelocks on the already held global lock which protects the kick_handler list. This can happen from any invoked handler, because the kick_handler is missing an irq_enter/pair preventing a kick_handler invocation recursion.
The obvious fix for now is to add an irq_enter/exit() pair around the spin locked section in the kick handler, so irq_exit() invocations of subhandlers wont lead to softirq execution.
The more elegant fix would be to replace that global lock which is not really scalable with an RCU protected list of subscribed handlers.
The most elegant fix would be to have separate entry points for various exceptions, but that seems to be an architectural issue.
Thanks,
tglx
Re: [PATCH 4/5] metag: kick: prevent nested kick handlers
Quoted 31 lines · Thomas Gleixner
Quoted 4 lines · James Hogan
Quoted 3 lines · Thomas Gleixner
Quoted 1 line · James Hogan
The main kick trigger handler iterates a list of kick handlers and
calls
Quoted 1 line
Quoted 1 line
Quoted 1 line
each one. This is done with the
kick_handlers_lockspin lock held,but
Quoted 1 line
Quoted 1 line
Quoted 1 line
this causes a problem on SMP where IPIs are implemented with
kicks. A
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
reschedule IPI calls
scheduler_ipi()which usesirq_enter()andirq_exit(). This results in the scheduler being invoked withkick_handlers_lockheld which can result in a nested kick trigger attempting to acquire the lock, resulting in deadlock.
irq_enter()andirq_exit()can nest, so call them from the mainkick
Quoted 4 lines
Quoted 4 lines
Quoted 2 lines
interrupt handler so that softirqs are only handled after
kick_handlers_lockis released.This changelog is confusing. What I decode from the patch is, that
you
Quoted 1 line
Quoted 1 line
are adding a missing irq_enter/exit pair to the
kick_handler, right?
Quoted 7 lines
Yes. Previously the outermost pair of irq_enter/exit was inside the spin lock critical section (inside
scheduler_ipi). so soft-irqs (apparently including the scheduler) would run fromirq_exitwith the spinlock still held. Now it waits until the new outermostirq_exit(), after the spin lock is released.
Quoted 5 lines
Quoted 2 lines
I should probably have increased the number of lines of diff context.
http://lxr.linux.no/#linux+v3.10/arch/metag/kernel/kick.c#L66No, you should have written the changelog less confusing. I can see the context by just looking at the current and the patched code.
Yes, i agree, sorry it wasn't clear.
Quoted 5 lines
The explanation you provided right now that the outermost pair of irq_enter/exit was inside the spin locked section matches the patch and makes sense.
So the real issue is, that all of your various interrupts come from
Only kicks, which are the hardware IPI mechanism (in practice only used for SMP IPIs and for communication with non-linux hw threads). Timer/peripheral interrupts come via different handlers.
Quoted 5 lines
the same place, i.e.
kick_handler. Thekick_handlerlocks a global lock and calls the handlers which have registered. These handlers (partially calling generic code) might invoke irq_enter/exit pairs, which leads to the underlying problem.
Quoted 7 lines
If
irq_exit()results in a nest count of hard interrupts = 0, then it invokes eventually pending soft interrupts. The softirq callbacks can issue an IPI of some sorts and because this IPI will end up in the kick handler, the systems livelocks on the already held global lock which protects thekick_handlerlist. This can happen from any invoked handler, because thekick_handleris missing an irq_enter/pair preventing akick_handlerinvocation recursion.
Yes, that's my understanding of the problem.
Quoted 7 lines
The obvious fix for now is to add an irq_enter/exit() pair around the spin locked section in the kick handler, so
irq_exit()invocations of subhandlers wont lead to softirq execution.The more elegant fix would be to replace that global lock which is not really scalable with an RCU protected list of subscribed handlers.
Yes, I considered switching it to an RCU list (thanks for confirming that), but this patch looks sensible to me even with rcu, so I figured that can be a separate patch.
Quoted 2 lines
The most elegant fix would be to have separate entry points for various exceptions, but that seems to be an architectural issue.
thanks for taking the time to review and for the other acks. i'll rewrite the commit message of this patch and resend.
cheers
James
PATCH 5/5 metag: cpu hotplug: route_irq: preserve irq mask
In reply to James HoganThe route_irq() function needs to preserve the irq mask by using the _irqsave/irqrestore variants of raw spin lock functions instead of the _irq variants. This is because it is called from __cpu_disable() (via migrate_irqs()), which is called with IRQs disabled, so using the _irq variants re-enables IRQs.
This appears to have been causing occasional hits of the BUG_ON(!irqs_disabled()) in __irq_work_run() during CPU hotplug soak testing:
BUG: failure at kernel/irq_work.c:122/__irq_work_run()!
--- arch/metag/kernel/irq.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
arch/metag/kernel/irq.c +3 -2
@@ -279,11 +279,12 @@ static void route_irq(struct irq_data *data, unsigned int irq, unsigned int cpu) {struct irq_desc *desc =irq_to_desc (irq);struct irq_chip *chip =irq_data_get_irq_chip (data); +unsigned long flags; -raw_spin_lock_irq (&desc->lock); +raw_spin_lock_irqsave (&desc->lock, flags);if (chip->irq_set_affinity) chip->irq_set_affinity (data,cpumask_of (cpu), false); -raw_spin_unlock_irq (&desc->lock); +raw_spin_unlock_irqrestore (&desc->lock, flags); }/*
Signature · 1 line
1.8.1.2
Re: [PATCH 5/5] metag: cpu hotplug: route_irq: preserve irq mask
Quoted 6 lines · James Hogan
The
route_irq()function needs to preserve the irq mask by using the _irqsave/irqrestore variants of raw spin lock functions instead of the_irqvariants. This is because it is called from__cpu_disable()(viamigrate_irqs()), which is called with IRQs disabled, so using the_irqvariants re-enables IRQs.
Quoted 6 lines
This appears to have been causing occasional hits of the BUG_ON(!irqs_disabled()) in
__irq_work_run()during CPU hotplug soak testing:BUG: failure at kernel/irq_work.c:122/__irq_work_run()!Signed-off-by James Hogan <[email protected]>
Quoted 28 lines
--- arch/metag/kernel/irq.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)arch/metag/kernel/irq.c +3
-2@@ -279,11 +279,12 @@ static void route_irq(struct irq_data *data, unsigned int irq, unsigned int cpu) {struct irq_desc *desc =irq_to_desc (irq);struct irq_chip *chip =irq_data_get_irq_chip (data); +unsigned long flags; -raw_spin_lock_irq (&desc->lock); +raw_spin_lock_irqsave (&desc->lock, flags);if (chip->irq_set_affinity) chip->irq_set_affinity (data,cpumask_of (cpu), false); -raw_spin_unlock_irq (&desc->lock); +raw_spin_unlock_irqrestore (&desc->lock, flags); }/* Signature · 1 line
1.8.1.2
Re: [PATCH 5/5] metag: cpu hotplug: route_irq: preserve irq mask
In reply to James HoganQuoted 6 lines · James Hogan
The
route_irq()function needs to preserve the irq mask by using the _irqsave/irqrestore variants of raw spin lock functions instead of the_irqvariants. This is because it is called from__cpu_disable()(viamigrate_irqs()), which is called with IRQs disabled, so using the_irqvariants re-enables IRQs.
Quoted 7 lines
This appears to have been causing occasional hits of the BUG_ON(!irqs_disabled()) in
__irq_work_run()during CPU hotplug soak testing:BUG: failure at kernel/irq_work.c:122/__irq_work_run()!Signed-off-by James Hogan <[email protected]>---
Regards,
Srivatsa S. Bhat
Quoted 23 lines
arch/metag/kernel/irq.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)arch/metag/kernel/irq.c +3
-2@@ -279,11 +279,12 @@ static void route_irq(struct irq_data *data, unsigned int irq, unsigned int cpu) {struct irq_desc *desc =irq_to_desc (irq);struct irq_chip *chip =irq_data_get_irq_chip (data); +unsigned long flags; -raw_spin_lock_irq (&desc->lock); +raw_spin_lock_irqsave (&desc->lock, flags);if (chip->irq_set_affinity) chip->irq_set_affinity (data,cpumask_of (cpu), false); -raw_spin_unlock_irq (&desc->lock); +raw_spin_unlock_irqrestore (&desc->lock, flags); }/*
Re: [PATCH 5/5] metag: cpu hotplug: route_irq: preserve irq mask
Quoted 12 lines · Srivatsa S. Bhat
Quoted 11 lines · James Hogan
The
route_irq()function needs to preserve the irq mask by using the _irqsave/irqrestore variants of raw spin lock functions instead of the_irqvariants. This is because it is called from__cpu_disable()(viamigrate_irqs()), which is called with IRQs disabled, so using the_irqvariants re-enables IRQs.This appears to have been causing occasional hits of the BUG_ON(!irqs_disabled()) in
__irq_work_run()during CPU hotplug soak testing:BUG: failure at kernel/irq_work.c:122/__irq_work_run()!
Quoted 4 lines
Quoted 2 lines
Signed-off-by James Hogan <[email protected]>---
Reviewed-by Srivatsa S. Bhat <[email protected]>
Thanks for taking the time to review these patches
Cheers
James