ARM: Kconfig: allow full nohz CPU accounting
tip-bot for Kevin Hilman
merge botCommit-ID: 31c1fc8187158cb80ccd57c19e024c55af901797 Gitweb: http://git.kernel.org/tip/31c1fc8187158cb80ccd57c19e024c55af901797 Author: Kevin Hilman <[email protected]> AuthorDate: Mon, 16 Sep 2013 15:28:22 -0700 Committer: Frederic Weisbecker <[email protected]> CommitDate: Mon, 30 Sep 2013 15:37:05 +0200
ARM: Kconfig: allow full nohz CPU accounting
With the 64-bit requirement removed from VIRT_CPU_ACCOUNTING_GEN, allow ARM platforms to enable it. Since VIRT_CPU_ACCOUNTING_GEN is a dependency for full NO_HZ, this allows ARM platforms to enable full NO_HZ as well.
--- arch/arm/Kconfig | 1 + 1 file changed, 1 insertion(+)
arch/arm/Kconfig +1 -0
@@ -54,6 +54,7 @@ config ARM select HAVE_REGS_AND_STACK_ACCESS_API select HAVE_SYSCALL_TRACEPOINTS select HAVE_UID16 + select HAVE_VIRT_CPU_ACCOUNTING_GEN select IRQ_FORCED_THREADING select KTIME_SCALAR select MODULES_USE_ELF_REL
Quoted 7 lines · tip-bot for Kevin Hilman
Commit-ID: 31c1fc8187158cb80ccd57c19e024c55af901797 Gitweb: http://git.kernel.org/tip/31c1fc8187158cb80ccd57c19e024c55af901797 Author: Kevin Hilman <[email protected]> AuthorDate: Mon, 16 Sep 2013 15:28:22 -0700 Committer: Frederic Weisbecker <[email protected]> CommitDate: Mon, 30 Sep 2013 15:37:05 +0200
Quoted 6 lines
ARM:
Kconfig: allow full nohz CPU accountingWith the 64-bit requirement removed from
VIRT_CPU_ACCOUNTING_GEN, allow ARM platforms to enable it. SinceVIRT_CPU_ACCOUNTING_GENis a dependency for fullNO_HZ, this allows ARM platforms to enable fullNO_HZas well.
I realize that arm doesn't implement irq work raise. It hooks on the timer to execute pending irq work. You'll need to implement arch_irq_work_raise() and implement some sort of self IPI to trigger irq work. The full dynticks subsystem relies on that to work correctly. It often use that to re-evaluate and possibly restart the tick after scheduler, posix timer updates, etc...
That notwithstanding that if you have no tick, tick based irq work can't work.
Thanks.
Quoted 23 lines
Signed-off-by Kevin Hilman <[email protected]>Cc Ingo Molnar <[email protected]>Cc Russell King <[email protected]>Cc Paul E. McKenney <[email protected]>Cc Arm Linux <[email protected]>Signed-off-by Frederic Weisbecker <[email protected]>--- arch/arm/Kconfig | 1 + 1 file changed, 1 insertion(+)arch/arm/Kconfig +1
-0@@ -54,6 +54,7 @@ config ARM select HAVE_REGS_AND_STACK_ACCESS_API select HAVE_SYSCALL_TRACEPOINTS select HAVE_UID16 + select HAVE_VIRT_CPU_ACCOUNTING_GEN select IRQ_FORCED_THREADING select KTIME_SCALAR select MODULES_USE_ELF_REL
Quoted 15 lines · Frederic Weisbecker
Quoted 7 lines · tip-bot for Kevin Hilman
Commit-ID: 31c1fc8187158cb80ccd57c19e024c55af901797 Gitweb: http://git.kernel.org/tip/31c1fc8187158cb80ccd57c19e024c55af901797 Author: Kevin Hilman <[email protected]> AuthorDate: Mon, 16 Sep 2013 15:28:22 -0700 Committer: Frederic Weisbecker <[email protected]> CommitDate: Mon, 30 Sep 2013 15:37:05 +0200Quoted 6 lines
ARM:
Kconfig: allow full nohz CPU accountingWith the 64-bit requirement removed from
VIRT_CPU_ACCOUNTING_GEN, allow ARM platforms to enable it. SinceVIRT_CPU_ACCOUNTING_GENis a dependency for fullNO_HZ, this allows ARM platforms to enable fullNO_HZas well.
Quoted 9 lines
I realize that arm doesn't implement irq work raise. It hooks on the timer to execute pending irq work. You'll need to implement
arch_irq_work_raise()and implement some sort of self IPI to trigger irq work. The full dynticks subsystem relies on that to work correctly. It often use that to re-evaluate and possibly restart the tick after scheduler, posix timer updates, etc...That notwithstanding that if you have no tick, tick based irq work can't work.
Something like this? The only problem is that this latches on to the SMP support for IPIs. If CONFIG_SMP is disabled we won't get arch_irq_work_raise() but perhaps that isn't so big of a deal? Or we should think about exposing IPI support to UP systems on ARM.
----8<-----
arch/arm/include/asm/hardirq.h | 2 +- arch/arm/kernel/smp.c | 18 ++++++++++++++++++ 2 files changed, 19 insertions(+), 1 deletion(-)
arch/arm/include/asm/hardirq.h +1 -1
@@ -5,7 +5,7 @@#include <linux/threads.h> #include <asm/irq.h> -#define NR_IPI 6 +#define NR_IPI 7 typedef struct { unsigned int __softirq_pending;
arch/arm/kernel/smp.c +18 -0
@@ -25,6 +25,7 @@#include <linux/clockchips.h> #include <linux/completion.h> #include <linux/cpufreq.h> +#include <linux/irq_work.h> #include <linux/atomic.h> #include <asm/smp.h>
@@ -66,6 +67,7 @@ enum ipi_msg_type {IPI_CALL_FUNC ,IPI_CALL_FUNC_SINGLE ,IPI_CPU_STOP , + IPI_IRQ_WORK, };static DECLARE_COMPLETION (cpu_running);
@@ -448,6 +450,13 @@ void arch_send_call_function_single_ipi(int cpu)smp_cross_call (cpumask_of (cpu),IPI_CALL_FUNC_SINGLE ); } +#ifdef CONFIG_IRQ_WORK +void arch_irq_work_raise (void ) +{ +smp_cross_call (cpumask_of (smp_processor_id ()),IPI_IRQ_WORK ); +} +#endif +static const char *ipi_types[NR_IPI ] = { #define S(x,s) [x] = sS (IPI_WAKEUP ,"CPU wakeup interrupts" ),
@@ -456,6 +465,7 @@ static const char *ipi_types[NR_IPI] = {S (IPI_CALL_FUNC ,"Function call interrupts" ),S (IPI_CALL_FUNC_SINGLE ,"Single function call interrupts" ),S (IPI_CPU_STOP ,"CPU stop interrupts" ), + S(IPI_IRQ_WORK, "IRQ work interrupts"), };void show_ipi_list (struct seq_file *p,int prec)
@@ -565,6 +575,14 @@ void handle_IPI(int ipinr, struct pt_regs *regs)irq_exit ();break ; +#ifdef CONFIG_IRQ_WORK +case IPI_IRQ_WORK : +irq_enter (); +irq_work_run (); +irq_exit (); +break ; +#endif +default :printk (KERN_CRIT "CPU%u: Unknown IPI message 0x%x\n" , cpu, ipinr);
Signature · 2 lines
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation
2013/10/18 Stephen Boyd <[email protected]>:
Quoted 96 lines · Frederic Weisbecker
Quoted 15 lines · tip-bot for Kevin Hilman
Quoted 7 lines
Commit-ID: 31c1fc8187158cb80ccd57c19e024c55af901797 Gitweb: http://git.kernel.org/tip/31c1fc8187158cb80ccd57c19e024c55af901797 Author: Kevin Hilman <[email protected]> AuthorDate: Mon, 16 Sep 2013 15:28:22 -0700 Committer: Frederic Weisbecker <[email protected]> CommitDate: Mon, 30 Sep 2013 15:37:05 +0200Quoted 6 lines
ARM:
Kconfig: allow full nohz CPU accountingWith the 64-bit requirement removed from
VIRT_CPU_ACCOUNTING_GEN, allow ARM platforms to enable it. SinceVIRT_CPU_ACCOUNTING_GENis a dependency for fullNO_HZ, this allows ARM platforms to enable fullNO_HZas well.Quoted 9 lines
I realize that arm doesn't implement irq work raise. It hooks on the timer to execute pending irq work. You'll need to implement
arch_irq_work_raise()and implement some sort of self IPI to trigger irq work. The full dynticks subsystem relies on that to work correctly. It often use that to re-evaluate and possibly restart the tick after scheduler, posix timer updates, etc...That notwithstanding that if you have no tick, tick based irq work can't work.
Something like this? The only problem is that this latches on to the SMP support for IPIs. If
CONFIG_SMPis disabled we won't getarch_irq_work_raise()but perhaps that isn't so big of a deal? Or we should think about exposing IPI support to UP systems on ARM.----8<-----
arch/arm/include/asm/hardirq.h | 2 +- arch/arm/kernel/smp.c | 18 ++++++++++++++++++ 2 files changed, 19 insertions(+), 1 deletion(-)arch/arm/include/asm/hardirq.h +1
-1@@ -5,7 +5,7 @@#include <linux/threads.h> #include <asm/irq.h> -#define NR_IPI 6 +#define NR_IPI 7 typedef struct { unsigned int __softirq_pending;arch/arm/kernel/smp.c +11
-0@@ -25,6 +25,7 @@#include <linux/clockchips.h> #include <linux/completion.h> #include <linux/cpufreq.h> +#include <linux/irq_work.h> #include <linux/atomic.h> #include <asm/smp.h> @@ -66,6 +67,7 @@ enum ipi_msg_type {IPI_CALL_FUNC ,IPI_CALL_FUNC_SINGLE ,IPI_CPU_STOP , + IPI_IRQ_WORK, };static DECLARE_COMPLETION (cpu_running);@@ -448,6 +450,13 @@ void arch_send_call_function_single_ipi(int cpu)smp_cross_call (cpumask_of (cpu),IPI_CALL_FUNC_SINGLE ); } +#ifdef CONFIG_IRQ_WORK +void arch_irq_work_raise (void ) +{ +smp_cross_call (cpumask_of (smp_processor_id ()),IPI_IRQ_WORK ); +} +#endif +static const char *ipi_types[NR_IPI ] = { #define S(x,s) [x] = sS (IPI_WAKEUP ,"CPU wakeup interrupts" ),@@ -456,6 +465,7 @@ static const char *ipi_types[NR_IPI] = {S (IPI_CALL_FUNC ,"Function call interrupts" ),S (IPI_CALL_FUNC_SINGLE ,"Single function call interrupts" ),S (IPI_CPU_STOP ,"CPU stop interrupts" ), + S(IPI_IRQ_WORK, "IRQ work interrupts"), };void show_ipi_list (struct seq_file *p,int prec)@@ -565,6 +575,14 @@ void handle_IPI(int ipinr, struct pt_regs *regs)irq_exit ();break ; +#ifdef CONFIG_IRQ_WORK
arch/arm/kernel/smp.c void handle_IPI(int ipinr, struct pt_regs *regs) · line 588
Quoted 10 lines
arch/arm/kernel/smp.c +7
-0+case IPI_IRQ_WORK : +irq_enter (); +irq_work_run (); +irq_exit (); +break ; +#endif +default :printk (KERN_CRIT "CPU%u: Unknown IPI message 0x%x\n" , cpu, ipinr);
Yeah that looks good! Indeed it's no big deal if it only support SMP for now since full dynticks only works on SMP. It might work on UP one day but we are not yet rushing on that :)
Hi Stephen,
Quoted 22 lines · Stephen Boyd
Quoted 21 lines · Frederic Weisbecker
Quoted 7 lines · tip-bot for Kevin Hilman
Commit-ID: 31c1fc8187158cb80ccd57c19e024c55af901797 Gitweb: http://git.kernel.org/tip/31c1fc8187158cb80ccd57c19e024c55af901797 Author: Kevin Hilman <[email protected]> AuthorDate: Mon, 16 Sep 2013 15:28:22 -0700 Committer: Frederic Weisbecker <[email protected]> CommitDate: Mon, 30 Sep 2013 15:37:05 +0200Quoted 6 lines
ARM:
Kconfig: allow full nohz CPU accountingWith the 64-bit requirement removed from
VIRT_CPU_ACCOUNTING_GEN, allow ARM platforms to enable it. SinceVIRT_CPU_ACCOUNTING_GENis a dependency for fullNO_HZ, this allows ARM platforms to enable fullNO_HZas well.I realize that arm doesn't implement irq work raise. It hooks on the timer to execute pending irq work. You'll need to implement
arch_irq_work_raise()and implement some sort of self IPI to trigger irq work. The full dynticks subsystem relies on that to work correctly. It often use that to re-evaluate and possibly restart the tick after scheduler, posix timer updates, etc...
Quoted 8 lines
Quoted 3 lines
That notwithstanding that if you have no tick, tick based irq work can't work.
Something like this? The only problem is that this latches on to the SMP support for IPIs. If
CONFIG_SMPis disabled we won't getarch_irq_work_raise()but perhaps that isn't so big of a deal? Or we should think about exposing IPI support to UP systems on ARM.
Care to submit a proper patch for this to LAKML? or if you prefer I can
do it (keeping your authorship.) We need this for proper full NOHZ
support on ARM. Thanks.
Kevin
Quoted 27 lines · Kevin Hilman
Hi Stephen,
Quoted 24 lines · Stephen Boyd
Quoted 23 lines · Frederic Weisbecker
Quoted 7 lines · tip-bot for Kevin Hilman
Commit-ID: 31c1fc8187158cb80ccd57c19e024c55af901797 Gitweb: http://git.kernel.org/tip/31c1fc8187158cb80ccd57c19e024c55af901797 Author: Kevin Hilman <[email protected]> AuthorDate: Mon, 16 Sep 2013 15:28:22 -0700 Committer: Frederic Weisbecker <[email protected]> CommitDate: Mon, 30 Sep 2013 15:37:05 +0200Quoted 6 lines
ARM:
Kconfig: allow full nohz CPU accountingWith the 64-bit requirement removed from
VIRT_CPU_ACCOUNTING_GEN, allow ARM platforms to enable it. SinceVIRT_CPU_ACCOUNTING_GENis a dependency for fullNO_HZ, this allows ARM platforms to enable fullNO_HZas well.I realize that arm doesn't implement irq work raise. It hooks on the timer to execute pending irq work. You'll need to implement
arch_irq_work_raise()and implement some sort of self IPI to trigger irq work. The full dynticks subsystem relies on that to work correctly. It often use that to re-evaluate and possibly restart the tick after scheduler, posix timer updates, etc...That notwithstanding that if you have no tick, tick based irq work can't work.
Quoted 7 lines
Quoted 4 lines
Something like this? The only problem is that this latches on to the SMP support for IPIs. If
CONFIG_SMPis disabled we won't getarch_irq_work_raise()but perhaps that isn't so big of a deal? Or we should think about exposing IPI support to UP systems on ARM.Care to submit a proper patch for this to LAKML? or if you prefer I can
do it (keeping your authorship.) We need this for proper full NOHZ
support on ARM. Thanks.
Sure no problem. Let me write up some sort of commit text and I'll send it off again.
Signature · 2 lines
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation
PATCH 1/1 ARM: Support arch_irq_work_raise() via self IPIs
In reply to Kevin HilmanThis will allow the scheduler tick to be restarted if we're in full NOHZ mode.
--- arch/arm/include/asm/hardirq.h | 2 +- arch/arm/kernel/smp.c | 18 ++++++++++++++++++ 2 files changed, 19 insertions(+), 1 deletion(-)
arch/arm/include/asm/hardirq.h +1 -1
@@ -5,7 +5,7 @@#include <linux/threads.h> #include <asm/irq.h> -#define NR_IPI 6 +#define NR_IPI 7 typedef struct { unsigned int __softirq_pending;
arch/arm/kernel/smp.c +18 -0
@@ -25,6 +25,7 @@#include <linux/clockchips.h> #include <linux/completion.h> #include <linux/cpufreq.h> +#include <linux/irq_work.h> #include <linux/atomic.h> #include <asm/smp.h>
@@ -66,6 +67,7 @@ enum ipi_msg_type {IPI_CALL_FUNC ,IPI_CALL_FUNC_SINGLE ,IPI_CPU_STOP , + IPI_IRQ_WORK, };static DECLARE_COMPLETION (cpu_running);
@@ -448,6 +450,13 @@ void arch_send_call_function_single_ipi(int cpu)smp_cross_call (cpumask_of (cpu),IPI_CALL_FUNC_SINGLE ); } +#ifdef CONFIG_IRQ_WORK +void arch_irq_work_raise (void ) +{ +smp_cross_call (cpumask_of (smp_processor_id ()),IPI_IRQ_WORK ); +} +#endif +static const char *ipi_types[NR_IPI ] = { #define S(x,s) [x] = sS (IPI_WAKEUP ,"CPU wakeup interrupts" ),
@@ -456,6 +465,7 @@ static const char *ipi_types[NR_IPI] = {S (IPI_CALL_FUNC ,"Function call interrupts" ),S (IPI_CALL_FUNC_SINGLE ,"Single function call interrupts" ),S (IPI_CPU_STOP ,"CPU stop interrupts" ), + S(IPI_IRQ_WORK, "IRQ work interrupts"), };void show_ipi_list (struct seq_file *p,int prec)
@@ -565,6 +575,14 @@ void handle_IPI(int ipinr, struct pt_regs *regs)irq_exit ();break ; +#ifdef CONFIG_IRQ_WORK +case IPI_IRQ_WORK : +irq_enter (); +irq_work_run (); +irq_exit (); +break ; +#endif +default :printk (KERN_CRIT "CPU%u: Unknown IPI message 0x%x\n" , cpu, ipinr);
Signature · 2 lines
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum, hosted by The Linux Foundation
Re: [PATCH] ARM: Support arch_irq_work_raise() via self IPIs
Quoted 6 lines · Stephen Boyd
This will allow the scheduler tick to be restarted if we're in full NOHZ mode.
Cc Kevin Hilman <[email protected]>Cc Frederic Weisbecker <[email protected]>Signed-off-by Stephen Boyd <[email protected]>
Minor nit, but I'd prefer a more verbose changelog (I forget things quickly and like to rely on changelogs for my memory.) Probably worth adding something like: "By default, irq_work is tied to the tick processing (update_process_times()) but in full NOHZ mode, no tick means no IRQ work. In order for IRQ work to be done in full NOHZ mode, a self-IPI is used to process IRQ work."
Other than the changelog nit, patch looks good, feel free to add
If Russell is OK with this, it can go to his patch system.
Kevin
Stephen Boyd
Reviewed-by Kevin HilmanRe: [PATCH] ARM: Support arch_irq_work_raise() via self IPIs
Quoted 16 lines · Kevin Hilman
Quoted 6 lines · Stephen Boyd
This will allow the scheduler tick to be restarted if we're in full NOHZ mode.
Cc Kevin Hilman <[email protected]>Cc Frederic Weisbecker <[email protected]>Signed-off-by Stephen Boyd <[email protected]>Minor nit, but I'd prefer a more verbose changelog (I forget things quickly and like to rely on changelogs for my memory.) Probably worth adding something like: "By default,
irq_workis tied to the tick processing (update_process_times()) but in full NOHZ mode, no tick means no IRQ work. In order for IRQ work to be done in full NOHZ mode, a self-IPI is used to process IRQ work."
Quoted 6 lines
Other than the changelog nit, patch looks good, feel free to add
Reviewed-by Kevin Hilman <[email protected]>If Russell is OK with this, it can go to his patch system.
Fair enough. This is what I came up with. I'll send it off to the patch tracker in about 12 hours if nobody else has anymore comments.
----8<-----
ARM: Support arch_irq_work_raise() via self IPIs
By default, IRQ work is run from the tick interrupt (see irq_work_run() in update_process_times()). When we're in full NOHZ mode, restarting the tick requires the use of IRQ work and if the only place we run IRQ work is in the tick interrupt we have an unbreakable cycle. Implement arch_irq_work_raise() via self IPIs to break this cycle and get the tick started again. Note that we implement this via IPIs which are only available on SMP builds. This shouldn't be a problem because full NOHZ is only supported on SMP builds anyway.
Signature · 2 lines
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation
Re: [PATCH] ARM: Support arch_irq_work_raise() via self IPIs
Quoted 31 lines · Stephen Boyd
Quoted 16 lines · Kevin Hilman
Quoted 6 lines · Stephen Boyd
This will allow the scheduler tick to be restarted if we're in full NOHZ mode.
Cc Kevin Hilman <[email protected]>Cc Frederic Weisbecker <[email protected]>Signed-off-by Stephen Boyd <[email protected]>Minor nit, but I'd prefer a more verbose changelog (I forget things quickly and like to rely on changelogs for my memory.) Probably worth adding something like: "By default,
irq_workis tied to the tick processing (update_process_times()) but in full NOHZ mode, no tick means no IRQ work. In order for IRQ work to be done in full NOHZ mode, a self-IPI is used to process IRQ work."Quoted 6 lines
Other than the changelog nit, patch looks good, feel free to add
Reviewed-by Kevin Hilman <[email protected]>If Russell is OK with this, it can go to his patch system.
Fair enough. This is what I came up with. I'll send it off to the patch tracker in about 12 hours if nobody else has anymore comments.
----8<-----
ARM: Supportarch_irq_work_raise()via self IPIs
Quoted 9 lines
By default, IRQ work is run from the tick interrupt (see
irq_work_run()inupdate_process_times()). When we're in full NOHZ mode, restarting the tick requires the use of IRQ work and if the only place we run IRQ work is in the tick interrupt we have an unbreakable cycle. Implementarch_irq_work_raise()via self IPIs to break this cycle and get the tick started again. Note that we implement this via IPIs which are only available on SMP builds. This shouldn't be a problem because full NOHZ is only supported on SMP builds anyway.
Nice description!
Thanks!
Quoted 7 lines
Signed-off-by Stephen Boyd <[email protected]>Reviewed-by Kevin Hilman <[email protected]>Cc Frederic Weisbecker <[email protected]>Signature · 2 lines
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation
Re: [PATCH] ARM: Support arch_irq_work_raise() via self IPIs
In reply to Stephen BoydQuoted 19 lines · Stephen Boyd
Quoted 18 lines · Kevin Hilman
Quoted 6 lines · Stephen Boyd
This will allow the scheduler tick to be restarted if we're in full NOHZ mode.
Cc Kevin Hilman <[email protected]>Cc Frederic Weisbecker <[email protected]>Signed-off-by Stephen Boyd <[email protected]>Minor nit, but I'd prefer a more verbose changelog (I forget things quickly and like to rely on changelogs for my memory.) Probably worth adding something like: "By default,
irq_workis tied to the tick processing (update_process_times()) but in full NOHZ mode, no tick means no IRQ work. In order for IRQ work to be done in full NOHZ mode, a self-IPI is used to process IRQ work."Other than the changelog nit, patch looks good, feel free to add
Quoted 8 lines
Quoted 4 lines
Reviewed-by Kevin Hilman <[email protected]>If Russell is OK with this, it can go to his patch system.
Fair enough. This is what I came up with. I'll send it off to the patch tracker in about 12 hours if nobody else has anymore comments.
Nice description, thanks for the respin.
Kevin
Re: [PATCH] ARM: Support arch_irq_work_raise() via self IPIs
In reply to Stephen BoydQuoted 41 lines · Stephen Boyd
Quoted 16 lines · Kevin Hilman
Quoted 6 lines · Stephen Boyd
This will allow the scheduler tick to be restarted if we're in full NOHZ mode.
Cc Kevin Hilman <[email protected]>Cc Frederic Weisbecker <[email protected]>Signed-off-by Stephen Boyd <[email protected]>Minor nit, but I'd prefer a more verbose changelog (I forget things quickly and like to rely on changelogs for my memory.) Probably worth adding something like: "By default,
irq_workis tied to the tick processing (update_process_times()) but in full NOHZ mode, no tick means no IRQ work. In order for IRQ work to be done in full NOHZ mode, a self-IPI is used to process IRQ work."Quoted 6 lines
Other than the changelog nit, patch looks good, feel free to add
Reviewed-by Kevin Hilman <[email protected]>If Russell is OK with this, it can go to his patch system.
Fair enough. This is what I came up with. I'll send it off to the patch tracker in about 12 hours if nobody else has anymore comments.
----8<-----
ARM: Supportarch_irq_work_raise()via self IPIsBy default, IRQ work is run from the tick interrupt (see
irq_work_run()inupdate_process_times()). When we're in full NOHZ mode, restarting the tick requires the use of IRQ work and if the only place we run IRQ work is in the tick interrupt we have an unbreakable cycle. Implementarch_irq_work_raise()via self IPIs to break this cycle and get the tick started again. Note that we implement this via IPIs which are only available on SMP builds. This shouldn't be a problem because full NOHZ is only supported on SMP builds anyway.
Quoted 3 lines
Signed-off-by Stephen Boyd <[email protected]>Reviewed-by Kevin Hilman <[email protected]>Cc Frederic Weisbecker <[email protected]>
Hm, so I think this just landed in -next, which seems... late for 3.13. Anyway, it causes boot failures on Cubieboards (Allwinner A10), I just bisected it down to this patch (fails to boot multi_v7_defconfig, no console output at all).
Unfortunately I'm not getting any debug_ll output from the board at the moment, so I can't actually get to a panic stack or other error info. :( I hope I can get back to you with one later today.
-Olof
Re: [PATCH] ARM: Support arch_irq_work_raise() via self IPIs
Quoted 47 lines · Olof Johansson
Quoted 41 lines · Stephen Boyd
Quoted 16 lines · Kevin Hilman
Quoted 6 lines · Stephen Boyd
This will allow the scheduler tick to be restarted if we're in full NOHZ mode.
Cc Kevin Hilman <[email protected]>Cc Frederic Weisbecker <[email protected]>Signed-off-by Stephen Boyd <[email protected]>Minor nit, but I'd prefer a more verbose changelog (I forget things quickly and like to rely on changelogs for my memory.) Probably worth adding something like: "By default,
irq_workis tied to the tick processing (update_process_times()) but in full NOHZ mode, no tick means no IRQ work. In order for IRQ work to be done in full NOHZ mode, a self-IPI is used to process IRQ work."Quoted 6 lines
Other than the changelog nit, patch looks good, feel free to add
Reviewed-by Kevin Hilman <[email protected]>If Russell is OK with this, it can go to his patch system.
Fair enough. This is what I came up with. I'll send it off to the patch tracker in about 12 hours if nobody else has anymore comments.
----8<-----
ARM: Supportarch_irq_work_raise()via self IPIsBy default, IRQ work is run from the tick interrupt (see
irq_work_run()inupdate_process_times()). When we're in full NOHZ mode, restarting the tick requires the use of IRQ work and if the only place we run IRQ work is in the tick interrupt we have an unbreakable cycle. Implementarch_irq_work_raise()via self IPIs to break this cycle and get the tick started again. Note that we implement this via IPIs which are only available on SMP builds. This shouldn't be a problem because full NOHZ is only supported on SMP builds anyway.Quoted 3 lines
Signed-off-by Stephen Boyd <[email protected]>Reviewed-by Kevin Hilman <[email protected]>Cc Frederic Weisbecker <[email protected]>
Quoted 8 lines
Hm, so I think this just landed in -next, which seems... late for 3.13. Anyway, it causes boot failures on Cubieboards (Allwinner A10), I just bisected it down to this patch (fails to boot
multi_v7_defconfig, no console output at all).Unfortunately I'm not getting any
debug_lloutput from the board at the moment, so I can't actually get to a panic stack or other error info. :( I hope I can get back to you with one later today.
Turns out that I hit this on the BeagleBone as well, and I got a trace out of it. Seems like this might be broken on all non-SMP platforms built with CONFIG_SMP=y?
[ 0.206322] Unable to handle kernel NULL pointer dereference at virtual address 00000000 [ 0.214752] pgd = c0004000 [ 0.217628] [00000000] *pgd=00000000 [ 0.221421] Internal error: Oops: 80000005 [#1] SMP ARM [ 0.226890] Modules linked in: [ 0.230136] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 3.12.0-rc6-00018-g8d45144-dirty #16 [ 0.238646] task: de05b440 ti: de05c000 task.ti: de05c000 [ 0.244294] PC is at 0x0 [ 0.247010] LR is at arch_irq_work_raise+0x3c/0x48 [ 0.252034] pc : [<00000000>] lr : [<c0019590>] psr: 60000193 [ 0.252034] sp : de05dd60 ip : 00000001 fp : 00000000 [ 0.264029] r10: c085e2f0 r9 : de05c000 r8 : c07be0a4 [ 0.269496] r7 : de05c000 r6 : de05c000 r5 : c07c5778 r4 : c0824554 [ 0.276305] r3 : 00000000 r2 : 00000000 r1 : 00000006 r0 : c0529a58 [ 0.283116] Flags: nZCv IRQs off FIQs on Mode SVC_32 ISA ARM Segment kernel [ 0.290818] Control: 10c5387d Table: 80004019 DAC: 00000017 [ 0.296821] Process swapper/0 (pid: 1, stack limit = 0xde05c248) [ 0.303094] Stack: (0xde05dd60 to 0xde05e000) [ 0.307677] dd60: c07b9dbc c00cb2dc 00000001 c08242c0 c08242c0 60000113 c07be0a8 c00b0590 [ 0.316191] dd80: de05c000 c085e2f0 c08242c0 c08242c0 c1414c28 c00b07cc de05b440 c1414c28 [ 0.324705] dda0: c08242c0 c00b0af8 c0862bb0 c0862db0 c1414cd8 de05c028 c0824840 de05ddb8 [ 0.333218] ddc0: 00000000 00000009 00000001 00000024 c07be0a8 c07be0a4 de05c000 c085e2f0 [ 0.341731] dde0: 00000000 c004a4b0 00000010 de00d2dc 00000054 00000100 00000024 00000000 [ 0.350244] de00: de05c028 0000000a ffff8ae7 00200040 00000016 de05c000 60000193 de05c000 [ 0.358756] de20: 00000054 00000000 00000000 00000000 00000000 c004a704 00000000 de05c008 [ 0.367269] de40: c07ba254 c004aa1c c07c5778 c0014b70 fa200000 00000054 de05de80 c0861244 [ 0.375782] de60: 00000000 c0008634 de05b440 c051c778 20000113 ffffffff de05deb4 c051d0a4 [ 0.384295] de80: 00000001 00000001 00000000 de05b440 c082afac de057ac0 de057ac0 de0443c0 [ 0.392807] dea0: 00000000 00000000 00000000 00000000 c082afbc de05dec8 c009f2a0 c051c778 [ 0.401321] dec0: 20000113 ffffffff 00000000 c016edb0 00000000 000002b0 de057ac0 de057ac0 [ 0.409833] dee0: 00000000 c016ee40 c0875e50 de05df2e de057ac0 00000000 00000013 00000000 [ 0.418345] df00: 00000000 c016f054 de043600 de0443c0 c008eb38 de004ec0 c0875e50 c008eb44 [ 0.426858] df20: 00000012 00000000 00000000 3931f0f8 00000000 00000000 00000014 c0822e84 [ 0.435370] df40: 00000000 c008ed2c 00000000 00000000 00000000 c07b7490 c07b7490 c075ab3c [ 0.443882] df60: 00000000 c00701ac 00000002 00000000 c0070160 dffadb73 7bf8edb4 00000000 [ 0.452394] df80: c051092c 00000000 00000000 00000000 00000000 00000000 00000000 c0510934 [ 0.460907] dfa0: de05aa40 00000000 c051092c c0013ce8 00000000 00000000 00000000 00000000 [ 0.469419] dfc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 0.477930] dfe0: 00000000 00000000 00000000 00000000 00000013 00000000 07efffe5 4dfac6f5 [ 0.486468] [<c0019590>] (arch_irq_work_raise+0x3c/0x48) from [<c00cb2dc>] (irq_work_queue+0xe4/0xf8) [ 0.496073] [<c00cb2dc>] (irq_work_queue+0xe4/0xf8) from [<c00b0590>] (rcu_accelerate_cbs+0x1d4/0x1d8) [ 0.505756] [<c00b0590>] (rcu_accelerate_cbs+0x1d4/0x1d8) from [<c00b07cc>] (rcu_start_gp+0x34/0x48) [ 0.515259] [<c00b07cc>] (rcu_start_gp+0x34/0x48) from [<c00b0af8>] (rcu_process_callbacks+0x318/0x608) [ 0.525040] [<c00b0af8>] (rcu_process_callbacks+0x318/0x608) from [<c004a4b0>] (__do_softirq+0x114/0x2a0) [ 0.534989] [<c004a4b0>] (__do_softirq+0x114/0x2a0) from [<c004a704>] (do_softirq+0x6c/0x74) [ 0.543776] [<c004a704>] (do_softirq+0x6c/0x74) from [<c004aa1c>] (irq_exit+0xac/0x100) [ 0.552126] [<c004aa1c>] (irq_exit+0xac/0x100) from [<c0014b70>] (handle_IRQ+0x54/0xb4) [ 0.560466] [<c0014b70>] (handle_IRQ+0x54/0xb4) from [<c0008634>] (omap3_intc_handle_irq+0x60/0x74) [ 0.569885] [<c0008634>] (omap3_intc_handle_irq+0x60/0x74) from [<c051d0a4>] (__irq_svc+0x44/0x5c) [ 0.579196] Exception stack(0xde05de80 to 0xde05dec8) [ 0.584491] de80: 00000001 00000001 00000000 de05b440 c082afac de057ac0 de057ac0 de0443c0 [ 0.593004] dea0: 00000000 00000000 00000000 00000000 c082afbc de05dec8 c009f2a0 c051c778 [ 0.601510] dec0: 20000113 ffffffff [ 0.605202] [<c051d0a4>] (__irq_svc+0x44/0x5c) from [<c051c778>] (_raw_spin_unlock_irq+0x28/0x2c) [ 0.614447] [<c051c778>] (_raw_spin_unlock_irq+0x28/0x2c) from [<c016edb0>] (proc_alloc_inum+0x30/0xa8) [ 0.624220] [<c016edb0>] (proc_alloc_inum+0x30/0xa8) from [<c016ee40>] (proc_register+0x18/0x130) [ 0.633454] [<c016ee40>] (proc_register+0x18/0x130) from [<c016f054>] (proc_mkdir_data+0x44/0x6c) [ 0.642702] [<c016f054>] (proc_mkdir_data+0x44/0x6c) from [<c008eb44>] (register_irq_proc+0x6c/0x128) [ 0.652292] [<c008eb44>] (register_irq_proc+0x6c/0x128) from [<c008ed2c>] (init_irq_proc+0x74/0xb0) [ 0.661706] [<c008ed2c>] (init_irq_proc+0x74/0xb0) from [<c075ab3c>] (kernel_init_freeable+0x84/0x1c8) [ 0.671390] [<c075ab3c>] (kernel_init_freeable+0x84/0x1c8) from [<c0510934>] (kernel_init+0x8/0x150) [ 0.680892] [<c0510934>] (kernel_init+0x8/0x150) from [<c0013ce8>] (ret_from_fork+0x14/0x2c) [ 0.689674] Code: bad PC value [ 0.692984] ---[ end trace 1b75b31a2719ed1c ]---
Re: [PATCH] ARM: Support arch_irq_work_raise() via self IPIs
Quoted 50 lines · Olof Johansson
Quoted 49 lines · Olof Johansson
Quoted 39 lines · Stephen Boyd
Quoted 15 lines · Kevin Hilman
Quoted 6 lines · Stephen Boyd
This will allow the scheduler tick to be restarted if we're in full NOHZ mode.
Cc Kevin Hilman <[email protected]>Cc Frederic Weisbecker <[email protected]>Signed-off-by Stephen Boyd <[email protected]>Minor nit, but I'd prefer a more verbose changelog (I forget things quickly and like to rely on changelogs for my memory.) Probably worth adding something like: "By default,
irq_workis tied to the tick processing (update_process_times()) but in full NOHZ mode, no tick means no IRQ work. In order for IRQ work to be done in full NOHZ mode, a self-IPI is used to process IRQ work."Quoted 6 lines
Other than the changelog nit, patch looks good, feel free to add
Reviewed-by Kevin Hilman <[email protected]>If Russell is OK with this, it can go to his patch system.
Fair enough. This is what I came up with. I'll send it off to the patch tracker in about 12 hours if nobody else has anymore comments.
----8<-----
ARM: Supportarch_irq_work_raise()via self IPIsBy default, IRQ work is run from the tick interrupt (see
irq_work_run()inupdate_process_times()). When we're in full NOHZ mode, restarting the tick requires the use of IRQ work and if the only place we run IRQ work is in the tick interrupt we have an unbreakable cycle. Implementarch_irq_work_raise()via self IPIs to break this cycle and get the tick started again. Note that we implement this via IPIs which are only available on SMP builds. This shouldn't be a problem because full NOHZ is only supported on SMP builds anyway.Quoted 3 lines
Signed-off-by Stephen Boyd <[email protected]>Reviewed-by Kevin Hilman <[email protected]>Cc Frederic Weisbecker <[email protected]>Hm, so I think this just landed in -next, which seems... late for 3.13. Anyway, it causes boot failures on Cubieboards (Allwinner A10), I just bisected it down to this patch (fails to boot
multi_v7_defconfig, no console output at all).
Quoted 6 lines
Quoted 3 lines
Unfortunately I'm not getting any
debug_lloutput from the board at the moment, so I can't actually get to a panic stack or other error info. :( I hope I can get back to you with one later today.Turns out that I hit this on the BeagleBone as well, and I got a trace out of it. Seems like this might be broken on all non-SMP platforms built with
CONFIG_SMP=y?
Ah yes. We don't have IPI support on UP kernels but SMP_ON_UP exposes arch_irq_work_raise(). How about this? We should just skip this function if we're running on UP.
---8<----
arch/arm/kernel/smp.c +2 -1
@@ -453,7 +453,8 @@ void arch_send_call_function_single_ipi(int cpu)#ifdef CONFIG_IRQ_WORK void arch_irq_work_raise (void ) { -smp_cross_call (cpumask_of (smp_processor_id ()),IPI_IRQ_WORK ); +if (is_smp ()) +smp_cross_call (cpumask_of (smp_processor_id ()),IPI_IRQ_WORK ); }#endif
Signature · 2 lines
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation
Olof Johansson
Tested-byRe: [PATCH] ARM: Support arch_irq_work_raise() via self IPIs
Quoted 54 lines · Stephen Boyd
Quoted 53 lines · Olof Johansson
Quoted 52 lines · Olof Johansson
Quoted 39 lines · Stephen Boyd
Quoted 15 lines · Kevin Hilman
Quoted 6 lines · Stephen Boyd
This will allow the scheduler tick to be restarted if we're in full NOHZ mode.
Cc Kevin Hilman <[email protected]>Cc Frederic Weisbecker <[email protected]>Signed-off-by Stephen Boyd <[email protected]>Minor nit, but I'd prefer a more verbose changelog (I forget things quickly and like to rely on changelogs for my memory.) Probably worth adding something like: "By default,
irq_workis tied to the tick processing (update_process_times()) but in full NOHZ mode, no tick means no IRQ work. In order for IRQ work to be done in full NOHZ mode, a self-IPI is used to process IRQ work."Quoted 6 lines
Other than the changelog nit, patch looks good, feel free to add
Reviewed-by Kevin Hilman <[email protected]>If Russell is OK with this, it can go to his patch system.
Fair enough. This is what I came up with. I'll send it off to the patch tracker in about 12 hours if nobody else has anymore comments.
----8<-----
ARM: Supportarch_irq_work_raise()via self IPIsBy default, IRQ work is run from the tick interrupt (see
irq_work_run()inupdate_process_times()). When we're in full NOHZ mode, restarting the tick requires the use of IRQ work and if the only place we run IRQ work is in the tick interrupt we have an unbreakable cycle. Implementarch_irq_work_raise()via self IPIs to break this cycle and get the tick started again. Note that we implement this via IPIs which are only available on SMP builds. This shouldn't be a problem because full NOHZ is only supported on SMP builds anyway.Quoted 3 lines
Signed-off-by Stephen Boyd <[email protected]>Reviewed-by Kevin Hilman <[email protected]>Cc Frederic Weisbecker <[email protected]>Hm, so I think this just landed in -next, which seems... late for 3.13. Anyway, it causes boot failures on Cubieboards (Allwinner A10), I just bisected it down to this patch (fails to boot
multi_v7_defconfig, no console output at all).Unfortunately I'm not getting any
debug_lloutput from the board at the moment, so I can't actually get to a panic stack or other error info. :( I hope I can get back to you with one later today.
Quoted 7 lines
Quoted 3 lines
Turns out that I hit this on the BeagleBone as well, and I got a trace out of it. Seems like this might be broken on all non-SMP platforms built with
CONFIG_SMP=y?Ah yes. We don't have IPI support on UP kernels but
SMP_ON_UPexposesarch_irq_work_raise(). How about this? We should just skip this function if we're running on UP.
Yep.
Send it to Russell's tracker, plz.
-Olof
Re: [PATCH] ARM: Support arch_irq_work_raise() via self IPIs
Quoted 11 lines · Olof Johansson
Quoted 4 lines · Stephen Boyd
Ah yes. We don't have IPI support on UP kernels but
SMP_ON_UPexposesarch_irq_work_raise(). How about this? We should just skip this function if we're running on UP.Yep.
Tested-by Olof Johansson <[email protected]>Send it to Russell's tracker, plz.
Thanks. 7887/1.
Signature · 2 lines
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation
Re: [PATCH] ARM: Support arch_irq_work_raise() via self IPIs
Quoted 6 lines · Stephen Boyd
Quoted 5 lines · Olof Johansson
Quoted 4 lines · Stephen Boyd
Ah yes. We don't have IPI support on UP kernels but
SMP_ON_UPexposesarch_irq_work_raise(). How about this? We should just skip this function if we're running on UP.
Quoted 8 lines
Quoted 6 lines
Yep.
Tested-by Olof Johansson <[email protected]>Send it to Russell's tracker, plz.
Thanks. 7887/1.
It's probably going to miss Stephen's pull from my tree this evening (if he even does one) so... that gives me two options: either drop the original commit or apply this.
I think I'll apply this and put it in place just after the original commit which caused the breakage.
Thanks.
Re: [PATCH] ARM: Support arch_irq_work_raise() via self IPIs
Quoted 11 lines · Russell King - ARM Linux
Quoted 10 lines · Stephen Boyd
Quoted 9 lines · Olof Johansson
Quoted 4 lines · Stephen Boyd
Ah yes. We don't have IPI support on UP kernels but
SMP_ON_UPexposesarch_irq_work_raise(). How about this? We should just skip this function if we're running on UP.Yep.
Tested-by Olof Johansson <[email protected]>
Quoted 8 lines
Quoted 4 lines
Quoted 2 lines
Send it to Russell's tracker, plz.
Thanks. 7887/1.
It's probably going to miss Stephen's pull from my tree this evening (if he even does one) so... that gives me two options: either drop the original commit or apply this.
It's Saturday for Stephen, so he won't do another one until Sunday night our time.
Quoted 2 lines
I think I'll apply this and put it in place just after the original commit which caused the breakage.
Thanks, sounds good to me.
-Olof