Clean up SMP IPI support
Subsystems (7)
cover Clean up SMP IPI support
This series is aimed at cleaning up our SMP IPI support. As has been recently pointed out, we have a spinlock in percpu data, which is not very nice.
If we switch to using the GIC SGIs, one for each IPI function, then we don't need the mask of ipi bits indicating what work needs to be done. This allows us to get rid of the troublesome spinlock in send_ipi_message().
However, we end up with slightly more expense on the IPI receiver side, as they no longer have a word to read to process a set of pending IPIs. This isn't much of a down-side as you hardly end up with masses of pending IPIs. Modern Cortex A9 platforms usually only clock up the reschedule IPI.
This leaves us with just ipi_count in the ipi per-cpu data structure. Other platforms put this in the global irq_stat array, and break this down into individual IPIs. So, let's do the same.
One down-side of this patchset is that platforms which use SGI 1 to wake their secondary CPUs will receive an "Unknown IPI message" kernel critical message at boot time - as SGI 1 is no longer recognised as a Linux IPI interrupt. We _could_ make this 'IPI_BOOT', something which is only used for booting and therefore simply ignored.
PATCH 1/9 ARM: remove obsolete smp_cross_call_done()
smp_cross_call_done() was removed long ago (see 78d236c - remove useless smp_cross_call_done()). Remove those which have been subsequently merged.
--- arch/arm/mach-tegra/include/mach/smp.h | 7 ------- 1 files changed, 0 insertions(+), 7 deletions(-)
arch/arm/mach-tegra/include/mach/smp.h +0 -7
@@ -12,11 +12,4 @@ static inline void smp_cross_call(const struct cpumask *mask) gic_raise_softirq(mask, 1); } -/* -* Do nothing on MPcore. -*/ -static inline void smp_cross_call_done (cpumask_t callmap) -{ -} -#endif
Signature · 1 line
1.6.2.5
PATCH 2/9 ARM: smp: pass an ipi number to smp_cross_call()
In reply to Russell King - ARM LinuxThis allows us to use smp_cross_call() to trigger a number of different software generated interrupts, rather than combining them all on one SGI. Recover the SGI number via do_IPI.
--- arch/arm/include/asm/smp.h | 4 ++-- arch/arm/kernel/entry-armv.S | 2 +- arch/arm/kernel/smp.c | 10 ++-------- arch/arm/mach-msm/include/mach/smp.h | 4 ++-- arch/arm/mach-omap2/omap-smp.c | 2 +- arch/arm/mach-realview/include/mach/smp.h | 4 ++-- arch/arm/mach-realview/platsmp.c | 2 +- arch/arm/mach-s5pv310/include/mach/smp.h | 4 ++-- arch/arm/mach-s5pv310/platsmp.c | 2 +- arch/arm/mach-tegra/include/mach/smp.h | 4 ++-- arch/arm/mach-ux500/include/mach/smp.h | 4 ++-- arch/arm/mach-ux500/platsmp.c | 2 +- arch/arm/mach-vexpress/include/mach/smp.h | 4 ++-- arch/arm/mach-vexpress/platsmp.c | 2 +- arch/arm/plat-omap/include/plat/smp.h | 4 ++-- 15 files changed, 24 insertions(+), 30 deletions(-)
arch/arm/include/asm/smp.h +2 -2
@@ -38,7 +38,7 @@ extern void show_ipi_list(struct seq_file *p);/* * Called from assembly code, this handles an IPI. */ -asmlinkage void do_IPI(struct pt_regs *regs); +asmlinkage void do_IPI(int ipinr, struct pt_regs *regs);/* * Setup the set of possible CPUs (via set_cpu_possible)
@@ -53,7 +53,7 @@ extern void smp_store_cpu_info(unsigned int cpuid);/* * Raise an IPI cross call on CPUs in callmap. */ -extern void smp_cross_call (const struct cpumask *mask); +extern void smp_cross_call (const struct cpumask *mask,int ipi);/* * Boot a secondary CPU, and assign it the specified idle task.
arch/arm/kernel/entry-armv.S +1 -1
@@ -48,7 +48,7 @@ */ ALT_SMP(test_for_ipi r0, r6, r5, lr) ALT_UP_B(9997f) - movne r0, sp + movne r1, sp adrne lr, BSYM(1b) bne do_IPI
arch/arm/kernel/smp.c +2 -8
@@ -404,7 +404,7 @@ static void send_ipi_message(const struct cpumask *mask, enum ipi_msg_type msg)/* * Call the platform specific cross-CPU call function. */ -smp_cross_call (mask); +smp_cross_call (mask,1 );local_irq_restore (flags); }
@@ -537,14 +537,8 @@ static void ipi_cpu_stop(unsigned int cpu)/* * Main handler for inter-processor interrupts -* -* For ARM, the ipimask now only identifies a single -* category of IPI (Bit 1 IPIs have been replaced by a -* different mechanism): -* -* Bit 0 - Inter-processor function call */ -asmlinkage void __exception do_IPI(struct pt_regs *regs) +asmlinkage void __exception do_IPI(int ipinr, struct pt_regs *regs) {unsigned int cpu =smp_processor_id ();struct ipi_data *ipi = &per_cpu (ipi_data, cpu);
arch/arm/mach-msm/include/mach/smp.h +2 -2
@@ -31,9 +31,9 @@#include <asm/hardware/gic.h> -static inline void smp_cross_call (const struct cpumask *mask) +static inline void smp_cross_call (const struct cpumask *mask,int ipi) { -gic_raise_softirq (mask,1 ); +gic_raise_softirq (mask, ipi); }#endif
arch/arm/mach-omap2/omap-smp.c +1 -1
@@ -76,7 +76,7 @@ int __cpuinit boot_secondary(unsigned int cpu, struct task_struct *idle)omap_modify_auxcoreboot0 (0x200 ,0xfffffdff );flush_cache_all ();smp_wmb (); -smp_cross_call (cpumask_of (cpu)); +smp_cross_call (cpumask_of (cpu),1 );/* * Now the secondary core is starting up let it run its
arch/arm/mach-realview/include/mach/smp.h +2 -2
@@ -7,9 +7,9 @@/* * We use IRQ1 as the IPI */ -static inline void smp_cross_call (const struct cpumask *mask) +static inline void smp_cross_call (const struct cpumask *mask,int ipi) { -gic_raise_softirq (mask,1 ); +gic_raise_softirq (mask, ipi); }#endif
arch/arm/mach-realview/platsmp.c +1 -1
@@ -116,7 +116,7 @@ int __cpuinit boot_secondary(unsigned int cpu, struct task_struct *idle)* Use smp_cross_call() for this, since there's little * point duplicating the code here */ -smp_cross_call (cpumask_of (cpu)); +smp_cross_call (cpumask_of (cpu),1 ); timeout = jiffies + (1 *HZ );while (time_before (jiffies, timeout)) {
arch/arm/mach-s5pv310/include/mach/smp.h +2 -2
@@ -14,9 +14,9 @@ extern void __iomem *gic_cpu_base_addr;/* * We use IRQ1 as the IPI */ -static inline void smp_cross_call (const struct cpumask *mask) +static inline void smp_cross_call (const struct cpumask *mask,int ipi) { -gic_raise_softirq (mask,1 ); +gic_raise_softirq (mask, ipi); }#endif
arch/arm/mach-s5pv310/platsmp.c +1 -1
@@ -97,7 +97,7 @@ int __cpuinit boot_secondary(unsigned int cpu, struct task_struct *idle)* the boot monitor to read the system wide flags register, * and branch to the address found there. */ -smp_cross_call (cpumask_of (cpu)); +smp_cross_call (cpumask_of (cpu),1 ); timeout = jiffies + (1 *HZ );while (time_before (jiffies, timeout)) {
arch/arm/mach-tegra/include/mach/smp.h +2 -2
@@ -7,9 +7,9 @@/* * We use IRQ1 as the IPI */ -static inline void smp_cross_call (const struct cpumask *mask) +static inline void smp_cross_call (const struct cpumask *mask,int ipi) { -gic_raise_softirq (mask,1 ); +gic_raise_softirq (mask, ipi); }#endif
arch/arm/mach-ux500/include/mach/smp.h +2 -2
@@ -18,8 +18,8 @@ extern void u8500_secondary_startup(void);/* * We use IRQ1 as the IPI */ -static inline void smp_cross_call (const struct cpumask *mask) +static inline void smp_cross_call (const struct cpumask *mask,int ipi) { -gic_raise_softirq (mask,1 ); +gic_raise_softirq (mask, ipi); }#endif
arch/arm/mach-ux500/platsmp.c +1 -1
@@ -78,7 +78,7 @@ int __cpuinit boot_secondary(unsigned int cpu, struct task_struct *idle)__cpuc_flush_dcache_area ((void *)&pen_release,sizeof (pen_release));outer_clean_range (__pa (&pen_release),__pa (&pen_release) +1 ); -smp_cross_call (cpumask_of (cpu)); +smp_cross_call (cpumask_of (cpu),1 ); timeout = jiffies + (1 *HZ );while (time_before (jiffies, timeout)) {
arch/arm/mach-vexpress/include/mach/smp.h +2 -2
@@ -7,8 +7,8 @@/* * We use IRQ1 as the IPI */ -static inline void smp_cross_call (const struct cpumask *mask) +static inline void smp_cross_call (const struct cpumask *mask,int ipi) { -gic_raise_softirq (mask,1 ); +gic_raise_softirq (mask, ipi); }#endif
arch/arm/mach-vexpress/platsmp.c +1 -1
@@ -92,7 +92,7 @@ int __cpuinit boot_secondary(unsigned int cpu, struct task_struct *idle)* the boot monitor to read the system wide flags register, * and branch to the address found there. */ -smp_cross_call (cpumask_of (cpu)); +smp_cross_call (cpumask_of (cpu),1 ); timeout = jiffies + (1 *HZ );while (time_before (jiffies, timeout)) {
arch/arm/plat-omap/include/plat/smp.h +2 -2
@@ -29,9 +29,9 @@ extern u32 omap_read_auxcoreboot0(void);/* * We use Soft IRQ1 as the IPI */ -static inline void smp_cross_call (const struct cpumask *mask) +static inline void smp_cross_call (const struct cpumask *mask,int ipi) { -gic_raise_softirq (mask,1 ); +gic_raise_softirq (mask, ipi); }#endif
Signature · 1 line
1.6.2.5
PATCH 2/9 ARM: smp: pass an ipi number to smp_cross_call()
arch/arm/mach-msm/include/mach/smp.h line 37
Quoted 12 lines · Russell King - ARM Linux
arch/arm/mach-msm/include/mach/smp.h +2
-2@@ -31,9 +31,9 @@#include <asm/hardware/gic.h> -static inline void smp_cross_call (const struct cpumask *mask) +static inline void smp_cross_call (const struct cpumask *mask,int ipi) { -gic_raise_softirq (mask,1 ); +gic_raise_softirq (mask, ipi); }
BTW, why don't we make smp_cross_call() generic, it looks like all platforms define the same thing.
Unless we get SMP systems in the future without a GIC.
Signature · 1 line
Catalin
PATCH 3/9 ARM: smp: avoid using bitmasks and locks for IPIs, use hardware instead
In reply to Russell King - ARM LinuxAvoid using bitmasks and locks in the percpu area for IPIs, and instead use individual software generated interrupts to identify the reason for the IPI. This avoids the problems of having spinlocks in the percpu area.
--- arch/arm/kernel/smp.c | 87 ++++++++++++++---------------------------------- 1 files changed, 26 insertions(+), 61 deletions(-)
arch/arm/kernel/smp.c +26 -61
@@ -48,20 +48,15 @@ struct secondary_data secondary_data;/* * structures for inter-processor calls -* - A collection of single bit ipi messages. */ struct ipi_data { -spinlock_t lock;unsigned long ipi_count; -unsigned long bits; }; -static DEFINE_PER_CPU(struct ipi_data, ipi_data) = { - .lock =SPIN_LOCK_UNLOCKED , -}; +static DEFINE_PER_CPU(struct ipi_data, ipi_data);enum ipi_msg_type { -IPI_TIMER , +IPI_TIMER =2 ,IPI_RESCHEDULE ,IPI_CALL_FUNC ,IPI_CALL_FUNC_SINGLE ,
@@ -389,22 +384,13 @@ void __init smp_prepare_boot_cpu(void)static void send_ipi_message (const struct cpumask *mask,enum ipi_msg_type msg) {unsigned long flags; -unsigned int cpu;local_irq_save (flags); -for_each_cpu (cpu, mask) { -struct ipi_data *ipi = &per_cpu (ipi_data, cpu); - -spin_lock (&ipi->lock); - ipi->bits |=1 << msg; -spin_unlock (&ipi->lock); - } -/* * Call the platform specific cross-CPU call function. */ -smp_cross_call (mask,1 ); +smp_cross_call (mask, msg);local_irq_restore (flags); }
@@ -546,56 +532,35 @@ asmlinkage void __exception do_IPI(int ipinr, struct pt_regs *regs) ipi->ipi_count++; -for (;;) { -unsigned long msgs; - -spin_lock (&ipi->lock); - msgs = ipi->bits; - ipi->bits =0 ; -spin_unlock (&ipi->lock); - -if (!msgs) -break ; - -do { -unsigned nextmsg; - - nextmsg = msgs & -msgs; - msgs &= ~nextmsg; - nextmsg =ffz (~nextmsg); - -switch (nextmsg) { -case IPI_TIMER : -ipi_timer (); -break ; +switch (ipinr) { +case IPI_TIMER : +ipi_timer (); +break ; -case IPI_RESCHEDULE : -/* -* nothing more to do - eveything is -* done on the interrupt return path -*/ -break ; +case IPI_RESCHEDULE : +/* +* nothing more to do - eveything is +* done on the interrupt return path +*/ +break ; -case IPI_CALL_FUNC : -generic_smp_call_function_interrupt (); -break ; +case IPI_CALL_FUNC : +generic_smp_call_function_interrupt (); +break ; -case IPI_CALL_FUNC_SINGLE : -generic_smp_call_function_single_interrupt (); -break ; +case IPI_CALL_FUNC_SINGLE : +generic_smp_call_function_single_interrupt (); +break ; -case IPI_CPU_STOP : -ipi_cpu_stop (cpu); -break ; +case IPI_CPU_STOP : +ipi_cpu_stop (cpu); +break ; -default : -printk (KERN_CRIT "CPU%u: Unknown IPI message 0x%x\n" , - cpu, nextmsg); -break ; - } - }while (msgs); +default : +printk (KERN_CRIT "CPU%u: Unknown IPI message 0x%x\n" , + cpu, ipinr); +break ; } -set_irq_regs (old_regs); }
Signature · 1 line
1.6.2.5
PATCH 3/9 ARM: smp: avoid using bitmasks and locks for IPIs, use hardware instead
Quoted 14 lines · Russell King - ARM Linux
Avoid using bitmasks and locks in the percpu area for IPIs, and instead use individual software generated interrupts to identify the reason for the IPI. This avoids the problems of having spinlocks in the percpu area.
Signed-off-by Russell King <[email protected]>--- arch/arm/kernel/smp.c | 87 ++++++++++++++---------------------------------- 1 files changed, 26 insertions(+), 61 deletions(-)arch/arm/kernel/smp.c +0
-0
[...]
arch/arm/kernel/smp.c struct secondary_data secondary_data; · line 62
Quoted 6 lines
enum
ipi_msg_type{arch/arm/kernel/smp.c +1
-1-IPI_TIMER , +IPI_TIMER =2 ,IPI_RESCHEDULE ,IPI_CALL_FUNC , IPI_CALL_FUNC_SINGLE,
Maybe a comment here on why this starts at 2. People may wonder in the future.
Signature · 1 line
Catalin
PATCH 3/9 ARM: smp: avoid using bitmasks and locks for IPIs, use hardware instead
Quoted 23 lines · Catalin Marinas
Quoted 14 lines · Russell King - ARM Linux
Avoid using bitmasks and locks in the percpu area for IPIs, and instead use individual software generated interrupts to identify the reason for the IPI. This avoids the problems of having spinlocks in the percpu area.
Signed-off-by Russell King <[email protected]>--- arch/arm/kernel/smp.c | 87 ++++++++++++++---------------------------------- 1 files changed, 26 insertions(+), 61 deletions(-)arch/arm/kernel/smp.c +0
-0[...]
arch/arm/kernel/smp.c struct secondary_data secondary_data; · line 62
Quoted 6 lines
enum
ipi_msg_type{arch/arm/kernel/smp.c +1
-1-IPI_TIMER , +IPI_TIMER =2 ,IPI_RESCHEDULE ,IPI_CALL_FUNC , IPI_CALL_FUNC_SINGLE,
Quoted 2 lines
Maybe a comment here on why this starts at 2. People may wonder in the future.
I would like to get rid of the 'starts at 2' but we can only do this once we're sure that the use of SGI1 to wake up the CPUs is not something that is a hard requirement for boot loaders.
I've not yet tested whether Realview and Versatile Express are fine with a different SGI number.
PATCH 3/9 ARM: smp: avoid using bitmasks and locks for IPIs, use hardware instead
Quoted 27 lines · Russell King - ARM Linux
Quoted 23 lines · Catalin Marinas
Quoted 14 lines · Russell King - ARM Linux
Avoid using bitmasks and locks in the percpu area for IPIs, and instead use individual software generated interrupts to identify the reason for the IPI. This avoids the problems of having spinlocks in the percpu area.
Signed-off-by Russell King <[email protected]>--- arch/arm/kernel/smp.c | 87 ++++++++++++++---------------------------------- 1 files changed, 26 insertions(+), 61 deletions(-)arch/arm/kernel/smp.c +0
-0[...]
arch/arm/kernel/smp.c struct secondary_data secondary_data; · line 62
Quoted 6 lines
enum
ipi_msg_type{arch/arm/kernel/smp.c +1
-1-IPI_TIMER , +IPI_TIMER =2 ,IPI_RESCHEDULE ,IPI_CALL_FUNC , IPI_CALL_FUNC_SINGLE,Quoted 2 lines
Maybe a comment here on why this starts at 2. People may wonder in the future.
Quoted 6 lines
I would like to get rid of the 'starts at 2' but we can only do this once we're sure that the use of SGI1 to wake up the CPUs is not something that is a hard requirement for boot loaders.
I've not yet tested whether Realview and Versatile Express are fine with a different SGI number.
AFAIK, the ARM boot monitor doesn't care about the IPI number, it just needs something to take it out of the WFI.
BTW, in boot_secondary() we could issue a SEV as well from the primary CPU. That's mainly useful when you don't have a full boot monitor to set up the GIC and it uses WFE in a loop (I have some simple wrapper for this, mainly used on software models).
Signature · 1 line
Catalin
PATCH 4/9 ARM: include local timer irq stats only when local timers configured
In reply to Russell King - ARM Linux--- arch/arm/include/asm/hardirq.h | 2 ++ arch/arm/kernel/irq.c | 2 ++ arch/arm/kernel/smp.c | 24 ++++++++++++------------ 3 files changed, 16 insertions(+), 12 deletions(-)
arch/arm/include/asm/hardirq.h +2 -0
@@ -7,7 +7,9 @@typedef struct {unsigned int __softirq_pending; +#ifdef CONFIG_LOCAL_TIMERS unsigned int local_timer_irqs; +#endif }____cacheline_aligned irq_cpustat_t ;#include <linux/irq_cpustat.h> /* Standard mappings for irq_cpustat_t above */
arch/arm/kernel/irq.c +2 -0
@@ -93,6 +93,8 @@ unlock:#endif #ifdef CONFIG_SMP show_ipi_list (p); +#endif +#ifdef CONFIG_LOCAL_TIMERS show_local_irqs (p);#endif seq_printf (p,"Err: %10lu\n" , irq_err_count);
arch/arm/kernel/smp.c +12 -12
@@ -417,18 +417,6 @@ void show_ipi_list(struct seq_file *p)seq_putc (p,'\n' ); } -void show_local_irqs (struct seq_file *p) -{ -unsigned int cpu; - -seq_printf (p,"LOC: " ); - -for_each_present_cpu (cpu) -seq_printf (p,"%10u " , irq_stat[cpu].local_timer_irqs); - -seq_putc (p,'\n' ); -} -/* * Timer (local or broadcast) support */
@@ -455,6 +443,18 @@ asmlinkage void __exception do_local_timer(struct pt_regs *regs) set_irq_regs(old_regs); } + +void show_local_irqs (struct seq_file *p) +{ +unsigned int cpu; + +seq_printf (p,"LOC: " ); + +for_each_present_cpu (cpu) +seq_printf (p,"%10u " , irq_stat[cpu].local_timer_irqs); + +seq_putc (p,'\n' ); +}#endif #ifdef CONFIG_GENERIC_CLOCKEVENTS_BROADCAST
Signature · 1 line
1.6.2.5
PATCH 5/9 ARM: smp: provide accessors for irq_stat data
In reply to Russell King - ARM LinuxProvide __inc_irq_stat() and __get_irq_stat() to increment and read the irq stat counters.
--- arch/arm/include/asm/hardirq.h | 3 +++ arch/arm/kernel/smp.c | 4 ++-- 2 files changed, 5 insertions(+), 2 deletions(-)
arch/arm/include/asm/hardirq.h +3 -0
@@ -14,6 +14,9 @@ typedef struct {#include <linux/irq_cpustat.h> /* Standard mappings for irq_cpustat_t above */ +#define __inc_irq_stat (cpu, member) __IRQ_STAT(cpu, member)++ +#define __get_irq_stat (cpu, member) __IRQ_STAT(cpu, member) +#if NR_IRQS >512 #define HARDIRQ_BITS 10#elif NR_IRQS >256
arch/arm/kernel/smp.c +2 -2
@@ -437,7 +437,7 @@ asmlinkage void __exception do_local_timer(struct pt_regs *regs)int cpu =smp_processor_id ();if (local_timer_ack ()) { - irq_stat[cpu].local_timer_irqs++; +__inc_irq_stat (cpu, local_timer_irqs);ipi_timer (); }
@@ -451,7 +451,7 @@ void show_local_irqs(struct seq_file *p)seq_printf (p,"LOC: " );for_each_present_cpu (cpu) -seq_printf (p,"%10u " , irq_stat[cpu].local_timer_irqs); +seq_printf (p,"%10u " ,__get_irq_stat (cpu, local_timer_irqs));seq_putc (p,'\n' ); }
Signature · 1 line
1.6.2.5
PATCH 6/9 ARM: smp: move ipi_count into irq_stat structure
In reply to Russell King - ARM LinuxMove the ipi_count into irq_stat, which allows the ipi_data structure to be entirely removed.
--- arch/arm/include/asm/hardirq.h | 3 +++ arch/arm/kernel/smp.c | 14 ++------------ 2 files changed, 5 insertions(+), 12 deletions(-)
arch/arm/include/asm/hardirq.h +3 -0
@@ -10,6 +10,9 @@ typedef struct {#ifdef CONFIG_LOCAL_TIMERS unsigned int local_timer_irqs;#endif +#ifdef CONFIG_SMP +unsigned int ipi_irqs; +#endif }____cacheline_aligned irq_cpustat_t;#include <linux/irq_cpustat.h> /* Standard mappings for irq_cpustat_t above */
arch/arm/kernel/smp.c +2 -12
@@ -46,15 +46,6 @@*/ struct secondary_data secondary_data; -/* -* structures for inter-processor calls -*/ -struct ipi_data { -unsigned long ipi_count; -}; - -static DEFINE_PER_CPU(struct ipi_data, ipi_data); -enum ipi_msg_type {IPI_TIMER =2 ,IPI_RESCHEDULE ,
@@ -412,7 +403,7 @@ void show_ipi_list(struct seq_file *p)seq_puts (p,"IPI:" );for_each_present_cpu (cpu) -seq_printf (p," %10lu" ,per_cpu (ipi_data, cpu).ipi_count); +seq_printf (p," %10u" ,__get_irq_stat (cpu, ipi_irqs));seq_putc (p,'\n' ); }
@@ -527,10 +518,9 @@ static void ipi_cpu_stop(unsigned int cpu) asmlinkage void __exception do_IPI(int ipinr, struct pt_regs *regs) {unsigned int cpu =smp_processor_id (); -struct ipi_data *ipi = &per_cpu (ipi_data, cpu);struct pt_regs *old_regs =set_irq_regs (regs); - ipi->ipi_count++; +__inc_irq_stat (cpu, ipi_irqs);switch (ipinr) {case IPI_TIMER :
Signature · 1 line
1.6.2.5
PATCH 7/9 ARM: fix /proc/interrupts formatting
In reply to Russell King - ARM LinuxAs per x86, align the initial column according to how many IRQs we have. Also, provide an english explaination for the 'LOC:' and 'IPI:' lines.
--- arch/arm/include/asm/mach/irq.h | 2 +- arch/arm/include/asm/smp.h | 4 ++-- arch/arm/kernel/fiq.c | 5 +++-- arch/arm/kernel/irq.c | 16 ++++++++++------ arch/arm/kernel/smp.c | 14 +++++++------- 5 files changed, 23 insertions(+), 18 deletions(-)
arch/arm/include/asm/mach/irq.h +1 -1
@@ -20,7 +20,7 @@ struct seq_file;extern unsigned int arch_nr_irqs;extern void (*init_arch_irq)(void );extern void init_FIQ (void ); -extern int show_fiq_list (struct seq_file *,void *); +extern int show_fiq_list (struct seq_file *,int );/* * This is for easy migration, but should be changed in the source
arch/arm/include/asm/smp.h +2 -2
@@ -33,7 +33,7 @@ struct seq_file;/* * generate IPI list text */ -extern void show_ipi_list (struct seq_file *p); +extern void show_ipi_list (struct seq_file *,int );/* * Called from assembly code, this handles an IPI.
@@ -97,6 +97,6 @@ extern void arch_send_call_function_ipi_mask(const struct cpumask *mask);/* * show local interrupt info */ -extern void show_local_irqs (struct seq_file *); +extern void show_local_irqs (struct seq_file *,int );#endif /* ifndef __ASM_ARM_SMP_H */
arch/arm/kernel/fiq.c +3 -2
@@ -67,10 +67,11 @@ static struct fiq_handler default_owner = {static struct fiq_handler *current_fiq = &default_owner; -int show_fiq_list (struct seq_file *p,void *v) +int show_fiq_list (struct seq_file *p,int prec) {if (current_fiq != &default_owner) -seq_printf (p,"FIQ: %s\n" , current_fiq->name); +seq_printf (p,"%*s: %s\n" , prec,"FIQ" , + current_fiq->name);return 0 ; }
arch/arm/kernel/irq.c +10 -6
@@ -57,11 +57,15 @@ int show_interrupts(struct seq_file *p, void *v)struct irq_desc *desc;struct irqaction * action;unsigned long flags; +int prec, n; + +for (prec =3 , n =1000 ; prec <10 && n <= nr_irqs; prec++) + n *=10 ;if (i ==0 ) {char cpuname[12 ]; -seq_printf (p," " ); +seq_printf (p,"%*s " , prec,"" );for_each_present_cpu (cpu) {sprintf (cpuname,"CPU%d" , cpu);seq_printf (p," %10s" , cpuname);
@@ -76,7 +80,7 @@ int show_interrupts(struct seq_file *p, void *v)if (!action) goto unlock; -seq_printf (p,"%3d: " , i); +seq_printf (p,"%*d: " , prec, i);for_each_present_cpu (cpu)seq_printf (p,"%10u " ,kstat_irqs_cpu (i, cpu));seq_printf (p," %10s" , desc->chip->name ? :"-" );
@@ -89,15 +93,15 @@ unlock:raw_spin_unlock_irqrestore (&desc->lock, flags); }else if (i == nr_irqs) {#ifdef CONFIG_FIQ -show_fiq_list (p, v); +show_fiq_list (p, prec);#endif #ifdef CONFIG_SMP -show_ipi_list (p); +show_ipi_list (p, prec);#endif #ifdef CONFIG_LOCAL_TIMERS -show_local_irqs (p); +show_local_irqs (p, prec);#endif -seq_printf (p,"Err: %10lu\n" , irq_err_count); +seq_printf (p,"%*s: %10lu\n" , prec,"Err" , irq_err_count); }return 0 ; }
arch/arm/kernel/smp.c +7 -7
@@ -396,16 +396,16 @@ void arch_send_call_function_single_ipi(int cpu)send_ipi_message (cpumask_of (cpu),IPI_CALL_FUNC_SINGLE ); } -void show_ipi_list (struct seq_file *p) +void show_ipi_list (struct seq_file *p,int prec) {unsigned int cpu; -seq_puts (p,"IPI:" ); +seq_printf (p,"%*s: " , prec,"IPI" );for_each_present_cpu (cpu) -seq_printf (p," %10u" ,__get_irq_stat (cpu, ipi_irqs)); +seq_printf (p,"%10u " ,__get_irq_stat (cpu, ipi_irqs)); -seq_putc (p,'\n' ); +seq_printf (p," Inter-processor interrupts\n" ); }/*
@@ -435,16 +435,16 @@ asmlinkage void __exception do_local_timer(struct pt_regs *regs) set_irq_regs(old_regs); } -void show_local_irqs (struct seq_file *p) +void show_local_irqs (struct seq_file *p,int prec) {unsigned int cpu; -seq_printf (p,"LOC: " ); +seq_printf (p,"%*s: " , prec,"LOC" );for_each_present_cpu (cpu)seq_printf (p,"%10u " ,__get_irq_stat (cpu, local_timer_irqs)); -seq_putc (p,'\n' ); +seq_printf (p," Local timer interrupts\n" ); }#endif
Signature · 1 line
1.6.2.5
PATCH 8/9 ARM: smp: provide individual IPI interrupt statistics
In reply to Russell King - ARM LinuxThis separates out the individual IPI interrupt counts from the total IPI count, which allows better visibility of what IPIs are being used for.
--- arch/arm/include/asm/hardirq.h | 4 +++- arch/arm/kernel/irq.c | 5 +++++ arch/arm/kernel/smp.c | 25 +++++++++++++++++++------ 3 files changed, 27 insertions(+), 7 deletions(-)
arch/arm/include/asm/hardirq.h +3 -1
@@ -5,13 +5,15 @@#include <linux/threads.h> #include <asm/irq.h> +#define NR_IPI 5 +typedef struct {unsigned int __softirq_pending;#ifdef CONFIG_LOCAL_TIMERS unsigned int local_timer_irqs;#endif #ifdef CONFIG_SMP -unsigned int ipi_irqs; +unsigned int ipi_irqs[NR_IPI ];#endif }____cacheline_aligned irq_cpustat_t ;
arch/arm/kernel/irq.c +5 -0
@@ -62,6 +62,11 @@ int show_interrupts(struct seq_file *p, void *v)for (prec =3 , n =1000 ; prec <10 && n <= nr_irqs; prec++) n *=10 ; +#ifdef CONFIG_SMP +if (prec <4 ) + prec =4 ; +#endif +if (i ==0 ) {char cpuname[12 ];
arch/arm/kernel/smp.c +19 -6
@@ -396,16 +396,28 @@ void arch_send_call_function_single_ipi(int cpu)send_ipi_message (cpumask_of (cpu),IPI_CALL_FUNC_SINGLE ); } +static const char *ipi_types[NR_IPI ] = { +#define S(x,s) [x - IPI_TIMER] = s +S (IPI_TIMER ,"Timer broadcast interrupts" ), +S (IPI_RESCHEDULE ,"Rescheduling interrupts" ), +S (IPI_CALL_FUNC ,"Function call interrupts" ), +S (IPI_CALL_FUNC_SINGLE ,"Single function call interrupts" ), +S (IPI_CPU_STOP ,"CPU stop interrupts" ), +}; +void show_ipi_list (struct seq_file *p,int prec) { -unsigned int cpu; +unsigned int cpu, i; -seq_printf (p,"%*s: " , prec,"IPI" ); +for (i =0 ; i <NR_IPI ; i++) { +seq_printf (p,"%*s%u: " , prec -1 ,"IPI" , i); -for_each_present_cpu (cpu) -seq_printf (p,"%10u " ,__get_irq_stat (cpu, ipi_irqs)); +for_each_present_cpu (cpu) +seq_printf (p,"%10u " , +__get_irq_stat (cpu, ipi_irqs[i])); -seq_printf (p," Inter-processor interrupts\n" ); +seq_printf (p," %s\n" , ipi_types[i]); + } }/*
@@ -520,7 +532,8 @@ asmlinkage void __exception do_IPI(int ipinr, struct pt_regs *regs)unsigned int cpu =smp_processor_id ();struct pt_regs *old_regs =set_irq_regs (regs); -__inc_irq_stat (cpu, ipi_irqs); +if (ipinr >=IPI_TIMER && ipinr <IPI_TIMER +NR_IPI ) +__inc_irq_stat (cpu, ipi_irqs[ipinr -IPI_TIMER ]);switch (ipinr) {case IPI_TIMER :
Signature · 1 line
1.6.2.5
PATCH 9/9 ARM: smp: collect IPI and local timer IRQs for /proc/stat
In reply to Russell King - ARM LinuxThe IPI and local timer interrupts weren't being properly accounted for in /proc/stat. Collect them from the irq_stat structure, and return their sum.
--- arch/arm/include/asm/hardirq.h | 8 ++++++++ arch/arm/kernel/smp.c | 15 +++++++++++++++ 2 files changed, 23 insertions(+), 0 deletions(-)
arch/arm/include/asm/hardirq.h +8 -0
@@ -22,6 +22,14 @@ typedef struct {#define __inc_irq_stat (cpu, member) __IRQ_STAT(cpu, member)++#define __get_irq_stat (cpu, member) __IRQ_STAT(cpu, member) +#ifdef CONFIG_SMP +u64 smp_irq_stat_cpu (unsigned int cpu); +#else +#define smp_irq_stat_cpu (cpu) 0 +#endif + +#define arch_irq_stat_cpu smp_irq_stat_cpu +#if NR_IRQS >512 #define HARDIRQ_BITS 10#elif NR_IRQS >256
arch/arm/kernel/smp.c +15 -0
@@ -420,6 +420,21 @@ void show_ipi_list(struct seq_file *p, int prec) } } +u64 smp_irq_stat_cpu (unsigned int cpu) +{ +u64 sum =0 ; +int i; + +for (i =0 ; i <NR_IPI ; i++) + sum +=__get_irq_stat (cpu, ipi_irqs[i]); + +#ifdef CONFIG_LOCAL_TIMERS + sum +=__get_irq_stat (cpu, local_timer_irqs); +#endif + +return sum; +} +/* * Timer (local or broadcast) support */
Signature · 1 line
1.6.2.5
cover Clean up SMP IPI support
In reply to Russell King - ARM LinuxQuoted 8 lines · Russell King - ARM Linux
This series is aimed at cleaning up our SMP IPI support. As has been recently pointed out, we have a spinlock in percpu data, which is not very nice.
If we switch to using the GIC SGIs, one for each IPI function, then we don't need the mask of ipi bits indicating what work needs to be done. This allows us to get rid of the troublesome spinlock in
send_ipi_message().
FWIW, the patches look fine to me (though I have tested them):
cover Clean up SMP IPI support
Quoted 11 lines · Catalin Marinas
Quoted 8 lines · Russell King - ARM Linux
This series is aimed at cleaning up our SMP IPI support. As has been recently pointed out, we have a spinlock in percpu data, which is not very nice.
If we switch to using the GIC SGIs, one for each IPI function, then we don't need the mask of ipi bits indicating what work needs to be done. This allows us to get rid of the troublesome spinlock in
send_ipi_message().FWIW, the patches look fine to me (though I have tested them):
^^^^
I meant I have *not* tested them.
Signature · 1 line
Catalin
cover Clean up SMP IPI support
Quoted 6 lines · Catalin Marinas
Quoted 5 lines · Catalin Marinas
Quoted 4 lines · Russell King - ARM Linux
This series is aimed at cleaning up our SMP IPI support. As has been recently pointed out, we have a spinlock in percpu data, which is not very nice.
Quoted 8 lines
Quoted 6 lines
Quoted 4 lines
If we switch to using the GIC SGIs, one for each IPI function, then we don't need the mask of ipi bits indicating what work needs to be done. This allows us to get rid of the troublesome spinlock in
send_ipi_message().FWIW, the patches look fine to me (though I have tested them):
^^^^I meant I have *not* tested them.
They work fine on my Versatile Express, but I've not checked the Realview EB yet.