powerpc/mpic: don't reset affinity for secondary MPIC on boot

This patch implements the first approach, because it can work on machines that have a secondary controller that needs to deliver interrupts to a destination other than CPU 0. The disadvantage is that it requires the system to set up the affinity register correctly on bootup.

I would rather, for non primary, set it to a cpu provided as either a new argument or an mpic struct member initially set to 1 with an accessor to change it if necessary.

Or should we define a flag to have it read it at init time from the chip ?

Kexec/kdump currently fails on the IBM QS2x blades when the kexec happens on a CPU other than the initial boot CPU. It turns out that this is the result of mpic_init trying to set affinity of each interrupt vector to the current boot CPU.

As far as I can tell, the same problem is likely to exist on any secondary MPIC, because they have to deliver interrupts to the first output all the time. There are two potential solutions for this: either not set up affinity at all for secondary MPICs, or assume that a single CPU output is connected to the upstream interrupt controller and hardcode affinity to that per architecture.

This patch implements the second approach, defaulting to the first output. An architecture that has a secondary MPIC connected upstream using a different output needs to set the mpic default_dest to the correct value before calling mpic_init.

Signed-off-by Arnd Bergmann <[email protected]>

---
 include/asm/mpic.h |    3 +++
 sysdev/mpic.c      |    9 +++++++--
 2 files changed, 10 insertions(+), 2 deletions(-)

Quoted 3 lines · Benjamin Herrenschmidt

I would rather, for non primary, set it to a cpu provided as either a new argument or an mpic struct member initially set to 1 with an accessor to change it if necessary.

How about this one?

Quoted 1 line

Or should we define a flag to have it read it at init time from the chip ?

I don't understand. This is init time, where we write it to the chip, what do you mean with reading from it? If I understand you correctly, we can't trust that value to start with.

	Arnd <><

arch/powerpc/include/asm/mpic.h +3 -0

index fe566a3..543d51c 100644
@@ -295,6 +295,9 @@ struct mpic
 	/* Protected sources */
 	unsigned long		*protected;
 
+	/* destination for non-primary MPICs */
+	int			default_dest;
+
 #ifdef CONFIG_MPIC_WEIRD
 	/* Pointer to HW info array */
 	u32			*hw_set;

arch/powerpc/sysdev/mpic.c +7 -2

index 8e3478c..cf154c9 100644
@@ -1220,6 +1220,7 @@ void __init mpic_set_default_senses(struct mpic *mpic, u8 *senses, int count)
 void __init mpic_init(struct mpic *mpic)
 {
 	int i;
+	int cpu;
 
 	BUG_ON(mpic->num_sources == 0);
 

@@ -1262,6 +1263,11 @@ void __init mpic_init(struct mpic *mpic)
 
 	mpic_pasemi_msi_init(mpic);
 
+	if (mpic->flags & MPIC_PRIMARY)
+		cpu = hard_smp_processor_id();
+	else
+		cpu = mpic->default_dest;
+
 	for (i = 0; i < mpic->num_sources; i++) {
 		/* start with vector = source number, and masked */
 		u32 vecpri = MPIC_VECPRI_MASK | i |

@@ -1272,8 +1278,7 @@ void __init mpic_init(struct mpic *mpic)
 			continue;
 		/* init hw */
 		mpic_irq_write(i, MPIC_INFO(IRQ_VECTOR_PRI), vecpri);
-		mpic_irq_write(i, MPIC_INFO(IRQ_DESTINATION),
-			       1 << hard_smp_processor_id());
+		mpic_irq_write(i, MPIC_INFO(IRQ_DESTINATION), 1 << cpu);
 	}
 
 	/* Init spurious vector */

Oh just that for powermac for example, I know I'm resetting the thing, so can't rely on init values, and on some BML embedded boxes too, while on things like cell I don't off hand know what the right CPU number is to hit the right C3PO interrupt, so I'm better off reading what SLOF did :-)

But if you think on Cell we can just hard wire in the platform code, then I'm ok.

Quoted 5 lines · Benjamin Herrenschmidt

Oh just that for powermac for example, I know I'm resetting the thing, so can't rely on init values, and on some BML embedded boxes too, while on things like cell I don't off hand know what the right CPU number is to hit the right C3PO interrupt, so I'm better off reading what SLOF did :-)

The current code will always use destination 0 for any MPIC, because mpic_init is run befew smp_init and it currently uses the local CPU.

If an SMP powermac requires a specific destination setting in the secondary MPIC, either the current mpic_init code is broken there in the same way as on cell, or more broken if it requires something other than destination 0.

Quoted 2 lines

But if you think on Cell we can just hard wire in the platform code, then I'm ok.

On cell, I know that destination 0 is always correct because of the wiring inside of Axon.