Skip to contentSkip to conversation

powerpc/64s: catch external interrupts going to host in POWER9

PATCH In review Nicholas Piggin 2 participants 5 messages Apr 12, 2017 – Apr 13, 2017 PowerPC

lore
Share & tools
lore

Nicholas Piggin

author reviewer
#0 lore raw

PATCH 1/1 powerpc/64s: catch external interrupts going to host in POWER9

After setting LPES0 in the host on POWER9, the host external interrupt handler no longer works correctly, because it's set to HV mode (HSRR) for POWER7/8 with LPES0 clear. We don't expect to get any EE in the host with XIVE, but it seems preferable to catch unexpected interrupts in case there are bugs or unexpected behaviour.

Signed-off-by Nicholas Piggin <[email protected]>

---

Hi,

I was testing the LPES0 code on POWER9 under mambo, which exploded because I didn't use --enable-xive_interrupts so the host was getting EEs.

Errant 0x500 in the host will end up hrfid'ing to uninitialized HSRR[01] which ends up dying in interesting ways. Should we add this patch to Ben's xive topic branch that sets LPES0? (Or do you rebase topic branches? It could be rolled up with that particular patch if so).

Thanks,
Nick

 arch/powerpc/kernel/exceptions-64s.S | 26 +++++++++++++++++++++++---
 1 file changed, 23 insertions(+), 3 deletions(-)
arch/powerpc/kernel/exceptions-64s.S +23 -3
index 857bf7c5b946..2f26a0553a4a 100644
@@ -718,9 +718,21 @@ hardware_interrupt_hv:
 		_MASKABLE_EXCEPTION_PSERIES(0x500, hardware_interrupt_common,
 					    EXC_HV, SOFTEN_TEST_HV)
 	FTR_SECTION_ELSE
+		/*
+		 * The POWER9 XIVE interrupt controller should be configured
+		 * to send all interrupts to the host as HVI, even with the
+		 * OPAL XICS emulation, so HVMODE should never see a 0x500
+		 * interrupt. However we catch it in case of a bug.
+		 *
+		 * POWER9 sets the LPES0 LPCR bit in the host, which
+		 * delivers external interrupts to SRR[01] with MSR_HV
+		 * unchanged (intended for guest delivery), so these need
+		 * to be caught as EXC_STD interrupts in the host.
+		 */
 		_MASKABLE_EXCEPTION_PSERIES(0x500, hardware_interrupt_common,
 					    EXC_STD, SOFTEN_TEST_PR)
-	ALT_FTR_SECTION_END_IFSET(CPU_FTR_HVMODE | CPU_FTR_ARCH_206)
+	ALT_FTR_SECTION_END(CPU_FTR_HVMODE|CPU_FTR_ARCH_206|CPU_FTR_ARCH_300,
+			    CPU_FTR_HVMODE|CPU_FTR_ARCH_206)
 EXC_REAL_END(hardware_interrupt, 0x500, 0x100)
 
 EXC_VIRT_BEGIN(hardware_interrupt, 0x4500, 0x100)

@@ -730,13 +742,21 @@ hardware_interrupt_relon_hv:
 		_MASKABLE_RELON_EXCEPTION_PSERIES(0x500, hardware_interrupt_common, EXC_HV, SOFTEN_TEST_HV)
 	FTR_SECTION_ELSE
 		_MASKABLE_RELON_EXCEPTION_PSERIES(0x500, hardware_interrupt_common, EXC_STD, SOFTEN_TEST_PR)
-	ALT_FTR_SECTION_END_IFSET(CPU_FTR_HVMODE)
+	ALT_FTR_SECTION_END(CPU_FTR_HVMODE|CPU_FTR_ARCH_206|CPU_FTR_ARCH_300,
+			    CPU_FTR_HVMODE|CPU_FTR_ARCH_206)
 EXC_VIRT_END(hardware_interrupt, 0x4500, 0x100)
 
 TRAMP_KVM(PACA_EXGEN, 0x500)
 TRAMP_KVM_HV(PACA_EXGEN, 0x500)
-EXC_COMMON_ASYNC(hardware_interrupt_common, 0x500, do_IRQ)
 
+EXC_COMMON_BEGIN(hardware_interrupt_common)
+BEGIN_FTR_SECTION
+	/* See POWER9 comment above */
+	b	unknown_host_ee_common
+END_FTR_SECTION_IFSET(CPU_FTR_HVMODE | CPU_FTR_ARCH_300)
+	STD_EXCEPTION_COMMON_ASYNC(0x500, hardware_interrupt_common, do_IRQ)
+
+EXC_COMMON_ASYNC(unknown_host_ee_common, 0x500, unknown_exception)
 
 EXC_REAL(alignment, 0x600, 0x100)
 EXC_VIRT(alignment, 0x4600, 0x100, 0x600)
Signature · 1 line
2.11.0
#1 lore raw
Quoted 8 lines · Nicholas Piggin

After setting LPES0 in the host on POWER9, the host external interrupt handler no longer works correctly, because it's set to HV mode (HSRR) for POWER7/8 with LPES0 clear. We don't expect to get any EE in the host with XIVE, but it seems preferable to catch unexpected interrupts in case there are bugs or unexpected behaviour.

Quoted 1 line
Signed-off-by Nicholas Piggin <[email protected]>

---

No. Let's just get LPES back to P8 value in the host, we don't care as we don't get those EEs on normal systems. Then make sure KVM properly sets it the way we want when setting up the guest LPCR (which it should be doing with my patches).

Much simpler patch...

Cheers,
Ben.

Quoted 68 lines

Hi,

I was testing the LPES0 code on POWER9 under mambo, which exploded because I didn't use --enable-xive_interrupts so the host was getting EEs.

Errant 0x500 in the host will end up hrfid'ing to uninitialized HSRR[01] which ends up dying in interesting ways. Should we add this patch to Ben's xive topic branch that sets LPES0? (Or do you rebase topic branches? It could be rolled up with that particular patch if so).

Thanks,
Nick

 arch/powerpc/kernel/exceptions-64s.S | 26 +++++++++++++++++++++++---

 1 file changed, 23 insertions(+), 3 deletions(-)
arch/powerpc/kernel/exceptions-64s.S +0 -0
index 857bf7c5b946..2f26a0553a4a 100644
@@ -718,9 +718,21 @@ hardware_interrupt_hv:
 >  		_MASKABLE_EXCEPTION_PSERIES(0x500, hardware_interrupt_common,
 >  					    EXC_HV, SOFTEN_TEST_HV)
Quoted 14 lines

  FTR_SECTION_ELSE

arch/powerpc/kernel/exceptions-64s.S +11 -0
+		/*
+		 * The POWER9 XIVE interrupt controller should be configured
+		 * to send all interrupts to the host as HVI, even with the
+		 * OPAL XICS emulation, so HVMODE should never see a 0x500
+		 * interrupt. However we catch it in case of a bug.
+		 *
+		 * POWER9 sets the LPES0 LPCR bit in the host, which
+		 * delivers external interrupts to SRR[01] with MSR_HV
+		 * unchanged (intended for guest delivery), so these need
+		 * to be caught as EXC_STD interrupts in the host.
+		 */

  _MASKABLE_EXCEPTION_PSERIES(0x500, hardware_interrupt_common, EXC_STD, SOFTEN_TEST_PR)

arch/powerpc/kernel/exceptions-64s.S hardware_interrupt_hv: · line 735

Quoted 3 lines
arch/powerpc/kernel/exceptions-64s.S +2 -1
-	ALT_FTR_SECTION_END_IFSET(CPU_FTR_HVMODE | CPU_FTR_ARCH_206)
+	ALT_FTR_SECTION_END(CPU_FTR_HVMODE|CPU_FTR_ARCH_206|CPU_FTR_ARCH_300,
+			    CPU_FTR_HVMODE|CPU_FTR_ARCH_206)

 EXC_REAL_END(hardware_interrupt, 0x500, 0x100)
 EXC_VIRT_BEGIN(hardware_interrupt, 0x4500, 0x100)

arch/powerpc/kernel/exceptions-64s.S +0 -0
@@ -730,13 +742,21 @@ hardware_interrupt_relon_hv:
 >  		_MASKABLE_RELON_EXCEPTION_PSERIES(0x500, hardware_interrupt_common, EXC_HV, SOFTEN_TEST_HV)

arch/powerpc/kernel/exceptions-64s.S hardware_interrupt_relon_hv: · line 746

Quoted 5 lines

  FTR_SECTION_ELSE
  _MASKABLE_RELON_EXCEPTION_PSERIES(0x500, hardware_interrupt_common, EXC_STD, SOFTEN_TEST_PR)

arch/powerpc/kernel/exceptions-64s.S +2 -1
-	ALT_FTR_SECTION_END_IFSET(CPU_FTR_HVMODE)
+	ALT_FTR_SECTION_END(CPU_FTR_HVMODE|CPU_FTR_ARCH_206|CPU_FTR_ARCH_300,
+			    CPU_FTR_HVMODE|CPU_FTR_ARCH_206)

 EXC_VIRT_END(hardware_interrupt, 0x4500, 0x100)

 TRAMP_KVM(PACA_EXGEN, 0x500) TRAMP_KVM_HV(PACA_EXGEN, 0x500) -EXC_COMMON_ASYNC(hardware_interrupt_common, 0x500, do_IRQ)

arch/powerpc/kernel/exceptions-64s.S +2 -0
+EXC_COMMON_BEGIN(hardware_interrupt_common)
+BEGIN_FTR_SECTION
Quoted 2 lines
arch/powerpc/kernel/exceptions-64s.S +1 -0
+	/* See POWER9 comment above */

arch/powerpc/kernel/exceptions-64s.S hardware_interrupt_relon_hv: · line 755

Quoted 1 line
arch/powerpc/kernel/exceptions-64s.S +1 -0
+	b	unknown_host_ee_common

+END_FTR_SECTION_IFSET(CPU_FTR_HVMODE | CPU_FTR_ARCH_300)

Quoted 1 line
arch/powerpc/kernel/exceptions-64s.S +1 -0
+	STD_EXCEPTION_COMMON_ASYNC(0x500, hardware_interrupt_common, do_IRQ)
arch/powerpc/kernel/exceptions-64s.S +2 -0
+
+EXC_COMMON_ASYNC(unknown_host_ee_common, 0x500, unknown_exception)


 EXC_REAL(alignment, 0x600, 0x100)
 EXC_VIRT(alignment, 0x4600, 0x100, 0x600)

author reviewer
#2 lore raw
Quoted 7 lines · On Wed, 12 Apr 2017 23:45:42 +1000 Benjamin Herrenschmidt…
Quoted 6 lines · Nicholas Piggin

After setting LPES0 in the host on POWER9, the host external interrupt handler no longer works correctly, because it's set to HV mode (HSRR) for POWER7/8 with LPES0 clear. We don't expect to get any EE in the host with XIVE, but it seems preferable to catch unexpected interrupts in case there are bugs or unexpected behaviour.

Quoted 8 lines
Quoted 2 lines
Quoted 1 line
Signed-off-by Nicholas Piggin <[email protected]>

---

No. Let's just get LPES back to P8 value in the host, we don't care as we don't get those EEs on normal systems. Then make sure KVM properly sets it the way we want when setting up the guest LPCR (which it should be doing with my patches).
Much simpler patch...

Yeah sure that sounds good. How's this then?

---
 arch/powerpc/kernel/exceptions-64s.S | 14 +++++++++++++-
 1 file changed, 13 insertions(+), 1 deletion(-)
arch/powerpc/kernel/exceptions-64s.S +13 -1
index 857bf7c5b946..c78165e5fb77 100644
@@ -735,8 +735,20 @@ EXC_VIRT_END(hardware_interrupt, 0x4500, 0x100)
 
 TRAMP_KVM(PACA_EXGEN, 0x500)
 TRAMP_KVM_HV(PACA_EXGEN, 0x500)
-EXC_COMMON_ASYNC(hardware_interrupt_common, 0x500, do_IRQ)
 
+EXC_COMMON_BEGIN(hardware_interrupt_common)
+BEGIN_FTR_SECTION
+	/*
+	 * The POWER9 XIVE interrupt controller should be configured to send
+	 * all interrupts to the host as HVI, even with the OPAL XICS
+	 * emulation, so HVMODE should never see a 0x500 interrupt. However we
+	 * catch it in case of a bug.
+	 */
+	b	unknown_host_ee_common
+END_FTR_SECTION_IFSET(CPU_FTR_HVMODE | CPU_FTR_ARCH_300)
+	STD_EXCEPTION_COMMON_ASYNC(0x500, hardware_interrupt_common, do_IRQ)
+
+EXC_COMMON_ASYNC(unknown_host_ee_common, 0x500, unknown_exception)
 
 EXC_REAL(alignment, 0x600, 0x100)
 EXC_VIRT(alignment, 0x4600, 0x100, 0x600)
Signature · 1 line
2.11.0
#3 lore raw
Quoted 1 line · Nicholas Piggin

Yeah sure that sounds good. How's this then?

I suppose so :-) When I was testing all that I had a "b ." at 0x500 and 0x4500 and I didn't hit them :)

author reviewer
#4 lore raw
Quoted 5 lines · On Thu, 13 Apr 2017 07:34:51 +1000 Benjamin Herrenschmidt…
Quoted 1 line · Nicholas Piggin

Yeah sure that sounds good. How's this then?

I suppose so :-) When I was testing all that I had a "b ." at 0x500 and 0x4500 and I didn't hit them :)

If only for the benefit of poorly configured systemsim users like me. Could remove it when the dust settles.