ppc/pnv: Fix NMI system reset SRR1 value
PATCH 1/1 ppc/pnv: Fix NMI system reset SRR1 value
Commit a77fed5bd926 ("ppc/pnv: Add support for NMI interface") got the SRR1 setting wrong for sresets that hit outside of power-save states.
Fix this, better documenting the source for the bit definitions.
---
Thanks to Cedric for pointing out concerns with a previous MCE patch that unearthed this as well. Linux does not actually care what these SRR1[42:45] bits look like for non-powersave sresets, but we should follow documented behaviour as far as possible.
hw/ppc/pnv.c | 26 ++++++++++++++++++++------ 1 file changed, 20 insertions(+), 6 deletions(-)
hw/ppc/pnv.c +20 -6
@@ -1986,12 +1986,26 @@ static void pnv_cpu_do_nmi_on_cpu(CPUState *cs, run_on_cpu_data arg)cpu_synchronize_state (cs);ppc_cpu_do_system_reset (cs); -/* -* SRR1[42:45] is set to 0100 which the ISA defines as implementation -* dependent. POWER processors use this for xscom triggered interrupts, -* which come from the BMC or NMI IPIs. -*/ - env->spr[SPR_SRR1 ] |=PPC_BIT (43 ); +if (env->spr[SPR_SRR1 ] &PPC_BITMASK (46 ,47 )) { +/* +* Power-save wakeups, as indicated by non-zero SRR1[46:47] put the +* wakeup reason in SRR1[42:45], system reset is indicated with 0b0100 +* (PPC_BIT(43)). +*/ +if (!(env->spr[SPR_SRR1 ] &PPC_BIT (43 ))) { +warn_report ("ppc_cpu_do_system_reset does not set system reset wakeup reason" ); + env->spr[SPR_SRR1 ] |=PPC_BIT (43 ); + } + }else { +/* +* For non-powersave system resets, SRR1[42:45] are defined to be +* implementation-dependent. The POWER9 User Manual specifies that +* an external (SCOM driven, which may come from a BMC nmi command or +* another CPU requesting a NMI IPI) system reset exception should be +* 0b0010 (PPC_BIT(44)). +*/ + env->spr[SPR_SRR1 ] |=PPC_BIT (44 ); + } }static void pnv_nmi (NMIState *n,int cpu_index,Error **errp)
Signature · 1 line
2.23.0
Quoted 9 lines · Nicholas Piggin
Commit
a77fed5bd926("ppc/pnv: Add support for NMI interface") got the SRR1 setting wrong for sresets that hit outside of power-save states.Fix this, better documenting the source for the bit definitions.
Fixes a77fed5bd926 ("ppc/pnv: Add support for NMI interface") got theCc Cédric Le Goater <[email protected]>Cc David Gibson <[email protected]>Signed-off-by Nicholas Piggin <[email protected]>
We should introduce some defines like the SRR1_WAKE ones in Linux and cleanup powerpc_reset_wakeup(). This function uses cryptic values. That can be done later on as a followup.
Quoted 6 lines
---
Thanks to Cedric for pointing out concerns with a previous MCE patch that unearthed this as well. Linux does not actually care what these SRR1[42:45] bits look like for non-powersave sresets, but we should follow documented behaviour as far as possible.
We should introduce some defines like the SRR1_WAKE ones in Linux and cleanup powerpc_reset_wakeup(). This function uses cryptic values. That can be done later on as a followup.
I am currently after a bug which results in a CPU hard lockup because of a pending interrupt. It occurs on a SMP PowerNV machine when it is stressed with IO, such as scp of a big file.
I am suspecting more and more an issue with an interrupt being handled when the CPU is coming out of idle. I haven't seen anything wrong in the models. Unless this maybe :
/* Pretend to be returning from doze always as we don't lose state */
*msr |= (0x1ull << (63 - 47));I am not sure how in sync it is with PSSCR.
Thanks,
C.
Quoted 41 lines
hw/ppc/pnv.c | 26 ++++++++++++++++++++------ 1 file changed, 20 insertions(+), 6 deletions(-)hw/ppc/pnv.c +20
-6@@ -1986,12 +1986,26 @@ static void pnv_cpu_do_nmi_on_cpu(CPUState *cs, run_on_cpu_data arg)cpu_synchronize_state (cs);ppc_cpu_do_system_reset (cs); -/* -* SRR1[42:45] is set to 0100 which the ISA defines as implementation -* dependent. POWER processors use this for xscom triggered interrupts, -* which come from the BMC or NMI IPIs. -*/ - env->spr[SPR_SRR1 ] |=PPC_BIT (43 ); +if (env->spr[SPR_SRR1 ] &PPC_BITMASK (46 ,47 )) { +/* +* Power-save wakeups, as indicated by non-zero SRR1[46:47] put the +* wakeup reason in SRR1[42:45], system reset is indicated with 0b0100 +* (PPC_BIT(43)). +*/ +if (!(env->spr[SPR_SRR1 ] &PPC_BIT (43 ))) { +warn_report ("ppc_cpu_do_system_reset does not set system reset wakeup reason" ); + env->spr[SPR_SRR1 ] |=PPC_BIT (43 ); + } + }else { +/* +* For non-powersave system resets, SRR1[42:45] are defined to be +* implementation-dependent. The POWER9 User Manual specifies that +* an external (SCOM driven, which may come from a BMC nmi command or +* another CPU requesting a NMI IPI) system reset exception should be +* 0b0010 (PPC_BIT(44)). +*/ + env->spr[SPR_SRR1 ] |=PPC_BIT (44 ); + } }static void pnv_nmi (NMIState *n,int cpu_index,Error **errp)
Excerpts from Cédric Le Goater's message of May 8, 2020 3:14 am:
Quoted 11 lines · Nicholas Piggin
Quoted 9 lines
Commit
a77fed5bd926("ppc/pnv: Add support for NMI interface") got the SRR1 setting wrong for sresets that hit outside of power-save states.Fix this, better documenting the source for the bit definitions.
Fixes a77fed5bd926 ("ppc/pnv: Add support for NMI interface") got theCc Cédric Le Goater <[email protected]>Cc David Gibson <[email protected]>Signed-off-by Nicholas Piggin <[email protected]>
Quoted 5 lines
We should introduce some defines like the
SRR1_WAKEones in Linux and cleanuppowerpc_reset_wakeup(). This function uses cryptic values. That can be done later on as a followup.Reviewed-by Cédric Le Goater <[email protected]>
Thanks.
Quoted 12 lines
Quoted 6 lines
---
Thanks to Cedric for pointing out concerns with a previous MCE patch that unearthed this as well. Linux does not actually care what these SRR1[42:45] bits look like for non-powersave sresets, but we should follow documented behaviour as far as possible.
We should introduce some defines like the
SRR1_WAKEones in Linux and cleanuppowerpc_reset_wakeup(). This function uses cryptic values. That can be done later on as a followup.
Quoted 6 lines
I am currently after a bug which results in a CPU hard lockup because of a pending interrupt. It occurs on a SMP PowerNV machine when it is stressed with IO, such as scp of a big file.
I am suspecting more and more an issue with an interrupt being handled when the CPU is coming out of idle. I haven't seen anything wrong in
So you can't hit it when booting Linux with powersave=off?
Do we model stop with EC=0 properly? Looks like helper_pminsn seems to be doing the right thing there.
Quoted 6 lines
the models. Unless this maybe :
/* Pretend to be returning from doze always as we don't lose state */ *msr |= (0x1ull << (63 - 47));I am not sure how in sync it is with PSSCR.
That should be okay, the hardware can always enter a shallower state than was asked for. Linux will handle it. For testing purpose, we could model deeper states by scribbling on registers and indicating state loss.
Aide from SRR1 sleep state value, Linux uses the SRR1 wake reason value to run the interrupt handler, but even if we got SRR1 wrong, Linux eventually enables MSR[EE] so the interrupt should get replayed then (this is what Linux used to do until we added the wake-reason processing for improved performance).
But we do appear to get those right in powerpc_reset_wakeup().
Thanks,
Nick
Quoted 7 lines
Quoted 5 lines
of a pending interrupt. It occurs on a SMP PowerNV machine when it is stressed with IO, such as scp of a big file.
I am suspecting more and more an issue with an interrupt being handled when the CPU is coming out of idle. I haven't seen anything wrong in
So you can't hit it when booting Linux with powersave=off?
no. I uploaded 32GB steadily at 3.0MB/s on a smp 2 machine.
When powersave is on, a P8 or P9 machine will miss an interrupt quite quickly. This assert can catch a symptom of the failure :
@@ -75,6 +83,9 @@ void ppc_set_irq(PowerPCCPU *cpu, int n_
if (level) {
env->pending_interrupts |= 1 << n_IRQ;
cpu_interrupt(cs, CPU_INTERRUPT_HARD);
+ if (!(env->pending_interrupts & (1 << n_IRQ))) {
+ g_assert_not_reached();
+ }
} else {
env->pending_interrupts &= ~(1 << n_IRQ);
if (env->pending_interrupts == 0) {env->pending_interrupts is reseted in ppc_set_irq() setting it. I think it is the CPU handling the external IO interrupt which is kicked to wake up in cpu_interrupt(). The IRQ level goes out of sync with what the device expects and things go bad very quickly after.
But this is post mortem. I need to find the right spot where to put an assert() to analyze. But, adding too much traces closes the window ...
Quoted 2 lines
Do we model stop with EC=0 properly? Looks like
helper_pminsnseems to be doing the right thing there.
Yes. It seems so. The CPUs enter nap and come out with PACA_IRQ_EE set.
Quoted 11 lines
Quoted 6 lines
the models. Unless this maybe :
/* Pretend to be returning from doze always as we don't lose state */ *msr |= (0x1ull << (63 - 47));I am not sure how in sync it is with PSSCR.
That should be okay, the hardware can always enter a shallower state than was asked for. Linux will handle it. For testing purpose, we could model deeper states by scribbling on registers and indicating state loss.
Quoted 7 lines
Aide from SRR1 sleep state value, Linux uses the SRR1 wake reason value to run the interrupt handler, but even if we got SRR1 wrong, Linux eventually enables MSR[EE] so the interrupt should get replayed then (this is what Linux used to do until we added the wake-reason processing for improved performance).
But we do appear to get those right in
powerpc_reset_wakeup().
yes. Still digging.
Thanks,
C.
David Gibson
AppliedQuoted 9 lines · Nicholas Piggin
Commit
a77fed5bd926("ppc/pnv: Add support for NMI interface") got the SRR1 setting wrong for sresets that hit outside of power-save states.Fix this, better documenting the source for the bit definitions.
Fixes a77fed5bd926 ("ppc/pnv: Add support for NMI interface") got theCc Cédric Le Goater <[email protected]>Cc David Gibson <[email protected]>Signed-off-by Nicholas Piggin <[email protected]>
Applied to ppc-for-5.1, thanks.
Quoted 47 lines
---
Thanks to Cedric for pointing out concerns with a previous MCE patch that unearthed this as well. Linux does not actually care what these SRR1[42:45] bits look like for non-powersave sresets, but we should follow documented behaviour as far as possible.
hw/ppc/pnv.c | 26 ++++++++++++++++++++------ 1 file changed, 20 insertions(+), 6 deletions(-)hw/ppc/pnv.c +20
-6@@ -1986,12 +1986,26 @@ static void pnv_cpu_do_nmi_on_cpu(CPUState *cs, run_on_cpu_data arg)cpu_synchronize_state (cs);ppc_cpu_do_system_reset (cs); -/* -* SRR1[42:45] is set to 0100 which the ISA defines as implementation -* dependent. POWER processors use this for xscom triggered interrupts, -* which come from the BMC or NMI IPIs. -*/ - env->spr[SPR_SRR1 ] |=PPC_BIT (43 ); +if (env->spr[SPR_SRR1 ] &PPC_BITMASK (46 ,47 )) { +/* +* Power-save wakeups, as indicated by non-zero SRR1[46:47] put the +* wakeup reason in SRR1[42:45], system reset is indicated with 0b0100 +* (PPC_BIT(43)). +*/ +if (!(env->spr[SPR_SRR1 ] &PPC_BIT (43 ))) { +warn_report ("ppc_cpu_do_system_reset does not set system reset wakeup reason" ); + env->spr[SPR_SRR1 ] |=PPC_BIT (43 ); + } + }else { +/* +* For non-powersave system resets, SRR1[42:45] are defined to be +* implementation-dependent. The POWER9 User Manual specifies that +* an external (SCOM driven, which may come from a BMC nmi command or +* another CPU requesting a NMI IPI) system reset exception should be +* 0b0010 (PPC_BIT(44)). +*/ + env->spr[SPR_SRR1 ] |=PPC_BIT (44 ); + } }static void pnv_nmi (NMIState *n,int cpu_index,Error **errp)
Signature · 4 lines
David Gibson | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson
Quoted 2 lines · On Thu, 7 May 2020 23:51:54 +1000 David Gibson <[email protected]>…
Quoted 1 line · Nicholas Piggin
Commit
a77fed5bd926("ppc/pnv: Add support for NMI interface") got the
Please note that the culprit patch was merged with a different SHA1:
https://git.qemu.org/?p=qemu.git;a=commit;h=01b552b05b0f21f8ff57a508f7ad26f7abbcd123
Quoted 5 lines
Quoted 5 lines
SRR1 setting wrong for sresets that hit outside of power-save states.
Fix this, better documenting the source for the bit definitions.
Fixes a77fed5bd926 ("ppc/pnv: Add support for NMI interface") got the
Quoted 53 lines
Quoted 3 lines
Cc Cédric Le Goater <[email protected]>Cc David Gibson <[email protected]>Signed-off-by Nicholas Piggin <[email protected]>Applied to ppc-for-5.1, thanks.
Quoted 47 lines
---
Thanks to Cedric for pointing out concerns with a previous MCE patch that unearthed this as well. Linux does not actually care what these SRR1[42:45] bits look like for non-powersave sresets, but we should follow documented behaviour as far as possible.
hw/ppc/pnv.c | 26 ++++++++++++++++++++------ 1 file changed, 20 insertions(+), 6 deletions(-)hw/ppc/pnv.c +20
-6@@ -1986,12 +1986,26 @@ static void pnv_cpu_do_nmi_on_cpu(CPUState *cs, run_on_cpu_data arg)cpu_synchronize_state (cs);ppc_cpu_do_system_reset (cs); -/* -* SRR1[42:45] is set to 0100 which the ISA defines as implementation -* dependent. POWER processors use this for xscom triggered interrupts, -* which come from the BMC or NMI IPIs. -*/ - env->spr[SPR_SRR1 ] |=PPC_BIT (43 ); +if (env->spr[SPR_SRR1 ] &PPC_BITMASK (46 ,47 )) { +/* +* Power-save wakeups, as indicated by non-zero SRR1[46:47] put the +* wakeup reason in SRR1[42:45], system reset is indicated with 0b0100 +* (PPC_BIT(43)). +*/ +if (!(env->spr[SPR_SRR1 ] &PPC_BIT (43 ))) { +warn_report ("ppc_cpu_do_system_reset does not set system reset wakeup reason" ); + env->spr[SPR_SRR1 ] |=PPC_BIT (43 ); + } + }else { +/* +* For non-powersave system resets, SRR1[42:45] are defined to be +* implementation-dependent. The POWER9 User Manual specifies that +* an external (SCOM driven, which may come from a BMC nmi command or +* another CPU requesting a NMI IPI) system reset exception should be +* 0b0010 (PPC_BIT(44)). +*/ + env->spr[SPR_SRR1 ] |=PPC_BIT (44 ); + } }static void pnv_nmi (NMIState *n,int cpu_index,Error **errp)
Quoted 10 lines · Greg Kurz
Quoted 2 lines · On Thu, 7 May 2020 23:51:54 +1000 David Gibson <[email protected]>…
Quoted 1 line · Nicholas Piggin
Commit
a77fed5bd926("ppc/pnv: Add support for NMI interface") got thePlease note that the culprit patch was merged with a different SHA1:
https://git.qemu.org/?p=qemu.git;a=commit;h=01b552b05b0f21f8ff57a508f7ad26f7abbcd123
Quoted 7 lines
Quoted 5 lines
Quoted 5 lines
SRR1 setting wrong for sresets that hit outside of power-save states.
Fix this, better documenting the source for the bit definitions.
Fixes a77fed5bd926 ("ppc/pnv: Add support for NMI interface") got theFixes 01b552b05b0f ("ppc/pnv: Add support for NMI interface")
Updated in my tree, thanks.
Quoted 55 lines
Quoted 54 lines
Quoted 4 lines
Cc Cédric Le Goater <[email protected]>Cc David Gibson <[email protected]>Signed-off-by Nicholas Piggin <[email protected]>Applied to ppc-for-5.1, thanks.
Quoted 47 lines
---
Thanks to Cedric for pointing out concerns with a previous MCE patch that unearthed this as well. Linux does not actually care what these SRR1[42:45] bits look like for non-powersave sresets, but we should follow documented behaviour as far as possible.
hw/ppc/pnv.c | 26 ++++++++++++++++++++------ 1 file changed, 20 insertions(+), 6 deletions(-)hw/ppc/pnv.c +20
-6@@ -1986,12 +1986,26 @@ static void pnv_cpu_do_nmi_on_cpu(CPUState *cs, run_on_cpu_data arg)cpu_synchronize_state (cs);ppc_cpu_do_system_reset (cs); -/* -* SRR1[42:45] is set to 0100 which the ISA defines as implementation -* dependent. POWER processors use this for xscom triggered interrupts, -* which come from the BMC or NMI IPIs. -*/ - env->spr[SPR_SRR1 ] |=PPC_BIT (43 ); +if (env->spr[SPR_SRR1 ] &PPC_BITMASK (46 ,47 )) { +/* +* Power-save wakeups, as indicated by non-zero SRR1[46:47] put the +* wakeup reason in SRR1[42:45], system reset is indicated with 0b0100 +* (PPC_BIT(43)). +*/ +if (!(env->spr[SPR_SRR1 ] &PPC_BIT (43 ))) { +warn_report ("ppc_cpu_do_system_reset does not set system reset wakeup reason" ); + env->spr[SPR_SRR1 ] |=PPC_BIT (43 ); + } + }else { +/* +* For non-powersave system resets, SRR1[42:45] are defined to be +* implementation-dependent. The POWER9 User Manual specifies that +* an external (SCOM driven, which may come from a BMC nmi command or +* another CPU requesting a NMI IPI) system reset exception should be +* 0b0010 (PPC_BIT(44)). +*/ + env->spr[SPR_SRR1 ] |=PPC_BIT (44 ); + } }static void pnv_nmi (NMIState *n,int cpu_index,Error **errp)
Signature · 4 lines
David Gibson | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson
Patchew URL: https://patchew.org/QEMU/[email protected]/
Patchew URL: https://patchew.org/QEMU/[email protected]/
Hi,
This series seems to have some coding style problems. See output below for more information:
Message-id: [email protected]
Subject: [PATCH] ppc/pnv: Fix NMI system reset SRR1 value
Type: series
=== TEST SCRIPT BEGIN ===
#!/bin/bash
git rev-parse base > /dev/null || exit 0
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram./scripts/checkpatch.pl --mailback base..
=== TEST SCRIPT END ===
Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
Switched to a new branch 'test'
fbe72cb ppc/pnv: Fix NMI system reset SRR1 value
=== OUTPUT BEGIN ===
ERROR: code indent should never use tabs
#35: FILE: hw/ppc/pnv.c:1991:
+^I * Power-save wakeups, as indicated by non-zero SRR1[46:47] put the$
ERROR: code indent should never use tabs
#36: FILE: hw/ppc/pnv.c:1992:
+^I * wakeup reason in SRR1[42:45], system reset is indicated with 0b0100$
ERROR: code indent should never use tabs
#37: FILE: hw/ppc/pnv.c:1993:
+^I * (PPC_BIT(43)).$
ERROR: code indent should never use tabs
#38: FILE: hw/ppc/pnv.c:1994:
+^I */$
ERROR: line over 90 characters
#40: FILE: hw/ppc/pnv.c:1996:
+ warn_report("ppc_cpu_do_system_reset does not set system reset wakeup reason");
ERROR: code indent should never use tabs
#45: FILE: hw/ppc/pnv.c:2001:
+^I * For non-powersave system resets, SRR1[42:45] are defined to be$
ERROR: code indent should never use tabs
#46: FILE: hw/ppc/pnv.c:2002:
+^I * implementation-dependent. The POWER9 User Manual specifies that$
ERROR: code indent should never use tabs
#47: FILE: hw/ppc/pnv.c:2003:
+^I * an external (SCOM driven, which may come from a BMC nmi command or$
ERROR: code indent should never use tabs
#48: FILE: hw/ppc/pnv.c:2004:
+^I * another CPU requesting a NMI IPI) system reset exception should be$
ERROR: code indent should never use tabs
#49: FILE: hw/ppc/pnv.c:2005:
+^I * 0b0010 (PPC_BIT(44)).$
total: 10 errors, 0 warnings, 32 lines checked
Commit fbe72cb9d465 (ppc/pnv: Fix NMI system reset SRR1 value) has style problems, please review. If any of these errors are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
=== OUTPUT END ===
Test command exited with code: 1
The full log is available at
http://patchew.org/logs/[email protected]/testing.checkpatch/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to [email protected]