Implement MPIPL for PowerNV
v2 cover Implement MPIPL for PowerNV
Overview
=========
Implemented MPIPL (Memory Preserving IPL, aka fadump) on PowerNV machine in QEMU.
Note: It's okay if this isn't merged as there might be less users. Sending for archieval purpose, as the patches can be referred for how fadump/mpipl can be implemented in baremetal/PowerNV/any other arch QEMU.
Fadump is an alternative dump mechanism to kdump, in which we the firmware does a memory preserving boot, and the second/crashkernel is booted fresh like a normal system reset, instead of the crashed kernel loading the second/crashkernel in case of kdump.
MPIPL in PowerNV, is similar to fadump in Pseries. The idea is same, memory preserving, where in PowerNV we are assisted by SBE (Self Boot Engine) & Hostboot, while in Pseries we are assisted by PHyp (Power Hypervisor)
For implementing in baremetal/powernv QEMU, we need to export a "ibm,opal/dump" node in the device tree, to tell the kernel we support MPIPL
Once kernel sees the support, and "fadump=on" is passed on commandline, kernel will register memory regions to preserve with Skiboot.
Kernel sends these data using OPAL calls, after which skiboot/opal saves the memory region details to MDST and MDDT tables (S-source, D-destination)
Skiboot then triggers the "S0 Interrupt" to the SBE (Self Boot Engine), along with OPAL's relocated base address.
SBE then stops all core clocks, and only does particular ISteps for a memory preserving boot.
Then, hostboot comes up, and with help of the relocated base address, it accesses MDST & MDDT tables (S-source and D-destination), and preserves the memory regions according to the data in these tables.
And after preserving, it writes the preserved memory region details to MDRT tables (R-Result), for the kernel to know where/whether a memory region is preserved.
Both SBE's and hostboot responsiblities have in implemented in the SBE code in QEMU.
Then in the second kernel/crashkernel boot, OPAL passes the "mpipl-boot" property for the kernel to know that a dump is active, which kernel then exports in /proc/vmcore
Git Tree for Testing
====================
https://github.com/adi-g15-ibm/qemu/tree/fadump-powernv-v2
Aditya Gupta (9):
hw/ppc: Move SBE host doorbell function to top of file hw/ppc: Implement S0 SBE interrupt as cpu_pause then host reset hw/ppc: Handle stash command in PowerNV SBE pnv/mpipl: Preserve memory regions as per MDST/MDDT tables pnv/mpipl: Preserve CPU registers after crash pnv/mpipl: Set thread entry size to be allocated by firmware pnv/mpipl: Write the preserved CPU and MDRT state pnv/mpipl: Enable MPIPL support
tests/functional: Add test for MPIPL in PowerNV
hw/ppc/meson.build | 1 + hw/ppc/pnv.c | 90 ++++++ hw/ppc/pnv_mpipl.c | 388 ++++++++++++++++++++++++++ hw/ppc/pnv_sbe.c | 74 ++++- include/hw/ppc/pnv.h | 7 + include/hw/ppc/pnv_mpipl.h | 167 +++++++++++ tests/functional/ppc64/test_fadump.py | 35 +-- 7 files changed, 731 insertions(+), 31 deletions(-) create mode 100644 hw/ppc/pnv_mpipl.c create mode 100644 include/hw/ppc/pnv_mpipl.h
Signature · 1 line
2.52.0
PATCH v2 1/9 hw/ppc: Move SBE host doorbell function to top of file
Moved 'pnv_sbe_set_host_doorbell' as-it-is to above 'pnv_sbe_power9_xscom_ctrl_write'.
This is done since in a future patch, S0 interrupt implementation uses 'pnv_sbe_set_host_doorbell', hence the host doorbell function needs to be declared/defined before 'pnv_sbe_power9_xscom_ctrl_write' where we implement the S0 interrupt.
No functional change.
--- hw/ppc/pnv_sbe.c | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-)
hw/ppc/pnv_sbe.c +9 -9
@@ -80,6 +80,15 @@#define SBE_CONTROL_REG_S0 PPC_BIT(14)#define SBE_CONTROL_REG_S1 PPC_BIT(15) +static void pnv_sbe_set_host_doorbell (PnvSBE *sbe,uint64_t val) +{ + val &=SBE_HOST_RESPONSE_MASK ;/* Is this right? What does HW do? */ + sbe->host_doorbell = val; + +trace_pnv_sbe_reg_set_host_doorbell (val); +qemu_set_irq (sbe->psi_irq, !!val); +} +struct sbe_msg {uint64_t reg[4 ]; };
@@ -125,15 +134,6 @@ static const MemoryRegionOps pnv_sbe_power9_xscom_ctrl_ops = { .endianness = DEVICE_BIG_ENDIAN, }; -static void pnv_sbe_set_host_doorbell (PnvSBE *sbe,uint64_t val) -{ - val &=SBE_HOST_RESPONSE_MASK ;/* Is this right? What does HW do? */ - sbe->host_doorbell = val; - -trace_pnv_sbe_reg_set_host_doorbell (val); -qemu_set_irq (sbe->psi_irq, !!val); -} -/* SBE Target Type */ #define SBE_TARGET_TYPE_PROC 0x00#define SBE_TARGET_TYPE_EX 0x01
Signature · 1 line
2.52.0
PATCH v2 2/9 hw/ppc: Implement S0 SBE interrupt as cpu_pause then host reset
In reply to Aditya GuptaDuring MPIPL (aka fadump), OPAL triggers the S0 SBE interrupt to trigger MPIPL.
Currently S0 interrupt is unimplemented in QEMU.
Implement S0 interrupt as 'pause_vcpus' + 'guest_reset' in QEMU, as the SBE's implementation of S0 seems to be basically "stop all clocks" and then "host reset".
pause_vcpus is done in a later patch when register preserving support is added
See 'stopClocksS0' in SBE source code for more information.
Also log both S0 and S1 interrupts.
--- hw/ppc/meson.build | 1 + hw/ppc/pnv_mpipl.c | 26 ++++++++++++++++++++++++++ hw/ppc/pnv_sbe.c | 29 +++++++++++++++++++++++++++++ include/hw/ppc/pnv.h | 6 ++++++ include/hw/ppc/pnv_mpipl.h | 19 +++++++++++++++++++ 5 files changed, 81 insertions(+) create mode 100644 hw/ppc/pnv_mpipl.c create mode 100644 include/hw/ppc/pnv_mpipl.h
hw/ppc/meson.build +1 -0
@@ -56,6 +56,7 @@ ppc_ss.add(when: 'CONFIG_POWERNV', if_true: files( 'pnv_pnor.c', 'pnv_nest_pervasive.c', 'pnv_n1_chiplet.c', + 'pnv_mpipl.c', )) # PowerPC 4xx boards ppc_ss.add(when: 'CONFIG_PPC405', if_true: files(
hw/ppc/pnv_mpipl.c new file +26 -0
@@ -0,0 +1,26 @@ +/* +* Emulation of MPIPL (Memory Preserving Initial Program Load), aka fadump +* +* SPDX-License-Identifier: GPL-2.0-or-later +*/ + +#include "qemu/osdep.h" +#include "system/runstate.h" +#include "hw/ppc/pnv.h" +#include "hw/ppc/pnv_mpipl.h" + +void do_mpipl_preserve (PnvMachineState *pnv) +{ +/* Mark next boot as Memory-preserving boot */ + pnv->mpipl_state.is_next_boot_mpipl = true; + +/* +* Do a guest reset. +* Next reset will see 'is_next_boot_mpipl' as true, and trigger MPIPL +* +* Requirement: +* GUEST_RESET is expected to NOT clear the memory, as is the case when +* this is merged +*/ +qemu_system_reset_request (SHUTDOWN_CAUSE_GUEST_RESET ); +}
hw/ppc/pnv_sbe.c +29 -0
@@ -21,11 +21,14 @@#include "qapi/error.h" #include "qemu/log.h" #include "qemu/module.h" +#include "system/cpus.h" +#include "system/runstate.h" #include "hw/irq.h" #include "hw/qdev-properties.h" #include "hw/ppc/pnv.h" #include "hw/ppc/pnv_xscom.h" #include "hw/ppc/pnv_sbe.h" +#include "hw/ppc/pnv_mpipl.h" #include "trace.h" /*
@@ -113,11 +116,37 @@ static uint64_t pnv_sbe_power9_xscom_ctrl_read(void *opaque, hwaddr addr,static void pnv_sbe_power9_xscom_ctrl_write (void *opaque,hwaddr addr,uint64_t val,unsigned size) { +PnvMachineState *pnv =PNV_MACHINE (qdev_get_machine ()); +PnvSBE *sbe = opaque;uint32_t offset = addr >>3 ;trace_pnv_sbe_xscom_ctrl_write (addr, val);switch (offset) { +case SBE_CONTROL_REG_RW : +switch (val) { +case SBE_CONTROL_REG_S0 : +qemu_log_mask (LOG_UNIMP ,"SBE: S0 Interrupt triggered\n" ); + +pnv_sbe_set_host_doorbell (sbe, sbe->host_doorbell |SBE_HOST_RESPONSE_MASK ); + +/* Preserve memory regions and CPU state, if MPIPL is registered */ +do_mpipl_preserve (pnv); + +/* +* Control may not come back here as 'do_mpipl_preserve' triggers +* a guest reboot +*/ +break ; +case SBE_CONTROL_REG_S1 : +qemu_log_mask (LOG_UNIMP ,"SBE: S1 Interrupt triggered\n" ); +break ; +default : +qemu_log_mask (LOG_UNIMP , +"SBE: CONTROL_REG_RW: Unknown value: Ox%." + HWADDR_PRIx"\n" , val); + } +break ;default :qemu_log_mask (LOG_UNIMP ,"SBE Unimplemented register: Ox%" HWADDR_PRIx"\n" , addr >>3 );
include/hw/ppc/pnv.h +6 -0
@@ -25,6 +25,7 @@#include "hw/sysbus.h" #include "hw/ipmi/ipmi.h" #include "hw/ppc/pnv_pnor.h" +#include "hw/ppc/pnv_mpipl.h" #define TYPE_PNV_CHIP "pnv-chip"
@@ -111,6 +112,8 @@ struct PnvMachineState {bool big_core;bool lpar_per_core; + +MpiplPreservedState mpipl_state; };PnvChip *pnv_get_chip (PnvMachineState *pnv,uint32_t chip_id);
@@ -290,4 +293,7 @@ void pnv_bmc_set_pnor(IPMIBmc *bmc, PnvPnor *pnor);#define PNV11_OCC_SENSOR_BASE (chip) PNV10_OCC_SENSOR_BASE(chip) +/* MPIPL helpers */ +void do_mpipl_preserve (PnvMachineState *pnv); +#endif /* PPC_PNV_H */
include/hw/ppc/pnv_mpipl.h new file +19 -0
@@ -0,0 +1,19 @@ +/* +* Emulation of MPIPL (Memory Preserving Initial Program Load), aka fadump +* +* SPDX-License-Identifier: GPL-2.0-or-later +*/ + +#ifndef PNV_MPIPL_H +#define PNV_MPIPL_H + +#include "qemu/osdep.h" + +typedef struct MpiplPreservedState MpiplPreservedState ; + +/* Preserved state to be saved in PnvMachineState */ +struct MpiplPreservedState { +bool is_next_boot_mpipl; +}; + +#endif
Signature · 1 line
2.52.0
Re: [PATCH v2 2/9] hw/ppc: Implement S0 SBE interrupt as cpu_pause then host reset
Quoted 2 lines · Aditya Gupta
During MPIPL (aka fadump), OPAL triggers the S0 SBE interrupt to trigger MPIPL.
As mentioned in cover letter, add more clarity by mentioning that kernel initiates fadump with OPAL call. Skiboot then triggers S0 SBE interrupt...
- Hari
Re: [PATCH v2 2/9] hw/ppc: Implement S0 SBE interrupt as cpu_pause then host reset
Quoted 9 lines · Hari Bathini
Quoted 2 lines · Aditya Gupta
During MPIPL (aka fadump), OPAL triggers the S0 SBE interrupt to trigger MPIPL.
As mentioned in cover letter, add more clarity by mentioning that kernel initiates fadump with OPAL call. Skiboot then triggers S0 SBE interrupt...
Will add here also. Thanks Hari !
- Aditya G
Quoted 2 lines
- Hari
Re: [PATCH v2 2/9] hw/ppc: Implement S0 SBE interrupt as cpu_pause then host reset
Quoted 4 lines · Aditya Gupta
Quoted 3 lines · Hari Bathini
On 06/12/25 11:26 am, Aditya Gupta wrote:
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines
During MPIPL (aka fadump), OPAL triggers the S0 SBE interrupt to trigger MPIPL.
As mentioned in cover letter, add more clarity by mentioning that kernel initiates fadump with OPAL call. Skiboot then triggers S0 SBE interrupt...
Will add here also. Thanks Hari !
Does this look okay Hari ?
During MPIPL (aka fadump), after a kernel crash, the kernel does opal_cec_reboot2 opal call, signifying an abnormal termination. When OPAL recieves this opal call, it further triggers SBE S0 interrupt, to trigger a MPIPL boot.
Currently S0 interrupt is unimplemented in QEMU.
Re: [PATCH v2 2/9] hw/ppc: Implement S0 SBE interrupt as cpu_pause then host reset
Quoted 14 lines · Aditya Gupta
Quoted 4 lines · Aditya Gupta
Quoted 3 lines · Hari Bathini
On 06/12/25 11:26 am, Aditya Gupta wrote:
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines
During MPIPL (aka fadump), OPAL triggers the S0 SBE interrupt to trigger MPIPL.
As mentioned in cover letter, add more clarity by mentioning that kernel initiates fadump with OPAL call. Skiboot then triggers S0 SBE interrupt...
Will add here also. Thanks Hari !
Quoted 9 lines
Does this look okay Hari ?
During MPIPL (aka fadump), after a kernel crash, the kernel does
opal_cec_reboot2opal call, signifying an abnormal termination. When OPAL recieves this opal call, it further triggers SBE S0 interrupt, to trigger a MPIPL boot.Currently S0 interrupt is unimplemented in QEMU.
Yeah. That offers better insight.
- Hari
PATCH v2 3/9 hw/ppc: Handle stash command in PowerNV SBE
In reply to Aditya GuptaEarlier since the SBE_CMD_STASH_MPIPL_CONFIG command was not handled, so skiboot used to not get any response from SBE:
[ 106.350742821,3] SBE: Message timeout [chip id = 0], cmd = d7, subcmd = 7
[ 106.352067746,3] SBE: Failed to send stash MPIPL config [chip id = 0x0, rc = 254]Fix this by handling the command in PowerNV SBE, and sending a response so skiboot knows SBE has handled the STASH command
The stashed skiboot base is later used to access the relocated MDST/MDDT tables when MPIPL is implemented.
The purpose of stashing relocated base address is explained in following skiboot commit:
author Vasant Hegde <[email protected]> Fri Jul 12 16:47:51 2019 +0530 committer Oliver O'Halloran <[email protected]> Thu Aug 15 17:53:39 2019 +1000
SBE: Send OPAL relocated base address to SBE
OPAL relocates itself during boot. During memory preserving IPL hostboot needs to access relocated OPAL base address to get MDST, MDDT tables. Hence send relocated base address to SBE via 'stash MPIPL config' chip-op. During next IPL SBE will send stashed data to hostboot... so that hostboot can access these data.
--- hw/ppc/pnv_sbe.c | 27 +++++++++++++++++++++++++++ include/hw/ppc/pnv_mpipl.h | 3 +++ 2 files changed, 30 insertions(+)
hw/ppc/pnv_sbe.c +27 -0
@@ -233,6 +233,7 @@ static void sbe_timer(void *opaque)static void do_sbe_msg (PnvSBE *sbe) { +PnvMachineState *pnv =PNV_MACHINE (qdev_get_machine ());struct sbe_msg msg;uint16_t cmd, ctrl_flags, seq_id;int i;
@@ -265,6 +266,32 @@ static void do_sbe_msg(PnvSBE *sbe)timer_del (sbe->timer); }break ; +case SBE_CMD_STASH_MPIPL_CONFIG : +/* key = sbe->mbox[1] */ +switch (sbe->mbox[1 ]) { +case SBE_STASH_KEY_SKIBOOT_BASE : + pnv->mpipl_state.skiboot_base = sbe->mbox[2 ]; +qemu_log_mask (LOG_UNIMP , +"Stashing skiboot base: 0x%" HWADDR_PRIx"\n" , + pnv->mpipl_state.skiboot_base); + +/* +* Set the response register. +* +* Currently setting the same sequence number in +* response as we got in the request. +*/ + sbe->mbox[4 ] = sbe->mbox[0 ];/* sequence number */ +pnv_sbe_set_host_doorbell (sbe, + sbe->host_doorbell |SBE_HOST_RESPONSE_WAITING ); + +break ; +default : +qemu_log_mask (LOG_UNIMP , +"SBE: CMD_STASH_MPIPL_CONFIG: Unimplemented key: 0x" TARGET_FMT_lx"\n" , + sbe->mbox[1 ]); + } +break ;default :qemu_log_mask (LOG_UNIMP ,"SBE Unimplemented command: 0x%x\n" , cmd); }
include/hw/ppc/pnv_mpipl.h +3 -0
@@ -8,11 +8,14 @@#define PNV_MPIPL_H #include "qemu/osdep.h" +#include "exec/hwaddr.h" typedef struct MpiplPreservedState MpiplPreservedState ;/* Preserved state to be saved in PnvMachineState */ struct MpiplPreservedState { +/* skiboot_base will be valid only after OPAL sends relocated base to SBE */ +hwaddr skiboot_base;bool is_next_boot_mpipl; };
Signature · 1 line
2.52.0
PATCH v2 4/9 pnv/mpipl: Preserve memory regions as per MDST/MDDT tables
In reply to Aditya GuptaImplement copying of memory region, as mentioned by MDST and MDDT tables.
Copy the memory regions from source to destination in chunks of 32MB
Note, qemu can fail preserving a particular entry due to any reason, such as:
* region length mis-matching in MDST & MDDT
* failed copy due to access/decode/etc memory issues
HDAT doesn't specify any field in MDRT to notify host about such errors.
Though HDAT section "15.3.1.3 Memory Dump Results Table (MDRT)" says:
The Memory Dump Results Table is a list of the memory ranges that have been included in the dump
Based on above statement, it looks like MDRT should include only those regions which are successfully captured in the dump, hence, regions which qemu fails to dump, just get skipped, and will not have a corresponding entry in MDRT
--- hw/ppc/pnv_mpipl.c | 157 +++++++++++++++++++++++++++++++++++++ include/hw/ppc/pnv_mpipl.h | 83 ++++++++++++++++++++ 2 files changed, 240 insertions(+)
hw/ppc/pnv_mpipl.c +157 -0
@@ -5,12 +5,169 @@*/ #include "qemu/osdep.h" +#include "qemu/log.h" +#include "qemu/units.h" +#include "system/address-spaces.h" #include "system/runstate.h" #include "hw/ppc/pnv.h" #include "hw/ppc/pnv_mpipl.h" +#include <math.h> + +#define MDST_TABLE_RELOCATED \ + (pnv->mpipl_state.skiboot_base + MDST_TABLE_OFF) +#define MDDT_TABLE_RELOCATED \ + (pnv->mpipl_state.skiboot_base + MDDT_TABLE_OFF) + +/* +* Preserve the memory regions as pointed by MDST table +* +* During this, the memory region pointed by entries in MDST, are 'copied' +* as it is to the memory region pointed by corresponding entry in MDDT +* +* Notes: All reads should consider data coming from skiboot as bigendian, +* and data written should also be in big-endian +*/ +static bool pnv_mpipl_preserve_mem (PnvMachineState *pnv) +{ + g_autofree MdstTableEntry *mdst = g_malloc(MDST_TABLE_SIZE); + g_autofree MddtTableEntry *mddt = g_malloc(MDDT_TABLE_SIZE); + g_autofree MdrtTableEntry *mdrt = g_malloc0(MDRT_TABLE_SIZE); +AddressSpace *default_as = &address_space_memory; +MemTxResult io_result; +MemTxAttrs attrs; +uint64_t src_addr, dest_addr; +uint32_t src_len; +uint64_t num_chunks; +int mdrt_idx =0 ; + +/* Mark the memory transactions as privileged memory access */ + attrs.user =0 ; + attrs.memory =1 ; + +if (pnv->mpipl_state.mdrt_table) { +/* +* MDRT table allocated from some past crash, free the memory to +* prevent memory leak +*/ +g_free (pnv->mpipl_state.mdrt_table); + pnv->mpipl_state.num_mdrt_entries =0 ; + } + + io_result =address_space_read (default_as,MDST_TABLE_RELOCATED , attrs, + mdst,MDST_TABLE_SIZE ); +if (io_result !=MEMTX_OK ) { +qemu_log_mask (LOG_GUEST_ERROR , +"MPIPL: Failed to read MDST table at: 0x%lx\n" , +MDST_TABLE_RELOCATED ); + +return false; + } + + io_result =address_space_read (default_as,MDDT_TABLE_RELOCATED , attrs, + mddt,MDDT_TABLE_SIZE ); +if (io_result !=MEMTX_OK ) { +qemu_log_mask (LOG_GUEST_ERROR , +"MPIPL: Failed to read MDDT table at: 0x%lx\n" , +MDDT_TABLE_RELOCATED ); + +return false; + } + +/* Try to read all entries */ +for (int i =0 ; i <MDST_MAX_ENTRIES ; ++i) { + g_autofree uint8_t *copy_buffer = NULL; +bool is_copy_failed = false; + +/* Considering entry with address and size as 0, as end of table */ +if ((mdst[i].addr ==0 ) && (mdst[i].size ==0 )) { +break ; + } + +if (mdst[i].size != mddt[i].size) { +qemu_log_mask (LOG_TRACE , +"Warning: Invalid entry, size mismatch in MDST & MDDT\n" ); +continue ; + } + +if (mdst[i].data_region != mddt[i].data_region) { +qemu_log_mask (LOG_TRACE , +"Warning: Invalid entry, region mismatch in MDST & MDDT\n" ); +continue ; + } + + src_addr =be64_to_cpu (mdst[i].addr) & ~HRMOR_BIT ; + dest_addr =be64_to_cpu (mddt[i].addr) & ~HRMOR_BIT ; + src_len =be32_to_cpu (mddt[i].size); + +#define COPY_CHUNK_SIZE ((size_t)(32 * MiB)) + is_copy_failed = false; + copy_buffer =g_try_malloc (COPY_CHUNK_SIZE ); +if (copy_buffer ==NULL ) { +qemu_log_mask (LOG_GUEST_ERROR , +"MPIPL: Failed allocating memory (size: %zu) for copying" +" reserved memory regions\n" ,COPY_CHUNK_SIZE ); + } + + num_chunks =ceil ((src_len *1.0f ) /COPY_CHUNK_SIZE ); +for (uint64_t chunk_id =0 ; chunk_id < num_chunks; ++chunk_id) { +/* Take minimum of bytes left to copy, and chunk size */ +uint64_t copy_len =MIN ( + src_len - (chunk_id *COPY_CHUNK_SIZE ), +COPY_CHUNK_SIZE + ); + +/* Copy the source region to destination */ + io_result =address_space_read (default_as, src_addr, attrs, + copy_buffer, copy_len); +if (io_result !=MEMTX_OK ) { +qemu_log_mask (LOG_GUEST_ERROR , +"MPIPL: Failed to read region at: 0x%lx\n" , src_addr); + is_copy_failed = true; +break ; + } + + io_result =address_space_write (default_as, dest_addr, attrs, + copy_buffer, copy_len); +if (io_result !=MEMTX_OK ) { +qemu_log_mask (LOG_GUEST_ERROR , +"MPIPL: Failed to write region at: 0x%lx\n" , dest_addr); + is_copy_failed = true; +break ; + } + + src_addr +=COPY_CHUNK_SIZE ; + dest_addr +=COPY_CHUNK_SIZE ; + } +#undef COPY_CHUNK_SIZE + +if (is_copy_failed) { +/* +* HDAT doesn't specify an error code in MDRT for failed copy, +* and doesn't specify how this is to be handled +* Hence just skip adding an entry in MDRT, as done for size +* mismatch or other inconsistency between MDST/MDDT +*/ +continue ; + } + +/* Populate entry in MDRT table if preserving successful */ + mdrt[mdrt_idx].src_addr =cpu_to_be64 (src_addr); + mdrt[mdrt_idx].dest_addr =cpu_to_be64 (dest_addr); + mdrt[mdrt_idx].size =cpu_to_be32 (src_len); + mdrt[mdrt_idx].data_region = mdst[i].data_region; + ++mdrt_idx; + } + + pnv->mpipl_state.mdrt_table =g_steal_pointer (&mdrt); + pnv->mpipl_state.num_mdrt_entries = mdrt_idx; + +return true; +}void do_mpipl_preserve (PnvMachineState *pnv) { +pnv_mpipl_preserve_mem (pnv); +/* Mark next boot as Memory-preserving boot */ pnv->mpipl_state.is_next_boot_mpipl = true;
include/hw/ppc/pnv_mpipl.h +83 -0
96 lines not on this page · raw message
Signature · 1 line
2.52.0
Re: [PATCH v2 4/9] pnv/mpipl: Preserve memory regions as per MDST/MDDT tables
Quoted 13 lines · Aditya Gupta
Implement copying of memory region, as mentioned by MDST and MDDT tables.
Copy the memory regions from source to destination in chunks of 32MB
Note, qemu can fail preserving a particular entry due to any reason, such as:
* region length mis-matching in MDST & MDDT
* failed copy due to access/decode/etc memory issuesHDAT doesn't specify any field in MDRT to notify host about such errors.
Quoted 8 lines
Though HDAT section "15.3.1.3 Memory Dump Results Table (MDRT)" says:
The Memory Dump Results Table is a list of the memory ranges that have been included in the dumpBased on above statement, it looks like MDRT should include only those regions which are successfully captured in the dump, hence, regions which qemu fails to dump, just get skipped, and will not have a corresponding entry in MDRT
Yeah. No error status is looked for by kernel. It tries to export /proc/vmcore with how much ever firmware succeeds in preserving..
Quoted 99 lines
Signed-off-by Aditya Gupta <[email protected]>--- hw/ppc/pnv_mpipl.c | 157 +++++++++++++++++++++++++++++++++++++ include/hw/ppc/pnv_mpipl.h | 83 ++++++++++++++++++++ 2 files changed, 240 insertions(+)hw/ppc/pnv_mpipl.c +81
-0@@ -5,12 +5,169 @@*/ #include "qemu/osdep.h" +#include "qemu/log.h" +#include "qemu/units.h" +#include "system/address-spaces.h" #include "system/runstate.h" #include "hw/ppc/pnv.h" #include "hw/ppc/pnv_mpipl.h" +#include <math.h> + +#define MDST_TABLE_RELOCATED \ + (pnv->mpipl_state.skiboot_base + MDST_TABLE_OFF) +#define MDDT_TABLE_RELOCATED \ + (pnv->mpipl_state.skiboot_base + MDDT_TABLE_OFF) + +/* +* Preserve the memory regions as pointed by MDST table +* +* During this, the memory region pointed by entries in MDST, are 'copied' +* as it is to the memory region pointed by corresponding entry in MDDT +* +* Notes: All reads should consider data coming from skiboot as bigendian, +* and data written should also be in big-endian +*/ +static bool pnv_mpipl_preserve_mem (PnvMachineState *pnv) +{ + g_autofree MdstTableEntry *mdst = g_malloc(MDST_TABLE_SIZE); + g_autofree MddtTableEntry *mddt = g_malloc(MDDT_TABLE_SIZE); + g_autofree MdrtTableEntry *mdrt = g_malloc0(MDRT_TABLE_SIZE); +AddressSpace *default_as = &address_space_memory; +MemTxResult io_result; +MemTxAttrs attrs; +uint64_t src_addr, dest_addr; +uint32_t src_len; +uint64_t num_chunks; +int mdrt_idx =0 ; + +/* Mark the memory transactions as privileged memory access */ + attrs.user =0 ; + attrs.memory =1 ; + +if (pnv->mpipl_state.mdrt_table) { +/* +* MDRT table allocated from some past crash, free the memory to +* prevent memory leak +*/ +g_free (pnv->mpipl_state.mdrt_table); + pnv->mpipl_state.num_mdrt_entries =0 ; + } + + io_result =address_space_read (default_as,MDST_TABLE_RELOCATED , attrs, + mdst,MDST_TABLE_SIZE ); +if (io_result !=MEMTX_OK ) { +qemu_log_mask (LOG_GUEST_ERROR , +"MPIPL: Failed to read MDST table at: 0x%lx\n" , +MDST_TABLE_RELOCATED ); + +return false; + } + + io_result =address_space_read (default_as,MDDT_TABLE_RELOCATED , attrs, + mddt,MDDT_TABLE_SIZE ); +if (io_result !=MEMTX_OK ) { +qemu_log_mask (LOG_GUEST_ERROR , +"MPIPL: Failed to read MDDT table at: 0x%lx\n" , +MDDT_TABLE_RELOCATED ); + +return false; + } + +/* Try to read all entries */ +for (int i =0 ; i <MDST_MAX_ENTRIES ; ++i) { + g_autofree uint8_t *copy_buffer = NULL; +bool is_copy_failed = false; + +/* Considering entry with address and size as 0, as end of table */ +if ((mdst[i].addr ==0 ) && (mdst[i].size ==0 )) { +break ; + } + +if (mdst[i].size != mddt[i].size) { +qemu_log_mask (LOG_TRACE , +"Warning: Invalid entry, size mismatch in MDST & MDDT\n" ); +continue ; + } +
hw/ppc/pnv_mpipl.c line 101
Quoted 10 lines
hw/ppc/pnv_mpipl.c +9
-0+if (mdst[i].data_region != mddt[i].data_region) { +qemu_log_mask (LOG_TRACE , +"Warning: Invalid entry, region mismatch in MDST & MDDT\n" ); +continue ; + } + + src_addr =be64_to_cpu (mdst[i].addr) & ~HRMOR_BIT ; + dest_addr =be64_to_cpu (mddt[i].addr) & ~HRMOR_BIT ; + src_len =be32_to_cpu (mddt[i].size);
data_len sounds more appropriate instead of src_len here, especially as we are using mddt length..
- Hari
Re: [PATCH v2 4/9] pnv/mpipl: Preserve memory regions as per MDST/MDDT tables
Quoted 9 lines · Hari Bathini
Quoted 6 lines · Aditya Gupta
<...snip...>
Though HDAT section "15.3.1.3 Memory Dump Results Table (MDRT)" says:
The Memory Dump Results Table is a list of the memory ranges that have been included in the dump
Quoted 7 lines
Quoted 4 lines
Based on above statement, it looks like MDRT should include only those regions which are successfully captured in the dump, hence, regions which qemu fails to dump, just get skipped, and will not have a corresponding entry in MDRT
Yeah. No error status is looked for by kernel. It tries to export
/proc/vmcorewith how much ever firmware succeeds in preserving..
Thanks for confirming. We are handling it correct way in qemu then.
Quoted 10 lines
hw/ppc/pnv_mpipl.c line 100
Quoted 7 lines
<...snip...>
hw/ppc/pnv_mpipl.c +3
-0+ src_addr =be64_to_cpu (mdst[i].addr) & ~HRMOR_BIT ; + dest_addr =be64_to_cpu (mddt[i].addr) & ~HRMOR_BIT ; + src_len =be32_to_cpu (mddt[i].size);
data_lensounds more appropriate instead ofsrc_lenhere, especially as we are using mddt length..
Makes sense, Will do it.
Thanks,
- Aditya G
Quoted 2 lines
- Hari
PATCH v2 5/9 pnv/mpipl: Preserve CPU registers after crash
In reply to Aditya GuptaKernel expects the platform to provide CPU registers after pausing execution of the CPUs.
Currently only exporting the registers, used by Linux, for generating the /proc/vmcore
--- hw/ppc/pnv_mpipl.c | 102 +++++++++++++++++++++++++++++++++++++ include/hw/ppc/pnv_mpipl.h | 62 ++++++++++++++++++++++ 2 files changed, 164 insertions(+)
hw/ppc/pnv_mpipl.c +102 -0
@@ -8,6 +8,8 @@#include "qemu/log.h" #include "qemu/units.h" #include "system/address-spaces.h" +#include "system/cpus.h" +#include "system/hw_accel.h" #include "system/runstate.h" #include "hw/ppc/pnv.h" #include "hw/ppc/pnv_mpipl.h"
@@ -17,6 +19,8 @@ (pnv->mpipl_state.skiboot_base + MDST_TABLE_OFF)#define MDDT_TABLE_RELOCATED \ (pnv->mpipl_state.skiboot_base + MDDT_TABLE_OFF) +#define PROC_DUMP_RELOCATED \ + (pnv->mpipl_state.skiboot_base + PROC_DUMP_AREA_OFF)/* * Preserve the memory regions as pointed by MDST table
@@ -164,9 +168,107 @@ static bool pnv_mpipl_preserve_mem(PnvMachineState *pnv)return true; } +static void do_store_cpu_regs (CPUState *cpu,MpiplPreservedCPUState *state) +{ +CPUPPCState *env =cpu_env (cpu); +MpiplRegDataHdr *regs_hdr = &state->hdr; +MpiplRegEntry *reg_entries = state->reg_entries; +MpiplRegEntry *curr_reg_entry; +uint32_t num_saved_regs =0 ; + +cpu_synchronize_state (cpu); + + regs_hdr->pir =cpu_to_be32 (env->spr[SPR_PIR ]); + +/* QEMU CPUs are not in Power Saving Mode */ + regs_hdr->core_state =0xff ; + + regs_hdr->off_regentries =0 ; + regs_hdr->num_regentries =cpu_to_be32 (NUM_REGS_PER_CPU ); + + regs_hdr->alloc_size =cpu_to_be32 (sizeof (MpiplRegEntry)); + regs_hdr->act_size =cpu_to_be32 (sizeof (MpiplRegEntry)); + +#define REG_TYPE_GPR 0x1 +#define REG_TYPE_SPR 0x2 +#define REG_TYPE_TIMA 0x3 + +/* +* ID numbers used by f/w while populating certain registers +* +* Copied these defines from the linux kernel +*/ +#define REG_ID_NIP 0x7D0 +#define REG_ID_MSR 0x7D1 +#define REG_ID_CCR 0x7D2 + + curr_reg_entry = reg_entries; + +#define REG_ENTRY (type, num, val) \ + do { \ + curr_reg_entry->reg_type = cpu_to_be32(type); \ + curr_reg_entry->reg_num = cpu_to_be32(num); \ + curr_reg_entry->reg_val = cpu_to_be64(val); \ + ++curr_reg_entry; \ + ++num_saved_regs; \ + } while (0) + +/* Save the GPRs */ +for (int gpr_id =0 ; gpr_id <32 ; ++gpr_id) { +REG_ENTRY (REG_TYPE_GPR , gpr_id, env->gpr[gpr_id]); + } + +REG_ENTRY (REG_TYPE_SPR ,REG_ID_NIP , env->nip); +REG_ENTRY (REG_TYPE_SPR ,REG_ID_MSR , env->msr); + +/* +* Ensure the number of registers saved match the number of +* registers per cpu +* +* This will help catch an error if in future a new register entry +* is added/removed while not modifying NUM_PER_CPU_REGS +*/ +assert (num_saved_regs ==NUM_REGS_PER_CPU ); +} + +static void pnv_mpipl_preserve_cpu_state (PnvMachineState *pnv) +{ +MachineState *machine =MACHINE (pnv); +uint32_t num_cpus = machine->smp.cpus; +MpiplPreservedCPUState *state; +CPUState *cpu; + +if (pnv->mpipl_state.cpu_states) { +/* +* CPU States might have been allocated from some past crash, free the +* memory to preven memory leak +*/ +g_free (pnv->mpipl_state.cpu_states); + pnv->mpipl_state.num_cpu_states =0 ; + } + + pnv->mpipl_state.cpu_states =g_malloc_n (num_cpus, +sizeof (MpiplPreservedCPUState)); + pnv->mpipl_state.num_cpu_states = num_cpus; + + state = pnv->mpipl_state.cpu_states; + +/* Preserve the Processor Dump Area */ +cpu_physical_memory_read (PROC_DUMP_RELOCATED , &pnv->mpipl_state.proc_area, +sizeof (MpiplProcDumpArea)); + +CPU_FOREACH (cpu) { +do_store_cpu_regs (cpu, state); + ++state; + } +} +void do_mpipl_preserve (PnvMachineState *pnv) { +pause_all_vcpus (); +pnv_mpipl_preserve_mem (pnv); +pnv_mpipl_preserve_cpu_state (pnv);/* Mark next boot as Memory-preserving boot */ pnv->mpipl_state.is_next_boot_mpipl = true;
include/hw/ppc/pnv_mpipl.h +62 -0
86 lines not on this page · raw message
Signature · 1 line
2.52.0
Re: [PATCH v2 5/9] pnv/mpipl: Preserve CPU registers after crash
Quoted 160 lines · Aditya Gupta
Kernel expects the platform to provide CPU registers after pausing execution of the CPUs.
Currently only exporting the registers, used by Linux, for generating the
/proc/vmcoreSigned-off-by Aditya Gupta <[email protected]>--- hw/ppc/pnv_mpipl.c | 102 +++++++++++++++++++++++++++++++++++++ include/hw/ppc/pnv_mpipl.h | 62 ++++++++++++++++++++++ 2 files changed, 164 insertions(+)hw/ppc/pnv_mpipl.c +102
-0@@ -8,6 +8,8 @@#include "qemu/log.h" #include "qemu/units.h" #include "system/address-spaces.h" +#include "system/cpus.h" +#include "system/hw_accel.h" #include "system/runstate.h" #include "hw/ppc/pnv.h" #include "hw/ppc/pnv_mpipl.h" @@ -17,6 +19,8 @@ (pnv->mpipl_state.skiboot_base + MDST_TABLE_OFF)#define MDDT_TABLE_RELOCATED \ (pnv->mpipl_state.skiboot_base + MDDT_TABLE_OFF) +#define PROC_DUMP_RELOCATED \ + (pnv->mpipl_state.skiboot_base + PROC_DUMP_AREA_OFF)/* * Preserve the memory regions as pointed by MDST table @@ -164,9 +168,107 @@ static bool pnv_mpipl_preserve_mem(PnvMachineState *pnv)return true; } +static void do_store_cpu_regs (CPUState *cpu,MpiplPreservedCPUState *state) +{ +CPUPPCState *env =cpu_env (cpu); +MpiplRegDataHdr *regs_hdr = &state->hdr; +MpiplRegEntry *reg_entries = state->reg_entries; +MpiplRegEntry *curr_reg_entry; +uint32_t num_saved_regs =0 ; + +cpu_synchronize_state (cpu); + + regs_hdr->pir =cpu_to_be32 (env->spr[SPR_PIR ]); + +/* QEMU CPUs are not in Power Saving Mode */ + regs_hdr->core_state =0xff ; + + regs_hdr->off_regentries =0 ; + regs_hdr->num_regentries =cpu_to_be32 (NUM_REGS_PER_CPU ); + + regs_hdr->alloc_size =cpu_to_be32 (sizeof (MpiplRegEntry)); + regs_hdr->act_size =cpu_to_be32 (sizeof (MpiplRegEntry)); + +#define REG_TYPE_GPR 0x1 +#define REG_TYPE_SPR 0x2 +#define REG_TYPE_TIMA 0x3 + +/* +* ID numbers used by f/w while populating certain registers +* +* Copied these defines from the linux kernel +*/ +#define REG_ID_NIP 0x7D0 +#define REG_ID_MSR 0x7D1 +#define REG_ID_CCR 0x7D2 + + curr_reg_entry = reg_entries; + +#define REG_ENTRY (type, num, val) \ + do { \ + curr_reg_entry->reg_type = cpu_to_be32(type); \ + curr_reg_entry->reg_num = cpu_to_be32(num); \ + curr_reg_entry->reg_val = cpu_to_be64(val); \ + ++curr_reg_entry; \ + ++num_saved_regs; \ + } while (0) + +/* Save the GPRs */ +for (int gpr_id =0 ; gpr_id <32 ; ++gpr_id) { +REG_ENTRY (REG_TYPE_GPR , gpr_id, env->gpr[gpr_id]); + } + +REG_ENTRY (REG_TYPE_SPR ,REG_ID_NIP , env->nip); +REG_ENTRY (REG_TYPE_SPR ,REG_ID_MSR , env->msr); + +/* +* Ensure the number of registers saved match the number of +* registers per cpu +* +* This will help catch an error if in future a new register entry +* is added/removed while not modifying NUM_PER_CPU_REGS +*/ +assert (num_saved_regs ==NUM_REGS_PER_CPU ); +} + +static void pnv_mpipl_preserve_cpu_state (PnvMachineState *pnv) +{ +MachineState *machine =MACHINE (pnv); +uint32_t num_cpus = machine->smp.cpus; +MpiplPreservedCPUState *state; +CPUState *cpu; + +if (pnv->mpipl_state.cpu_states) { +/* +* CPU States might have been allocated from some past crash, free the +* memory to preven memory leak +*/ +g_free (pnv->mpipl_state.cpu_states); + pnv->mpipl_state.num_cpu_states =0 ; + } + + pnv->mpipl_state.cpu_states =g_malloc_n (num_cpus, +sizeof (MpiplPreservedCPUState)); + pnv->mpipl_state.num_cpu_states = num_cpus; + + state = pnv->mpipl_state.cpu_states; + +/* Preserve the Processor Dump Area */ +cpu_physical_memory_read (PROC_DUMP_RELOCATED , &pnv->mpipl_state.proc_area, +sizeof (MpiplProcDumpArea)); + +CPU_FOREACH (cpu) { +do_store_cpu_regs (cpu, state); + ++state; + } +} +void do_mpipl_preserve (PnvMachineState *pnv) { +pause_all_vcpus (); +pnv_mpipl_preserve_mem (pnv); +pnv_mpipl_preserve_cpu_state (pnv);/* Mark next boot as Memory-preserving boot */ pnv->mpipl_state.is_next_boot_mpipl = true;include/hw/ppc/pnv_mpipl.h +6
-0@@ -16,6 +16,12 @@ typedef struct MdstTableEntry MdstTableEntry;typedef struct MdrtTableEntry MdrtTableEntry ;typedef struct MpiplPreservedState MpiplPreservedState ; +typedef struct MpiplRegDataHdr MpiplRegDataHdr ; +typedef struct MpiplRegEntry MpiplRegEntry ; +typedef struct MpiplProcDumpArea MpiplProcDumpArea ; +typedef struct MpiplPreservedState MpiplPreservedState ; +typedef struct MpiplPreservedCPUState MpiplPreservedCPUState ; +/* Following offsets are copied from skiboot source code */ /* Use 768 bytes for SPIRAH */ #define SPIRAH_OFF 0x00010000
include/hw/ppc/pnv_mpipl.h typedef struct MpiplPreservedState MpiplPreservedState; · line 57
Quoted 6 lines
include/hw/ppc/pnv_mpipl.h +2
-0@@ -46,6 +52,8 @@ typedef struct MpiplPreservedState MpiplPreservedState;#define __packed __attribute__((packed)) +#define NUM_REGS_PER_CPU 34/*(32 GPRs, NIP, MSR)*/ +
Any limitation with saving the other SPRs? At least, LR is one other relevant SPR that needs to be saved for some meaningful context..
- Hari
Re: [PATCH v2 5/9] pnv/mpipl: Preserve CPU registers after crash
Quoted 153 lines · Hari Bathini
Quoted 149 lines · Aditya Gupta
Kernel expects the platform to provide CPU registers after pausing execution of the CPUs.
Currently only exporting the registers, used by Linux, for generating the
/proc/vmcoreSigned-off-by Aditya Gupta <[email protected]>---
hw/ppc/pnv_mpipl.c | 102 +++++++++++++++++++++++++++++++++++++include/hw/ppc/pnv_mpipl.h| 62 ++++++++++++++++++++++2 files changed, 164 insertions(+)hw/ppc/pnv_mpipl.c +77
-0@@ -8,6 +8,8 @@ #include "qemu/log.h" #include "qemu/units.h" #include "system/address-spaces.h" +#include "system/cpus.h" +#include "system/hw_accel.h" #include "system/runstate.h" #include "hw/ppc/pnv.h" #include "hw/ppc/pnv_mpipl.h"@@ -17,6 +19,8 @@ (pnv->mpipl_state.skiboot_base + MDST_TABLE_OFF) #define MDDT_TABLE_RELOCATED \ (pnv->mpipl_state.skiboot_base + MDDT_TABLE_OFF) +#define PROC_DUMP_RELOCATED \ + (pnv->mpipl_state.skiboot_base + PROC_DUMP_AREA_OFF) /** Preserve the memory regions as pointed by MDST table @@ -164,9 +168,107 @@ static bool pnv_mpipl_preserve_mem(PnvMachineState *pnv) return true; } +static void do_store_cpu_regs (CPUState *cpu,MpiplPreservedCPUState *state) +{ + CPUPPCState *env = cpu_env(cpu); + MpiplRegDataHdr *regs_hdr = &state->hdr; + MpiplRegEntry *reg_entries = state->reg_entries; + MpiplRegEntry *curr_reg_entry; + uint32_t num_saved_regs = 0; + + cpu_synchronize_state(cpu); + + regs_hdr->pir = cpu_to_be32(env->spr[SPR_PIR]); + + /* QEMU CPUs are not in Power Saving Mode */ + regs_hdr->core_state = 0xff; + + regs_hdr->off_regentries = 0; + regs_hdr->num_regentries = cpu_to_be32(NUM_REGS_PER_CPU); + + regs_hdr->alloc_size = cpu_to_be32(sizeof(MpiplRegEntry)); + regs_hdr->act_size = cpu_to_be32(sizeof(MpiplRegEntry)); + +#define REG_TYPE_GPR 0x1 +#define REG_TYPE_SPR 0x2 +#define REG_TYPE_TIMA 0x3 + +/* +* ID numbers used by f/w while populating certain registers +* +* Copied these defines from the linux kernel +*/ +#define REG_ID_NIP 0x7D0 +#define REG_ID_MSR 0x7D1 +#define REG_ID_CCR 0x7D2 + + curr_reg_entry = reg_entries; + +#define REG_ENTRY (type, num, val) \ + do { \ + curr_reg_entry->reg_type = cpu_to_be32(type); \ + curr_reg_entry->reg_num = cpu_to_be32(num); \ + curr_reg_entry->reg_val = cpu_to_be64(val); \ + ++curr_reg_entry; \ + ++num_saved_regs; \ + } while (0) + + /* Save the GPRs */ + for (int gpr_id = 0; gpr_id < 32; ++gpr_id) { + REG_ENTRY(REG_TYPE_GPR, gpr_id, env->gpr[gpr_id]); + } + + REG_ENTRY(REG_TYPE_SPR, REG_ID_NIP, env->nip); + REG_ENTRY(REG_TYPE_SPR, REG_ID_MSR, env->msr); + + /* + * Ensure the number of registers saved match the number of + * registers per cpu + * + * This will help catch an error if in future a new register entry + * is added/removed while not modifying NUM_PER_CPU_REGS + */ + assert(num_saved_regs == NUM_REGS_PER_CPU); +} + +static void pnv_mpipl_preserve_cpu_state (PnvMachineState *pnv) +{ + MachineState *machine = MACHINE(pnv); + uint32_t num_cpus = machine->smp.cpus; + MpiplPreservedCPUState *state; + CPUState *cpu; + + if (pnv->mpipl_state.cpu_states) { + /* +* CPU States might have been allocated from some past crash, free the
+ * memory to preven memory leak
+ */
+g_free(pnv->mpipl_state.cpu_states);
+pnv->mpipl_state.num_cpu_states= 0;
+ }hw/ppc/pnv_mpipl.c +9
-0+ + pnv->mpipl_state.cpu_states = g_malloc_n(num_cpus, + sizeof(MpiplPreservedCPUState)); + pnv->mpipl_state.num_cpu_states = num_cpus; + + state = pnv->mpipl_state.cpu_states; + + /* Preserve the Processor Dump Area */ + cpu_physical_memory_read(PROC_DUMP_RELOCATED, &pnv-Quoted 1 line
mpipl_state.proc_area,
+ sizeof(MpiplProcDumpArea));
hw/ppc/pnv_mpipl.c +7
-0+ + CPU_FOREACH(cpu) { + do_store_cpu_regs(cpu, state); + ++state; + } +} +void
do_mpipl_preserve(PnvMachineState *pnv){
+pause_all_vcpus();
+
pnv_mpipl_preserve_mem(pnv);
+pnv_mpipl_preserve_cpu_state(pnv);
/* Mark next boot as Memory-preserving boot */pnv->mpipl_state.is_next_boot_mpipl= true;include/hw/ppc/pnv_mpipl.h +0
-0@@ -16,6 +16,12 @@ typedef struct MdstTableEntry MdstTableEntry; typedef struct MdrtTableEntry MdrtTableEntry;
Quoted 7 lines
include/hw/ppc/pnv_mpipl.h typedef struct MdstTableEntry MdstTableEntry; · line 22
Quoted 7 lines
include/hw/ppc/pnv_mpipl.h +4
-0typedef struct MpiplPreservedState MpiplPreservedState; +typedef struct MpiplRegDataHdr MpiplRegDataHdr ; +typedef struct MpiplRegEntry MpiplRegEntry ; +typedef struct MpiplProcDumpArea MpiplProcDumpArea ; +typedef struct MpiplPreservedState MpiplPreservedState ;
btw, the above typedef is already available from a previous patch..
Quoted 8 lines
Quoted 7 lines
include/hw/ppc/pnv_mpipl.h +2
-0+typedef struct MpiplPreservedCPUState MpiplPreservedCPUState ; + /* Following offsets are copied from skiboot source code */ /* Use 768 bytes for SPIRAH */#define SPIRAH_OFF
0x00010000include/hw/ppc/pnv_mpipl.h +0
-0@@ -46,6 +52,8 @@ typedef struct MpiplPreservedState MpiplPreservedState; #define __packed __attribute__((packed))
Quoted 5 lines
include/hw/ppc/pnv_mpipl.h typedef struct MpiplPreservedState MpiplPreservedState; · line 56
Quoted 2 lines
include/hw/ppc/pnv_mpipl.h +2
-0+#define NUM_REGS_PER_CPU 34/*(32 GPRs, NIP, MSR)*/ +Any limitation with saving the other SPRs? At least, LR is one other relevant SPR that needs to be saved for some meaningful context..
- Hari
Re: [PATCH v2 5/9] pnv/mpipl: Preserve CPU registers after crash
Quoted 6 lines · Hari Bathini
Quoted 3 lines
<...snip...>
Quoted 1 line · Hari Bathini
Quoted 1 line
typedef struct MpiplPreservedState MpiplPreservedState;
Quoted 7 lines
Quoted 5 lines
include/hw/ppc/pnv_mpipl.h typedef struct MdstTableEntry MdstTableEntry; · line 22
Quoted 5 lines
include/hw/ppc/pnv_mpipl.h +4
-0+typedef struct MpiplRegDataHdr MpiplRegDataHdr ; +typedef struct MpiplRegEntry MpiplRegEntry ; +typedef struct MpiplProcDumpArea MpiplProcDumpArea ; +typedef struct MpiplPreservedState MpiplPreservedState ;btw, the above typedef is already available from a previous patch..
Will remove this duplicate.
Quoted 9 lines
Quoted 9 lines
Quoted 8 lines
include/hw/ppc/pnv_mpipl.h +2
-0+typedef struct MpiplPreservedCPUState MpiplPreservedCPUState ; +/* Following offsets are copied from skiboot source code */ /* Use 768 bytes for SPIRAH */
#define SPIRAH_OFF0x00010000include/hw/ppc/pnv_mpipl.h +0
-0@@ -46,6 +52,8 @@ typedef struct MpiplPreservedState MpiplPreservedState; #define __packed __attribute__((packed))
Quoted 5 lines
Quoted 5 lines
include/hw/ppc/pnv_mpipl.h typedef struct MpiplPreservedState MpiplPreservedState; · line 56
Quoted 2 lines
include/hw/ppc/pnv_mpipl.h +2
-0+#define NUM_REGS_PER_CPU 34/*(32 GPRs, NIP, MSR)*/ +Any limitation with saving the other SPRs? At least, LR is one other relevant SPR that needs to be saved for some meaningful context..
Few registers are not implemented in QEMU. But LR should be possible, will add more registers that can be added Hari.
Thanks,
- Aditya G
Quoted 2 lines
- Hari
PATCH v2 6/9 pnv/mpipl: Set thread entry size to be allocated by firmware
In reply to Aditya GuptaSet the "Thread Register State Entry Size" that is required by firmware (OPAL), to know size of memory to allocate to capture CPU state, in the event of a crash
--- hw/ppc/pnv.c | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+)
hw/ppc/pnv.c +24 -0
@@ -780,6 +780,30 @@ static void pnv_reset(MachineState *machine, ResetType type)_FDT ((fdt_pack (fdt))); } +if (!pnv->mpipl_state.is_next_boot_mpipl) { +/* +* Set the "Thread Register State Entry Size", so that firmware can +* allocate enough memory to capture CPU state in the event of a +* crash +*/ + +MpiplProcDumpArea proc_area; + + proc_area.version =PROC_DUMP_AREA_VERSION_P9 ; + proc_area.thread_size =cpu_to_be32 (sizeof (MpiplPreservedCPUState)); + +/* These are to be allocated & assigned by the firmware */ + proc_area.alloc_addr =0 ; + proc_area.alloc_size =0 ; + +/* These get assigned after crash, when QEMU preserves the registers */ + proc_area.dest_addr =0 ; + proc_area.act_size =0 ; + +cpu_physical_memory_write (PROC_DUMP_AREA_OFF , &proc_area, +sizeof (proc_area)); + } +cpu_physical_memory_write (PNV_FDT_ADDR , fdt,fdt_totalsize (fdt));/* Update machine->fdt with latest fdt */
Signature · 1 line
2.52.0
PATCH v2 7/9 pnv/mpipl: Write the preserved CPU and MDRT state
In reply to Aditya GuptaLogic for preserving the CPU registers and memory regions has been done in previous patches.
Write those data at the relevant memory address, such as PROC_DUMP_AREA for CPU registers, and MDRT for preserved memory regions.
Also export "mpipl-boot" device tree node, for kernel to know that it's a 'dump active' boot
--- hw/ppc/pnv.c | 31 ++++++++++++- hw/ppc/pnv_mpipl.c | 103 +++++++++++++++++++++++++++++++++++++++++++ include/hw/ppc/pnv.h | 1 + 3 files changed, 134 insertions(+), 1 deletion(-)
hw/ppc/pnv.c +30 -1
@@ -750,6 +750,7 @@ static void pnv_reset(MachineState *machine, ResetType type)PnvMachineState *pnv =PNV_MACHINE (machine);IPMIBmc *bmc;void *fdt; +int node_offset;qemu_devices_reset (type);
@@ -780,7 +781,35 @@ static void pnv_reset(MachineState *machine, ResetType type)_FDT ((fdt_pack (fdt))); } -if (!pnv->mpipl_state.is_next_boot_mpipl) { +/* +* If it's a MPIPL boot, add the "mpipl-boot" property, and reset the +* boolean for MPIPL boot for next boot +*/ +if (pnv->mpipl_state.is_next_boot_mpipl) { +void *fdt_copy =g_malloc0 (FDT_MAX_SIZE ); + +/* Write the preserved MDRT and CPU State Data */ +do_mpipl_write (pnv); + +/* Create a writable copy of the fdt */ +_FDT ((fdt_open_into (fdt, fdt_copy,FDT_MAX_SIZE ))); + + node_offset =fdt_path_offset (fdt_copy,"/ibm,opal/dump" ); +_FDT ((fdt_appendprop_u64 (fdt_copy, node_offset,"mpipl-boot" ,1 ))); + +/* Update the fdt, and free the original fdt */ +if (fdt != machine->fdt) { +/* +* Only free the fdt if it's not machine->fdt, to prevent +* double free, since we already free machine->fdt later +*/ +g_free (fdt); + } + fdt = fdt_copy; + +/* This boot is an MPIPL, reset the boolean for next boot */ + pnv->mpipl_state.is_next_boot_mpipl = false; + }else {/* * Set the "Thread Register State Entry Size", so that firmware can * allocate enough memory to capture CPU state in the event of a
hw/ppc/pnv_mpipl.c +103 -0
118 lines not on this page · raw message
include/hw/ppc/pnv.h +1 -0
@@ -295,5 +295,6 @@ void pnv_bmc_set_pnor(IPMIBmc *bmc, PnvPnor *pnor);/* MPIPL helpers */ void do_mpipl_preserve (PnvMachineState *pnv); +void do_mpipl_write (PnvMachineState *pnv);#endif /* PPC_PNV_H */
Signature · 1 line
2.52.0
PATCH v2 8/9 pnv/mpipl: Enable MPIPL support
In reply to Aditya GuptaWith all MPIPL support in place, export a "dump" node in device tree, signifying that PowerNV QEMU platform supports MPIPL
--- hw/ppc/pnv.c | 37 +++++++++++++++++++++++++++++++++++++ 1 file changed, 37 insertions(+)
hw/ppc/pnv.c +37 -0
@@ -54,6 +54,7 @@#include "hw/ppc/pnv_chip.h" #include "hw/ppc/pnv_xscom.h" #include "hw/ppc/pnv_pnor.h" +#include "hw/ppc/pnv_mpipl.h" #include "hw/isa/isa.h" #include "hw/char/serial-isa.h"
@@ -671,6 +672,39 @@ static void pnv_dt_power_mgt(PnvMachineState *pnv, void *fdt)_FDT (fdt_setprop_cell (fdt, off,"ibm,enabled-stop-levels" ,0xc0000000 )); } +static void pnv_dt_mpipl_dump (PnvMachineState *pnv,void *fdt) +{ +int off; + +/* +* Add "dump" node so kernel knows MPIPL (aka fadump) is supported +* +* Note: This is only needed to be done since we are passing device tree to +* opal +* +* In case HDAT is supported in future, then opal can add these nodes by +* itself based on system attribute having MPIPL_SUPPORTED bit set +*/ + off =fdt_add_subnode (fdt,0 ,"ibm,opal" ); +if (off == -FDT_ERR_EXISTS ) { + off =fdt_path_offset (fdt,"/ibm,opal" ); + } + +_FDT (off); + off =fdt_add_subnode (fdt, off,"dump" ); +_FDT (off); +_FDT ((fdt_setprop_string (fdt, off,"compatible" ,"ibm,opal-dump" ))); + +/* Add kernel and initrd as fw-load-area */ +uint64_t fw_load_area[4 ] = { +cpu_to_be64 (KERNEL_LOAD_ADDR ),cpu_to_be64 (KERNEL_MAX_SIZE ), +cpu_to_be64 (INITRD_LOAD_ADDR ),cpu_to_be64 (INITRD_MAX_SIZE ) + }; + +_FDT ((fdt_setprop (fdt, off,"fw-load-area" , + fw_load_area,sizeof (fw_load_area)))); +} +static void *pnv_dt_create (MachineState *machine) {PnvMachineClass *pmc =PNV_MACHINE_GET_CLASS (machine);
@@ -733,6 +767,9 @@ static void *pnv_dt_create(MachineState *machine) pmc->dt_power_mgt (pnv, fdt); } +/* Advertise support for MPIPL */ +pnv_dt_mpipl_dump (pnv, fdt); +return fdt; }
Signature · 1 line
2.52.0
Re: [PATCH v2 8/9] pnv/mpipl: Enable MPIPL support
Quoted 48 lines · Aditya Gupta
With all MPIPL support in place, export a "dump" node in device tree, signifying that PowerNV QEMU platform supports MPIPL
Signed-off-by Aditya Gupta <[email protected]>--- hw/ppc/pnv.c | 37 +++++++++++++++++++++++++++++++++++++ 1 file changed, 37 insertions(+)hw/ppc/pnv.c +24
-0@@ -54,6 +54,7 @@#include "hw/ppc/pnv_chip.h" #include "hw/ppc/pnv_xscom.h" #include "hw/ppc/pnv_pnor.h" +#include "hw/ppc/pnv_mpipl.h" #include "hw/isa/isa.h" #include "hw/char/serial-isa.h" @@ -671,6 +672,39 @@ static void pnv_dt_power_mgt(PnvMachineState *pnv, void *fdt)_FDT (fdt_setprop_cell (fdt, off,"ibm,enabled-stop-levels" ,0xc0000000 )); } +static void pnv_dt_mpipl_dump (PnvMachineState *pnv,void *fdt) +{ +int off; + +/* +* Add "dump" node so kernel knows MPIPL (aka fadump) is supported +* +* Note: This is only needed to be done since we are passing device tree to +* opal +* +* In case HDAT is supported in future, then opal can add these nodes by +* itself based on system attribute having MPIPL_SUPPORTED bit set +*/ + off =fdt_add_subnode (fdt,0 ,"ibm,opal" ); +if (off == -FDT_ERR_EXISTS ) { + off =fdt_path_offset (fdt,"/ibm,opal" ); + } + +_FDT (off); + off =fdt_add_subnode (fdt, off,"dump" ); +_FDT (off); +_FDT ((fdt_setprop_string (fdt, off,"compatible" ,"ibm,opal-dump" ))); +
hw/ppc/pnv.c static void pnv_dt_power_mgt(PnvMachineState *pnv, void *fdt) · line 708
Quoted 10 lines
hw/ppc/pnv.c +10
-0+/* Add kernel and initrd as fw-load-area */ +uint64_t fw_load_area[4 ] = { +cpu_to_be64 (KERNEL_LOAD_ADDR ),cpu_to_be64 (KERNEL_MAX_SIZE ), +cpu_to_be64 (INITRD_LOAD_ADDR ),cpu_to_be64 (INITRD_MAX_SIZE ) + }; + +_FDT ((fdt_setprop (fdt, off,"fw-load-area" , + fw_load_area,sizeof (fw_load_area)))); +} +
While the changelog only talks about dump node, fw-load-area is exported too. A comment explaining what and why it is used helps..
- Hari
Re: [PATCH v2 8/9] pnv/mpipl: Enable MPIPL support
Quoted 4 lines · Hari Bathini
<...snip...>
Quoted 13 lines
hw/ppc/pnv.c static void pnv_dt_power_mgt(PnvMachineState *pnv, void *fdt) · line 707
Quoted 10 lines
hw/ppc/pnv.c +10
-0+/* Add kernel and initrd as fw-load-area */ +uint64_t fw_load_area[4 ] = { +cpu_to_be64 (KERNEL_LOAD_ADDR ),cpu_to_be64 (KERNEL_MAX_SIZE ), +cpu_to_be64 (INITRD_LOAD_ADDR ),cpu_to_be64 (INITRD_MAX_SIZE ) + }; + +_FDT ((fdt_setprop (fdt, off,"fw-load-area" , + fw_load_area,sizeof (fw_load_area)))); +} +While the changelog only talks about dump node, fw-load-area is exported too. A comment explaining what and why it is used helps..
Sure hari, will add purpose of fw-load-area also in the description
Thanks,
- Aditya G
Quoted 2 lines
- Hari
PATCH v2 9/9 tests/functional: Add test for MPIPL in PowerNV
In reply to Aditya GuptaWith MPIPL support implemented, enable fadump's functional test for PowerNV
Also, current functional test for powernv uses op-build's Linux 5.10 image, which doesn't support adding "fadump=on" in argument due to this:
Kernel is locked down from Kernel configuration; see man kernel_lockdown.7
Hence, instead of op-build's image, use the newer fedora vmlinuz as used in FADump PSeries functional test
Also due to "bash#" string not showing up, rely on sh: no job control to check if testcase has reached till shell
--- tests/functional/ppc64/test_fadump.py | 35 ++++++++++----------------- 1 file changed, 13 insertions(+), 22 deletions(-)
tests/functional/ppc64/test_fadump.py +13 -22
@@ -14,6 +14,7 @@ class QEMUFadump(LinuxKernelTest): 1. test_fadump_pseries: PSeries 2. test_fadump_pseries_kvm: PSeries + KVM + 3. test_fadump_powernv: PowerNV """ timeout = 90
@@ -24,11 +25,6 @@ class QEMUFadump(LinuxKernelTest): msg_registered_failed = '' msg_dump_active = '' - ASSET_EPAPR_KERNEL = Asset( - ('https://github.com/open-power/op-build/releases/download/v2.7/' - 'zImage.epapr'), - '0ab237df661727e5392cee97460e8674057a883c5f74381a128fa772588d45cd') - ASSET_VMLINUZ_KERNEL = Asset( ('https://archives.fedoraproject.org/pub/archive/fedora-secondary/' 'releases/39/Everything/ppc64le/os/ppc/ppc64/vmlinuz'),
@@ -64,16 +60,14 @@ def do_test_fadump(self, is_kvm=False, is_powernv=False): # SLOF takes upto >20s in startup time, use VOF self.set_machine("pseries") self.vm.add_args("-machine", "x-vof=on") - self.vm.add_args("-m", "6G") + + self.vm.add_args("-m", "6G") self.vm.set_console() kernel_path = None - if is_powernv: - kernel_path = self.ASSET_EPAPR_KERNEL.fetch() - else: - kernel_path = self.ASSET_VMLINUZ_KERNEL.fetch() + kernel_path = self.ASSET_VMLINUZ_KERNEL.fetch() initrd_path = self.ASSET_FEDORA_INITRD.fetch()
@@ -104,16 +98,14 @@ def do_test_fadump(self, is_kvm=False, is_powernv=False): timeout=20 ) - # Ensure fadump is registered successfully, if registration - # succeeds, we get a log from rtas fadump: - # - # rtas fadump: Registration is successful! - self.wait_for_console_pattern( - "rtas fadump: Registration is successful!" - ) + # Ensure fadump is registered successfully + if not is_powernv: + self.wait_for_console_pattern( + "rtas fadump: Registration is successful!" + ) # Wait for the shell - self.wait_for_console_pattern("#") + self.wait_for_console_pattern("sh: no job control") # Mount /proc since not available in the initrd used exec_command(self, command="mount -t proc proc /proc")
@@ -137,7 +129,7 @@ def do_test_fadump(self, is_kvm=False, is_powernv=False): # that qemu didn't pass the 'ibm,kernel-dump' device tree node wait_for_console_pattern( test=self, - success_message="rtas fadump: Firmware-assisted dump is active", + success_message="fadump: Firmware-assisted dump is active", failure_message="fadump: Reserved " )
@@ -150,7 +142,7 @@ def do_test_fadump(self, is_kvm=False, is_powernv=False): self.wait_for_console_pattern("preserving crash data") # Wait for prompt - self.wait_for_console_pattern("sh-5.2#") + self.wait_for_console_pattern("Run /bin/sh as init process") # Mount /proc since not available in the initrd used exec_command_and_wait_for_pattern(self,
@@ -168,9 +160,8 @@ def do_test_fadump(self, is_kvm=False, is_powernv=False): def test_fadump_pseries(self): return self.do_test_fadump(is_kvm=False, is_powernv=False) - @skip("PowerNV Fadump not supported yet") def test_fadump_powernv(self): - return + return self.do_test_fadump(is_kvm=False, is_powernv=True) def test_fadump_pseries_kvm(self): """
Signature · 1 line
2.52.0
Quoted 25 lines · Aditya Gupta
Overview
=========Implemented MPIPL (Memory Preserving IPL, aka fadump) on PowerNV machine in QEMU.
Note: It's okay if this isn't merged as there might be less users. Sending for archieval purpose, as the patches can be referred for how fadump/mpipl can be implemented in baremetal/PowerNV/any other arch QEMU.
Fadump is an alternative dump mechanism to kdump, in which we the firmware does a memory preserving boot, and the second/crashkernel is booted fresh like a normal system reset, instead of the crashed kernel loading the second/crashkernel in case of kdump.
MPIPL in PowerNV, is similar to fadump in Pseries. The idea is same, memory preserving, where in PowerNV we are assisted by SBE (Self Boot Engine) & Hostboot, while in Pseries we are assisted by PHyp (Power Hypervisor)
For implementing in baremetal/powernv QEMU, we need to export a "ibm,opal/dump" node in the device tree, to tell the kernel we support MPIPL
Quoted 8 lines
Once kernel sees the support, and "fadump=on" is passed on commandline, kernel will register memory regions to preserve with Skiboot.
Kernel sends these data using OPAL calls, after which skiboot/opal saves the memory region details to MDST and MDDT tables (S-source, D-destination)
Skiboot then triggers the "S0 Interrupt" to the SBE (Self Boot Engine), along with OPAL's relocated base address.
This sounds as though skiboot triggers S0 Interrupt right after kernel registers through OPAL calls, which is not the case. Please make it explicit that kernel initiates fadump with another OPAL call after registering..
- Hari
Thanks for your reviews, and sorry for the delay Hari, got involved with other things.
Quoted 14 lines · Hari Bathini
Quoted 11 lines · Aditya Gupta
Overview
=========<...snip...>
Once kernel sees the support, and "fadump=on" is passed on commandline, kernel will register memory regions to preserve with Skiboot.
Kernel sends these data using OPAL calls, after which skiboot/opal saves the memory region details to MDST and MDDT tables (S-source, D-destination)
Quoted 7 lines
Quoted 2 lines
Skiboot then triggers the "S0 Interrupt" to the SBE (Self Boot Engine), along with OPAL's relocated base address.
This sounds as though skiboot triggers S0 Interrupt right after kernel registers through OPAL calls, which is not the case. Please make it explicit that kernel initiates fadump with another OPAL call after registering..
Right Hari, will reword the sentence and add this detail.
Thanks,
- Aditya G
Quoted 2 lines
- Hari
Quoted 43 lines · Aditya Gupta
Overview
=========Implemented MPIPL (Memory Preserving IPL, aka fadump) on PowerNV machine in QEMU.
Note: It's okay if this isn't merged as there might be less users. Sending for archieval purpose, as the patches can be referred for how fadump/mpipl can be implemented in baremetal/PowerNV/any other arch QEMU.
Fadump is an alternative dump mechanism to kdump, in which we the firmware does a memory preserving boot, and the second/crashkernel is booted fresh like a normal system reset, instead of the crashed kernel loading the second/crashkernel in case of kdump.
MPIPL in PowerNV, is similar to fadump in Pseries. The idea is same, memory preserving, where in PowerNV we are assisted by SBE (Self Boot Engine) & Hostboot, while in Pseries we are assisted by PHyp (Power Hypervisor)
For implementing in baremetal/powernv QEMU, we need to export a "ibm,opal/dump" node in the device tree, to tell the kernel we support MPIPL
Once kernel sees the support, and "fadump=on" is passed on commandline, kernel will register memory regions to preserve with Skiboot.
Kernel sends these data using OPAL calls, after which skiboot/opal saves the memory region details to MDST and MDDT tables (S-source, D-destination)
Skiboot then triggers the "S0 Interrupt" to the SBE (Self Boot Engine), along with OPAL's relocated base address.
SBE then stops all core clocks, and only does particular ISteps for a memory preserving boot.
Then, hostboot comes up, and with help of the relocated base address, it accesses MDST & MDDT tables (S-source and D-destination), and preserves the memory regions according to the data in these tables.
And after preserving, it writes the preserved memory region details to MDRT tables (R-Result), for the kernel to know where/whether a memory region is preserved.
Quoted 2 lines
Both SBE's and hostboot responsiblities have in implemented in the SBE code in QEMU.
s/have in implemented/are implemented/
Quoted 6 lines · Hari Bathini
Quoted 3 lines · Aditya Gupta
Overview
=========
Quoted 6 lines
Quoted 4 lines
<...snip...>
Both SBE's and hostboot responsiblities have in implemented in the SBE code in QEMU.
s/have in implemented/are implemented/
Will fix it Hari.
Thanks,
- Aditya G