Implement MPIPL for PowerNV
Get series with b4
b4 am [email protected]Versions newest first · linked by author and title
- v6 Latest10 patches 1 replyIn review
- v510 patches 4 repliesSuperseded
- v410 patches 4 repliesSuperseded
Earlier versions · 3
| Version | Posted | Patches | Replies | Status |
|---|---|---|---|---|
| v3 | 10 patches | 5 replies | Superseded | |
| v2 | 9 patches | 15 replies | Superseded | |
| v1 | 7 patches | 16 replies | Superseded |
Changes in v6 from the cover letter
* rebased to upstream commit bb230769b4d01de714bed686161ad39a8f4f3fd1
* #2,3,4: remove osdep.h include from .h file
* #3: use RAM_ADDR_FMT format for ram size
* #4,5: replace '__packed' with 'QEMU_PACKED'
* #8: add pnv_dt_create in pnv_reset, to create dt after mpipl
* #10: add sourabh as reviewer
Patches in v6 No applies recorded yet
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/10 | ppc/pnv: Move SBE host doorbell function to top of filehw/ | +9 | 0 replies | |
| 2/10 | ppc/mpipl: Implement S0 SBE interrupthw/ hw/ hw/ +2 more | +80 | 0 replies | |
| 3/10 | ppc/pnv: Handle stash command in PowerNV SBEhw/ include/ | +43 | 0 replies | |
| 4/10 | pnv/mpipl: Preserve memory regions as per MDST/MDDT tableshw/ include/ | +246 | 0 replies | |
| 5/10 | pnv/mpipl: Preserve CPU registers after crashhw/ include/ | +214 | 0 replies | |
| 6/10 | pnv/mpipl: Set thread entry size to be allocated by firmwarehw/ | +25 | 0 replies | |
| 7/10 | pnv/mpipl: Write the preserved CPU and MDRT statehw/ hw/ include/ | +179 | 0 replies | |
| 8/10 | pnv/mpipl: Enable MPIPL supporthw/ | +45 | 0 replies | |
| 9/10 | tests/functional: Add test for MPIPL in PowerNVtests/ | +13 | 0 replies | |
| 10/10 | MAINTAINERS: Add entry for MPIPL (PowerNV)MAINTAINERS | +9 | 0 replies |
Reviews trailers per patch
All 10 patches have a Reviewed-by or Acked-by.
| Patch | Hari Bathini | Sourabh Jain | Shivang Upadhyay |
|---|---|---|---|
| Series | T-b | ||
| 1/10 | R-b | R-b | |
| 2/10 | R-b | R-b | |
| 3/10 | R-b | R-b | |
| 4/10 | R-b | R-b | |
| 5/10 | R-b | R-b | |
| 6/10 | R-b | R-b | |
| 7/10 | R-b | R-b | |
| 8/10 | R-b | R-b | |
| 9/10 | R-b | R-b | |
| 10/10 | R-b | R-b |
Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes. Scroll the table sideways for all 3 reviewers.
Cover letter [PATCH v6 00/10] Implement MPIPL for PowerNV
Overview
=========
Implemented MPIPL (Memory Preserving IPL, aka fadump) on PowerNV machine in QEMU.
Fadump is an alternative dump mechanism to kdump, in which we the firmware does a memory preserving boot, and the second/crashkernel is booted fresh like a normal system reset, instead of the crashed kernel loading the second/crashkernel in case of kdump.
MPIPL in PowerNV, is similar to fadump in Pseries. The idea is same, memory preserving, where in PowerNV we are assisted by SBE (Self Boot Engine) & Hostboot, while in Pseries we are assisted by PHyp (Power Hypervisor)
For implementing in baremetal/powernv QEMU, we need to export a "ibm,opal/dump" node in the device tree, to tell the kernel we support MPIPL
Once kernel sees the support, and "fadump=on" is passed on commandline, kernel will register memory regions to preserve with Skiboot.
Kernel sends these data using OPAL calls, after which skiboot/opal saves the memory region details to MDST and MDDT tables (S-source, D-destination)
Then in the event of a kernel crash, the kernel initiates MPIPL with another OPAL code (opal_cec_reboot2), this request goes to Skiboot. Skiboot then triggers the "S0 Interrupt" to the SBE (Self Boot Engine), along with OPAL's relocated base address.
SBE then stops all core clocks, and only does particular ISteps for a memory preserving boot.
Then, hostboot comes up, and with help of the relocated base address, it accesses MDST & MDDT tables (S-source and D-destination), and preserves the memory regions according to the data in these tables.
And after preserving, it writes the preserved memory region details to MDRT tables (R-Result), for the kernel to know where/whether a memory region is preserved.
Both SBE's and hostboot responsiblities are implemented in the SBE code in QEMU.
Then in the second kernel/crashkernel boot, OPAL passes the "mpipl-boot" property for the kernel to know that a dump is active, which kernel then exports in /proc/vmcore
Testing
====================
1. Git tree for testing: https://gitlab.com/adi-g15-ibm/qemu/tree/fadump-powernv-v6
2. Gitlab pipeline: https://gitlab.com/adi-g15-ibm/qemu/-/pipelines/2476290852
3. QEMU ppc64 functional tests:
Ok: 13
Fail: 0
Skipped: 3 4. Analysing generated vmcore:
# file /proc/vmcore
/proc/vmcore: ELF 64-bit LSB core file, 64-bit PowerPC or cisco 7500, OpenPOWER ELF V2 ABI, version 1 (SYSV), SVR4-style # crash vmlinux-38fec10eb60d-network vmcore-powernv-24apr26
...
KERNEL: vmlinux-38fec10eb60d-network
DUMPFILE: vmcore-powernv-24apr26
CPUS: 2
DATE: Thu Jan 1 05:30:00 IST 1970
UPTIME: 00:01:38
LOAD AVERAGE: 0.13, 0.08, 0.03
TASKS: 84
NODENAME: buildroot
RELEASE: 6.14.0
VERSION: #1 SMP Thu Apr 3 08:06:13 CDT 2025
MACHINE: ppc64le (1000 Mhz)
MEMORY: 6 GB
PANIC: "Kernel panic - not syncing: sysrq triggered crash"
PID: 239
COMMAND: "sh"
TASK: c00000000933cb00 [THREAD_INFO: c00000000933cb00]
CPU: 0
STATE: TASK_RUNNING (PANIC) crash> bt
PID: 239 TASK: c00000000933cb00 CPU: 0 COMMAND: "sh"
R0: c000000000059604 R1: c0000000093df9c0 R2: c0000000020eaa00
R3: c0000000bffffff8 R4: c0000000071e0dc2 R5: 0000000000000002
R6: 5245424d554e0a32 R7: 4d5f584944415228 R8: 454b0a313d29554d
R9: 0000000000000000 R10: 4152430a303d5445 R11: 313d454d49544853
R12: 3837383030373737 R13: c00000017ffff480 R14: 00000001172c0460
R15: 0000000000000000 R16: 0000000000000000 R17: 00007fffe17bac78
R18: 000000012cd00700 R19: 0000000000000000 R20: 000000012cd006f8
R21: fcffffffffffffff R22: c00000000933cb00 R23: 0000000000000000
R24: a8aaaaaaaaaaaaaa R25: c000000003bb7eb8 R26: c000000003e5eab8
R27: c0000000c0000000 R28: 0000000000000000 R29: 0000000000000002
R30: c000000003e5eab8 R31: c000000003e16b30
NIP: c000000000059720 MSR: 9000000000001033 OR3: 0000000000000000
CTR: 0000000000000000 LR: c00000000002f69c XER: 0000000020040006
CCR: 0000000028002202 MQ: 0000000000000003 DAR: 0000000000000000
DSISR: 0000000000000000 Syscall Result: 0000000000000000
[NIP : crash_fadump+560]
[LR : ppc_panic_fadump_handler+84]
#0 [c0000000093df9c0] crash_fadump at c00000000005966c
#1 [c0000000093dfa20] ppc_panic_fadump_handler at c00000000002f69c
#2 [c0000000093dfa40] notifier_call_chain at c0000000001ab390
#3 [c0000000093dfaa0] atomic_notifier_call_chain at c0000000001ab4a4
#4 [c0000000093dfac0] panic at c000000000163598
#5 [c0000000093dfb60] sysrq_handle_crash at c000000000beafd4
#6 [c0000000093dfbc0] __handle_sysrq at c000000000beb9b4
#7 [c0000000093dfc60] write_sysrq_trigger at c000000000bec34c
#8 [c0000000093dfce0] proc_reg_write at c00000000071027c
#9 [c0000000093dfd10] vfs_write at c000000000620a28
#10 [c0000000093dfdc0] ksys_write at c000000000621010
#11 [c0000000093dfe10] system_call_exception at c0000000000324b8
#12 [c0000000093dfe50] system_call_vectored_common at c00000000000bff0
crash> kmem -i
PAGES TOTAL PERCENTAGE
TOTAL MEM 92396 5.6 GB ----
FREE 89710 5.5 GB 97% of TOTAL MEM
USED 2686 167.9 MB 2% of TOTAL MEM
BUFFERS 0 0 0% of TOTAL MEM
CACHED 1500 93.8 MB 1% of TOTAL MEM
SLAB 674 42.1 MB 0% of TOTAL MEM
TOTAL HUGE 0 0 ----
HUGE FREE 0 0 0% of TOTAL HUGE
TOTAL SWAP 0 0 ----
SWAP USED 0 0 0% of TOTAL SWAP
SWAP FREE 0 0 0% of TOTAL SWAP
COMMIT LIMIT 46198 2.8 GB ----
COMMITTED 0 0 0% of TOTAL LIMIT
crash> # ps and kmem -i worksChangelog
====================
v5 -> v6:
* rebased to upstream commit bb230769b4d01de714bed686161ad39a8f4f3fd1
* #2,3,4: remove osdep.h include from .h file
* #3: use RAM_ADDR_FMT format for ram size
* #4,5: replace '__packed' with 'QEMU_PACKED'
* #8: add pnv_dt_create in pnv_reset, to create dt after mpipl
* #10: add sourabh as reviewer
v4 -> v5:
* #4/10: set chunk_id=0 before copying
* #7/10: remove unnecessary bool check, ie. 'if (b1) b2=b1 else b2=!b1' => 'b2=b1'
v3 -> v4:
* #2/10: s/recieves/receives
* #7/10: remove empty line at EOF
v2 -> v3:
* rebase to upstream, changes in patches below
* #2/10: no code change. add comment that skiboot triggers S0
* #3/10: stash command: handle invalid skiboot_base sent by guest
* #4/10: s/src_len/data_len/
* #4/10: use TARGET_FMT_lx/PRIx64 instead of %lx to prevent build errors
* #4/10: stop copying chunks once copying a chunk fails
* #5/10: use address_space_{read,write} instead of cpu_physical_memory_{read,write}
* #5/10: add more SPRs to be saved, same set of SPRs as spapr FADump, except CR and FPSCR
* #7/10: only export "mpipl-boot" property if preserving cpu states and writing MDRT was successful, otherwise continue with normal reboot
* #7/10: use address_space_{read,write} instead of cpu_physical_memory_{read,write}
* #8/10: reword commit description to mention fw-load-area, no code change
* #10/10: add entry in MAINTAINERS file
Aditya Gupta (10):
ppc/pnv: Move SBE host doorbell function to top of file ppc/mpipl: Implement S0 SBE interrupt
ppc/pnv: Handle stash command in PowerNV SBE pnv/mpipl: Preserve memory regions as per MDST/MDDT tables pnv/mpipl: Preserve CPU registers after crash pnv/mpipl: Set thread entry size to be allocated by firmware pnv/mpipl: Write the preserved CPU and MDRT state pnv/mpipl: Enable MPIPL support
tests/functional: Add test for MPIPL in PowerNV
MAINTAINERS: Add entry for MPIPL (PowerNV)
MAINTAINERS | 9 + hw/ppc/meson.build | 1 + hw/ppc/pnv.c | 108 +++++- hw/ppc/pnv_mpipl.c | 482 ++++++++++++++++++++++++++ hw/ppc/pnv_sbe.c | 85 ++++- include/hw/ppc/pnv.h | 6 + include/hw/ppc/pnv_mpipl.h | 168 +++++++++ tests/functional/ppc64/test_fadump.py | 35 +- 8 files changed, 862 insertions(+), 32 deletions(-) create mode 100644 hw/ppc/pnv_mpipl.c create mode 100644 include/hw/ppc/pnv_mpipl.h
Signature · 1 line
2.53.0