Skip to contentSkip to conversation

Support for Self Save API in OPAL

PATCH In review Pratik Rajesh Sampat 3 participants 8 messages Apr 23, 2020 – Apr 29, 2020

lore
Share & tools
lore
#0 lore raw

v8 cover Support for Self Save API in OPAL

v7: https://lists.ozlabs.org/pipermail/skiboot/2020-April/016763.html
Changelog
v6 --> v7
1. Simplified approach. Instead of advertising support bitmask for each SPR, advertising support only for the presence of the new self-save API Complete design specification is detailed below in the cover-leter.
2. Patch re-organization, move introducing self-save [2] after STOP API versioning [3].

Background
==========

The power management framework on POWER systems include core idle states that lose context. Deep idle states namely "winkle" on POWER8 and "stop4" and "stop5" on POWER9 can be entered by a CPU to save different levels of power, as a consequence of which all the hypervisor resources such as SPRs and SCOMs are lost.

For most SPRs, saving and restoration of content for SPRs and SCOMs is handled by the hypervisor kernel prior to entering an post exit from an idle state respectively. However, there is a small set of critical SPRs and XSCOMs that are expected to contain sane values even before the control is transferred to the hypervisor kernel at system reset vector.

For this purpose, microcode firmware provides a mechanism to restore values on certain SPRs. The communication mechanism between the hypervisor kernel and the microcode is a standard interface called sleep-winkle-engine (SLW) on Power8 and Stop-API on Power9 which is abstracted by OPAL calls from the hypervisor kernel. The Stop-API provides an interface known as the self-restore API, to which the SPR number and a predefined value to be restored on wake-up from a deep stop state is supplied.

Motivation to introduce a new Stop-API
======================================

The self-restore API expects not just the SPR number but also the value with which the SPR is restored. This is good for those SPRs such as HSPRG0 whose values do not change at runtime, since for them, the kernel can invoke the self-restore API at boot time once the values of these SPRs are determined.

However, there are use-cases where-in the value to be saved cannot be known or cannot be updated in the layer it currently is. The shortcomings and the new use-cases which cannot be served by the existing self-restore API, serves as motivation for a new API:

Shortcoming1:
------------
In a special wakeup scenario when a CPU is woken up in stop4/5 and after the task is done, the HCODE puts it back to stop. The value of PSSCR is passed to the HCODE via the self-restore API. The kernel currently provides the value of the deepest stop state due to being conservative. Thus if a core that was in stop4 was woken up due to special wakeup, the HCODE will now put it back to stop5 thus increasing the subsequent wakeup latency to ~200us.

Shortcoming2:
------------
The value of LPCR is dynamic based on if the CPU is entered a stop state during cpu idle versus cpu hotplug.
Today, an additional self-restore call is made before entering CPU-Hotplug to clear the PECE1 bit in stop-API so that if we are woken up by a special wakeup on an offlined CPU, we go back to stop with the the bit cleared.
There is a overhead of an extra call

New Use-case:
-------------
In the case where the hypervisor is running on an ultravisor environment, the boot time is too late in the cycle to make the self-restore API calls, as these cannot be invoked from an non-secure context anymore

To address these shortcomings, the firmware provides another API known as the self-save API. The self-save API only takes the SPR number as a parameter and will ensure that on wakeup from a deep-stop state the SPR is restored with the value that it contained prior to entering the deep-stop.

Contrast between self-save and self-restore APIs
================================================

		  Before entering
                  deep idle     |---------------|
                  ------------> | HCODE A       |                
                  |             |---------------|
   |---------|    |
   |   CPU   |----|
   |---------|    |             
                  |             |---------------|
                  |------------>| HCODE B       |
                  On waking up  |---------------|

from deep idle

When a self-restore API is invoked, the HCODE inserts instructions into "HCODE B" region of the above figure to restore the content of the SPR to the said value. The "HCODE B" region gets executed soon after the CPU wakes up from a deep idle state, thus executing the inserted instructions, thereby restoring the contents of the SPRs to the required values.

When a self-save API is invoked, the HCODE inserts instructions into the "HCODE A" region of the above figure to save the content of the SPR into some location in memory. It also inserts instructions into the "HCODE B" region to restore the content of the SPR to the corresponding value saved in the memory by the instructions in "HCODE A" region.

Thus, in contrast with self-restore, the self-save API *does not* need a value to be passed to it, since it ensures that the value of SPR before entering deep stop is saved, and subsequently the same value is restored.

Self-save and self-restore are complementary features since, self-restore can help in restoring a different value in the SPR on wakeup from a deep-idle state than what it had before entering the deep idle state. This was used in POWER8 for HSPRG0 to distinguish a wakeup from Winkle vs Fastsleep.

Limitations of self-save
========================
Ideally all SPRs should be available for self-save, but HID0 is very tricky to implement in microcode due to various endianess quirks. Couple of implementation schemes were buggy and hence HID0 was left out to be self-restore only.

The fallout of this limitation is as follows:

* In Non PEF environment, no issue. Linux will use self-restore for HID0 as it does today and no functional impact.

* In PEF environment, the HID0 restore value is decided by OPAL during boot and it is setup for LE hypervisor with radix MMU. This is the default and current working configuration of a PEF environment. However if there is a change, then HV Linux will try to change the HID0 value to something different than what OPAL decided, at which time deep-stop states will be disabled under this new PEF environment.

A simple and workable design is achieved by scoping the power management deep-stop state support only to a known default PEF environment. Any deviation will affect *only* deep stop-state support (stop4,5) in that environment and not have any functional impediment to the environment itself.

In future, if there is a need to support changing of HID0 to various values under PEF environment and support deep-stop states, it can be worked out via an ultravisor call or improve the microcode design to include HID0 in self-save. These future scheme would be an extension and does not break or make the current implementation scheme redundant.

Design Choices
==============

Presenting the design choices in front of us:

Design-Choice 1:
----------------
Only expose one of self-save or self-restore for all the SPRs. Prefer Self-save

Pros:
- Simplifies the design heavily, since the Kernel can unambiguously make one API call for all the SPRs on discovering the presence of the API type.

Cons:
- Breaks backward compatibility if OPAL always chooses to expose only the self-save API as the older kernels assume the existence of self-restore.

- The set of SPRs supported by self-save and self-restore are not identical. Eg: HID0 is not supported by self-save API. PSSCR support via self-restore is not robust during special-wakeup.

- As discussed above, self-save and self-restore are complementary. Thus OPAL apriory choosing one over the other for all SPRs takes away the flexibility from the kernel.

Design-Choice 2:
----------------
Expose two arrays of SPRs: One set of SPRs that are supported by self-save. Another set of SPRs supported by self-restore. These two sets do not intersect. Further, if an SPR is supported by both self-save and self-restore APIs, expose it only via self-save.

Pros:
- For an SPR the choice for the kernel is unambiguous.

Cons:
- Breaks backward compatibility if OPAL always chooses to expose the legacy SPRs only via the self-save API as the older kernels assume the existence of self-restore.

- By making the decision early on, we take away the flexibility from the kernel to use an API of its choice for an SPR.

Design-Choice 3
---------------
Expose two arrays of SPRs. One set of SPRs that are supported by self-save API. Another set of SPRs supported by self-restore API. Let the kernel choose which API to invoke. Even if it wants to always prefer self-save over self-restore, let that be kernel's choice.

Pros:
- Keeps the design flexible to allow the kernel to take a decision based on its functional and performance requirements. Thus, the kernel for instance can make a choice to invoke self-restore API (when available) for SPRs whose values do not evolve at runtime, and invoke the self-save API (when available)
for SPRs whose values will change during runtime.

- Design is backward compatible with older kernels.

Cons:
- The Kernel code will have additional complexity for parsing two lists of SPRs and making a choice w.r.t invocation of a specific stop-api.

Design-Choice 4
===============
Expose only the availability of the self-save feature not the subsequent SPR list. The kernel will have to make a hard choice of preference, although can still have fallback available.
This makes the design although less flexible for the kernel however keeps it simple while still adhering to correctness and backward compatibility.

Patches Organization
====================
Design choice 4 has been chosen as an implementation to demonstrate in this patch series.

Patch 1:
Commit adds support calling into the self save firmware API. Also adds abstraction for making platform agnostic calls.

Patch 2:
Commit adds API to determine the version of the STOP API. This helps to identify support for self save in the firmware

Patch 3:
commit adds wrappers for the self save api for which an opal call can be made. The commit also advertises the self-save feature if the STOP API versions are consistent with the firmware.

Pratik Rajesh Sampat (1):
Self save API integration

Prem Shanker Jha (2):
Self Save: Introducing Support for SPR Self Save API to verify the STOP API and image compatibility

 doc/opal-api/opal-slw-self-save-reg-181.rst |  51 ++
 doc/opal-api/opal-slw-set-reg-100.rst       |   5 +
 doc/power-management.rst                    |  48 +
 hw/slw.c                                    | 106 +++
 include/opal-api.h                          |   3 +-
 include/p9_stop_api.H                       | 122 ++-
 include/skiboot.h                           |   3 +
 libpore/p9_cpu_reg_restore_instruction.H    |  11 +-
 libpore/p9_hcd_memmap_base.H                |   7 +
 libpore/p9_stop_api.C                       | 964 +++++++++++---------
 libpore/p9_stop_api.H                       | 141 ++-
 libpore/p9_stop_data_struct.H               |   4 +-
 libpore/p9_stop_util.H                      |  27 +-
 13 files changed, 1060 insertions(+), 432 deletions(-)
 create mode 100644 doc/opal-api/opal-slw-self-save-reg-181.rst
Signature · 1 line
2.25.1
#1 lore raw

PATCH v8 1/3 Self Save: Introducing Support for SPR Self Save

From: Prem Shanker Jha <[email protected]>

The commit is a merger of commits that makes the following changes:
1. Commit fixes some issues with code found during integration test

  -  replacement of addi with xor instruction during self save API.
  -  fixing instruction generation for MFMSR during self save
  -  data struct updates in STOP API
  -  error RC updates for hcode image build
  -  HOMER parser updates.
  -  removed self save support for URMOR and HRMOR
  -  code changes for compilation with OPAL
  -  populating CME Image header with unsecure HOMER address.

Key_Cronus_Test=PM_REGRESS

Change-Id: I7cedcc466267c4245255d8d75c01ed695e316720
Reviewed-on: http://rchgit01.rchland.ibm.com/gerrit1/66580

Reviewed-on: http://rchgit01.rchland.ibm.com/gerrit1/66587

2. The commit also incorporates changes that make STOP API project agnostic changes include defining wrapper functions which call legacy API. It also adds duplicate enum members which start with prefix PROC instead of P9.

Key_Cronus_Test=PM_REGRESS

Change-Id: If87970f3e8cf9b507f33eb1be249e03eb3836a5e
RTC: 201128
Reviewed-on: http://rchgit01.rchland.ibm.com/gerrit1/71307

Reviewed-on: http://rchgit01.rchland.ibm.com/gerrit1/71314

---
 include/p9_stop_api.H                    |  79 +-
 libpore/p9_cpu_reg_restore_instruction.H |   4 +
 libpore/p9_stop_api.C                    | 954 +++++++++++++----------
 libpore/p9_stop_api.H                    | 115 ++-
 libpore/p9_stop_data_struct.H            |   4 +-
 libpore/p9_stop_util.H                   |   7 +-
 6 files changed, 721 insertions(+), 442 deletions(-)
include/p9_stop_api.H +77 -2
index 79abd000..9d3bc1e5 100644
@@ -63,6 +63,26 @@ typedef enum
     P9_STOP_SPR_PMCR    =    884,   // core register
     P9_STOP_SPR_HID     =   1008,   // core register
     P9_STOP_SPR_MSR     =   2000,   // thread register
+
+    //enum members which are project agnostic
+    PROC_STOP_SPR_DAWR    =    180,   // thread register
+    PROC_STOP_SPR_CIABR   =    187,   // thread register
+    PROC_STOP_SPR_DAWRX   =    188,   // thread register
+    PROC_STOP_SPR_HSPRG0  =    304,   // thread register
+    PROC_STOP_SPR_HRMOR   =    313,   // core register
+    PROC_STOP_SPR_LPCR    =    318,   // thread register
+    PROC_STOP_SPR_HMEER   =    337,   // core register
+    PROC_STOP_SPR_PTCR    =    464,   // core register
+    PROC_STOP_SPR_USPRG0  =    496,   // thread register
+    PROC_STOP_SPR_USPRG1  =    497,   // thread register
+    PROC_STOP_SPR_URMOR   =    505,   // core register
+    PROC_STOP_SPR_SMFCTRL =    511,   // thread register
+    PROC_STOP_SPR_LDBAR   =    850,   // thread register
+    PROC_STOP_SPR_PSSCR   =    855,   // thread register
+    PROC_STOP_SPR_PMCR    =    884,   // core register
+    PROC_STOP_SPR_HID     =   1008,   // core register
+    PROC_STOP_SPR_MSR     =   2000,   // thread register
+
 } CpuReg_t;
 
 /**
@@ -85,6 +105,8 @@ typedef enum
     STOP_SAVE_SCOM_ENTRY_UPDATE_FAILED   = 12,
     STOP_SAVE_INVALID_FUSED_CORE_STATUS  = 13,
     STOP_SAVE_FAIL                       = 14,  // for internal failure within firmware.
+    STOP_SAVE_SPR_ENTRY_MISSING          =  15,
+    STOP_SAVE_SPR_BIT_POS_RESERVE        =  16,
 } StopReturnCode_t;
 
 /**
@@ -101,7 +123,20 @@ typedef enum
     P9_STOP_SCOM_RESET      = 6,
     P9_STOP_SCOM_OR_APPEND  = 7,
     P9_STOP_SCOM_AND_APPEND = 8,
-    P9_STOP_SCOM_OP_MAX     = 9
+    P9_STOP_SCOM_OP_MAX     = 9,
+
+    //enum members which are project agnostic
+    PROC_STOP_SCOM_OP_MIN     =   0,
+    PROC_STOP_SCOM_APPEND     =   1,
+    PROC_STOP_SCOM_REPLACE    =   2,
+    PROC_STOP_SCOM_OR         =   3,
+    PROC_STOP_SCOM_AND        =   4,
+    PROC_STOP_SCOM_NOOP       =   5,
+    PROC_STOP_SCOM_RESET      =   6,
+    PROC_STOP_SCOM_OR_APPEND  =   7,
+    PROC_STOP_SCOM_AND_APPEND =   8,
+    PROC_STOP_SCOM_OP_MAX     =   9,
+
 } ScomOperation_t;
 
 /**
@@ -114,9 +149,49 @@ typedef enum
     P9_STOP_SECTION_EQ_SCOM     = 2,
     P9_STOP_SECTION_L2          = 3,
     P9_STOP_SECTION_L3          = 4,
-    P9_STOP_SECTION_MAX         = 5
+    P9_STOP_SECTION_MAX         = 5,
+
+    //enum members which are project agnostic
+    PROC_STOP_SECTION_MIN         =   0,
+    PROC_STOP_SECTION_CORE_SCOM   =   1,
+    PROC_STOP_SECTION_EQ_SCOM     =   2,
+    PROC_STOP_SECTION_L2          =   3,
+    PROC_STOP_SECTION_L3          =   4,
+    PROC_STOP_SECTION_MAX         =   5,
+
 } ScomSection_t;
 
+
+
+/**
+ * @brief   List of major incompatibilities between API version.
+ * @note    STOP APIs assumes a specific HOMER layout, certain
+ * level of CME-SGPE hcode and certain version of self-save restore
+ * binary. A mismatch can break STOP function.
+ */
+
+/**
+ * @brief  Summarizes bit position allocated to SPRs in save bit mask vector.
+ */
+typedef enum
+{
+    BIT_POS_CIABR       =   0,
+    BIT_POS_DAWR        =   1,
+    BIT_POS_DAWRX       =   2,
+    BIT_POS_HSPRG0      =   3,
+    BIT_POS_LDBAR       =   4,
+    BIT_POS_LPCR        =   5,
+    BIT_POS_PSSCR       =   6,
+    BIT_POS_MSR         =   7,
+    BIT_POS_HID         =   21,
+    BIT_POS_HMEER       =   22,
+    BIT_POS_PMCR        =   23,
+    BIT_POS_PTCR        =   24,
+    BIT_POS_SMFCTRL     =   28,
+    BIT_POS_USPRG0      =   29,
+    BIT_POS_USPRG1      =   30,
+} SprBitPositionList_t;
+
 #ifdef __cplusplus
 extern "C" {
 #endif
libpore/p9_cpu_reg_restore_instruction.H +4 -0
index cf00ff5e..d69a4212 100644
@@ -62,6 +62,10 @@ enum
     MTSPR_CONST1        =   467,
     MTMSRD_CONST1       =   178,
     MFSPR_CONST         =   339,
+    BLR_INST            =   0x4e800020,
+    MTSPR_BASE_OPCODE   =   0x7c0003a6,
+    MFSPR_BASE_OPCODE   =   0x7c0002a6,
+    ATTN_OPCODE         =   0x00000200,
     OPCODE_18           =   18,
     SELF_SAVE_FUNC_ADD  =   0x2300,
     SELF_SAVE_OFFSET    =   0x180,
libpore/p9_stop_api.C +523 -431

1,124 lines not on this page · raw message

libpore/p9_stop_api.H +110 -5

157 lines not on this page · raw message

libpore/p9_stop_data_struct.H +2 -2
index 1e9721e0..4e73aab5 100644
@@ -67,9 +67,9 @@ enum
     SIZE_PER_SPR_RESTORE_INST   =   ((4 * sizeof(uint8_t)) / sizeof(uint32_t)),
     MAX_THREAD_LEVEL_SPRS       =   11,
     MAX_CORE_LEVEL_SPRS         =   6,
-    MAX_SPR_BIT_POS             =   31,
+    MAX_SPR_BIT_POS             =   30,
     SPR_BIT_POS_8               =    8,
-    SPR_BIT_POS_19              =   19,
+    SPR_BIT_POS_20              =   20,
     SPR_BIT_POS_25              =   25,
     SPR_BIT_POS_27              =   27,
 };
libpore/p9_stop_util.H +5 -2
index 3266fdef..79b4e959 100644
@@ -5,7 +5,7 @@
 /*                                                                        */
 /* OpenPOWER HostBoot Project                                             */
 /*                                                                        */
-/* Contributors Listed Below - COPYRIGHT 2016,2017                        */
+/* Contributors Listed Below - COPYRIGHT 2016,2018                        */
 /* [+] International Business Machines Corp.                              */
 /*                                                                        */
 /*                                                                        */
@@ -95,7 +95,10 @@ typedef struct
     uint64_t cpmrMagicWord;
     uint32_t buildDate;
     uint32_t version;
-    uint8_t  reserve1[7];
+    uint8_t  reserve1[4];
+    uint8_t  selfRestoreVer;
+    uint8_t  stopApiVer;
+    uint8_t  urmorFix;
     uint8_t  fusedModeStatus;
     uint32_t cmeImgOffset;
     uint32_t cmeImgLength;
Signature · 1 line
2.25.1
#3 lore raw

PATCH v8 2/3 API to verify the STOP API and image compatibility

In reply to Pratik Rajesh Sampat

From: Prem Shanker Jha <[email protected]>

Commit defines a new API primarily intended for OPAL to determine cpu register save API's compatibility with HOMER layout and self save restore. It can help OPAL determine if version of API integrated with OPAL is different from hostboot.

Change-Id: Ic0de45a336cfb8b6b6096a10ac1cd3ffbaa44fc0
Reviewed-on: http://rchgit01.rchland.ibm.com/gerrit1/77612

Reviewed-on: http://rchgit01.rchland.ibm.com/gerrit1/77614

---
 include/p9_stop_api.H                    | 25 ++++++++++
 libpore/p9_cpu_reg_restore_instruction.H |  7 ++-
 libpore/p9_hcd_memmap_base.H             |  7 +++
 libpore/p9_stop_api.C                    | 58 +++++++++++++++++++++++-
 libpore/p9_stop_api.H                    | 26 ++++++++++-
 libpore/p9_stop_util.H                   | 20 ++++----
 6 files changed, 130 insertions(+), 13 deletions(-)
include/p9_stop_api.H +25 -0
index 9d3bc1e5..cb5ffd6f 100644
@@ -107,6 +107,7 @@ typedef enum
     STOP_SAVE_FAIL                       = 14,  // for internal failure within firmware.
     STOP_SAVE_SPR_ENTRY_MISSING          =  15,
     STOP_SAVE_SPR_BIT_POS_RESERVE        =  16,
+    STOP_SAVE_API_IMG_INCOMPATIBLE       =  18,
 } StopReturnCode_t;
 
 /**
@@ -161,6 +162,14 @@ typedef enum
 
 } ScomSection_t;
 
+/**
+ * @brief   versions pertaining relvant to STOP API.
+ */
+typedef enum
+{
+    STOP_API_VER            =   0x00,
+    STOP_API_VER_CONTROL    =   0x02,
+} VersionList_t;
 
 
 /**
@@ -192,6 +201,14 @@ typedef enum
     BIT_POS_USPRG1      =   30,
 } SprBitPositionList_t;
 
+typedef enum
+{
+    SMF_SUPPORT_MISSING_IN_HOMER         =   0x01,
+    SELF_SUPPORT_MISSING_FOR_LE_HYP      =   0x02,
+    IPL_RUNTIME_CPU_SAVE_VER_MISMATCH    =   0x04,
+    SELF_RESTORE_VER_MISMATCH            =   0x08,
+} VersionIncompList_t;
+
 #ifdef __cplusplus
 extern "C" {
 #endif
@@ -230,6 +247,14 @@ StopReturnCode_t p9_stop_save_scom( void* const   i_pImage,
                                     const ScomOperation_t i_operation,
                                     const ScomSection_t i_section );
 
+/**
+ * @brief       verifies if API is compatible of current HOMER image.
+ * @param[in]   i_pImage        points to the start of HOMER image of P9 chip.
+ * @param[out]  o_inCompVector  list of incompatibilities found.
+ * @return      STOP_SAVE_SUCCESS if if API succeeds, error code otherwise.
+ */
+StopReturnCode_t proc_stop_api_discover_capability( void* const i_pImage, uint64_t* o_inCompVector );
+
 #ifdef __cplusplus
 } // extern "C"
 };  // namespace stopImageSection ends
libpore/p9_cpu_reg_restore_instruction.H +6 -1
index d69a4212..5f168855 100644
@@ -5,7 +5,7 @@
 /*                                                                        */
 /* OpenPOWER HostBoot Project                                             */
 /*                                                                        */
-/* Contributors Listed Below - COPYRIGHT 2015,2018                        */
+/* Contributors Listed Below - COPYRIGHT 2015,2020                        */
 /* [+] International Business Machines Corp.                              */
 /*                                                                        */
 /*                                                                        */
@@ -69,6 +69,11 @@ enum
     OPCODE_18           =   18,
     SELF_SAVE_FUNC_ADD  =   0x2300,
     SELF_SAVE_OFFSET    =   0x180,
+    SKIP_SPR_REST_INST  =   0x4800001c, //b . +0x01c
+    MFLR_R30            =   0x7fc802a6,
+    SKIP_SPR_SELF_SAVE  =   0x3bff0020, //addi r31 r31, 0x20
+    MTLR_INST           =   0x7fc803a6,  //mtlr r30
+    BRANCH_BE_INST      =   0x48000020,
 };
 
 #define MR_R0_TO_R10            0x7c0a0378UL //mr r10 r0
libpore/p9_hcd_memmap_base.H +7 -0
index 000fafef..ddb56728 100644
@@ -444,6 +444,13 @@ HCD_CONST(CME_QUAD_PSTATE_SIZE,                 HALF_KB)
 
 HCD_CONST(CME_REGION_SIZE,                      (64 * ONE_KB))
 
+
+// HOMER compatibility
+
+HCD_CONST(STOP_API_CPU_SAVE_VER,                0x02)
+HCD_CONST(SELF_SAVE_RESTORE_VER,                0x02)
+HCD_CONST(SMF_SUPPORT_SIGNATURE_OFFSET,         0x1300)
+HCD_CONST(SMF_SELF_SIGNATURE,                   (0x5f534d46))
 // Debug
 
 HCD_CONST(CPMR_TRACE_REGION_OFFSET,             (512 * ONE_KB))
libpore/p9_stop_api.C +57 -1

70 lines not on this page · raw message

libpore/p9_stop_api.H +25 -1
index 3f6420ff..983a3845 100644
@@ -5,7 +5,7 @@
 /*                                                                        */
 /* OpenPOWER HostBoot Project                                             */
 /*                                                                        */
-/* Contributors Listed Below - COPYRIGHT 2015,2018                        */
+/* Contributors Listed Below - COPYRIGHT 2015,2020                        */
 /* [+] International Business Machines Corp.                              */
 /*                                                                        */
 /*                                                                        */
@@ -114,6 +114,7 @@ typedef enum
     STOP_SAVE_FAIL                       =  14,  // for internal failure within firmware.
     STOP_SAVE_SPR_ENTRY_MISSING          =  15,
     STOP_SAVE_SPR_BIT_POS_RESERVE        =  16,
+    STOP_SAVE_API_IMG_INCOMPATIBLE       =  18,
 } StopReturnCode_t;
 
 /**
@@ -198,6 +199,21 @@ typedef enum
     BIT_POS_USPRG1      =   30,
 } SprBitPositionList_t;
 
+/**
+ * @brief   List of major incompatibilities between API version.
+ * @note    STOP APIs assumes a specific HOMER layout, certain
+ * level of CME-SGPE hcode and certain version of self-save restore
+ * binary. A mismatch can break STOP function.
+ */
+
+typedef enum
+{
+    SMF_SUPPORT_MISSING_IN_HOMER         =   0x01,
+    SELF_SUPPORT_MISSING_FOR_LE_HYP      =   0x02,
+    IPL_RUNTIME_CPU_SAVE_VER_MISMATCH    =   0x04,
+    SELF_RESTORE_VER_MISMATCH            =   0x08,
+} VersionIncompList_t;
+
 
 #ifdef __cplusplus
 extern "C" {
@@ -341,6 +357,14 @@ StopReturnCode_t proc_stop_save_cpureg(  void* const i_pImage,
  */
 StopReturnCode_t proc_stop_init_self_save(  void* const i_pImage, const uint32_t i_corePos );
 
+/**
+ * @brief       verifies if API is compatible of current HOMER image.
+ * @param[in]   i_pImage        points to the start of HOMER image of P9 chip.
+ * @param[out]  o_inCompVector  list of incompatibilities found.
+ * @return      STOP_SAVE_SUCCESS if if API succeeds, error code otherwise.
+ */
+StopReturnCode_t proc_stop_api_discover_capability( void* const i_pImage, uint64_t* o_inCompVector );
+
 #ifdef __cplusplus
 } // extern "C"
 };  // namespace stopImageSection ends
libpore/p9_stop_util.H +10 -10
index 79b4e959..1328a54b 100644
@@ -72,18 +72,18 @@ namespace stopImageSection
     ( (((WORD) >> 8) & 0x00FF) | (((WORD) << 8) & 0xFF00) )
 
 #define SWIZZLE_4_BYTE(WORD) \
-    ( (((WORD) >> 24) & 0x000000FF) | (((WORD) >>  8) & 0x0000FF00) | \
-      (((WORD) <<  8) & 0x00FF0000) | (((WORD) << 24) & 0xFF000000) )
+    ( (((WORD) & 0x000000FF) << 24) | (((WORD) & 0x0000FF00) <<  8) | \
+      (((WORD) & 0x00FF0000) >>  8) | (((WORD) & 0xFF000000) >> 24) )
 
 #define SWIZZLE_8_BYTE(WORD) \
-    ( (((WORD) >> 56) & 0x00000000000000FF) |  \
-      (((WORD) >> 40) & 0x000000000000FF00)| \
-      (((WORD) >> 24) & 0x0000000000FF0000) |  \
-      (((WORD) >>  8) & 0x00000000FF000000) |  \
-      (((WORD) <<  8) & 0x000000FF00000000) |  \
-      (((WORD) << 24) & 0x0000FF0000000000) | \
-      (((WORD) << 40) & 0x00FF000000000000) |  \
-      (((WORD) << 56) & 0xFF00000000000000) )
+    ( (((WORD) & 0x00000000000000ffULL) << 56) | \
+      (((WORD) & 0x000000000000ff00ULL) << 40) | \
+      (((WORD) & 0x0000000000ff0000ULL) << 24) | \
+      (((WORD) & 0x00000000ff000000ULL) <<  8) | \
+      (((WORD) & 0x000000ff00000000ULL) >>  8) | \
+      (((WORD) & 0x0000ff0000000000ULL) >> 24) | \
+      (((WORD) & 0x00ff000000000000ULL) >> 40) | \
+      (((WORD) & 0xff00000000000000ULL) >> 56) )
 #endif
 
 /**
Signature · 1 line
2.25.1
#5 lore raw

PATCH v8 3/3 Self save API integration

In reply to Pratik Rajesh Sampat

The commit makes the self save API available outside the firmware by defining an OPAL wrapper.
This wrapper has a similar interface to that of self restore and expects the cpu pir, SPR number, minus the value of that SPR to be passed in its paramters and returns OPAL_SUCCESS on success. It adds a device-tree node signifying support for self-save after verifying the stop API version compatibility.

The commit also documents both the self-save and the self-restore API calls along with their working and usage.

---
 doc/opal-api/opal-slw-self-save-reg-181.rst |  51 ++++++++++
 doc/opal-api/opal-slw-set-reg-100.rst       |   5 +
 doc/power-management.rst                    |  48 +++++++++
 hw/slw.c                                    | 106 ++++++++++++++++++++
 include/opal-api.h                          |   3 +-
 include/p9_stop_api.H                       |  18 ++++
 include/skiboot.h                           |   3 +
 7 files changed, 233 insertions(+), 1 deletion(-)
 create mode 100644 doc/opal-api/opal-slw-self-save-reg-181.rst
doc/opal-api/opal-slw-self-save-reg-181.rst new file +51 -0
new file mode 100644
index 00000000..f20e9b81
@@ -0,0 +1,51 @@
+.. OPAL_SLW_SELF_SAVE_REG:
+
+OPAL_SLW_SELF_SAVE_REG
+======================
+
+.. code-block:: c
+
+   #define OPAL_SLW_SELF_SAVE_REG			181
+
+   int64_t opal_slw_self_save_reg(uint64_t cpu_pir, uint64_t sprn);
+
+:ref:`OPAL_SLW_SELF_SAVE_REG` is used to inform low-level firmware to save
+the current contents of the SPR before entering a state of loss and
+also restore the content back on waking up from a deep stop state.
+
+An OPAL call `OPAL_SLW_SET_REG` exists which is similar in function as
+saving and restoring the SPR, with one difference being that the value of the
+SPR must also be supplied in the parameters.
+Complete reference: doc/opal-api/opal-slw-set-reg-100.rst
+
+Parameters
+----------
+
+``uint64_t cpu_pir``
+  This parameter specifies the pir of the cpu for which the call is being made.
+``uint64_t sprn``
+  This parameter specifies the spr number as mentioned in p9_stop_api.H
+  The list of SPRs supported is as follows.
+	P9_STOP_SPR_DAWR,
+	P9_STOP_SPR_HSPRG0,
+	P9_STOP_SPR_LDBAR,
+	P9_STOP_SPR_LPCR,
+	P9_STOP_SPR_PSSCR,
+	P9_STOP_SPR_MSR,
+	P9_STOP_SPR_HRMOR,
+	P9_STOP_SPR_HMEER,
+	P9_STOP_SPR_PMCR,
+	P9_STOP_SPR_PTCR
+
+  The property "ibm,opal-self-save" is supplied to the device tree to advterise
+  support.
+
+Returns
+-------
+
+:ref:`OPAL_UNSUPPORTED`
+  If spr restore is not supported by pore engine.
+:ref:`OPAL_PARAMETER`
+  Invalid handle for the pir/chip
+:ref:`OPAL_SUCCESS`
+  On success
doc/opal-api/opal-slw-set-reg-100.rst +5 -0
index 2e8f1bd6..ee3e68ce 100644
@@ -21,6 +21,11 @@ In Power 9, it uses p9_stop_save_cpureg(), api provided by self restore code,
 to inform the spr with their corresponding values with which they
 must be restored.
 
+An OPAL call `OPAL_SLW_SELF_SAVE_REG` exists which is similar in function
+saving and restoring the SPR, with one difference being that the value of the
+SPR doesn't need to be passed in the parameters, only with the SPR number
+the firmware can identify, save and restore the values for the same.
+Complete reference: doc/opal-api/opal-slw-self-save-reg-181.rst
 
 Parameters
 ----------
doc/power-management.rst +48 -0

51 lines not on this page · raw message

hw/slw.c +106 -0

135 lines not on this page · raw message

include/opal-api.h +2 -1
index e90cab1e..1607a89b 100644
@@ -227,7 +227,8 @@
 #define OPAL_SECVAR_ENQUEUE_UPDATE		178
 #define OPAL_PHB_SET_OPTION			179
 #define OPAL_PHB_GET_OPTION			180
-#define OPAL_LAST				180
+#define OPAL_SLW_SELF_SAVE_REG			181
+#define OPAL_LAST				181
 
 #define QUIESCE_HOLD			1 /* Spin all calls at entry */
 #define QUIESCE_REJECT			2 /* Fail all calls with OPAL_BUSY */
include/p9_stop_api.H +18 -0
index cb5ffd6f..09ce3dc1 100644
@@ -34,6 +34,8 @@
 ///
 /// @file   p9_stop_api.H
 /// @brief  describes STOP API which  create/manipulate STOP image.
+///         This header need not be consistent, however is a subset of the
+///         libpore/p9_stop_api.H counterpart
 ///
 // *HWP HW Owner    :  Greg Still <[email protected]>
 // *HWP FW Owner    :  Prem Shanker Jha <[email protected]>
@@ -58,6 +60,7 @@ typedef enum
     P9_STOP_SPR_HRMOR   =    313,   // core register
     P9_STOP_SPR_LPCR    =    318,   // thread register
     P9_STOP_SPR_HMEER   =    337,   // core register
+    P9_STOP_SPR_PTCR    =    464,   // core register
     P9_STOP_SPR_LDBAR   =    850,   // thread register
     P9_STOP_SPR_PSSCR   =    855,   // thread register
     P9_STOP_SPR_PMCR    =    884,   // core register
@@ -247,6 +250,21 @@ StopReturnCode_t p9_stop_save_scom( void* const   i_pImage,
                                     const ScomOperation_t i_operation,
                                     const ScomSection_t i_section );
 
+/**
+ * @brief       Facilitates self save and restore of a list of SPRs of a thread.
+ * @param[in]   i_pImage        points to the start of HOMER image of P9 chip.
+ * @param[in]   i_pir           PIR associated with thread
+ * @param[in]   i_saveRegVector bit vector representing SPRs that needs to be restored.
+ * @return      STOP_SAVE_SUCCESS if API succeeds, error code otherwise.
+ * @note        SPR save vector is a bit vector. For each SPR supported,
+ *              there is an associated bit position in the bit vector.Refer
+ *              to definition of SprBitPositionList_t to determine bit position
+ *              associated with a particular SPR.
+ */
+StopReturnCode_t
+p9_stop_save_cpureg_control( void* i_pImage, const uint64_t i_pir,
+                             const uint32_t  i_saveRegVector );
+
 /**
  * @brief       verifies if API is compatible of current HOMER image.
  * @param[in]   i_pImage        points to the start of HOMER image of P9 chip.
include/skiboot.h +3 -0
index 30ff500c..9ced240e 100644
@@ -306,6 +306,9 @@ extern void nx_p9_rng_late_init(void);
 /* SLW reinit function for switching core settings */
 extern int64_t slw_reinit(uint64_t flags);
 
+/* Self save SPR before entering the stop state */
+extern int64_t opal_slw_self_save_reg(uint64_t cpu_pir, uint64_t sprn);
+
 /* Patch SPR in SLW image */
 extern int64_t opal_slw_set_reg(uint64_t cpu_pir, uint64_t sprn, uint64_t val);
 
Signature · 1 line
2.25.1
Reviewed-by on v8 3/3
#6 lore raw

Re: [PATCH v8 3/3] Self save API integration

Quoted 8 lines · Pratik Rajesh Sampat

The commit makes the self save API available outside the firmware by defining an OPAL wrapper.
This wrapper has a similar interface to that of self restore and expects the cpu pir, SPR number, minus the value of that SPR to be passed in its paramters and returns OPAL_SUCCESS on success. It adds a device-tree node signifying support for self-save after verifying the stop API version compatibility.

Quoted 4 lines

The commit also documents both the self-save and the self-restore API calls along with their working and usage.

LGTM.

Quoted 357 lines
---
 doc/opal-api/opal-slw-self-save-reg-181.rst |  51 ++++++++++
 doc/opal-api/opal-slw-set-reg-100.rst       |   5 +
 doc/power-management.rst                    |  48 +++++++++
 hw/slw.c                                    | 106 ++++++++++++++++++++
 include/opal-api.h                          |   3 +-
 include/p9_stop_api.H                       |  18 ++++
 include/skiboot.h                           |   3 +
 7 files changed, 233 insertions(+), 1 deletion(-)
 create mode 100644 doc/opal-api/opal-slw-self-save-reg-181.rst
doc/opal-api/opal-slw-self-save-reg-181.rst new file +51 -0
new file mode 100644
index 00000000..f20e9b81
@@ -0,0 +1,51 @@
+.. OPAL_SLW_SELF_SAVE_REG:
+
+OPAL_SLW_SELF_SAVE_REG
+======================
+
+.. code-block:: c
+
+   #define OPAL_SLW_SELF_SAVE_REG			181
+
+   int64_t opal_slw_self_save_reg(uint64_t cpu_pir, uint64_t sprn);
+
+:ref:`OPAL_SLW_SELF_SAVE_REG` is used to inform low-level firmware to save
+the current contents of the SPR before entering a state of loss and
+also restore the content back on waking up from a deep stop state.
+
+An OPAL call `OPAL_SLW_SET_REG` exists which is similar in function as
+saving and restoring the SPR, with one difference being that the value of the
+SPR must also be supplied in the parameters.
+Complete reference: doc/opal-api/opal-slw-set-reg-100.rst
+
+Parameters
+----------
+
+``uint64_t cpu_pir``
+  This parameter specifies the pir of the cpu for which the call is being made.
+``uint64_t sprn``
+  This parameter specifies the spr number as mentioned in p9_stop_api.H
+  The list of SPRs supported is as follows.
+	P9_STOP_SPR_DAWR,
+	P9_STOP_SPR_HSPRG0,
+	P9_STOP_SPR_LDBAR,
+	P9_STOP_SPR_LPCR,
+	P9_STOP_SPR_PSSCR,
+	P9_STOP_SPR_MSR,
+	P9_STOP_SPR_HRMOR,
+	P9_STOP_SPR_HMEER,
+	P9_STOP_SPR_PMCR,
+	P9_STOP_SPR_PTCR
+
+  The property "ibm,opal-self-save" is supplied to the device tree to advterise
+  support.
+
+Returns
+-------
+
+:ref:`OPAL_UNSUPPORTED`
+  If spr restore is not supported by pore engine.
+:ref:`OPAL_PARAMETER`
+  Invalid handle for the pir/chip
+:ref:`OPAL_SUCCESS`
+  On success
doc/opal-api/opal-slw-set-reg-100.rst +5 -0
index 2e8f1bd6..ee3e68ce 100644
@@ -21,6 +21,11 @@ In Power 9, it uses p9_stop_save_cpureg(), api provided by self restore code,
 to inform the spr with their corresponding values with which they
 must be restored.
 
+An OPAL call `OPAL_SLW_SELF_SAVE_REG` exists which is similar in function
+saving and restoring the SPR, with one difference being that the value of the
+SPR doesn't need to be passed in the parameters, only with the SPR number
+the firmware can identify, save and restore the values for the same.
+Complete reference: doc/opal-api/opal-slw-self-save-reg-181.rst
 
 Parameters
 ----------
doc/power-management.rst +48 -0
index 76491a71..d6bd5358 100644
@@ -15,3 +15,51 @@ On boot, specific stop states can be disabled via setting a mask. For example,
 to disable all but stop 0,1,2, use ~0xE0000000. ::
 
   nvram -p ibm,skiboot --update-config opal-stop-state-disable-mask=0x1FFFFFFF
+
+Saving and restoring Special Purpose Registers(SPRs)
+----------------------------------------------------
+
+When a CPU wakes up from a deep stop state which can result in
+hypervisor state loss, all the SPRs are lost. The Linux Kernel expects
+a small set of SPRs to contain an expected value when the CPU wakes up
+from such a deep stop state. The microcode firmware provides the
+following two APIs, collectively known as the stop-APIs, to allow the
+kernel/OPAL to specify this set of SPRs and the value that they need
+to be restored with on waking up from a deep stop state.
+
+Self-restore:
+int64_t opal_slw_set_reg(uint64_t cpu_pir, uint64_t sprn, uint64_t val);
+The SPR number and the value of the that SPR must be restored with on
+wakeup from the deep-stop state must be specified. When this call is
+made, the microcode inserts instruction into the HOMER region to
+restore the content of the SPR to the specified value on wakeup from a
+deep-stop state. These instructions are executed by the CPU as soon as
+it wakes up from a deep stop state. The call is to be made once per
+SPR.
+
+Self-Save:
+int64_t opal_slw_self_save_reg(uint64_t cpu_pir, uint64_t sprn);
+Only the SPR number needs to be specified. When this call is made, the
+microcode inserts instructions into the HOMER region to save the
+current value of the SPR before the CPU goes to a deep stop state, and
+restores the value back when the CPU wakes up from a deep stop state.
+These instructions are correspondingly executed just before and after
+the CPU goes/comes out of a deep stop state. This call can be made
+once per SPR.
+
+The key difference between self-save and self-restore is the
+use-case. If the Kernel expects the SPR to contain a particular value
+on waking up from a deep-stop state, that wasn't the value of that SPR
+before entering deep stop-state, then self-restore is preferable.
+Also in a case where SPR does not change across the lifetime
+self-restore is more efficient as when the value is same the memeory location
+is not updated.
+
+When deep stop states are to be supported in an Ultravisor
+environment, since HOMER is in a secure region, the stop-api cannot
+update the HOMER if invoked from a context when the OPAL/Kernel is
+executing without the ultravisor privilege. In this scenario, at the
+time of early OPAL boot, while OPAL has ultravisor privileges, it can
+make the self-save stop-api call for all the supported SPRs, so that
+the microcode in the HOMER will always save and restore all the
+supported SPRs during entry/exit from a deep stop state.
hw/slw.c +106 -0
index beb129a8..8423eeaf 100644
@@ -35,6 +35,43 @@ static bool slw_current_le = false;
 enum wakeup_engine_states wakeup_engine_state = WAKEUP_ENGINE_NOT_PRESENT;
 bool has_deep_states = false;
 
+/**
+ * The struct and SPR list is partially consistent with libpore/p9_stop_api.c
+ */
+/**
+ * @brief summarizes attributes associated with a SPR register.
+ */
+typedef struct
+{
+    uint32_t iv_sprId;
+    bool     iv_isThreadScope;
+    uint32_t iv_saveMaskPos;
+
+} StopSprReg_t;
+
+/**
+ * @brief a true in the table below means register is of scope thread
+ * whereas a false meanse register is of scope core.
+ * The number is the bit position on a uint32_t mask
+ */
+
+static const StopSprReg_t g_sprRegister[] =
+{
+	{ P9_STOP_SPR_DAWR,      true,  1   },
+	{ P9_STOP_SPR_HSPRG0,    true,  3   },
+	{ P9_STOP_SPR_LDBAR,     true,  4,  },
+	{ P9_STOP_SPR_LPCR,      true,  5   },
+	{ P9_STOP_SPR_PSSCR,     true,  6   },
+	{ P9_STOP_SPR_MSR,       true,  7   },
+	{ P9_STOP_SPR_HRMOR,     false, 255 },
+	{ P9_STOP_SPR_HID,       false, 21  },
+	{ P9_STOP_SPR_HMEER,     false, 22  },
+	{ P9_STOP_SPR_PMCR,      false, 23  },
+	{ P9_STOP_SPR_PTCR,      false, 24  },
+};
+
+static const uint32_t MAX_SPR_SUPPORTED	= ARRAY_SIZE(g_sprRegister);
+
 DEFINE_LOG_ENTRY(OPAL_RC_SLW_INIT, OPAL_PLATFORM_ERR_EVT, OPAL_SLW,
 		 OPAL_PLATFORM_FIRMWARE, OPAL_PREDICTIVE_ERR_GENERAL,
 		 OPAL_NA);
@@ -720,11 +757,14 @@ void add_cpu_idle_state_properties(void)
 	struct cpu_idle_states *states;
 	struct proc_chip *chip;
 	int nr_states;
+	int rc;
 
 	bool can_sleep = true;
 	bool has_stop_inst = false;
+	bool has_self_save = true;
 	u8 i;
 
+	u64 compVector = -1;
 	fdt64_t *pm_ctrl_reg_val_buf;
 	fdt64_t *pm_ctrl_reg_mask_buf;
 	u32 supported_states_mask;
@@ -766,6 +806,20 @@ void add_cpu_idle_state_properties(void)
 	 */
 	chip = next_chip(NULL);
 	assert(chip);
+	rc = proc_stop_api_discover_capability((void *) chip->homer_base,
+					       &compVector);
+	if (rc == STOP_SAVE_ARG_INVALID_IMG) {
+		prlog(PR_DEBUG, "HOMER BASE INVALID\n");
+		return;
+	} else if (rc == STOP_SAVE_API_IMG_INCOMPATIBLE) {
+		prlog(PR_DEBUG, "STOP API running incompatible versions\n");
+		if ((compVector & SELF_RESTORE_VER_MISMATCH) == 0) {
+			prlog(PR_DEBUG, "Self-save API unsupported\n");
+			has_self_save = false;
+		}
+	}
+	if (has_self_save)
+		dt_add_property(power_mgt, "ibm,opal-self-save", NULL, 0);
 	if (chip->type == PROC_CHIP_P9_NIMBUS ||
 	    chip->type == PROC_CHIP_P9_CUMULUS ||
 	    chip->type == PROC_CHIP_P9P) {
@@ -1446,6 +1500,58 @@ int64_t opal_slw_set_reg(uint64_t cpu_pir, uint64_t sprn, uint64_t val)
 
 opal_call(OPAL_SLW_SET_REG, opal_slw_set_reg, 3);
 
+int64_t opal_slw_self_save_reg(uint64_t cpu_pir, uint64_t sprn)
+{
+	struct cpu_thread * c = find_cpu_by_pir(cpu_pir);
+	uint32_t save_reg_vector = 0;
+	struct proc_chip * chip;
+	int rc;
+	int index;
+
+	if (!c) {
+		prlog(PR_DEBUG, "SLW: Unknown thread with pir %x\n",
+		      (u32) cpu_pir);
+		return OPAL_PARAMETER;
+	}
+
+	chip = get_chip(c->chip_id);
+	if (!chip) {
+		prlog(PR_DEBUG, "SLW: Unknown chip for thread with pir %x\n",
+		      (u32) cpu_pir);
+		return OPAL_PARAMETER;
+	}
+	if (proc_gen != proc_gen_p9 || !has_deep_states) {
+		prlog(PR_DEBUG, "SLW: Self-save feature unsupported\n");
+		return OPAL_UNSUPPORTED;
+	}
+	if (wakeup_engine_state != WAKEUP_ENGINE_PRESENT) {
+		log_simple_error(&e_info(OPAL_RC_SLW_REG),
+			"SLW: wakeup_engine in bad state=%d chip=%x\n",
+			wakeup_engine_state, chip->id);
+		return OPAL_INTERNAL_ERROR;
+	}
+	for (index = 0; index < MAX_SPR_SUPPORTED; ++index) {
+		if (sprn == (CpuReg_t) g_sprRegister[index].iv_sprId) {
+			save_reg_vector = PPC_BIT32(
+				g_sprRegister[index].iv_saveMaskPos);
+			break;
+		}
+	}
+	if (save_reg_vector == 0)
+		return OPAL_INTERNAL_ERROR;
+	rc = p9_stop_save_cpureg_control((void *) chip->homer_base,
+						cpu_pir, save_reg_vector);
+
+	if (rc) {
+		log_simple_error(&e_info(OPAL_RC_SLW_REG),
+			"SLW: Failed to save vector %x for CPU %x\n",
+			save_reg_vector, c->pir);
+		return OPAL_INTERNAL_ERROR;
+	}
+	return OPAL_SUCCESS;
+}
+opal_call(OPAL_SLW_SELF_SAVE_REG, opal_slw_self_save_reg, 2);
+
 void slw_init(void)
 {
 	struct proc_chip *chip;
include/opal-api.h +2 -1
index e90cab1e..1607a89b 100644
@@ -227,7 +227,8 @@
 #define OPAL_SECVAR_ENQUEUE_UPDATE		178
 #define OPAL_PHB_SET_OPTION			179
 #define OPAL_PHB_GET_OPTION			180
-#define OPAL_LAST				180
+#define OPAL_SLW_SELF_SAVE_REG			181
+#define OPAL_LAST				181
 
 #define QUIESCE_HOLD			1 /* Spin all calls at entry */
 #define QUIESCE_REJECT			2 /* Fail all calls with OPAL_BUSY */
include/p9_stop_api.H +18 -0
index cb5ffd6f..09ce3dc1 100644
@@ -34,6 +34,8 @@
 ///
 /// @file   p9_stop_api.H
 /// @brief  describes STOP API which  create/manipulate STOP image.
+///         This header need not be consistent, however is a subset of the
+///         libpore/p9_stop_api.H counterpart
 ///
 // *HWP HW Owner    :  Greg Still <[email protected]>
 // *HWP FW Owner    :  Prem Shanker Jha <[email protected]>
@@ -58,6 +60,7 @@ typedef enum
     P9_STOP_SPR_HRMOR   =    313,   // core register
     P9_STOP_SPR_LPCR    =    318,   // thread register
     P9_STOP_SPR_HMEER   =    337,   // core register
+    P9_STOP_SPR_PTCR    =    464,   // core register
     P9_STOP_SPR_LDBAR   =    850,   // thread register
     P9_STOP_SPR_PSSCR   =    855,   // thread register
     P9_STOP_SPR_PMCR    =    884,   // core register
@@ -247,6 +250,21 @@ StopReturnCode_t p9_stop_save_scom( void* const   i_pImage,
                                     const ScomOperation_t i_operation,
                                     const ScomSection_t i_section );
 
+/**
+ * @brief       Facilitates self save and restore of a list of SPRs of a thread.
+ * @param[in]   i_pImage        points to the start of HOMER image of P9 chip.
+ * @param[in]   i_pir           PIR associated with thread
+ * @param[in]   i_saveRegVector bit vector representing SPRs that needs to be restored.
+ * @return      STOP_SAVE_SUCCESS if API succeeds, error code otherwise.
+ * @note        SPR save vector is a bit vector. For each SPR supported,
+ *              there is an associated bit position in the bit vector.Refer
+ *              to definition of SprBitPositionList_t to determine bit position
+ *              associated with a particular SPR.
+ */
+StopReturnCode_t
+p9_stop_save_cpureg_control( void* i_pImage, const uint64_t i_pir,
+                             const uint32_t  i_saveRegVector );
+
 /**
  * @brief       verifies if API is compatible of current HOMER image.
  * @param[in]   i_pImage        points to the start of HOMER image of P9 chip.
include/skiboot.h +3 -0
index 30ff500c..9ced240e 100644
@@ -306,6 +306,9 @@ extern void nx_p9_rng_late_init(void);
 /* SLW reinit function for switching core settings */
 extern int64_t slw_reinit(uint64_t flags);
 
+/* Self save SPR before entering the stop state */
+extern int64_t opal_slw_self_save_reg(uint64_t cpu_pir, uint64_t sprn);
+
 /* Patch SPR in SLW image */
 extern int64_t opal_slw_set_reg(uint64_t cpu_pir, uint64_t sprn, uint64_t val);
 
Signature · 1 line
2.25.1
Reviewed-by on v8 1/3
#2 lore raw

Re: [Skiboot] [PATCH v8 1/3] Self Save: Introducing Support for SPR Self Save

In reply to Pratik Rajesh Sampat
Quoted 26 lines · Pratik Rajesh Sampat

From: Prem Shanker Jha <[email protected]>

The commit is a merger of commits that makes the following changes:
1. Commit fixes some issues with code found during integration test

  -  replacement of addi with xor instruction during self save API.
  -  fixing instruction generation for MFMSR during self save
  -  data struct updates in STOP API
  -  error RC updates for hcode image build
  -  HOMER parser updates.
  -  removed self save support for URMOR and HRMOR
  -  code changes for compilation with OPAL
  -  populating CME Image header with unsecure HOMER address.

Key_Cronus_Test=PM_REGRESS

Change-Id: I7cedcc466267c4245255d8d75c01ed695e316720
Reviewed-on: http://rchgit01.rchland.ibm.com/gerrit1/66580

Quoted 5 lines

Reviewed-on: http://rchgit01.rchland.ibm.com/gerrit1/66587

Quoted 23 lines

2. The commit also incorporates changes that make STOP API project agnostic changes include defining wrapper functions which call legacy API. It also adds duplicate enum members which start with prefix PROC instead of P9.

Key_Cronus_Test=PM_REGRESS

Change-Id: If87970f3e8cf9b507f33eb1be249e03eb3836a5e
RTC: 201128
Reviewed-on: http://rchgit01.rchland.ibm.com/gerrit1/71307

Reviewed-on: http://rchgit01.rchland.ibm.com/gerrit1/71314

Quoted 8 lines
---
 include/p9_stop_api.H                    |  79 +-
 libpore/p9_cpu_reg_restore_instruction.H |   4 +
 libpore/p9_stop_api.C                    | 954 +++++++++++++----------
 libpore/p9_stop_api.H                    | 115 ++-
 libpore/p9_stop_data_struct.H            |   4 +-
 libpore/p9_stop_util.H                   |   7 +-
 6 files changed, 721 insertions(+), 442 deletions(-)

These code changes are from hcode component copied as is into OPAL project for integration and use of stop-api.

Hcode component in cooperation with hostboot would be loaded in memory before OPAL is loaded. This code will allow runtime changes and usage of various power management save restore functions.

This patch specifically enables self-save feature by the microcode.

--Vaidy

Reviewed-by on v8 2/3
#4 lore raw

Re: [PATCH v8 2/3] API to verify the STOP API and image compatibility

In reply to Pratik Rajesh Sampat
Quoted 15 lines · Pratik Rajesh Sampat

From: Prem Shanker Jha <[email protected]>

Commit defines a new API primarily intended for OPAL to determine cpu register save API's compatibility with HOMER layout and self save restore. It can help OPAL determine if version of API integrated with OPAL is different from hostboot.

Change-Id: Ic0de45a336cfb8b6b6096a10ac1cd3ffbaa44fc0
Reviewed-on: http://rchgit01.rchland.ibm.com/gerrit1/77612

Quoted 5 lines

Reviewed-on: http://rchgit01.rchland.ibm.com/gerrit1/77614

Quoted 8 lines
---
 include/p9_stop_api.H                    | 25 ++++++++++
 libpore/p9_cpu_reg_restore_instruction.H |  7 ++-
 libpore/p9_hcd_memmap_base.H             |  7 +++
 libpore/p9_stop_api.C                    | 58 +++++++++++++++++++++++-
 libpore/p9_stop_api.H                    | 26 ++++++++++-
 libpore/p9_stop_util.H                   | 20 ++++----
 6 files changed, 130 insertions(+), 13 deletions(-)

This stop-api code will help OPAL check and use self-save functions so that different versions of OPAL can be loaded and run with different versions of low level firmware stack.

--Vaidy

Reviewed-by on v8 3/3
#7 lore raw

Re: [Skiboot] [PATCH v8 3/3] Self save API integration

In reply to Pratik Rajesh Sampat
Quoted 8 lines · Pratik Rajesh Sampat

The commit makes the self save API available outside the firmware by defining an OPAL wrapper.
This wrapper has a similar interface to that of self restore and expects the cpu pir, SPR number, minus the value of that SPR to be passed in its paramters and returns OPAL_SUCCESS on success. It adds a device-tree node signifying support for self-save after verifying the stop API version compatibility.

Quoted 4 lines

The commit also documents both the self-save and the self-restore API calls along with their working and usage.

Quoted 10 lines
---
 doc/opal-api/opal-slw-self-save-reg-181.rst |  51 ++++++++++
 doc/opal-api/opal-slw-set-reg-100.rst       |   5 +
 doc/power-management.rst                    |  48 +++++++++
 hw/slw.c                                    | 106 ++++++++++++++++++++
 include/opal-api.h                          |   3 +-
 include/p9_stop_api.H                       |  18 ++++
 include/skiboot.h                           |   3 +
 7 files changed, 233 insertions(+), 1 deletion(-)
 create mode 100644 doc/opal-api/opal-slw-self-save-reg-181.rst

This patch enables OPAL interface to call stop-api and set self-save. Basically completes the infrastructure required to use the new self-save function provided by the microcode.

--Vaidy