powerpc/papr_scm: Add support for reporting nvdimm health
Subsystems (3)
v6 cover powerpc/papr_scm: Add support for reporting nvdimm health
The PAPR standard[1][3] provides mechanisms to query the health and performance stats of an NVDIMM via various hcalls as described in Ref[2]. Until now these stats were never available nor exposed to the user-space tools like 'ndctl'. This is partly due to PAPR platform not having support for ACPI and NFIT. Hence 'ndctl' is unable to query and report the dimm health status and a user had no way to determine the current health status of a NDVIMM.
To overcome this limitation, this patch-set updates papr_scm kernel module to query and fetch nvdimm health stats using hcalls described in Ref[2]. This health and performance stats are then exposed to userspace via syfs and PAPR-nvDimm-Specific-Methods(PDSM) issued by libndctl.
These changes coupled with proposed ndtcl changes located at Ref[4] should provide a way for the user to retrieve NVDIMM health status using ndtcl.
Below is a sample output using proposed kernel + ndctl for PAPR NVDIMM in a emulation environment:
# ndctl list -DH
[
{
"dev":"nmem0",
"health":{
"health_state":"fatal",
"shutdown_state":"dirty"
}
}
]Dimm health report output on a pseries guest lpar with vPMEM or HMS based nvdimms that are in perfectly healthy conditions:
# ndctl list -d nmem0 -H
[
{
"dev":"nmem0",
"health":{
"health_state":"ok",
"shutdown_state":"clean"
}
}
]PAPR nvDimm-Specific-Methods(PDSM)
==================================
PDSM requests are issued by vendor specific code in libndctl to execute certain operations or fetch information from NVDIMMS. PDSMs requests can be sent to papr_scm module via libndctl(userspace) and libnvdimm (kernel) using the ND_CMD_CALL ioctl command which can be handled in the dimm control function papr_scm_ndctl(). Current patchset proposes a single PDSM to retrieve NVDIMM health, defined in the newly introduced uapi header named 'papr_scm_pdsm.h'. Support for more PDSMs will be added in future.
Structure of the patch-set
==========================
The patchset starts with implementing support for fetching nvdimm health information from PHYP and partially exposing it to user-space via a nvdimm sysfs flag.
Second & Third patches deal with implementing support for servicing PDSM commands in papr_scm module.
Finally the Fourth patch implements support for servicing PDSM 'PAPR_SCM_PDSM_HEALTH' that returns the nvdimm health information to libndctl.
Changelog:
==========
v5..v6:
* Incorporate review comments from Mpe and Dan Williams.
* Changed the usage of term DSM to PDSM as former conflicted with usage in ACPI context.
* UAPI updates to remove usage of bool and marking the structs defined as 'packed'.
* Simplified the health-bitmap handling in papr_scm to use u64 instead of __be64 integers.
* Caching of the health information so reading the dimm-flag file doesn't result in costly hcalls everytime.
* Changed dimm-flag 'save_fail' to 'flush_fail'
* Moved the dimm flag file from 'papr_flags' to 'papr/flags'.
* Added a patch to document H_SCM_HEALTH hcall return values.
* Added sysfs ABI documentation for newly introduce dimm-flag sysfs file 'papr/flags'
v4..v5:
* Fixed a bug in new implementation of papr_scm_ndctl() that was triggering a false error condition.
v3..v4:
* Restructured papr_scm_ndctl() to dispatch ND_CMD_CALL commands to a new function named papr_scm_service_dsm() to serivice DSM requests. [Aneesh]
v2..v3:
* Updated the papr_scm_dsm.h header to be more confimant general kernel guidelines for UAPI headers. [Aneesh]
* Changed the definition of macro PAPR_SCM_DIMM_UNARMED_MASK to not include case when the nvdimm is unarmed because its a vPMEM nvdimm. [Aneesh]
v1..v2:
* Restructured the patch-set based on review comments on V1 patch-set to simplify the patch review. Multiple small patches have been combined into single patches to reduce cross referencing that was needed in earlier patch-set. Hence most of the patches in this patch-set as now new. [Aneesh]
* Removed the initial work done for fetch nvdimm performance statistics. These changes will be re-proposed in a separate patch-set. [Aneesh]
* Simplified handling of versioning of 'struct nd_papr_scm_dimm_health_stat_v1' as only one version of the structure is currently in existence.
References:
[1] "Power Architecture Platform Reference"
https://en.wikipedia.org/wiki/Power_Architecture_Platform_Reference
[2] commit 58b278f568f0
("powerpc: Provide initial documentation for PAPR hcalls")
[3] "Linux on Power Architecture Platform Reference"
https://members.openpowerfoundation.org/document/dl/469
[4] https://patchwork.kernel.org/project/linux-nvdimm/list/?series=244625
Vaibhav Jain (4):
powerpc: Document details on H_SCM_HEALTH hcall powerpc/papr_scm: Fetch nvdimm health information from PHYP ndctl/papr_scm,uapi: Add support for PAPR nvdimm specific methods powerpc/papr_scm: Implement support for PAPR_SCM_PDSM_HEALTH
Documentation/ABI/testing/sysfs-bus-papr-scm | 27 ++ Documentation/powerpc/papr_hcalls.rst | 43 ++- arch/powerpc/include/asm/papr_scm.h | 49 +++ arch/powerpc/include/uapi/asm/papr_scm_pdsm.h | 192 +++++++++++ arch/powerpc/platforms/pseries/papr_scm.c | 317 +++++++++++++++++- include/uapi/linux/ndctl.h | 1 + 6 files changed, 617 insertions(+), 12 deletions(-) create mode 100644 Documentation/ABI/testing/sysfs-bus-papr-scm create mode 100644 arch/powerpc/include/asm/papr_scm.h create mode 100644 arch/powerpc/include/uapi/asm/papr_scm_pdsm.h
Signature · 4 lines
2.25.3 _______________________________________________ Linux-nvdimm mailing list -- [email protected] To unsubscribe send an email to [email protected]
PATCH v6 1/4 powerpc: Document details on H_SCM_HEALTH hcall
Add documentation to 'papr_hcalls.rst' describing the bitmap flags that are returned from H_SCM_HEALTH hcall as per the PAPR-SCM specification.
---
Changelog:
v5..v6
* New patch in the series
--- Documentation/powerpc/papr_hcalls.rst | 43 ++++++++++++++++++++++++--- 1 file changed, 39 insertions(+), 4 deletions(-)
Documentation/powerpc/papr_hcalls.rst +39 -4
@@ -220,13 +220,48 @@ from the LPAR memory. **H_SCM_HEALTH** | Input: drcIndex -| Out: *health-bitmap, health-bit-valid-bitmap* +| Out: *health-bitmap (r4), health-bit-valid-bitmap (r5)* | Return Value: *H_Success, H_Parameter, H_Hardware* Given a DRC Index return the info on predictive failure and overall health of -the NVDIMM. The asserted bits in the health-bitmap indicate a single predictive -failure and health-bit-valid-bitmap indicate which bits in health-bitmap are -valid. +the NVDIMM. The asserted bits in the health-bitmap indicate one or more states +(described in table below) of the NVDIMM and health-bit-valid-bitmap indicate +which bits in health-bitmap are valid. + +Health Bitmap Flags: + ++------+-----------------------------------------------------------------------+ +| Bit | Definition | ++======+=======================================================================+ +| 00 | SCM device is unable to persist memory contents. | +| | If the system is powered down, nothing will be saved. | ++------+-----------------------------------------------------------------------+ +| 01 | SCM device failed to persist memory contents. Either contents were not| +| | saved successfully on power down or were not restored properly on | +| | power up. | ++------+-----------------------------------------------------------------------+ +| 02 | SCM device contents are persisted from previous IPL. The data from | +| | the last boot were successfully restored. | ++------+-----------------------------------------------------------------------+ +| 03 | SCM device contents are not persisted from previous IPL. There was no | +| | data to restore from the last boot. | ++------+-----------------------------------------------------------------------+ +| 04 | SCM device memory life remaining is critically low | ++------+-----------------------------------------------------------------------+ +| 05 | SCM device will be garded off next IPL due to failure | ++------+-----------------------------------------------------------------------+ +| 06 | SCM contents cannot persist due to current platform health status. A | +| | hardware failure may prevent data from being saved or restored. | ++------+-----------------------------------------------------------------------+ +| 07 | SCM device is unable to persist memory contents in certain conditions | ++------+-----------------------------------------------------------------------+ +| 08 | SCM device is encrypted | ++------+-----------------------------------------------------------------------+ +| 09 | SCM device has successfully completed a requested erase or secure | +| | erase procedure. | ++------+-----------------------------------------------------------------------+ +|10:63 | Reserved / Unused | ++------+-----------------------------------------------------------------------+ **H_SCM_PERFORMANCE_STATS**
Signature · 4 lines
2.25.3 _______________________________________________ Linux-nvdimm mailing list -- [email protected] To unsubscribe send an email to [email protected]
Michael Ellerman
reviewerRe: [PATCH v6 1/4] powerpc: Document details on H_SCM_HEALTH hcall
Quoted 35 lines · Vaibhav Jain
Add documentation to '
papr_hcalls.rst' describing the bitmap flags that are returned fromH_SCM_HEALTHhcall as per the PAPR-SCM specification.Cc Dan Williams <[email protected]>Cc Michael Ellerman <[email protected]>Cc "Aneesh Kumar K . V" <[email protected]>Signed-off-by Vaibhav Jain <[email protected]>---
Changelog:v5..v6
* New patch in the series--- Documentation/powerpc/papr_hcalls.rst | 43 ++++++++++++++++++++++++--- 1 file changed, 39 insertions(+), 4 deletions(-)Documentation/powerpc/papr_hcalls.rst +3
-4@@ -220,13 +220,48 @@ from the LPAR memory. **H_SCM_HEALTH** | Input: drcIndex -| Out: *health-bitmap, health-bit-valid-bitmap* +| Out: *health-bitmap (r4), health-bit-valid-bitmap (r5)* | Return Value: *H_Success, H_Parameter, H_Hardware* Given a DRC Index return the info on predictive failure and overall health of -the NVDIMM. The asserted bits in the health-bitmap indicate a single predictive -failure and health-bit-valid-bitmap indicate which bits in health-bitmap are -valid. +the NVDIMM. The asserted bits in the health-bitmap indicate one or more states +(described in table below) of the NVDIMM and health-bit-valid-bitmap indicate
Documentation/powerpc/papr_hcalls.rst from the LPAR memory. · line 238
Quoted 10 lines
Documentation/powerpc/papr_hcalls.rst +10
-0+which bits in health-bitmap are valid. + +Health Bitmap Flags: + ++------+-----------------------------------------------------------------------+ +| Bit | Definition | ++======+=======================================================================+ +| 00 | SCM device is unable to persist memory contents. | +| | If the system is powered down, nothing will be saved. | ++------+-----------------------------------------------------------------------+
Are these correct bit numbers or backward IBM big endian bit numbers?
ie. which bit is LSB?
cheers
_______________________________________________
Linux-nvdimm mailing list -- [email protected] To unsubscribe send an email to [email protected]
Re: [PATCH v6 1/4] powerpc: Document details on H_SCM_HEALTH hcall
Thanks for looking into this patch Mpe,
Quoted 50 lines · Michael Ellerman
Quoted 35 lines · Vaibhav Jain
Add documentation to '
papr_hcalls.rst' describing the bitmap flags that are returned fromH_SCM_HEALTHhcall as per the PAPR-SCM specification.Cc Dan Williams <[email protected]>Cc Michael Ellerman <[email protected]>Cc "Aneesh Kumar K . V" <[email protected]>Signed-off-by Vaibhav Jain <[email protected]>---
Changelog:v5..v6
* New patch in the series--- Documentation/powerpc/papr_hcalls.rst | 43 ++++++++++++++++++++++++--- 1 file changed, 39 insertions(+), 4 deletions(-)Documentation/powerpc/papr_hcalls.rst +3
-4@@ -220,13 +220,48 @@ from the LPAR memory. **H_SCM_HEALTH** | Input: drcIndex -| Out: *health-bitmap, health-bit-valid-bitmap* +| Out: *health-bitmap (r4), health-bit-valid-bitmap (r5)* | Return Value: *H_Success, H_Parameter, H_Hardware* Given a DRC Index return the info on predictive failure and overall health of -the NVDIMM. The asserted bits in the health-bitmap indicate a single predictive -failure and health-bit-valid-bitmap indicate which bits in health-bitmap are -valid. +the NVDIMM. The asserted bits in the health-bitmap indicate one or more states +(described in table below) of the NVDIMM and health-bit-valid-bitmap indicateDocumentation/powerpc/papr_hcalls.rst from the LPAR memory. · line 238
Quoted 10 lines
Documentation/powerpc/papr_hcalls.rst +10
-0+which bits in health-bitmap are valid. + +Health Bitmap Flags: + ++------+-----------------------------------------------------------------------+ +| Bit | Definition | ++======+=======================================================================+ +| 00 | SCM device is unable to persist memory contents. | +| | If the system is powered down, nothing will be saved. | ++------+-----------------------------------------------------------------------+Are these correct bit numbers or backward IBM big endian bit numbers?
Quoted 1 line
ie. which bit is LSB?
These bit numbers index to a 64-bit dword laid in IBM big endian format. So LSB would be at the located at a higher address. For example 0xC400000000000000 indicates bits 0, 1, and 5 are valid.
Quoted 5 lines
cheers
_______________________________________________
Linux-nvdimm mailing list -- [email protected] To unsubscribe send an email to [email protected]
_______________________________________________
Linux-nvdimm mailing list -- [email protected] To unsubscribe send an email to [email protected]
PATCH v6 2/4 powerpc/papr_scm: Fetch nvdimm health information from PHYP
In reply to Vaibhav JainImplement support for fetching nvdimm health information via H_SCM_HEALTH hcall as documented in Ref[1]. The hcall returns a pair of 64-bit big-endian integers, bitwise-and of which is then stored in 'struct papr_scm_priv' and subsequently partially exposed to user-space via newly introduced dimm specific attribute 'papr/flags'. Since the hcall is costly, the health information is cached and only re-queried, 60s after the previous successful hcall.
The patch also adds a new asm header named 'papr_scm.h' describing the interface between PHYP and guest kernel. A documentation text describing flags reported by the the new sysfs attribute 'papr/flags' is also introduced at Documentation/ABI/testing/sysfs-bus-papr-scm.
[1] commit 58b278f568f0 ("powerpc: Provide initial documentation for PAPR hcalls")
---
Changelog
v5..v6 :
* Change the flags sysfs attribute from 'papr_flags' to 'papr/flags' [Dan Williams]
* Include documentation for 'papr/flags' attr [Dan Williams]
* Change flag 'save_fail' to 'flush_fail' [Dan Williams]
* Caching of health bitmap to reduce expensive hcalls [Dan Williams]
* Removed usage of PPC_BIT from 'papr-scm.h' header [Mpe]
* Replaced two __be64 integers from papr_scm_priv to a single u64 integer [Mpe]
* Updated patch description to reflect the changes made in this version.
* Removed avoidable usage of 'papr_scm_priv.dimm_mutex' from flags_show() [Dan Williams]
v4..v5 :
* None
v3..v4 :
* None
v2..v3 :
* Removed PAPR_SCM_DIMM_HEALTH_NON_CRITICAL as a condition for NVDIMM unarmed [Aneesh]
v1..v2 :
* New patch in the series.
--- Documentation/ABI/testing/sysfs-bus-papr-scm | 27 ++++ arch/powerpc/include/asm/papr_scm.h | 49 ++++++++ arch/powerpc/platforms/pseries/papr_scm.c | 126 ++++++++++++++++++- 3 files changed, 200 insertions(+), 2 deletions(-) create mode 100644 Documentation/ABI/testing/sysfs-bus-papr-scm create mode 100644 arch/powerpc/include/asm/papr_scm.h
Documentation/ABI/testing/sysfs-bus-papr-scm new file +27 -0
@@ -0,0 +1,27 @@ +What: /sys/bus/nd/devices/nmemX/papr/flags +Date: Apr, 2020 +KernelVersion: v5.8 +Contact: linuxppc-dev <[email protected]>, [email protected], +Description: + (RO) Report flags indicating various states of a + papr-scm NVDIMM device. Each flag maps to a one or + more bits set in the dimm-health-bitmap retrieved in + response to H_SCM_HEALTH hcall. The details of the bit + flags returned in response to this hcall is available + at 'Documentation/powerpc/papr_hcalls.rst' . Below are + the flags reported in this sysfs file: + + * "not_armed" : Indicating that nvdimm contents will not + survive a power cycle. + * "flush_fail" : Indicating that nvdimm contents + couldn't be flushed during last + shutdown event. + * "restore_fail": Indicating that nvdimm contents + couldn't be restored during dimm + initialization. + * "encrypted" : Dimm contents are encrypted. + * "smart_notify": There is health event for the nvdimm. + * "scrubbed" : Indicating that contents of the + nvdimm have been scrubbed. + * "locked" : Indicating that nvdimm contents cant + be modified until next power cycle.
arch/powerpc/include/asm/papr_scm.h new file +49 -0
49 lines not on this page · raw message
arch/powerpc/platforms/pseries/papr_scm.c +124 -2
162 lines not on this page · raw message
Signature · 4 lines
2.25.3 _______________________________________________ Linux-nvdimm mailing list -- [email protected] To unsubscribe send an email to [email protected]
Michael Ellerman
reviewerRe: [PATCH v6 2/4] powerpc/papr_scm: Fetch nvdimm health information from PHYP
Quoted 67 lines · Vaibhav Jain
Implement support for fetching nvdimm health information via
H_SCM_HEALTHhcall as documented in Ref[1]. The hcall returns a pair of 64-bit big-endian integers, bitwise-and of which is then stored in 'structpapr_scm_priv' and subsequently partially exposed to user-space via newly introduced dimm specific attribute 'papr/flags'. Since the hcall is costly, the health information is cached and only re-queried, 60s after the previous successful hcall.The patch also adds a new asm header named '
papr_scm.h' describing the interface between PHYP and guest kernel. A documentation text describing flags reported by the the new sysfs attribute 'papr/flags' is also introduced atDocumentation/ABI/testing/sysfs-bus-papr-scm.[1] commit
58b278f568f0("powerpc: Provide initial documentation for PAPR hcalls")Cc Dan Williams <[email protected]>Cc Michael Ellerman <[email protected]>Cc "Aneesh Kumar K . V" <[email protected]>Signed-off-by Vaibhav Jain <[email protected]>---
Changelogv5..v6 :
* Change the flags sysfs attribute from 'papr_flags' to 'papr/flags' [Dan Williams]
* Include documentation for 'papr/flags' attr [Dan Williams]
* Change flag 'save_fail' to 'flush_fail' [Dan Williams]
* Caching of health bitmap to reduce expensive hcalls [Dan Williams]
* Removed usage ofPPC_BITfrom 'papr-scm.h' header [Mpe]
* Replaced two__be64integers frompapr_scm_privto a single u64 integer [Mpe]
* Updated patch description to reflect the changes made in this version.
* Removed avoidable usage of 'papr_scm_priv.dimm_mutex' fromflags_show()[Dan Williams]v4..v5 :
* Nonev3..v4 :
* Nonev2..v3 :
* RemovedPAPR_SCM_DIMM_HEALTH_NON_CRITICALas a condition for NVDIMM unarmed [Aneesh]v1..v2 :
* New patch in the series.--- Documentation/ABI/testing/sysfs-bus-papr-scm | 27 ++++ arch/powerpc/include/asm/papr_scm.h | 49 ++++++++ arch/powerpc/platforms/pseries/papr_scm.c | 126 ++++++++++++++++++- 3 files changed, 200 insertions(+), 2 deletions(-) create mode 100644 Documentation/ABI/testing/sysfs-bus-papr-scm create mode 100644 arch/powerpc/include/asm/papr_scm.hDocumentation/ABI/testing/sysfs-bus-papr-scm new file +4
-0@@ -0,0 +1,27 @@ +What: /sys/bus/nd/devices/nmemX/papr/flags +Date: Apr, 2020 +KernelVersion: v5.8 +Contact: linuxppc-dev <[email protected]>, [email protected],
Documentation/ABI/testing/sysfs-bus-papr-scm line 14
Quoted 10 lines
Documentation/ABI/testing/sysfs-bus-papr-scm new file +10
-0+Description: + (RO) Report flags indicating various states of a + papr-scm NVDIMM device. Each flag maps to a one or + more bits set in the dimm-health-bitmap retrieved in + response to H_SCM_HEALTH hcall. The details of the bit + flags returned in response to this hcall is available + at 'Documentation/powerpc/papr_hcalls.rst' . Below are + the flags reported in this sysfs file: + + * "not_armed" : Indicating that nvdimm contents will not
NVDIMM?
Documentation/ABI/testing/sysfs-bus-papr-scm line 20
Quoted 6 lines
Documentation/ABI/testing/sysfs-bus-papr-scm new file +6
-0+ survive a power cycle. + * "flush_fail" : Indicating that nvdimm contents + couldn't be flushed during last + shutdown event. + * "restore_fail": Indicating that nvdimm contents + couldn't be restored during dimm
DIMM?
Documentation/ABI/testing/sysfs-bus-papr-scm line 27
Quoted 7 lines
Documentation/ABI/testing/sysfs-bus-papr-scm new file +7
-0+ initialization. + * "encrypted" : Dimm contents are encrypted. + * "smart_notify": There is health event for the nvdimm. + * "scrubbed" : Indicating that contents of the + nvdimm have been scrubbed. + * "locked" : Indicating that nvdimm contents cant + be modified until next power cycle.
Some of these strings are not very meaningful to me, I would choose different values.
But I assume you are using these because they are defined somewhere else, where is that?
Quoted 43 lines
arch/powerpc/include/asm/papr_scm.h new file +37
-0@@ -0,0 +1,49 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* +* Structures and defines needed to manage nvdimms for spapr guests. +*/ +#ifndef _ASM_POWERPC_PAPR_SCM_H_ +#define _ASM_POWERPC_PAPR_SCM_H_ + +#include <linux/types.h> +#include <asm/bitsperlong.h> + +/* DIMM health bitmap bitmap indicators */ + +/* SCM device is unable to persist memory contents */ +#define PAPR_SCM_DIMM_UNARMED (1ULL << (63 - 0)) +/* SCM device failed to persist memory contents */ +#define PAPR_SCM_DIMM_SHUTDOWN_DIRTY (1ULL << (63 - 1)) +/* SCM device contents are persisted from previous IPL */ +#define PAPR_SCM_DIMM_SHUTDOWN_CLEAN (1ULL << (63 - 2)) +/* SCM device contents are not persisted from previous IPL */ +#define PAPR_SCM_DIMM_EMPTY (1ULL << (63 - 3)) +/* SCM device memory life remaining is critically low */ +#define PAPR_SCM_DIMM_HEALTH_CRITICAL (1ULL << (63 - 4)) +/* SCM device will be garded off next IPL due to failure */ +#define PAPR_SCM_DIMM_HEALTH_FATAL (1ULL << (63 - 5)) +/* SCM contents cannot persist due to current platform health status */ +#define PAPR_SCM_DIMM_HEALTH_UNHEALTHY (1ULL << (63 - 6)) +/* SCM device is unable to persist memory contents in certain conditions */ +#define PAPR_SCM_DIMM_HEALTH_NON_CRITICAL (1ULL << (63 - 7)) +/* SCM device is encrypted */ +#define PAPR_SCM_DIMM_ENCRYPTED (1ULL << (63 - 8)) +/* SCM device has been scrubbed and locked */ +#define PAPR_SCM_DIMM_SCRUBBED_AND_LOCKED (1ULL << (63 - 9)) + +/* Bits status indicators for health bitmap indicating unarmed dimm */ +#define PAPR_SCM_DIMM_UNARMED_MASK (PAPR_SCM_DIMM_UNARMED | \ + PAPR_SCM_DIMM_HEALTH_UNHEALTHY) +
arch/powerpc/include/asm/papr_scm.h line 47
Quoted 10 lines
arch/powerpc/include/asm/papr_scm.h new file +10
-0+/* Bits status indicators for health bitmap indicating unflushed dimm */ +#define PAPR_SCM_DIMM_BAD_SHUTDOWN_MASK (PAPR_SCM_DIMM_SHUTDOWN_DIRTY) + +/* Bits status indicators for health bitmap indicating unrestored dimm */ +#define PAPR_SCM_DIMM_BAD_RESTORE_MASK (PAPR_SCM_DIMM_EMPTY) + +/* Bit status indicators for smart event notification */ +#define PAPR_SCM_DIMM_SMART_EVENT_MASK (PAPR_SCM_DIMM_HEALTH_CRITICAL | \ + PAPR_SCM_DIMM_HEALTH_FATAL | \ + PAPR_SCM_DIMM_HEALTH_UNHEALTHY)
These are only ever used in papr_scm.c AFAICS, so put them in there.
Quoted 13 lines
arch/powerpc/platforms/pseries/papr_scm.c +1
-0@@ -14,6 +14,7 @@#include <linux/delay.h> #include <asm/plpar_wrappers.h> +#include <asm/papr_scm.h> #define BIND_ANY_ADDR (~0ul)
arch/powerpc/platforms/pseries/papr_scm.c struct papr_scm_priv { · line 45
Quoted 6 lines
arch/powerpc/platforms/pseries/papr_scm.c +3
-0@@ -39,6 +40,15 @@ struct papr_scm_priv {struct resource res;struct nd_region *region;struct nd_interleave_set nd_set; + +/* Protect dimm health data from concurrent read/writes */ +struct mutex dimm_mutex;
health_mutex then?
Quoted 9 lines
arch/powerpc/platforms/pseries/papr_scm.c +5
-0+/* Last time the health information of the dimm was updated */ +unsigned long lasthealth_jiffies; + +/* Health information for the dimm */ +u64 health_bitmap; };static int drc_pmem_bind (struct papr_scm_priv *p)
arch/powerpc/platforms/pseries/papr_scm.c static int drc_pmem_query_n_bind(struct papr_scm_priv *p) · line 162
Quoted 9 lines
arch/powerpc/platforms/pseries/papr_scm.c +6
-0@@ -144,6 +154,56 @@ static int drc_pmem_query_n_bind(struct papr_scm_priv *p)return drc_pmem_bind (p); } +/* Min interval in seconds between successive H_SCM_HEALTH hcalls */ +#define MIN_HEALTH_QUERY_INTERVAL 60 + +/* +* Issue hcall if needed to retrieve dimm health info. Information is cached +* and subsequent calls may return success without issueing the hcall.
^
issuingarch/powerpc/platforms/pseries/papr_scm.c static int drc_pmem_query_n_bind(struct papr_scm_priv *p) · line 166
Quoted 4 lines
arch/powerpc/platforms/pseries/papr_scm.c +4
-0+* Use 'force == true' to force issue of the hcall ignoring the cache +* timeout. +*/ +static int drc_pmem_query_health (struct papr_scm_priv *p,bool force)
The force API is a bit clunky.
I think it would make more sense if you had the caching logic in a wrapper function, or even in flags_show() directly.
And then have papr_scm_get_health() just call the underlying routine that always does the hcall.
I think you can safely read lasthealth_jiffies without the mutex.
arch/powerpc/platforms/pseries/papr_scm.c static int drc_pmem_query_n_bind(struct papr_scm_priv *p) · line 170
Quoted 4 lines
arch/powerpc/platforms/pseries/papr_scm.c +4
-0+{ +unsigned long ret[PLPAR_HCALL_BUFSIZE ]; +s64 rc; +unsigned long cache_timeout;
My eyes! ;)
unsigned long cache_timeout, ret[PLPAR_HCALL_BUFSIZE]; s64 rc;
Please :)
Quoted 17 lines
arch/powerpc/platforms/pseries/papr_scm.c +17
-0+ +/* Protect concurrent modifications to papr_scm_priv */ + rc =mutex_lock_interruptible (&p->dimm_mutex); +if (rc) +return rc; + +/* Jiffies offset for which the health data is assumed to be same */ + cache_timeout = p->lasthealth_jiffies + +msecs_to_jiffies (MIN_HEALTH_QUERY_INTERVAL *1000 ); + +/* Dont issue the hcall if health information is relatively new */ +if (!force &&time_after (cache_timeout, jiffies)) { + rc =0 ; + goto out; + } + +/* issue the hcall */
arch/powerpc/platforms/pseries/papr_scm.c static int drc_pmem_query_n_bind(struct papr_scm_priv *p) · line 197
Quoted 10 lines
arch/powerpc/platforms/pseries/papr_scm.c +10
-0+ rc =plpar_hcall (H_SCM_HEALTH , ret, p->drc_index); +if (rc !=H_SUCCESS ) { +dev_err (&p->pdev->dev, +"Failed to query health information, Err:%lld\n" , rc); + rc = -ENXIO ; + goto out; + } + + p->lasthealth_jiffies = jiffies; + p->health_bitmap = ret[0 ] & ret[1 ];
I said in v5 that you needn't store health_bitmap as __be64, but you do still need to convert it to CPU endian at some point.
I think ret[] should actually be __be64, and this is where you should convert it to CPU endian, before assigning to health_bitmap.
Quoted 23 lines
arch/powerpc/platforms/pseries/papr_scm.c +16
-0+ +dev_dbg (&p->pdev->dev, +"Queried dimm health info. Bitmap:0x%016lx Mask:0x%016lx\n" , + ret[0 ], ret[1 ]); + +out: +mutex_unlock (&p->dimm_mutex); +return rc; +}static int papr_scm_meta_get (struct papr_scm_priv *p,struct nd_cmd_get_config_data_hdr *hdr)@@ -304,6 +364,65 @@ static inline int papr_scm_node(int node)return min_node; } +static ssize_t flags_show (struct device *dev, +struct device_attribute *attr,char *buf) +{ +struct nvdimm *dimm =to_nvdimm (dev); +struct papr_scm_priv *p =nvdimm_provider_data (dimm); +int rc; +u64 health;
arch/powerpc/platforms/pseries/papr_scm.c static inline int papr_scm_node(int node) · line 383
Quoted 10 lines
arch/powerpc/platforms/pseries/papr_scm.c +10
-0+ + rc =drc_pmem_query_health (p, false); +if (rc) +return rc; + +/* +* Copy the LE byte-ordered health_bitmap locally, check for various +* masks and update the sysfs out buffer. +*/ + health = p->health_bitmap;
Why are you copying it locally? Because you don't hold the lock and you're worried it will mutate? The compiler may just ignore you and reload it anyway.
You'd need to use READ_ONCE(), or do it with the lock held. Or you can just decide that it's OK for an update to race vs the display.
arch/powerpc/platforms/pseries/papr_scm.c static inline int papr_scm_node(int node) · line 386
Quoted 3 lines
arch/powerpc/platforms/pseries/papr_scm.c +3
-0+ +if (health &PAPR_SCM_DIMM_UNARMED_MASK ) + rc +=sprintf (buf,"not_armed " );
I'd still be happier if you used snprintf() or seq_puts(), but I won't die in a ditch over it.
Quoted 11 lines
arch/powerpc/platforms/pseries/papr_scm.c +11
-0+if (health &PAPR_SCM_DIMM_BAD_SHUTDOWN_MASK ) + rc +=sprintf (buf + rc,"save_fail " ); + +if (health &PAPR_SCM_DIMM_BAD_RESTORE_MASK ) + rc +=sprintf (buf + rc,"restore_fail " ); + +if (health &PAPR_SCM_DIMM_ENCRYPTED ) + rc +=sprintf (buf + rc,"encrypted " ); + +if (health &PAPR_SCM_DIMM_SMART_EVENT_MASK ) + rc +=sprintf (buf + rc,"smart_notify " );
arch/powerpc/platforms/pseries/papr_scm.c static inline int papr_scm_node(int node) · line 408
Quoted 10 lines
arch/powerpc/platforms/pseries/papr_scm.c +10
-0+ +if (health &PAPR_SCM_DIMM_SCRUBBED_AND_LOCKED ) + rc +=sprintf (buf + rc,"scrubbed locked " ); + +if (rc >0 ) + rc +=sprintf (buf + rc,"\n" ); + +return rc; +} +DEVICE_ATTR_RO (flags);
cheers
_______________________________________________
Linux-nvdimm mailing list -- [email protected] To unsubscribe send an email to [email protected]
PATCH v6 3/4 ndctl/papr_scm,uapi: Add support for PAPR nvdimm specific methods
In reply to Vaibhav JainIntroduce support for Papr nvDimm Specific Methods (PDSM) in papr_scm modules and add the command family to the white list of NVDIMM command sets. Also advertise support for ND_CMD_CALL for the dimm command mask and implement necessary scaffolding in the module to handle ND_CMD_CALL ioctl and PDSM requests that we receive.
The layout of the PDSM request as we expect from libnvdimm/libndctl is described in newly introduced uapi header 'papr_scm_pdsm.h' which defines a new 'struct nd_pdsm_cmd_pkg' header. This header is used to communicate the PDSM request via member 'nd_pkg_papr_scm->nd_command' and size of payload that need to be sent/received for servicing the PDSM.
A new function is_cmd_valid() is implemented that reads the args to papr_scm_ndctl() and performs sanity tests on them. A new function papr_scm_service_pdsm() is introduced and is called from papr_scm_ndctl() in case of a PDSM request is received via ND_CMD_CALL command from libnvdimm.
---
Changelog
v5..v6 :
* Changed the usage of the term DSM to PDSM to distinguish it from the ACPI term [ Dan Williams ]
* Renamed papr_scm_dsm.h to papr_scm_pdsm.h and updated various struct to reflect the new terminology.
* Updated the patch description and title to reflect the new terminology.
* Squashed patch to introduce new command family in 'ndctl.h' with this patch [ Dan Williams ]
* Updated the papr_scm_pdsm method starting index from 0x10000 to 0x0 [ Dan Williams ]
* Removed redundant license text from the papr_scm_psdm.h file. [ Dan Williams ]
* s/envelop/envelope/ at various places [ Dan Williams ]
* Added '__packed' attribute to command package header to gaurd against different compiler adding paddings between the fields. [ Dan Williams]
* Converted various pr_debug to dev_debug [ Dan Williams ]
v4..v5 :
* None
v3..v4 :
* None
v2..v3 :
* Updated the patch prefix to 'ndctl/uapi' [Aneesh]
v1..v2 :
* None
--- arch/powerpc/include/uapi/asm/papr_scm_pdsm.h | 153 ++++++++++++++++++ arch/powerpc/platforms/pseries/papr_scm.c | 101 +++++++++++- include/uapi/linux/ndctl.h | 1 + 3 files changed, 249 insertions(+), 6 deletions(-) create mode 100644 arch/powerpc/include/uapi/asm/papr_scm_pdsm.h
arch/powerpc/include/uapi/asm/papr_scm_pdsm.h new file +153 -0
@@ -0,0 +1,153 @@ +/* SPDX-License-Identifier: GPL-2.0+ WITH Linux-syscall-note */ +/* +* PAPR-SCM Dimm specific methods (PDSM) and structs for libndctl +* +* (C) Copyright IBM 2020 +* +* Author: Vaibhav Jain <vaibhav at linux.ibm.com> +*/ + +#ifndef _UAPI_ASM_POWERPC_PAPR_SCM_PDSM_H_ +#define _UAPI_ASM_POWERPC_PAPR_SCM_PDSM_H_ + +#include <linux/types.h> + +#ifdef __KERNEL__ +#include <linux/ndctl.h> +#else +#include <ndctl.h> +#endif + +/* +* PDSM Envelope: +* +* The ioctl ND_CMD_CALL transfers data between user-space and kernel via +* 'envelopes' which consists of a header and user-defined payload sections. +* The header is described by 'struct nd_pdsm_cmd_pkg' which expects a +* payload following it and offset of which relative to the struct is provided +* by 'nd_pdsm_cmd_pkg.payload_offset'. * +* +* +-------------+---------------------+---------------------------+ +* | 64-Bytes | 8-Bytes | Max 184-Bytes | +* +-------------+---------------------+---------------------------+ +* | nd_pdsm_cmd_pkg | | +* |-------------+ | | +* | nd_cmd_pkg | | | +* +-------------+---------------------+---------------------------+ +* | nd_family | | | +* | nd_size_out | cmd_status | | +* | nd_size_in | payload_version | PAYLOAD | +* | nd_command | payload_offset -----> | +* | nd_fw_size | | | +* +-------------+---------------------+---------------------------+ +* +* PDSM Header: +* +* The header is defined as 'struct nd_pdsm_cmd_pkg' which embeds a +* 'struct nd_cmd_pkg' instance. The PDSM command is assigned to member +* 'nd_cmd_pkg.nd_command'. Apart from size information of the envelope which is +* contained in 'struct nd_cmd_pkg', the header also has members following +* members: +* +* 'cmd_status' : (Out) Errors if any encountered while servicing PDSM. +* 'payload_version' : (In/Out) Version number associated with the payload. +* 'payload_offset' : (In)Relative offset of payload from start of envelope. +* +* PDSM Payload: +* +* The layout of the PDSM Payload is defined by various structs shared between +* papr_scm and libndctl so that contents of payload can be interpreted. During +* servicing of a PDSM the papr_scm module will read input args from the payload +* field by casting its contents to an appropriate struct pointer based on the +* PDSM command. Similarly the output of servicing the PDSM command will be +* copied to the payload field using the same struct. +* +* 'libnvdimm' enforces a hard limit of 256 bytes on the envelope size, which +* leaves around 184 bytes for the envelope payload (ignoring any padding that +* the compiler may silently introduce). +* +* Payload Version: +* +* A 'payload_version' field is present in PDSM header that indicates a specific +* version of the structure present in PDSM Payload for a given PDSM command. +* This provides backward compatibility in case the PDSM Payload structure +* evolves and different structures are supported by 'papr_scm' and 'libndctl'. +* +* When sending a PDSM Payload to 'papr_scm', 'libndctl' should send the version +* of the payload struct it supports via 'payload_version' field. The 'papr_scm' +* module when servicing the PDSM envelope checks the 'payload_version' and then +* uses 'payload struct version' == MIN('payload_version field', +* 'max payload-struct-version supported by papr_scm') to service the PDSM. +* After servicing the PDSM, 'papr_scm' put the negotiated version of payload +* struct in returned 'payload_version' field. +* +* Libndctl on receiving the envelope back from papr_scm again checks the +* 'payload_version' field and based on it use the appropriate version dsm +* struct to parse the results. +* +* Backward Compatibility: +* +* Above scheme of exchanging different versioned PDSM struct between libndctl +* and papr_scm should provide backward compatibility until following two +* assumptions/conditions when defining new PDSM structs hold: +* +* Let T(X) = { set of attributes in PDSM struct 'T' versioned X } +* +* 1. T(X) is a proper subset of T(Y) if X > Y. +* i.e Each new version of PDSM struct should retain existing struct +* attributes from previous version +* +* 2. If an entity (libndctl or papr_scm) supports a PDSM struct T(X) then +* it should also support T(1), T(2)...T(X - 1). +* i.e When adding support for new version of a PDSM struct, libndctl +* and papr_scm should retain support of the existing PDSM struct +* version they support. +*/ + +#ifndef __packed +#define __packed __attribute__((packed)) +#endif + +/* Papr-scm-header + payload expected with ND_CMD_CALL ioctl from libnvdimm */ +struct nd_pdsm_cmd_pkg { +struct nd_cmd_pkg hdr;/* Package header containing sub-cmd */ +__s32 cmd_status;/* Out: Sub-cmd status returned back */ +__u16 payload_offset;/* In: offset from start of struct */ +__u16 payload_version;/* In/Out: version of the payload */ +__u8 payload[];/* In/Out: Sub-cmd data buffer */ +} __packed; + +/* +* Methods to be embedded in ND_CMD_CALL request. These are sent to the kernel +* via 'nd_pdsm_cmd_pkg.hdr.nd_command' member of the ioctl struct +*/ +enum papr_scm_pdsm { +PAPR_SCM_PDSM_MIN =0x0 , +PAPR_SCM_PDSM_MAX , +}; + +/* Helpers to evaluate the size of PDSM envelope */ +/* Calculate the papr_scm-header size */ +#define ND_PDSM_ENVELOPE_CONTENT_HDR_SIZE \ + (sizeof(struct nd_pdsm_cmd_pkg) - sizeof(struct nd_cmd_pkg)) + +/* Given a type calculate envelope-content size (papr_scm-header + payload) */ +#define ND_PDSM_ENVELOPE_CONTENT_SIZE (_type_) \ + (sizeof(_type_) + ND_PDSM_ENVELOPE_CONTENT_HDR_SIZE) + +/* Convert a libnvdimm nd_cmd_pkg to pdsm specific pkg */ +static struct nd_pdsm_cmd_pkg *nd_to_pdsm_cmd_pkg (struct nd_cmd_pkg *cmd) +{ +return (struct nd_pdsm_cmd_pkg *) cmd; +} + +/* Return the payload pointer for a given pcmd */ +static void *pdsm_cmd_to_payload (struct nd_pdsm_cmd_pkg *pcmd) +{ +if (pcmd->hdr.nd_size_in ==0 && pcmd->hdr.nd_size_out ==0 ) +return NULL ; +else +return (void *)((__u8 *) pcmd + pcmd->payload_offset); +} + +#endif /* _UAPI_ASM_POWERPC_PAPR_SCM_PDSM_H_ */
arch/powerpc/platforms/pseries/papr_scm.c +95 -6
137 lines not on this page · raw message
include/uapi/linux/ndctl.h +1 -0
@@ -244,6 +244,7 @@ struct nd_cmd_pkg {#define NVDIMM_FAMILY_HPE2 2#define NVDIMM_FAMILY_MSFT 3#define NVDIMM_FAMILY_HYPERV 4 +#define NVDIMM_FAMILY_PAPR_SCM 5#define ND_IOCTL_CALL _IOWR(ND_IOCTL, ND_CMD_CALL,\ struct nd_cmd_pkg)
Signature · 4 lines
2.25.3 _______________________________________________ Linux-nvdimm mailing list -- [email protected] To unsubscribe send an email to [email protected]
Michael Ellerman
reviewerRe: [PATCH v6 3/4] ndctl/papr_scm,uapi: Add support for PAPR nvdimm specific methods
Quoted 77 lines · Vaibhav Jain
Introduce support for Papr nvDimm Specific Methods (PDSM) in
papr_scmmodules and add the command family to the white list of NVDIMM command sets. Also advertise support forND_CMD_CALLfor the dimm command mask and implement necessary scaffolding in the module to handleND_CMD_CALLioctl and PDSM requests that we receive.The layout of the PDSM request as we expect from libnvdimm/libndctl is described in newly introduced uapi header '
papr_scm_pdsm.h' which defines a new 'structnd_pdsm_cmd_pkg' header. This header is used to communicate the PDSM request via member 'nd_pkg_papr_scm->nd_command' and size of payload that need to be sent/received for servicing the PDSM.A new function
is_cmd_valid()is implemented that reads the args topapr_scm_ndctl()and performs sanity tests on them. A new functionpapr_scm_service_pdsm()is introduced and is called frompapr_scm_ndctl()in case of a PDSM request is received viaND_CMD_CALLcommand from libnvdimm.Cc Dan Williams <[email protected]>Cc Michael Ellerman <[email protected]>Cc "Aneesh Kumar K . V" <[email protected]>Signed-off-by Vaibhav Jain <[email protected]>---
Changelogv5..v6 :
* Changed the usage of the term DSM to PDSM to distinguish it from the ACPI term [ Dan Williams ]
* Renamedpapr_scm_dsm.htopapr_scm_pdsm.hand updated various struct to reflect the new terminology.
* Updated the patch description and title to reflect the new terminology.
* Squashed patch to introduce new command family in 'ndctl.h' with this patch [ Dan Williams ]
* Updated thepapr_scm_pdsmmethod starting index from0x10000to0x0[ Dan Williams ]
* Removed redundant license text from thepapr_scm_psdm.hfile. [ Dan Williams ]
* s/envelop/envelope/ at various places [ Dan Williams ]
* Added '__packed' attribute to command package header to gaurd against different compiler adding paddings between the fields. [ Dan Williams]
* Converted variouspr_debugtodev_debug[ Dan Williams ]v4..v5 :
* Nonev3..v4 :
* Nonev2..v3 :
* Updated the patch prefix to 'ndctl/uapi' [Aneesh]v1..v2 :
* None--- arch/powerpc/include/uapi/asm/papr_scm_pdsm.h | 153 ++++++++++++++++++ arch/powerpc/platforms/pseries/papr_scm.c | 101 +++++++++++- include/uapi/linux/ndctl.h | 1 + 3 files changed, 249 insertions(+), 6 deletions(-) create mode 100644 arch/powerpc/include/uapi/asm/papr_scm_pdsm.harch/powerpc/include/uapi/asm/papr_scm_pdsm.h new file +9
-0@@ -0,0 +1,153 @@ +/* SPDX-License-Identifier: GPL-2.0+ WITH Linux-syscall-note */ +/* +* PAPR-SCM Dimm specific methods (PDSM) and structs for libndctl +* +* (C) Copyright IBM 2020 +* +* Author: Vaibhav Jain <vaibhav at linux.ibm.com> +*/ +
arch/powerpc/include/uapi/asm/papr_scm_pdsm.h line 19
Quoted 10 lines
arch/powerpc/include/uapi/asm/papr_scm_pdsm.h new file +10
-0+#ifndef _UAPI_ASM_POWERPC_PAPR_SCM_PDSM_H_ +#define _UAPI_ASM_POWERPC_PAPR_SCM_PDSM_H_ + +#include <linux/types.h> + +#ifdef __KERNEL__ +#include <linux/ndctl.h> +#else +#include <ndctl.h> +#endif
We shouldn't be checking __KERNEL__ in uapi headers, something has gone wrong if this is necessary.
Quoted 80 lines
arch/powerpc/include/uapi/asm/papr_scm_pdsm.h new file +80
-0+ +/* +* PDSM Envelope: +* +* The ioctl ND_CMD_CALL transfers data between user-space and kernel via +* 'envelopes' which consists of a header and user-defined payload sections. +* The header is described by 'struct nd_pdsm_cmd_pkg' which expects a +* payload following it and offset of which relative to the struct is provided +* by 'nd_pdsm_cmd_pkg.payload_offset'. * +* +* +-------------+---------------------+---------------------------+ +* | 64-Bytes | 8-Bytes | Max 184-Bytes | +* +-------------+---------------------+---------------------------+ +* | nd_pdsm_cmd_pkg | | +* |-------------+ | | +* | nd_cmd_pkg | | | +* +-------------+---------------------+---------------------------+ +* | nd_family | | | +* | nd_size_out | cmd_status | | +* | nd_size_in | payload_version | PAYLOAD | +* | nd_command | payload_offset -----> | +* | nd_fw_size | | | +* +-------------+---------------------+---------------------------+ +* +* PDSM Header: +* +* The header is defined as 'struct nd_pdsm_cmd_pkg' which embeds a +* 'struct nd_cmd_pkg' instance. The PDSM command is assigned to member +* 'nd_cmd_pkg.nd_command'. Apart from size information of the envelope which is +* contained in 'struct nd_cmd_pkg', the header also has members following +* members: +* +* 'cmd_status' : (Out) Errors if any encountered while servicing PDSM. +* 'payload_version' : (In/Out) Version number associated with the payload. +* 'payload_offset' : (In)Relative offset of payload from start of envelope. +* +* PDSM Payload: +* +* The layout of the PDSM Payload is defined by various structs shared between +* papr_scm and libndctl so that contents of payload can be interpreted. During +* servicing of a PDSM the papr_scm module will read input args from the payload +* field by casting its contents to an appropriate struct pointer based on the +* PDSM command. Similarly the output of servicing the PDSM command will be +* copied to the payload field using the same struct. +* +* 'libnvdimm' enforces a hard limit of 256 bytes on the envelope size, which +* leaves around 184 bytes for the envelope payload (ignoring any padding that +* the compiler may silently introduce). +* +* Payload Version: +* +* A 'payload_version' field is present in PDSM header that indicates a specific +* version of the structure present in PDSM Payload for a given PDSM command. +* This provides backward compatibility in case the PDSM Payload structure +* evolves and different structures are supported by 'papr_scm' and 'libndctl'. +* +* When sending a PDSM Payload to 'papr_scm', 'libndctl' should send the version +* of the payload struct it supports via 'payload_version' field. The 'papr_scm' +* module when servicing the PDSM envelope checks the 'payload_version' and then +* uses 'payload struct version' == MIN('payload_version field', +* 'max payload-struct-version supported by papr_scm') to service the PDSM. +* After servicing the PDSM, 'papr_scm' put the negotiated version of payload +* struct in returned 'payload_version' field. +* +* Libndctl on receiving the envelope back from papr_scm again checks the +* 'payload_version' field and based on it use the appropriate version dsm +* struct to parse the results. +* +* Backward Compatibility: +* +* Above scheme of exchanging different versioned PDSM struct between libndctl +* and papr_scm should provide backward compatibility until following two +* assumptions/conditions when defining new PDSM structs hold: +* +* Let T(X) = { set of attributes in PDSM struct 'T' versioned X } +* +* 1. T(X) is a proper subset of T(Y) if X > Y. +* i.e Each new version of PDSM struct should retain existing struct +* attributes from previous version +*
arch/powerpc/include/uapi/asm/papr_scm_pdsm.h line 109
Quoted 10 lines
arch/powerpc/include/uapi/asm/papr_scm_pdsm.h new file +10
-0+* 2. If an entity (libndctl or papr_scm) supports a PDSM struct T(X) then +* it should also support T(1), T(2)...T(X - 1). +* i.e When adding support for new version of a PDSM struct, libndctl +* and papr_scm should retain support of the existing PDSM struct +* version they support. +*/ + +#ifndef __packed +#define __packed __attribute__((packed)) +#endif
I'm not sure it's kosher to be defining __packed in a uapi header.
Quoted 17 lines
arch/powerpc/include/uapi/asm/papr_scm_pdsm.h new file +17
-0+ +/* Papr-scm-header + payload expected with ND_CMD_CALL ioctl from libnvdimm */ +struct nd_pdsm_cmd_pkg { +struct nd_cmd_pkg hdr;/* Package header containing sub-cmd */ +__s32 cmd_status;/* Out: Sub-cmd status returned back */ +__u16 payload_offset;/* In: offset from start of struct */ +__u16 payload_version;/* In/Out: version of the payload */ +__u8 payload[];/* In/Out: Sub-cmd data buffer */ +} __packed; + +/* +* Methods to be embedded in ND_CMD_CALL request. These are sent to the kernel +* via 'nd_pdsm_cmd_pkg.hdr.nd_command' member of the ioctl struct +*/ +enum papr_scm_pdsm { +PAPR_SCM_PDSM_MIN =0x0 , +PAPR_SCM_PDSM_MAX ,
arch/powerpc/include/uapi/asm/papr_scm_pdsm.h line 136
Quoted 10 lines
arch/powerpc/include/uapi/asm/papr_scm_pdsm.h new file +10
-0+}; + +/* Helpers to evaluate the size of PDSM envelope */ +/* Calculate the papr_scm-header size */ +#define ND_PDSM_ENVELOPE_CONTENT_HDR_SIZE \ + (sizeof(struct nd_pdsm_cmd_pkg) - sizeof(struct nd_cmd_pkg)) + +/* Given a type calculate envelope-content size (papr_scm-header + payload) */ +#define ND_PDSM_ENVELOPE_CONTENT_SIZE (_type_) \ + (sizeof(_type_) + ND_PDSM_ENVELOPE_CONTENT_HDR_SIZE)
Those seem unused? Do we really need to make them part of the uapi for all time?
arch/powerpc/include/uapi/asm/papr_scm_pdsm.h line 145
Quoted 8 lines
arch/powerpc/include/uapi/asm/papr_scm_pdsm.h new file +8
-0+/* Convert a libnvdimm nd_cmd_pkg to pdsm specific pkg */ +static struct nd_pdsm_cmd_pkg *nd_to_pdsm_cmd_pkg (struct nd_cmd_pkg *cmd) +{ +return (struct nd_pdsm_cmd_pkg *) cmd; +} + +/* Return the payload pointer for a given pcmd */ +static void *pdsm_cmd_to_payload (struct nd_pdsm_cmd_pkg *pcmd)
That should probably be static inline?
Quoted 157 lines
arch/powerpc/include/uapi/asm/papr_scm_pdsm.h new file +8
-0+{ +if (pcmd->hdr.nd_size_in ==0 && pcmd->hdr.nd_size_out ==0 ) +return NULL ; +else +return (void *)((__u8 *) pcmd + pcmd->payload_offset); +} + +#endif /* _UAPI_ASM_POWERPC_PAPR_SCM_PDSM_H_ */ arch/powerpc/platforms/pseries/papr_scm.c +95
-6@@ -15,13 +15,15 @@#include <asm/plpar_wrappers.h> #include <asm/papr_scm.h> +#include <asm/papr_scm_pdsm.h> #define BIND_ANY_ADDR (~0ul)#define PAPR_SCM_DIMM_CMD_MASK \ ((1ul << ND_CMD_GET_CONFIG_SIZE) | \ (1ul << ND_CMD_GET_CONFIG_DATA) | \ - (1ul << ND_CMD_SET_CONFIG_DATA)) + (1ul << ND_CMD_SET_CONFIG_DATA) | \ + (1ul << ND_CMD_CALL))struct papr_scm_priv {struct platform_device *pdev;@@ -306,16 +308,97 @@ static int papr_scm_meta_set(struct papr_scm_priv *p,return 0 ; } -int papr_scm_ndctl (struct nvdimm_bus_descriptor *nd_desc,struct nvdimm *nvdimm, -unsigned int cmd,void *buf,unsigned int buf_len,int *cmd_rc) +/* +* Validate the inputs args to dimm-control function and return '0' if valid. +* This also does initial sanity validation to ND_CMD_CALL sub-command packages. +*/ +static int is_cmd_valid (struct nvdimm *nvdimm,unsigned int cmd,void *buf, +unsigned int buf_len) { -struct nd_cmd_get_config_size *get_size_hdr; +unsigned long cmd_mask =PAPR_SCM_DIMM_CMD_MASK ; +struct nd_pdsm_cmd_pkg *pkg =nd_to_pdsm_cmd_pkg (buf);struct papr_scm_priv *p;/* Only dimm-specific calls are supported atm */ if (!nvdimm)return -EINVAL ; +/* get the provider date from struct nvdimm */ + p =nvdimm_provider_data (nvdimm); + +if (!test_bit (cmd, &cmd_mask)) { +dev_dbg (&p->pdev->dev,"Unsupported cmd=%u\n" , cmd); +return -EINVAL ; + }else if (cmd ==ND_CMD_CALL ) { + +/* Verify the envelope package */ +if (!buf || buf_len <sizeof (struct nd_pdsm_cmd_pkg )) { +dev_dbg (&p->pdev->dev,"Invalid pkg size=%u\n" , + buf_len); +return -EINVAL ; + } + +/* Verify that the PDSM family is valid */ +if (pkg->hdr.nd_family !=NVDIMM_FAMILY_PAPR_SCM ) { +dev_dbg (&p->pdev->dev,"Invalid pkg family=0x%llx\n" , + pkg->hdr.nd_family); +return -EINVAL ; + + } + +/* We except a payload with all PDSM commands */ +if (pdsm_cmd_to_payload (pkg) ==NULL ) { +dev_dbg (&p->pdev->dev, +"Empty payload for sub-command=0x%llx\n" , + pkg->hdr.nd_command); +return -EINVAL ; + } + } + +/* Command looks valid */ +return 0 ; +} + +static int papr_scm_service_pdsm (struct papr_scm_priv *p, +struct nd_pdsm_cmd_pkg *call_pkg) +{ +/* unknown subcommands return error in packages */ +if (call_pkg->hdr.nd_command <=PAPR_SCM_PDSM_MIN || + call_pkg->hdr.nd_command >=PAPR_SCM_PDSM_MAX ) { +dev_dbg (&p->pdev->dev,"Invalid PDSM request 0x%llx\n" , + call_pkg->hdr.nd_command); + call_pkg->cmd_status = -EINVAL ; +return 0 ; + } + +/* Depending on the DSM command call appropriate service routine */ +switch (call_pkg->hdr.nd_command) { +default : +dev_dbg (&p->pdev->dev,"Unsupported PDSM request 0x%llx\n" , + call_pkg->hdr.nd_command); + call_pkg->cmd_status = -ENOENT ; +return 0 ; + } +} + +int papr_scm_ndctl (struct nvdimm_bus_descriptor *nd_desc,struct nvdimm *nvdimm, +unsigned int cmd,void *buf,unsigned int buf_len,int *cmd_rc) +{ +struct nd_cmd_get_config_size *get_size_hdr; +struct papr_scm_priv *p; +struct nd_pdsm_cmd_pkg *call_pkg =NULL ; +int rc; + +/* Use a local variable in case cmd_rc pointer is NULL */ +if (cmd_rc ==NULL ) + cmd_rc = &rc; + + *cmd_rc =is_cmd_valid (nvdimm, cmd, buf, buf_len); +if (*cmd_rc) { +pr_debug ("Invalid cmd=0x%x. Err=%d\n" , cmd, *cmd_rc); +return *cmd_rc; + } + p =nvdimm_provider_data (nvdimm);switch (cmd) {@@ -336,13 +419,19 @@ int papr_scm_ndctl(struct nvdimm_bus_descriptor *nd_desc, struct nvdimm *nvdimm, *cmd_rc =papr_scm_meta_set (p, buf);break ; +case ND_CMD_CALL : + call_pkg =nd_to_pdsm_cmd_pkg (buf); + *cmd_rc =papr_scm_service_pdsm (p, call_pkg); +break ; +default : -return -EINVAL ; +dev_dbg (&p->pdev->dev,"Unknown command = %d\n" , cmd); + *cmd_rc = -EINVAL ; }dev_dbg (&p->pdev->dev,"returned with cmd_rc = %d\n" , *cmd_rc); -return 0 ; +return *cmd_rc; }static inline int papr_scm_node (int node)include/uapi/linux/ndctl.h +0
-0
include/uapi/linux/ndctl.h struct nd_cmd_pkg { · line 247
Quoted 4 lines
include/uapi/linux/ndctl.h +1
-0@@ -244,6 +244,7 @@ struct nd_cmd_pkg {#define NVDIMM_FAMILY_HPE2 2#define NVDIMM_FAMILY_MSFT 3#define NVDIMM_FAMILY_HYPERV 4 +#define NVDIMM_FAMILY_PAPR_SCM 5
Would like an ack from Dan for that change.
cheers
_______________________________________________
Linux-nvdimm mailing list -- [email protected] To unsubscribe send an email to [email protected]
Re: [PATCH v6 3/4] ndctl/papr_scm,uapi: Add support for PAPR nvdimm specific methods
Hi Mpe,
Thanks again for reviewing this patchset.
Quoted 83 lines · Michael Ellerman
Quoted 82 lines · Vaibhav Jain
Introduce support for Papr nvDimm Specific Methods (PDSM) in
papr_scmmodules and add the command family to the white list of NVDIMM command sets. Also advertise support forND_CMD_CALLfor the dimm command mask and implement necessary scaffolding in the module to handleND_CMD_CALLioctl and PDSM requests that we receive.The layout of the PDSM request as we expect from libnvdimm/libndctl is described in newly introduced uapi header '
papr_scm_pdsm.h' which defines a new 'structnd_pdsm_cmd_pkg' header. This header is used to communicate the PDSM request via member 'nd_pkg_papr_scm->nd_command' and size of payload that need to be sent/received for servicing the PDSM.A new function
is_cmd_valid()is implemented that reads the args topapr_scm_ndctl()and performs sanity tests on them. A new functionpapr_scm_service_pdsm()is introduced and is called frompapr_scm_ndctl()in case of a PDSM request is received viaND_CMD_CALLcommand from libnvdimm.Cc Dan Williams <[email protected]>Cc Michael Ellerman <[email protected]>Cc "Aneesh Kumar K . V" <[email protected]>Signed-off-by Vaibhav Jain <[email protected]>---
Changelogv5..v6 :
* Changed the usage of the term DSM to PDSM to distinguish it from the ACPI term [ Dan Williams ]
* Renamedpapr_scm_dsm.htopapr_scm_pdsm.hand updated various struct to reflect the new terminology.
* Updated the patch description and title to reflect the new terminology.
* Squashed patch to introduce new command family in 'ndctl.h' with this patch [ Dan Williams ]
* Updated thepapr_scm_pdsmmethod starting index from0x10000to0x0[ Dan Williams ]
* Removed redundant license text from thepapr_scm_psdm.hfile. [ Dan Williams ]
* s/envelop/envelope/ at various places [ Dan Williams ]
* Added '__packed' attribute to command package header to gaurd against different compiler adding paddings between the fields. [ Dan Williams]
* Converted variouspr_debugtodev_debug[ Dan Williams ]v4..v5 :
* Nonev3..v4 :
* Nonev2..v3 :
* Updated the patch prefix to 'ndctl/uapi' [Aneesh]v1..v2 :
* None--- arch/powerpc/include/uapi/asm/papr_scm_pdsm.h | 153 ++++++++++++++++++ arch/powerpc/platforms/pseries/papr_scm.c | 101 +++++++++++- include/uapi/linux/ndctl.h | 1 + 3 files changed, 249 insertions(+), 6 deletions(-) create mode 100644 arch/powerpc/include/uapi/asm/papr_scm_pdsm.harch/powerpc/include/uapi/asm/papr_scm_pdsm.h new file +14
-0@@ -0,0 +1,153 @@ +/* SPDX-License-Identifier: GPL-2.0+ WITH Linux-syscall-note */ +/* +* PAPR-SCM Dimm specific methods (PDSM) and structs for libndctl +* +* (C) Copyright IBM 2020 +* +* Author: Vaibhav Jain <vaibhav at linux.ibm.com> +*/ + +#ifndef _UAPI_ASM_POWERPC_PAPR_SCM_PDSM_H_ +#define _UAPI_ASM_POWERPC_PAPR_SCM_PDSM_H_ + +#include <linux/types.h> +
Quoted 8 lines
arch/powerpc/include/uapi/asm/papr_scm_pdsm.h line 19
Quoted 5 lines
arch/powerpc/include/uapi/asm/papr_scm_pdsm.h new file +5
-0+#ifdef __KERNEL__ +#include <linux/ndctl.h> +#else +#include <ndctl.h> +#endif We shouldn't be checking
__KERNEL__in uapi headers, something has gone wrong if this is necessary.
This header file is shared with userspace libndctl changes as proposed at
https://lore.kernel.org/linux-nvdimm/[email protected]/
The definition of "struct nd_cmd_pkg" on which 'struct nd_pdsm_cmd_pkg' depenends on, is available in different locations in kernel and libndctl
Quoted 85 lines
Quoted 85 lines
arch/powerpc/include/uapi/asm/papr_scm_pdsm.h new file +84
-0+ +/* +* PDSM Envelope: +* +* The ioctl ND_CMD_CALL transfers data between user-space and kernel via +* 'envelopes' which consists of a header and user-defined payload sections. +* The header is described by 'struct nd_pdsm_cmd_pkg' which expects a +* payload following it and offset of which relative to the struct is provided +* by 'nd_pdsm_cmd_pkg.payload_offset'. * +* +* +-------------+---------------------+---------------------------+ +* | 64-Bytes | 8-Bytes | Max 184-Bytes | +* +-------------+---------------------+---------------------------+ +* | nd_pdsm_cmd_pkg | | +* |-------------+ | | +* | nd_cmd_pkg | | | +* +-------------+---------------------+---------------------------+ +* | nd_family | | | +* | nd_size_out | cmd_status | | +* | nd_size_in | payload_version | PAYLOAD | +* | nd_command | payload_offset -----> | +* | nd_fw_size | | | +* +-------------+---------------------+---------------------------+ +* +* PDSM Header: +* +* The header is defined as 'struct nd_pdsm_cmd_pkg' which embeds a +* 'struct nd_cmd_pkg' instance. The PDSM command is assigned to member +* 'nd_cmd_pkg.nd_command'. Apart from size information of the envelope which is +* contained in 'struct nd_cmd_pkg', the header also has members following +* members: +* +* 'cmd_status' : (Out) Errors if any encountered while servicing PDSM. +* 'payload_version' : (In/Out) Version number associated with the payload. +* 'payload_offset' : (In)Relative offset of payload from start of envelope. +* +* PDSM Payload: +* +* The layout of the PDSM Payload is defined by various structs shared between +* papr_scm and libndctl so that contents of payload can be interpreted. During +* servicing of a PDSM the papr_scm module will read input args from the payload +* field by casting its contents to an appropriate struct pointer based on the +* PDSM command. Similarly the output of servicing the PDSM command will be +* copied to the payload field using the same struct. +* +* 'libnvdimm' enforces a hard limit of 256 bytes on the envelope size, which +* leaves around 184 bytes for the envelope payload (ignoring any padding that +* the compiler may silently introduce). +* +* Payload Version: +* +* A 'payload_version' field is present in PDSM header that indicates a specific +* version of the structure present in PDSM Payload for a given PDSM command. +* This provides backward compatibility in case the PDSM Payload structure +* evolves and different structures are supported by 'papr_scm' and 'libndctl'. +* +* When sending a PDSM Payload to 'papr_scm', 'libndctl' should send the version +* of the payload struct it supports via 'payload_version' field. The 'papr_scm' +* module when servicing the PDSM envelope checks the 'payload_version' and then +* uses 'payload struct version' == MIN('payload_version field', +* 'max payload-struct-version supported by papr_scm') to service the PDSM. +* After servicing the PDSM, 'papr_scm' put the negotiated version of payload +* struct in returned 'payload_version' field. +* +* Libndctl on receiving the envelope back from papr_scm again checks the +* 'payload_version' field and based on it use the appropriate version dsm +* struct to parse the results. +* +* Backward Compatibility: +* +* Above scheme of exchanging different versioned PDSM struct between libndctl +* and papr_scm should provide backward compatibility until following two +* assumptions/conditions when defining new PDSM structs hold: +* +* Let T(X) = { set of attributes in PDSM struct 'T' versioned X } +* +* 1. T(X) is a proper subset of T(Y) if X > Y. +* i.e Each new version of PDSM struct should retain existing struct +* attributes from previous version +* +* 2. If an entity (libndctl or papr_scm) supports a PDSM struct T(X) then +* it should also support T(1), T(2)...T(X - 1). +* i.e When adding support for new version of a PDSM struct, libndctl +* and papr_scm should retain support of the existing PDSM struct
Quoted 9 lines
arch/powerpc/include/uapi/asm/papr_scm_pdsm.h line 109
Quoted 6 lines
arch/powerpc/include/uapi/asm/papr_scm_pdsm.h new file +6
-0+* version they support. +*/ + +#ifndef __packed +#define __packed __attribute__((packed)) +#endif I'm not sure it's kosher to be defining
__packedin a uapi header.
Similar reason as before. Userspace libndctl doesnt provide a definition of __packed hence adding it here.
Quoted 22 lines
Quoted 22 lines
arch/powerpc/include/uapi/asm/papr_scm_pdsm.h new file +22
-0+ +/* Papr-scm-header + payload expected with ND_CMD_CALL ioctl from libnvdimm */ +struct nd_pdsm_cmd_pkg { +struct nd_cmd_pkg hdr;/* Package header containing sub-cmd */ +__s32 cmd_status;/* Out: Sub-cmd status returned back */ +__u16 payload_offset;/* In: offset from start of struct */ +__u16 payload_version;/* In/Out: version of the payload */ +__u8 payload[];/* In/Out: Sub-cmd data buffer */ +} __packed; + +/* +* Methods to be embedded in ND_CMD_CALL request. These are sent to the kernel +* via 'nd_pdsm_cmd_pkg.hdr.nd_command' member of the ioctl struct +*/ +enum papr_scm_pdsm { +PAPR_SCM_PDSM_MIN =0x0 , +PAPR_SCM_PDSM_MAX , +}; + +/* Helpers to evaluate the size of PDSM envelope */ +/* Calculate the papr_scm-header size */ +#define ND_PDSM_ENVELOPE_CONTENT_HDR_SIZE \
Quoted 8 lines
arch/powerpc/include/uapi/asm/papr_scm_pdsm.h line 136
Quoted 5 lines
arch/powerpc/include/uapi/asm/papr_scm_pdsm.h new file +5
-0+ (sizeof (struct nd_pdsm_cmd_pkg ) -sizeof (struct nd_cmd_pkg )) + +/* Given a type calculate envelope-content size (papr_scm-header + payload) */ +#define ND_PDSM_ENVELOPE_CONTENT_SIZE (_type_) \ + (sizeof(_type_) + ND_PDSM_ENVELOPE_CONTENT_HDR_SIZE)Those seem unused? Do we really need to make them part of the uapi for all time?
This helper macro is used in proposed libndctl changes to calculate returning payload buffer size from the kernel.
https://lore.kernel.org/linux-nvdimm/[email protected]/
Quoted 11 lines
arch/powerpc/include/uapi/asm/papr_scm_pdsm.h line 145
Quoted 9 lines
arch/powerpc/include/uapi/asm/papr_scm_pdsm.h new file +8
-0+/* Convert a libnvdimm nd_cmd_pkg to pdsm specific pkg */ +static struct nd_pdsm_cmd_pkg *nd_to_pdsm_cmd_pkg (struct nd_cmd_pkg *cmd) +{ +return (struct nd_pdsm_cmd_pkg *) cmd; +} + +/* Return the payload pointer for a given pcmd */ +static void *pdsm_cmd_to_payload (struct nd_pdsm_cmd_pkg *pcmd)That should probably be static inline?
Agree, will do.
Quoted 152 lines
Quoted 152 lines
arch/powerpc/include/uapi/asm/papr_scm_pdsm.h new file +8
-0+{ +if (pcmd->hdr.nd_size_in ==0 && pcmd->hdr.nd_size_out ==0 ) +return NULL ; +else +return (void *)((__u8 *) pcmd + pcmd->payload_offset); +} + +#endif /* _UAPI_ASM_POWERPC_PAPR_SCM_PDSM_H_ */ arch/powerpc/platforms/pseries/papr_scm.c +95
-6@@ -15,13 +15,15 @@#include <asm/plpar_wrappers.h> #include <asm/papr_scm.h> +#include <asm/papr_scm_pdsm.h> #define BIND_ANY_ADDR (~0ul)#define PAPR_SCM_DIMM_CMD_MASK \ ((1ul << ND_CMD_GET_CONFIG_SIZE) | \ (1ul << ND_CMD_GET_CONFIG_DATA) | \ - (1ul << ND_CMD_SET_CONFIG_DATA)) + (1ul << ND_CMD_SET_CONFIG_DATA) | \ + (1ul << ND_CMD_CALL))struct papr_scm_priv {struct platform_device *pdev;@@ -306,16 +308,97 @@ static int papr_scm_meta_set(struct papr_scm_priv *p,return 0 ; } -int papr_scm_ndctl (struct nvdimm_bus_descriptor *nd_desc,struct nvdimm *nvdimm, -unsigned int cmd,void *buf,unsigned int buf_len,int *cmd_rc) +/* +* Validate the inputs args to dimm-control function and return '0' if valid. +* This also does initial sanity validation to ND_CMD_CALL sub-command packages. +*/ +static int is_cmd_valid (struct nvdimm *nvdimm,unsigned int cmd,void *buf, +unsigned int buf_len) { -struct nd_cmd_get_config_size *get_size_hdr; +unsigned long cmd_mask =PAPR_SCM_DIMM_CMD_MASK ; +struct nd_pdsm_cmd_pkg *pkg =nd_to_pdsm_cmd_pkg (buf);struct papr_scm_priv *p;/* Only dimm-specific calls are supported atm */ if (!nvdimm)return -EINVAL ; +/* get the provider date from struct nvdimm */ + p =nvdimm_provider_data (nvdimm); + +if (!test_bit (cmd, &cmd_mask)) { +dev_dbg (&p->pdev->dev,"Unsupported cmd=%u\n" , cmd); +return -EINVAL ; + }else if (cmd ==ND_CMD_CALL ) { + +/* Verify the envelope package */ +if (!buf || buf_len <sizeof (struct nd_pdsm_cmd_pkg )) { +dev_dbg (&p->pdev->dev,"Invalid pkg size=%u\n" , + buf_len); +return -EINVAL ; + } + +/* Verify that the PDSM family is valid */ +if (pkg->hdr.nd_family !=NVDIMM_FAMILY_PAPR_SCM ) { +dev_dbg (&p->pdev->dev,"Invalid pkg family=0x%llx\n" , + pkg->hdr.nd_family); +return -EINVAL ; + + } + +/* We except a payload with all PDSM commands */ +if (pdsm_cmd_to_payload (pkg) ==NULL ) { +dev_dbg (&p->pdev->dev, +"Empty payload for sub-command=0x%llx\n" , + pkg->hdr.nd_command); +return -EINVAL ; + } + } + +/* Command looks valid */ +return 0 ; +} + +static int papr_scm_service_pdsm (struct papr_scm_priv *p, +struct nd_pdsm_cmd_pkg *call_pkg) +{ +/* unknown subcommands return error in packages */ +if (call_pkg->hdr.nd_command <=PAPR_SCM_PDSM_MIN || + call_pkg->hdr.nd_command >=PAPR_SCM_PDSM_MAX ) { +dev_dbg (&p->pdev->dev,"Invalid PDSM request 0x%llx\n" , + call_pkg->hdr.nd_command); + call_pkg->cmd_status = -EINVAL ; +return 0 ; + } + +/* Depending on the DSM command call appropriate service routine */ +switch (call_pkg->hdr.nd_command) { +default : +dev_dbg (&p->pdev->dev,"Unsupported PDSM request 0x%llx\n" , + call_pkg->hdr.nd_command); + call_pkg->cmd_status = -ENOENT ; +return 0 ; + } +} + +int papr_scm_ndctl (struct nvdimm_bus_descriptor *nd_desc,struct nvdimm *nvdimm, +unsigned int cmd,void *buf,unsigned int buf_len,int *cmd_rc) +{ +struct nd_cmd_get_config_size *get_size_hdr; +struct papr_scm_priv *p; +struct nd_pdsm_cmd_pkg *call_pkg =NULL ; +int rc; + +/* Use a local variable in case cmd_rc pointer is NULL */ +if (cmd_rc ==NULL ) + cmd_rc = &rc; + + *cmd_rc =is_cmd_valid (nvdimm, cmd, buf, buf_len); +if (*cmd_rc) { +pr_debug ("Invalid cmd=0x%x. Err=%d\n" , cmd, *cmd_rc); +return *cmd_rc; + } + p =nvdimm_provider_data (nvdimm);switch (cmd) {@@ -336,13 +419,19 @@ int papr_scm_ndctl(struct nvdimm_bus_descriptor *nd_desc, struct nvdimm *nvdimm, *cmd_rc =papr_scm_meta_set (p, buf);break ; +case ND_CMD_CALL : + call_pkg =nd_to_pdsm_cmd_pkg (buf); + *cmd_rc =papr_scm_service_pdsm (p, call_pkg); +break ; +default : -return -EINVAL ; +dev_dbg (&p->pdev->dev,"Unknown command = %d\n" , cmd); + *cmd_rc = -EINVAL ; }dev_dbg (&p->pdev->dev,"returned with cmd_rc = %d\n" , *cmd_rc); -return 0 ; +return *cmd_rc; }
Quoted 13 lines
include/uapi/linux/ndctl.h struct nd_cmd_pkg { · line 247
Quoted 10 lines
arch/powerpc/platforms/pseries/papr_scm.c +0
-0static inline int papr_scm_node (int node)include/uapi/linux/ndctl.h +1
-0@@ -244,6 +244,7 @@ struct nd_cmd_pkg {#define NVDIMM_FAMILY_HPE2 2#define NVDIMM_FAMILY_MSFT 3#define NVDIMM_FAMILY_HYPERV 4 +#define NVDIMM_FAMILY_PAPR_SCM 5Would like an ack from Dan for that change.
Agree
Quoted 4 lines
cheers
_______________________________________________
Linux-nvdimm mailing list -- [email protected] To unsubscribe send an email to [email protected]
~ Vaibhav
_______________________________________________
Linux-nvdimm mailing list -- [email protected] To unsubscribe send an email to [email protected]
PATCH v6 4/4 powerpc/papr_scm: Implement support for PAPR_SCM_PDSM_HEALTH
In reply to Vaibhav JainThis patch implements support for PDSM request 'PAPR_SCM_PDSM_HEALTH' that returns a newly introduced 'struct nd_papr_pdsm_health' instance containing dimm health information back to user space in response to ND_CMD_CALL. This functionality is implemented in newly introduced papr_scm_get_health() that queries the scm-dimm health information and then copies this information to the package payload whose layout is defined by 'struct nd_papr_pdsm_health'.
The patch also introduces a new member 'struct papr_scm_priv.health' thats an instance of 'struct nd_papr_pdsm_health' to cache the health information of a nvdimm. As a result functions drc_pmem_query_health() and flags_show() are updated to populate and use this new struct instead of a u64 integer that was earlier used.
---
Changelog:
v5..v6:
* Added attribute '__packed' to 'struct nd_papr_pdsm_health_v1' to gaurd against possibility of different compilers adding different paddings to the struct [ Dan Williams ]
* Updated 'struct nd_papr_pdsm_health_v1' to use __u8 instead of 'bool' and also updated drc_pmem_query_health() to take this into account. [ Dan Williams ]
v4..v5:
* None
v3..v4:
* Call the DSM_PAPR_SCM_HEALTH service function from papr_scm_service_dsm() instead of papr_scm_ndctl(). [Aneesh]
v2..v3:
* Updated struct nd_papr_scm_dimm_health_stat_v1 to use '__xx' types as its exported to the userspace [Aneesh]
* Changed the constants DSM_PAPR_SCM_DIMM_XX indicating dimm health from enum to #defines [Aneesh]
v1..v2:
* New patch in the series
--- arch/powerpc/include/uapi/asm/papr_scm_pdsm.h | 39 ++++++ arch/powerpc/platforms/pseries/papr_scm.c | 122 +++++++++++++++--- 2 files changed, 145 insertions(+), 16 deletions(-)
arch/powerpc/include/uapi/asm/papr_scm_pdsm.h +39 -0
@@ -123,6 +123,7 @@ struct nd_pdsm_cmd_pkg {*/ enum papr_scm_pdsm {PAPR_SCM_PDSM_MIN =0x0 , +PAPR_SCM_PDSM_HEALTH ,PAPR_SCM_PDSM_MAX , };
@@ -150,4 +151,42 @@ static void *pdsm_cmd_to_payload(struct nd_pdsm_cmd_pkg *pcmd) return (void *)((__u8 *) pcmd + pcmd->payload_offset); } +/* Various scm-dimm health indicators */ +#define PAPR_PDSM_DIMM_HEALTHY 0 +#define PAPR_PDSM_DIMM_UNHEALTHY 1 +#define PAPR_PDSM_DIMM_CRITICAL 2 +#define PAPR_PDSM_DIMM_FATAL 3 + +/* +* Struct exchanged between kernel & ndctl in for PAPR_SCM_PDSM_HEALTH +* Various flags indicate the health status of the dimm. +* +* dimm_unarmed : Dimm not armed. So contents wont persist. +* dimm_bad_shutdown : Previous shutdown did not persist contents. +* dimm_bad_restore : Contents from previous shutdown werent restored. +* dimm_scrubbed : Contents of the dimm have been scrubbed. +* dimm_locked : Contents of the dimm cant be modified until CEC reboot +* dimm_encrypted : Contents of dimm are encrypted. +* dimm_health : Dimm health indicator. One of PAPR_PDSM_DIMM_XXXX +*/ +struct nd_papr_pdsm_health_v1 { +__u8 dimm_unarmed; +__u8 dimm_bad_shutdown; +__u8 dimm_bad_restore; +__u8 dimm_scrubbed; +__u8 dimm_locked; +__u8 dimm_encrypted; +__u16 dimm_health; +} __packed; + +/* +* Typedef the current struct for dimm_health so that any application +* or kernel recompiled after introducing a new version automatically +* supports the new version. +*/ +#define nd_papr_pdsm_health nd_papr_pdsm_health_v1 + +/* Current version number for the dimm health struct */ +#define ND_PAPR_PDSM_HEALTH_VERSION 1 +#endif /* _UAPI_ASM_POWERPC_PAPR_SCM_PDSM_H_ */
arch/powerpc/platforms/pseries/papr_scm.c +106 -16
182 lines not on this page · raw message
Signature · 4 lines
2.25.3 _______________________________________________ Linux-nvdimm mailing list -- [email protected] To unsubscribe send an email to [email protected]
Quoted 65 lines · Vaibhav Jain
The PAPR standard[1][3] provides mechanisms to query the health and performance stats of an NVDIMM via various hcalls as described in Ref[2]. Until now these stats were never available nor exposed to the user-space tools like 'ndctl'. This is partly due to PAPR platform not having support for ACPI and NFIT. Hence 'ndctl' is unable to query and report the dimm health status and a user had no way to determine the current health status of a NDVIMM.
To overcome this limitation, this patch-set updates
papr_scmkernel module to query and fetch nvdimm health stats using hcalls described in Ref[2]. This health and performance stats are then exposed to userspace via syfs and PAPR-nvDimm-Specific-Methods(PDSM) issued by libndctl.These changes coupled with proposed ndtcl changes located at Ref[4] should provide a way for the user to retrieve NVDIMM health status using ndtcl.
Below is a sample output using proposed kernel + ndctl for PAPR NVDIMM in a emulation environment:
# ndctl list -DH [ { "dev":"nmem0", "health":{ "health_state":"fatal", "shutdown_state":"dirty" } } ]Dimm health report output on a pseries guest lpar with vPMEM or HMS based nvdimms that are in perfectly healthy conditions:
# ndctl list -d nmem0 -H [ { "dev":"nmem0", "health":{ "health_state":"ok", "shutdown_state":"clean" } } ]PAPR nvDimm-Specific-Methods(PDSM)
==================================PDSM requests are issued by vendor specific code in libndctl to execute certain operations or fetch information from NVDIMMS. PDSMs requests can be sent to
papr_scmmodule via libndctl(userspace) and libnvdimm (kernel) using theND_CMD_CALLioctl command which can be handled in the dimm control functionpapr_scm_ndctl(). Current patchset proposes a single PDSM to retrieve NVDIMM health, defined in the newly introduced uapi header named 'papr_scm_pdsm.h'. Support for more PDSMs will be added in future.Structure of the patch-set
==========================The patchset starts with implementing support for fetching nvdimm health information from PHYP and partially exposing it to user-space via a nvdimm sysfs flag.
Quoted 6 lines
Second & Third patches deal with implementing support for servicing PDSM commands in
papr_scmmodule.Finally the Fourth patch implements support for servicing PDSM '
PAPR_SCM_PDSM_HEALTH' that returns the nvdimm health information to libndctl.
You can add to the series.
-aneesh
_______________________________________________
Linux-nvdimm mailing list -- [email protected] To unsubscribe send an email to [email protected]