Add support for reporting papr-scm nvdimm health
cover Add support for reporting papr-scm nvdimm health
This patch-set proposes changes to libndctl to add support for reporting health for nvdimms that support the PAPR standard[1]. The standard defines heathenism (HCALL) through which a guest kernel can query and fetch health and performance stats of an nvdimm attached to the hypervisor[2]. Until now 'ndctl' was unable to report these stats for papr_scm dimms on PPC64 guests due to absence of ACPI/NFIT, a limitation which this patch-set tries to address.
The patch-set introduces support for the new PAPR-SCM DSM family defined at
[3] via a new dimm-ops named 'papr_scm_dimm_ops'. Infrastructure to probe and distinguish papr-scm dimms from other dimm families that may support ACPI/NFIT is implemented by updating the 'struct ndctl_dimm' initialization routines to bifurcate based on the nvdimm type. We also introduce two new dimm-ops for handling initialization of dimm specific data for specific DSM families.
These changes coupled with proposed ndtcl changes located at Ref[4] should provide a way for the user to retrieve NVDIMM health status using ndtcl for papr_scm guests. Below is a sample output using proposed kernel + ndctl changes:
# ndctl list -DH
[
{
"dev":"nmem0",
"health":{
"health_state":"fatal",
"shutdown_state":"dirty"
}
}
]Structure of the patchset
=========================
We start with a re-factoring patch that splits the 'add_dimm()' function into two functions one that take care of allocating and initializing 'struct ndctl_dimm' and another that takes care of initializing nfit specific dimm attributes.
Patch-2 introduces new dimm ops 'dimm_init()' & 'dimm_uninit()' to handle DSM family specific initialization of 'struct ndctl_dimm'.
Patch-3 introduces probe function of papr_scm nvdimms and assigning 'papr_scm_dimm_ops' to 'dimm->ops' if needed. Subsequent patch-4 adds support for parsing papr_scm nvdimm flags and updating 'dimm->flags' with them.
Patches-5,6 implements scaffolding to add support for PAPR_SCM DSM commands and pull in their definitions from the kernel. This implementation resides in newly added 'papr_scm.c' file.
Finally Patched-7,8 add support for issuing and handling the result of 'struct ndctl_cmd' to request dimm health stats from papr_scm kernel module and returning appropriate health status to libndctl for reporting.
References:
[1]: "Power Architecture Platform Reference"
https://en.wikipedia.org/wiki/Power_Architecture_Platform_Reference
[2]: "[DOC,v2] powerpc: Provide initial documentation for PAPR hcalls"
https://patchwork.ozlabs.org/patch/1154292/ [3]: "[PATCH 5/8] powerpc/uapi: Introduce uapi header 'papr_scm_dsm.h' for papr_scm DSMs" https://patchwork.ozlabs.org/patch/1241352/ [4]: "powerpc/papr_scm: Add support for reporting nvdimm health" https://patchwork.ozlabs.org/project/linuxppc-dev/list/?series=159701
Vaibhav Jain (8):
libndctl: Refactor out add_dimm() to handle NFIT specific init
libndctl: Introduce a new dimm-ops dimm_init() & dimm_uninit()
libncdtl: Add initial support for NVDIMM_FAMILY_PAPR_SCM dimm family
libndctl: Add support for parsing of_pmem dimm flags and monitor mode libndctl,papr_scm: Add definitions for PAPR_SCM DSM commands libndctl,papr_scm: Implement scaffolding to issue and handle DSM cmds libndctl,papr_scm: Implement support for DSM_PAPR_SCM_HEALTH libndctl,papr_scm: Add support for reporting bad shutdown
ndctl/lib/Makefile.am | 1 + ndctl/lib/libndctl.c | 263 +++++++++++++++++++++++++---------- ndctl/lib/papr_scm.c | 292 +++++++++++++++++++++++++++++++++++++++ ndctl/lib/papr_scm_dsm.h | 143 +++++++++++++++++++ ndctl/lib/private.h | 7 + ndctl/libndctl.h | 1 + ndctl/ndctl.h | 1 + 7 files changed, 634 insertions(+), 74 deletions(-) create mode 100644 ndctl/lib/papr_scm.c create mode 100644 ndctl/lib/papr_scm_dsm.h
Signature · 4 lines
2.24.1 _______________________________________________ Linux-nvdimm mailing list -- [email protected] To unsubscribe send an email to [email protected]
PATCH 1/8 libndctl: Refactor out add_dimm() to handle NFIT specific init
Presently add_dimm() only probes dimms that support NFIT/ACPI. Hence this patch refactors this functionality into two functions namely add_dimm() and add_nfit_dimm(). Function add_dimm() performs allocation and common 'struct ndctl_dimm' initialization and depending on whether the dimm-bus supports NIFT, calls add_nfit_dimm(). Once the probe is completed based on the value of 'ndctl_dimm.cmd_family' appropriate dimm-ops are assigned to the dimm.
In case dimm-bus is of unknown type or doesn't support NFIT the initialization still continues, with no dimm-ops assigned to the 'struct ndctl_dimm' there-by limiting the functionality available.
This patch shouldn't introduce any behavioral change.
--- ndctl/lib/libndctl.c | 193 +++++++++++++++++++++++++------------------ 1 file changed, 112 insertions(+), 81 deletions(-)
ndctl/lib/libndctl.c +112 -81
@@ -1401,82 +1401,15 @@ static int ndctl_bind(struct ndctl_ctx *ctx, struct kmod_module *module,static int ndctl_unbind (struct ndctl_ctx *ctx,const char *devpath);static struct kmod_module *to_module (struct ndctl_ctx *ctx,const char *alias); -static void *add_dimm (void *parent,int id,const char *dimm_base) +static int add_nfit_dimm (struct ndctl_dimm *dimm,const char *dimm_base) { -int formats, i; -struct ndctl_dimm *dimm; +int i, rc =-1 ;char buf[SYSFS_ATTR_SIZE ]; -struct ndctl_bus *bus = parent; -struct ndctl_ctx *ctx = bus->ctx; +struct ndctl_ctx *ctx = dimm->bus->ctx;char *path =calloc (1 ,strlen (dimm_base) +100 );if (!path) -return NULL ; - -sprintf (path,"%s/nfit/formats" , dimm_base); -if (sysfs_read_attr (ctx, path, buf) <0 ) - formats =1 ; -else - formats =clamp (strtoul (buf,NULL ,0 ),1UL ,2UL ); - - dimm =calloc (1 ,sizeof (*dimm) +sizeof (int ) * formats); -if (!dimm) - goto err_dimm; - dimm->bus = bus; - dimm->id = id; - -sprintf (path,"%s/dev" , dimm_base); -if (sysfs_read_attr (ctx, path, buf) <0 ) - goto err_read; -if (sscanf (buf,"%d:%d" , &dimm->major, &dimm->minor) !=2 ) - goto err_read; - -sprintf (path,"%s/commands" , dimm_base); -if (sysfs_read_attr (ctx, path, buf) <0 ) - goto err_read; - dimm->cmd_mask =parse_commands (buf,1 ); - - dimm->dimm_buf =calloc (1 ,strlen (dimm_base) +50 ); -if (!dimm->dimm_buf) - goto err_read; - dimm->buf_len =strlen (dimm_base) +50 ; - - dimm->dimm_path =strdup (dimm_base); -if (!dimm->dimm_path) - goto err_read; - -sprintf (path,"%s/modalias" , dimm_base); -if (sysfs_read_attr (ctx, path, buf) <0 ) - goto err_read; - dimm->module =to_module (ctx, buf); - - dimm->handle =-1 ; - dimm->phys_id =-1 ; - dimm->serial =-1 ; - dimm->vendor_id =-1 ; - dimm->device_id =-1 ; - dimm->revision_id =-1 ; - dimm->health_eventfd =-1 ; - dimm->dirty_shutdown = -ENOENT ; - dimm->subsystem_vendor_id =-1 ; - dimm->subsystem_device_id =-1 ; - dimm->subsystem_revision_id =-1 ; - dimm->manufacturing_date =-1 ; - dimm->manufacturing_location =-1 ; - dimm->cmd_family =-1 ; - dimm->nfit_dsm_mask =ULONG_MAX ; -for (i =0 ; i < formats; i++) - dimm->format[i] =-1 ; - -sprintf (path,"%s/flags" , dimm_base); -if (sysfs_read_attr (ctx, path, buf) <0 ) { - dimm->locked =-1 ; - dimm->aliased =-1 ; - }else -parse_dimm_flags (dimm, buf); - -if (!ndctl_bus_has_nfit (bus)) - goto out; +return -1 ;/* * 'unique_id' may not be available on older kernels, so don't
@@ -1542,24 +1475,15 @@ static void *add_dimm(void *parent, int id, const char *dimm_base)sprintf (path,"%s/nfit/family" , dimm_base);if (sysfs_read_attr (ctx, path, buf) ==0 ) dimm->cmd_family =strtoul (buf,NULL ,0 ); -if (dimm->cmd_family ==NVDIMM_FAMILY_INTEL ) - dimm->ops = intel_dimm_ops; -if (dimm->cmd_family ==NVDIMM_FAMILY_HPE1 ) - dimm->ops = hpe1_dimm_ops; -if (dimm->cmd_family ==NVDIMM_FAMILY_MSFT ) - dimm->ops = msft_dimm_ops; -if (dimm->cmd_family ==NVDIMM_FAMILY_HYPERV ) - dimm->ops = hyperv_dimm_ops;sprintf (path,"%s/nfit/dsm_mask" , dimm_base);if (sysfs_read_attr (ctx, path, buf) ==0 ) dimm->nfit_dsm_mask =strtoul (buf,NULL ,0 ); - dimm->formats = formats;sprintf (path,"%s/nfit/format" , dimm_base);if (sysfs_read_attr (ctx, path, buf) ==0 ) dimm->format[0 ] =strtoul (buf,NULL ,0 ); -for (i =1 ; i < formats; i++) { +for (i =1 ; i < dimm->formats; i++) {sprintf (path,"%s/nfit/format%d" , dimm_base, i);if (sysfs_read_attr (ctx, path, buf) ==0 ) dimm->format[i] =strtoul (buf,NULL ,0 );
@@ -1570,7 +1494,114 @@ static void *add_dimm(void *parent, int id, const char *dimm_base)parse_nfit_mem_flags (dimm, buf); dimm->health_eventfd =open (path,O_RDONLY |O_CLOEXEC ); + rc =0 ; + err_read: + +free (path); +return rc; +} + +static void *add_dimm (void *parent,int id,const char *dimm_base) +{ +int formats, i, rc = -ENODEV ; +struct ndctl_dimm *dimm =NULL ; +char buf[SYSFS_ATTR_SIZE ]; +struct ndctl_bus *bus = parent; +struct ndctl_ctx *ctx = bus->ctx; +char *path =calloc (1 ,strlen (dimm_base) +100 ); + +if (!path) +return NULL ; + +sprintf (path,"%s/nfit/formats" , dimm_base); +if (sysfs_read_attr (ctx, path, buf) <0 ) + formats =1 ; +else + formats =clamp (strtoul (buf,NULL ,0 ),1UL ,2UL ); + + dimm =calloc (1 ,sizeof (*dimm) +sizeof (int ) * formats); +if (!dimm) + goto err_dimm; + dimm->bus = bus; + dimm->id = id; + +sprintf (path,"%s/dev" , dimm_base); +if (sysfs_read_attr (ctx, path, buf) <0 ) + goto err_read; +if (sscanf (buf,"%d:%d" , &dimm->major, &dimm->minor) !=2 ) + goto err_read; + +sprintf (path,"%s/commands" , dimm_base); +if (sysfs_read_attr (ctx, path, buf) <0 ) + goto err_read; + dimm->cmd_mask =parse_commands (buf,1 ); + + dimm->dimm_buf =calloc (1 ,strlen (dimm_base) +50 ); +if (!dimm->dimm_buf) + goto err_read; + dimm->buf_len =strlen (dimm_base) +50 ; + + dimm->dimm_path =strdup (dimm_base); +if (!dimm->dimm_path) + goto err_read; + +sprintf (path,"%s/modalias" , dimm_base); +if (sysfs_read_attr (ctx, path, buf) <0 ) + goto err_read; + dimm->module =to_module (ctx, buf); + + dimm->handle =-1 ; + dimm->phys_id =-1 ; + dimm->serial =-1 ; + dimm->vendor_id =-1 ; + dimm->device_id =-1 ; + dimm->revision_id =-1 ; + dimm->health_eventfd =-1 ; + dimm->dirty_shutdown = -ENOENT ; + dimm->subsystem_vendor_id =-1 ; + dimm->subsystem_device_id =-1 ; + dimm->subsystem_revision_id =-1 ; + dimm->manufacturing_date =-1 ; + dimm->manufacturing_location =-1 ; + dimm->cmd_family =-1 ; + dimm->nfit_dsm_mask =ULONG_MAX ; +for (i =0 ; i < formats; i++) + dimm->format[i] =-1 ; + +sprintf (path,"%s/flags" , dimm_base); +if (sysfs_read_attr (ctx, path, buf) <0 ) { + dimm->locked =-1 ; + dimm->aliased =-1 ; + }else +parse_dimm_flags (dimm, buf); + +/* Check if the given dimm supports nfit */ +if (ndctl_bus_has_nfit (bus)) { + dimm->formats = formats; + rc =add_nfit_dimm (dimm, dimm_base); + } + +if (rc == -ENODEV ) { +/* Unprobed dimm with no family */ + rc =0 ; + goto out; + } + +/* Assign dimm-ops based on command family */ +if (dimm->cmd_family ==NVDIMM_FAMILY_INTEL ) + dimm->ops = intel_dimm_ops; +if (dimm->cmd_family ==NVDIMM_FAMILY_HPE1 ) + dimm->ops = hpe1_dimm_ops; +if (dimm->cmd_family ==NVDIMM_FAMILY_MSFT ) + dimm->ops = msft_dimm_ops; +if (dimm->cmd_family ==NVDIMM_FAMILY_HYPERV ) + dimm->ops = hyperv_dimm_ops; out: +if (rc) { +err (ctx,"Unable to probe dimm:%d. Err:%d\n" , id, rc); + goto err_read; + } +list_add (&bus->dimms, &dimm->list);free (path);
Signature · 4 lines
2.24.1 _______________________________________________ Linux-nvdimm mailing list -- [email protected] To unsubscribe send an email to [email protected]
PATCH 2/8 libndctl: Introduce a new dimm-ops dimm_init() & dimm_uninit()
In reply to Vaibhav JainThere are scenarios when a dimm-provider need to allocate some per-dimm data that can be quickly retrieved. This data can be used to cache data that spans multiple 'struct ndctl_cmd' submissions.
Unfortunately currently in libnvdimm there is no easiy way to implement this. Even if this data is some how store in an overloaded field of 'struct ndctl_dimm', managing its lifetime is a challenge.
To solve this problem this patch proposes a new member 'struct ndctl_dimm.dimm_user_data' to store per-dimm data thats specific to a dimm-provider. Also two new dimm-ops namely dimm_init() & dimm_uninit() are introduced that can be used to manage the lifetime of this per-dimm data.
Semantics
=========
int (*dimm_init)(struct ndctl_dimm *):
This callback will be called just after dimm-probe inside add_dimm() is completed. Dimm-providers should use this callback to allocate per-dimm data and assign it to 'struct ndctl_dimm.dimm_user_data' member. In case this function returns an error, dimm initialization is halted and errors out.
void (*dimm_uninit)(struct ndctl_dimm *):
This callback will be called during free_dimm() and is only called if previous call to 'dimm_ops->dimm_init()' had reported no error. Dimm-providers should use this callback to unallocate and cleanup 'dimm_user_data'.
--- ndctl/lib/libndctl.c | 13 ++++++++++++- ndctl/lib/private.h | 5 +++++ 2 files changed, 17 insertions(+), 1 deletion(-)
ndctl/lib/libndctl.c +12 -1
@@ -596,6 +596,10 @@ static void free_dimm(struct ndctl_dimm *dimm) {if (!dimm)return ; +/* If needed call the dimm uninitialization function */ +if (dimm->ops && dimm->ops->dimm_uninit) + dimm->ops->dimm_uninit (dimm); +free (dimm->unique_id);free (dimm->dimm_buf);free (dimm->dimm_path);
@@ -1596,8 +1600,15 @@ static void *add_dimm(void *parent, int id, const char *dimm_base) dimm->ops = msft_dimm_ops;if (dimm->cmd_family ==NVDIMM_FAMILY_HYPERV ) dimm->ops = hyperv_dimm_ops; - out: + +/* Call the dimm initialization function if needed */ +if (!rc && dimm->ops && dimm->ops->dimm_init) + rc = dimm->ops->dimm_init (dimm); + +out:if (rc) { +/* Ensure dimm_uninit() is not called during free_dimm() */ + dimm->ops =NULL ;err (ctx,"Unable to probe dimm:%d. Err:%d\n" , id, rc); goto err_read; }
ndctl/lib/private.h +5 -0
@@ -98,6 +98,7 @@ struct ndctl_dimm { } flags;int locked;int aliased; +void *dimm_user_data;struct list_node list;int formats;int format[0 ];
@@ -340,6 +341,10 @@ struct ndctl_dimm_ops {struct ndctl_cmd *(*new_ack_shutdown_count)(struct ndctl_dimm *);int (*fw_update_supported)(struct ndctl_dimm *);int (*xlat_firmware_status)(struct ndctl_cmd *); +/* Called just after dimm is initialized and probed */ +int (*dimm_init)(struct ndctl_dimm *); +/* Called just before struct ndctl_dimm is de-allocated */ +void (*dimm_uninit)(struct ndctl_dimm *); };struct ndctl_dimm_ops *const intel_dimm_ops;
Signature · 4 lines
2.24.1 _______________________________________________ Linux-nvdimm mailing list -- [email protected] To unsubscribe send an email to [email protected]
PATCH 3/8 libncdtl: Add initial support for NVDIMM_FAMILY_PAPR_SCM dimm family
In reply to Vaibhav JainThe patch-set adds necessary scaffolding in libndctl for dimms that support papr_scm specification[1]. Since there can be platforms that support Open-Firmware[2] but not the papr_scm specification, hence the changes proposed first add support for probing if the dimm bus supports Open-Firmware. This is done via querying for sysfs attribute 'of_node' in dimm device sysfs directory. If available newly introduced member 'struct ndctl_bus.has_of_node' is set.
During the probe of the dimm and execution of add_dimm(), the newly introduced add_of_pmem_dimm() is called if dimm bus reports supports Open-Firmware.
Function add_of_pmem_dimm() queries the 'compatible' device tree attribute and based on its value assign NVDIMM_FAMILY_PAPR_SCM to the dimm command family. In future based on the contents of 'compatible' attribute more of_pmem dimm families can be queried.
Presently the dimm-ops implementation for NVDIMM_FAMILY_PAPR_SCM is available in global variable 'papr_scm_dimm_ops' which is a NULL pointer. Subsequent patches will provide a working dimm-ops implementation pointed to by 'papr_scm_dimm_ops'.
References:
[1] : https://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux.git/commit/?id=58b278f568f0509497e2df7310bfd719156a60d1
[2] : https://en.wikipedia.org/wiki/Open_Firmware
--- ndctl/lib/libndctl.c | 42 ++++++++++++++++++++++++++++++++++++++++++ ndctl/lib/private.h | 2 ++ ndctl/libndctl.h | 1 + ndctl/ndctl.h | 1 + 4 files changed, 46 insertions(+)
ndctl/lib/libndctl.c +42 -0
@@ -858,6 +858,12 @@ static void *add_bus(void *parent, int id, const char *ctl_base) bus->revision =strtoul (buf,NULL ,0 ); } +sprintf (path,"%s/device/of_node/compatible" , ctl_base); +if (sysfs_read_attr (ctx, path, buf) <0 ) + bus->has_of_node =0 ; +else + bus->has_of_node =1 ; +sprintf (path,"%s/device/nfit/dsm_mask" , ctl_base);if (sysfs_read_attr (ctx, path, buf) <0 ) bus->nfit_dsm_mask =0 ;
@@ -966,6 +972,10 @@ NDCTL_EXPORT int ndctl_bus_has_nfit(struct ndctl_bus *bus)return bus->has_nfit; } +NDCTL_EXPORT int ndctl_bus_has_of_node(struct ndctl_bus *bus) +{ +return bus->has_of_node; +}/** * ndctl_bus_get_major - nd bus character device major number * @bus: ndctl_bus instance returned from ndctl_bus_get_{first|next}
@@ -1405,6 +1415,34 @@ static int ndctl_bind(struct ndctl_ctx *ctx, struct kmod_module *module,static int ndctl_unbind (struct ndctl_ctx *ctx,const char *devpath);static struct kmod_module *to_module (struct ndctl_ctx *ctx,const char *alias); +static int add_of_pmem_dimm (struct ndctl_dimm *dimm,const char *dimm_base) +{ +int rc = -ENODEV ; +char buf[SYSFS_ATTR_SIZE ]; +struct ndctl_ctx *ctx = dimm->bus->ctx; +char *path =calloc (1 ,strlen (dimm_base) +100 ); + +dbg (ctx,"Probing of_pmem dimm %d at %s\n" , dimm->id, dimm_base); + +if (!path) +return -ENOMEM ; + +sprintf (path,"%s/../of_node/compatible" , dimm_base); +if (sysfs_read_attr (ctx, path, buf) <0 ) + goto out; + + +dbg (ctx,"Compatible of_pmem dimm %d at %s\n" , dimm->id, buf); +if (strcmp (buf,"ibm,pmemory" ) ==0 ) { + dimm->cmd_family =NVDIMM_FAMILY_PAPR_SCM ; + rc =0 ; + goto out; + } +out: +free (path); +return rc; +} +static int add_nfit_dimm (struct ndctl_dimm *dimm,const char *dimm_base) {int i, rc =-1 ;
@@ -1583,6 +1621,8 @@ static void *add_dimm(void *parent, int id, const char *dimm_base)if (ndctl_bus_has_nfit (bus)) { dimm->formats = formats; rc =add_nfit_dimm (dimm, dimm_base); + }else if (ndctl_bus_has_of_node (bus)) { + rc =add_of_pmem_dimm (dimm, dimm_base); }if (rc == -ENODEV ) {
@@ -1600,6 +1640,8 @@ static void *add_dimm(void *parent, int id, const char *dimm_base) dimm->ops = msft_dimm_ops;if (dimm->cmd_family ==NVDIMM_FAMILY_HYPERV ) dimm->ops = hyperv_dimm_ops; +if (dimm->cmd_family ==NVDIMM_FAMILY_PAPR_SCM ) + dimm->ops = papr_scm_dimm_ops;/* Call the dimm initialization function if needed */ if (!rc && dimm->ops && dimm->ops->dimm_init)
ndctl/lib/private.h +2 -0
@@ -167,6 +167,7 @@ struct ndctl_bus {int dimms_init;int regions_init;int has_nfit; +int has_of_node;char *bus_path;char *bus_buf;size_t buf_len;
@@ -351,6 +352,7 @@ struct ndctl_dimm_ops * const intel_dimm_ops;struct ndctl_dimm_ops *const hpe1_dimm_ops;struct ndctl_dimm_ops *const msft_dimm_ops;struct ndctl_dimm_ops *const hyperv_dimm_ops; +struct ndctl_dimm_ops *const papr_scm_dimm_ops;static inline struct ndctl_bus *cmd_to_bus (struct ndctl_cmd *cmd) {
ndctl/libndctl.h +1 -0
@@ -119,6 +119,7 @@ struct ndctl_bus *ndctl_bus_get_next(struct ndctl_bus *bus); bus = ndctl_bus_get_next(bus))struct ndctl_ctx *ndctl_bus_get_ctx (struct ndctl_bus *bus);int ndctl_bus_has_nfit (struct ndctl_bus *bus); +int ndctl_bus_has_of_node (struct ndctl_bus *bus);unsigned int ndctl_bus_get_major (struct ndctl_bus *bus);unsigned int ndctl_bus_get_minor (struct ndctl_bus *bus);const char *ndctl_bus_get_devname (struct ndctl_bus *bus);
ndctl/ndctl.h +1 -0
@@ -263,6 +263,7 @@ struct nd_cmd_pkg {#define NVDIMM_FAMILY_HPE2 2#define NVDIMM_FAMILY_MSFT 3#define NVDIMM_FAMILY_HYPERV 4 +#define NVDIMM_FAMILY_PAPR_SCM 5#define ND_IOCTL_CALL _IOWR(ND_IOCTL, ND_CMD_CALL,\ struct nd_cmd_pkg)
Signature · 4 lines
2.24.1 _______________________________________________ Linux-nvdimm mailing list -- [email protected] To unsubscribe send an email to [email protected]
PATCH 4/8 libndctl: Add support for parsing of_pmem dimm flags and monitor mode
In reply to Vaibhav JainAdd support for parsing the dimm flags for of_pmem supporting dimms. The flag file is exported in sysfs as 'papr_flags' and its contents are space separated flags indicating the current state of the dimm. A newly introduced function parse_of_pmem_flags() reads the contents of this flag file and sets appropriate flag bits in 'struct ndctl_dimm.flags'. This function is called at the end of of_pmem probe function add_of_pmem_dimm().
Also we advertise support for monitor mode by allocating a file descriptor to the 'papr_flags' file and assigning it to 'struct ndctl_dimm.health_event_fd'.
--- ndctl/lib/libndctl.c | 33 ++++++++++++++++++++++++++++++++- 1 file changed, 32 insertions(+), 1 deletion(-)
ndctl/lib/libndctl.c +32 -1
@@ -801,6 +801,28 @@ static void parse_nfit_mem_flags(struct ndctl_dimm *dimm, char *flags) ndctl_dimm_get_devname(dimm), flags); } +static void parse_of_pmem_flags (struct ndctl_dimm *dimm,char *flags) +{ +char *start, *end; + + start = flags; +while ((end =strchr (start,' ' ))) { + *end ='\0' ; +if (strcmp (start,"not_armed" ) ==0 ) + dimm->flags.f_arm =1 ; +else if (strcmp (start,"save_fail" ) ==0 ) + dimm->flags.f_save =1 ; +else if (strcmp (start,"flush_fail" ) ==0 ) + dimm->flags.f_flush =1 ; +else if (strcmp (start,"smart_notify" ) ==0 ) + dimm->flags.f_notify =1 ; + start = end +1 ; + } +if (end != start) +dbg (ndctl_dimm_get_ctx (dimm),"%s: %s\n" , +ndctl_dimm_get_devname (dimm), flags); +} +static void parse_dimm_flags (struct ndctl_dimm *dimm,char *flags) {char *start, *end;
@@ -1436,8 +1458,17 @@ static int add_of_pmem_dimm(struct ndctl_dimm *dimm, const char *dimm_base)if (strcmp (buf,"ibm,pmemory" ) ==0 ) { dimm->cmd_family =NVDIMM_FAMILY_PAPR_SCM ; rc =0 ; - goto out; + goto out_monitor; } + +out_monitor: +/* read the flags and also allocate the monitor mode event_fd */ +sprintf (path,"%s/papr_flags" , dimm_base); +if (sysfs_read_attr (ctx, path, buf) ==0 ) +parse_of_pmem_flags (dimm, buf); + +/* Allocate monitor mode fd */ + dimm->health_eventfd =open (path,O_RDONLY |O_CLOEXEC ); out:free (path);return rc;
Signature · 4 lines
2.24.1 _______________________________________________ Linux-nvdimm mailing list -- [email protected] To unsubscribe send an email to [email protected]
PATCH 5/8 libndctl,papr_scm: Add definitions for PAPR_SCM DSM commands
In reply to Vaibhav JainPull the kernel definition of PAPR_SCM DSM command which is located in the kernel tree at Ref[1]. Also add an implementation of 'papr_scm_dimm_ops' in a new file named 'papr_scm.c'. For now only an implementation of 'dimm_ops.cmd_is_supported' is provided.
References:
[1]: arch/powerpc/include/uapi/asm/papr_scm_dsm.h
--- ndctl/lib/Makefile.am | 1 + ndctl/lib/papr_scm.c | 34 ++++++++++ ndctl/lib/papr_scm_dsm.h | 143 +++++++++++++++++++++++++++++++++++++++ 3 files changed, 178 insertions(+) create mode 100644 ndctl/lib/papr_scm.c create mode 100644 ndctl/lib/papr_scm_dsm.h
ndctl/lib/Makefile.am +1 -0
@@ -21,6 +21,7 @@ libndctl_la_SOURCES =\ hpe1.c \ msft.c \ hyperv.c \ + papr_scm.c \ ars.c \ firmware.c \ libndctl.c
ndctl/lib/papr_scm.c new file +34 -0
@@ -0,0 +1,34 @@ +/* +* Copyright (C) 2020 IBM Corporation +* +* This program is free software; you can redistribute it and/or modify it +* under the terms and conditions of the GNU Lesser General Public License, +* version 2.1, as published by the Free Software Foundation. +* +* This program is distributed in the hope it will be useful, but WITHOUT ANY +* WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS +* FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for +* more details. +*/ +#include <stdint.h> +#include <stdlib.h> +#include <limits.h> +#include <endian.h> +#include <util/log.h> +#include <ndctl.h> +#include <ndctl/libndctl.h> +#include <lib/private.h> +#include <papr_scm_dsm.h> + +static bool papr_cmd_is_supported (struct ndctl_dimm *dimm,int cmd) +{ +/* Handle this separately to support monitor mode */ +if (cmd ==ND_CMD_SMART ) +return true; + +return !!(dimm->cmd_mask & (1ULL << cmd)); +} + +struct ndctl_dimm_ops *const papr_scm_dimm_ops = &(struct ndctl_dimm_ops ) { + .cmd_is_supported = papr_cmd_is_supported, +};
ndctl/lib/papr_scm_dsm.h new file +143 -0
143 lines not on this page · raw message
Signature · 4 lines
2.24.1 _______________________________________________ Linux-nvdimm mailing list -- [email protected] To unsubscribe send an email to [email protected]
PATCH 6/8 libndctl,papr_scm: Implement scaffolding to issue and handle DSM cmds
In reply to Vaibhav JainThis patch implement necessary infrastructure inside 'papr_scm.c' to issue and handle DSM commands. Changed implemented are:
* Implement dimm initialization/un-initialization functions papr_dimm_init()/unint() to allocate a per-dimm 'struct dimm_priv' instance.
* New helper function allocate_cmd() to allocate command packages for a specific DSM command and payload size.
* New function update_dimm_state() to parse a given command payload and update per dimm 'struct dimm_priv'.
* Provide an implementation of 'dimm_ops->smart_get_flags' to send the submitted instance of 'struct ndctl_cmd' to update_dimm_state().
* Logging helpers for papr_scm that use the underlying libndctl provided logging.
--- ndctl/lib/papr_scm.c | 174 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 174 insertions(+)
ndctl/lib/papr_scm.c +174 -0
@@ -20,6 +20,29 @@#include <lib/private.h> #include <papr_scm_dsm.h> +/* Utility logging maros for simplify logging */ +#define PAPR_DBG (_dimm_, _format_str_, ...) dbg(_dimm_->bus->ctx, \ + "papr_scm:"#_format_str_, \ + ##__VA_ARGS__) +#define PAPR_INFO (_dimm_, _format_str_, ...) info(_dimm_->bus->ctx, \ + "papr_scm:"#_format_str_, \ + ##__VA_ARGS__) +#define PAPR_ERR (_dimm_, _format_str_, ...) err(_dimm_->bus->ctx, \ + "papr_scm:"#_format_str_, \ + ##__VA_ARGS__) +#define PAPR_NOTICE (_dimm_, _format_str_, ...) notice(_dimm_->bus->ctx, \ + "papr_scm:"#_format_str_, \ + ##__VA_ARGS__) + +/* Command flags to indicate if a given command is parsed of not */ +#define CMD_PKG_SUBMITTED 1 +#define CMD_PKG_PARSED 2 + +/* Per dimm data. Holds per-dimm data parsed from the cmd_pkgs */ +struct dimm_priv { +/* Empty for now */ +}; +static bool papr_cmd_is_supported (struct ndctl_dimm *dimm,int cmd) {/* Handle this separately to support monitor mode */
@@ -29,6 +52,157 @@ static bool papr_cmd_is_supported(struct ndctl_dimm *dimm, int cmd)return !!(dimm->cmd_mask & (1ULL << cmd)); } +static __u64 pcmd_to_dsm (const struct nd_papr_scm_cmd_pkg *pcmd) +{ +return pcmd->hdr.nd_command; +} + +/* Verify if the given command is supported and valid */ +static bool cmd_is_valid (struct ndctl_dimm *dimm,struct ndctl_cmd *cmd) +{ +const struct nd_papr_scm_cmd_pkg *pcmd =nd_to_papr_cmd_pkg (cmd->pkg); + +if (dimm ==NULL ) +return false; + +if (cmd ==NULL ) { +PAPR_ERR (dimm,"Invalid command\n" ); +return false; + } + +/* Verify the command family */ +if (pcmd->hdr.nd_family !=NVDIMM_FAMILY_PAPR_SCM ) { +PAPR_ERR (dimm,"Invalid command family:0x%016llx\n" , + pcmd->hdr.nd_family); +return false; + } + +/* Verify the DSM */ +if (pcmd_to_dsm (pcmd) <=DSM_PAPR_SCM_MIN || +pcmd_to_dsm (pcmd) >=DSM_PAPR_SCM_MAX ) { +PAPR_ERR (dimm,"Invalid command :0x%016llx\n" , + pcmd->hdr.nd_command); +return false; + } + +return true; +} + +/* Parse a command payload and update dimm flags/private data */ +static int update_dimm_stats (struct ndctl_dimm *dimm,struct ndctl_cmd *cmd) +{ +const struct nd_papr_scm_cmd_pkg *pcmd; + +if (!cmd_is_valid (dimm, cmd)) +return -EINVAL ; + +/* +* Silently prevent parsing of an already parsed ndctl_cmd else +* mark the command as parsed. +*/ +if (cmd->status >=CMD_PKG_PARSED ) { +return 0 ; + }else if (cmd->status <0 ) { +PAPR_ERR (dimm,"Command error %d\n" , cmd->status); +return -ENXIO ; + } + +/* Mark the command as parsed */ + cmd->status =CMD_PKG_PARSED ; + +/* Get the command dsm and handle it */ + pcmd =nd_to_papr_cmd_pkg (cmd->pkg); +switch (pcmd_to_dsm (pcmd)) { +default : +PAPR_ERR (dimm,"Unhandled dsm-command 0x%016llx\n" , +pcmd_to_dsm (pcmd)); +return -ENOENT ; + } +} + +/* Allocate a struct ndctl_cmd for given dsm command with payload size */ +static struct ndctl_cmd *allocate_cmd (struct ndctl_dimm *dimm, +__u64 dsm_cmd,size_t payload_size, +uint16_t payload_version) +{ +struct ndctl_cmd *cmd; +struct nd_papr_scm_cmd_pkg *pcmd; +size_t size; + + size =sizeof (struct ndctl_cmd ) + +sizeof (struct nd_papr_scm_cmd_pkg ) + payload_size; + cmd =calloc (1 , size); +if (!cmd) +return NULL ; + pcmd =nd_to_papr_cmd_pkg (cmd->pkg); + +ndctl_cmd_ref (cmd); + cmd->dimm = dimm; + cmd->type =ND_CMD_CALL ; + cmd->size = size; + cmd->status =CMD_PKG_SUBMITTED ; + cmd->firmware_status = (u32 *) &pcmd->cmd_status; + +/* Populate the nd_cmd_pkg contained in nd_papr_scm_cmd_pkg */ + pcmd->hdr.nd_family =NVDIMM_FAMILY_PAPR_SCM ; + pcmd->hdr.nd_command = dsm_cmd; + + pcmd->payload_version = payload_version; + pcmd->payload_offset =sizeof (struct nd_papr_scm_cmd_pkg ); + +/* Keep payload size empty. To be populated by called */ + pcmd->hdr.nd_fw_size =0 ; + pcmd->hdr.nd_size_out =0 ; + pcmd->hdr.nd_size_in =0 ; + +return cmd; +} + +static unsigned int papr_smart_get_flags (struct ndctl_cmd *cmd) +{ +/* In case of error return empty flags * */ +if (update_dimm_stats (cmd->dimm, cmd)) +return 0 ; + +/* Return empty flags for now as no DSM support */ +return 0 ; +} + +static int papr_dimm_init (struct ndctl_dimm *dimm) +{ +struct dimm_priv *p; + +if (dimm->dimm_user_data) { +PAPR_DBG (dimm,"Dimm already initialized !!\n" ); +return 0 ; + } + + p =calloc (1 ,sizeof (struct dimm_priv )); +if (!p) { +PAPR_ERR (dimm,"Unable to allocate memory for dimm-private\n" ); +return -1 ; + } + + dimm->dimm_user_data = p; +return 0 ; +} + +static void papr_dimm_uninit (struct ndctl_dimm *dimm) +{ +struct dimm_priv *p = dimm->dimm_user_data; + +if (!p) { +PAPR_DBG (dimm,"Dimm already un-initialized !!\n" ); +return ; + } + + dimm->dimm_user_data =NULL ; +free (p); +} +struct ndctl_dimm_ops *const papr_scm_dimm_ops = &(struct ndctl_dimm_ops ) { .cmd_is_supported = papr_cmd_is_supported, + .dimm_init = papr_dimm_init, + .dimm_uninit = papr_dimm_uninit, + .smart_get_flags = papr_smart_get_flags, };
Signature · 4 lines
2.24.1 _______________________________________________ Linux-nvdimm mailing list -- [email protected] To unsubscribe send an email to [email protected]
PATCH 7/8 libndctl,papr_scm: Implement support for DSM_PAPR_SCM_HEALTH
In reply to Vaibhav JainAdd support for reporting DIMM health by issuing DSM_PAPR_SCM_HEALTH DSM. It returns an instance of '
struct nd_papr_scm_dimm_health_stat' as defined in 'papr_scm_dsm.h'. The patch provides support for dimm-ops 'new_smart' & 'smart_get_health' as papr_new_smart_health() & papr_smart_get_health() respectively. This callbacks should enable ndctl to report DIMM health.
Also a new member 'struct dimm_priv.health' is introduced which holds the current health status of the dimm. This member is set inside newly added function 'update_dimm_health_v1()' which parses the v1 payload returned by the kernel after servicing DSM_PAPR_SCM_HEALTH. The function will also update dimm-flags viz 'struct ndctl_dimm.flags.f_*' based on the flags set in the returned payload.
--- ndctl/lib/papr_scm.c | 80 ++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 77 insertions(+), 3 deletions(-)
ndctl/lib/papr_scm.c +77 -3
@@ -40,7 +40,9 @@/* Per dimm data. Holds per-dimm data parsed from the cmd_pkgs */ struct dimm_priv { -/* Empty for now */ + +/* Cache the dimm health status */ +struct nd_papr_scm_dimm_health_stat health; };static bool papr_cmd_is_supported (struct ndctl_dimm *dimm,int cmd)
@@ -88,6 +90,43 @@ static bool cmd_is_valid(struct ndctl_dimm *dimm, struct ndctl_cmd *cmd)return true; } +/* +* Parse the nd_papr_scm_dimm_health_stat_v1 payload embedded in ndctl_cmd and +* update dimm health/flags +*/ +static int update_dimm_health_v1 (struct ndctl_dimm *dimm,struct ndctl_cmd *cmd) +{ +struct nd_papr_scm_cmd_pkg *pcmd =nd_to_papr_cmd_pkg (cmd->pkg); +struct dimm_priv *p = dimm->dimm_user_data; +const struct nd_papr_scm_dimm_health_stat_v1 *health = +papr_scm_pcmd_to_payload (pcmd); + +/* Update the dimm flags */ + dimm->flags.f_arm = health->dimm_unarmed; + dimm->flags.f_flush = health->dimm_bad_shutdown; + dimm->flags.f_restore = health->dimm_bad_restore; + dimm->flags.f_smart = (health->dimm_health !=0 ); + +/* Cache the dimm health information */ +memcpy (&p->health, health,sizeof (*health)); +return 0 ; +} + +/* Check payload version returned and pass the packet to appropriate handler */ +static int update_dimm_health (struct ndctl_dimm *dimm,struct ndctl_cmd *cmd) +{ +const struct nd_papr_scm_cmd_pkg *pcmd =nd_to_papr_cmd_pkg (cmd->pkg); + +if (pcmd->payload_version ==1 ) +return update_dimm_health_v1 (dimm, cmd); + +/* unknown version */ +PAPR_ERR (dimm,"Unknown payload version for dimm_health." +"Ver=%d, Supported=%d\n" , pcmd->payload_version, +ND_PAPR_SCM_DIMM_HEALTH_VERSION ); +return -EINVAL ; +} +/* Parse a command payload and update dimm flags/private data */ static int update_dimm_stats (struct ndctl_dimm *dimm,struct ndctl_cmd *cmd) {
@@ -113,6 +152,8 @@ static int update_dimm_stats(struct ndctl_dimm *dimm, struct ndctl_cmd *cmd)/* Get the command dsm and handle it */ pcmd =nd_to_papr_cmd_pkg (cmd->pkg);switch (pcmd_to_dsm (pcmd)) { +case DSM_PAPR_SCM_HEALTH : +return update_dimm_health (dimm, cmd);default :PAPR_ERR (dimm,"Unhandled dsm-command 0x%016llx\n" ,pcmd_to_dsm (pcmd));
@@ -158,14 +199,45 @@ static struct ndctl_cmd *allocate_cmd(struct ndctl_dimm *dimm,return cmd; } +static struct ndctl_cmd *papr_new_smart_health (struct ndctl_dimm *dimm) +{ +struct ndctl_cmd *cmd_ret; + + cmd_ret =allocate_cmd (dimm,DSM_PAPR_SCM_HEALTH , +sizeof (struct nd_papr_scm_dimm_health_stat ), +ND_PAPR_SCM_DIMM_HEALTH_VERSION ); +if (!cmd_ret) { +PAPR_ERR (dimm,"Unable to allocate smart_health command\n" ); +return NULL ; + } + + cmd_ret->pkg[0 ].nd_size_out =ND_PAPR_SCM_ENVELOPE_CONTENT_SIZE ( + struct nd_papr_scm_dimm_health_stat); + +return cmd_ret; +} + +static unsigned int papr_smart_get_health (struct ndctl_cmd *cmd) +{ +struct dimm_priv *p = cmd->dimm->dimm_user_data; + +/* +* Update the dimm stats and use some math to return one of +* defined ND_SMART_*_HEALTH values +*/ +if (update_dimm_stats (cmd->dimm, cmd) || !p->health.dimm_health) +return 0 ; +else +return 1 << (p->health.dimm_health -1 ); +} +static unsigned int papr_smart_get_flags (struct ndctl_cmd *cmd) {/* In case of error return empty flags * */ if (update_dimm_stats (cmd->dimm, cmd))return 0 ; -/* Return empty flags for now as no DSM support */ -return 0 ; +return ND_SMART_HEALTH_VALID ; }static int papr_dimm_init (struct ndctl_dimm *dimm)
@@ -205,4 +277,6 @@ struct ndctl_dimm_ops * const papr_scm_dimm_ops = &(struct ndctl_dimm_ops) { .dimm_init = papr_dimm_init, .dimm_uninit = papr_dimm_uninit, .smart_get_flags = papr_smart_get_flags, + .new_smart = papr_new_smart_health, + .smart_get_health = papr_smart_get_health, };
Signature · 4 lines
2.24.1 _______________________________________________ Linux-nvdimm mailing list -- [email protected] To unsubscribe send an email to [email protected]
PATCH 8/8 libndctl,papr_scm: Add support for reporting bad shutdown
In reply to Vaibhav JainProvide support for dimm-op 'smart_get_shutdown_state' implemented as papr_smart_get_shutdown_state() to report a bad shutdown that couldn't flush contents of nvdimm to flash memory properly.
--- ndctl/lib/papr_scm.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-)
ndctl/lib/papr_scm.c +11 -1
@@ -231,13 +231,22 @@ static unsigned int papr_smart_get_health(struct ndctl_cmd *cmd)return 1 << (p->health.dimm_health -1 ); } +static unsigned int papr_smart_get_shutdown_state (struct ndctl_cmd *cmd) +{ +struct dimm_priv *p = cmd->dimm->dimm_user_data; + +/* Update dimm state and return f_flush */ +return update_dimm_stats (cmd->dimm, cmd) ? +0 : p->health.dimm_bad_shutdown; +} +static unsigned int papr_smart_get_flags (struct ndctl_cmd *cmd) {/* In case of error return empty flags * */ if (update_dimm_stats (cmd->dimm, cmd))return 0 ; -return ND_SMART_HEALTH_VALID ; +return ND_SMART_HEALTH_VALID |ND_SMART_SHUTDOWN_VALID ; }static int papr_dimm_init (struct ndctl_dimm *dimm)
@@ -279,4 +288,5 @@ struct ndctl_dimm_ops * const papr_scm_dimm_ops = &(struct ndctl_dimm_ops) { .smart_get_flags = papr_smart_get_flags, .new_smart = papr_new_smart_health, .smart_get_health = papr_smart_get_health, + .smart_get_shutdown_state = papr_smart_get_shutdown_state, };
Signature · 4 lines
2.24.1 _______________________________________________ Linux-nvdimm mailing list -- [email protected] To unsubscribe send an email to [email protected]