Skip to contentSkip to conversation

libndctl: Update nvdimm flags after inject-smart

PATCH Superseded Vaibhav Jain 2 participants 3 messages Jul 13, 2021 – Jul 14, 2021 libnvdimm: non-volatile memory device

v1 of 5 Superseded Latest is v5 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 libndctl: Update nvdimm flags after inject-smart

Presently after performing a inject-smart the nvdimm flags reported are out of date as shown below where no 'smart_notify' or 'flush_fail' flags were reported even though they are set after injecting the smart error:

$ sudo inject-smart -fU nmem0
[
  {
    "dev":"nmem0",
    "health":{
      "health_state":"fatal",
      "shutdown_state":"dirty",
      "shutdown_count":0
    }
  }
]
$ sudo cat /sys/class/nd/ndctl0/device/nmem0/papr/flags
flush_fail smart_notify

This happens because nvdimm flags are only parsed once during its probe and not refreshed even after a inject-smart operation makes them out of date. To fix this the patch adds a new export from libndctl named as ndctl_refresh_dimm_flags() that can be called after inject-smart that forces a refresh of nvdimm flags. This ensures that correct nvdimm flags are reported after the inject-smart operation as shown below:

$ sudo ndctl inject-smart -fU nmem0
[
  {
    "dev":"nmem0",
    "flag_failed_flush":true,
    "flag_smart_event":true,
    "health":{
      "health_state":"fatal",
      "shutdown_state":"dirty",
      "shutdown_count":0
    }
  }
]

The patch refactors populate_dimm_attributes() to move the nvdimm flags parsing code to the newly introduced ndctl_refresh_dimm_flags() export. Since reading nvdimm flags requires constructing path using 'bus_prefix' which is only available during add_dimm(), the patch introduces a new member 'struct ndctl_dimm.bus_prefix' to cache its value. During ndctl_refresh_dimm_flags() the cached bus_prefix is used to read the contents of the nvdimm flag file and pass it on to the appropriate flag parsing function.

---
 ndctl/inject-smart.c   |  3 +++
 ndctl/lib/libndctl.c   | 45 +++++++++++++++++++++++++++++-------------
 ndctl/lib/libndctl.sym |  4 ++++
 ndctl/lib/private.h    |  1 +
 ndctl/libndctl.h       |  1 +
 5 files changed, 40 insertions(+), 14 deletions(-)
ndctl/inject-smart.c +3 -0
index ef0620f55531..6d9abe612e73 100644
@@ -463,6 +463,9 @@ static int dimm_inject_smart(struct ndctl_dimm *dimm)
 			goto out;
 	}
 
+	/* Force update of dimm flags after smart-inject */
+	ndctl_refresh_dimm_flags(dimm);
+
 	if (rc == 0) {
 		jdimms = json_object_new_array();
 		if (!jdimms)
ndctl/lib/libndctl.c +31 -14

82 lines not on this page · raw message

ndctl/lib/libndctl.sym +4 -0
index 58afb74229fe..a498535af283 100644
@@ -455,3 +455,7 @@ LIBNDCTL_25 {
 LIBNDCTL_26 {
 	ndctl_bus_nfit_translate_spa;
 } LIBNDCTL_25;
+
+LIBNDCTL_27 {
+	ndctl_refresh_dimm_flags;
+} LIBNDCTL_26;
ndctl/lib/private.h +1 -0
index 8f4510e562c4..a4b920ca0c94 100644
@@ -75,6 +75,7 @@ struct ndctl_dimm {
 	char *unique_id;
 	char *dimm_path;
 	char *dimm_buf;
+	char *bus_prefix;
 	int health_eventfd;
 	int buf_len;
 	int id;
ndctl/libndctl.h +1 -0
index 3a5013007038..54539ac7a6c2 100644
@@ -221,6 +221,7 @@ int ndctl_dimm_is_active(struct ndctl_dimm *dimm);
 int ndctl_dimm_is_enabled(struct ndctl_dimm *dimm);
 int ndctl_dimm_disable(struct ndctl_dimm *dimm);
 int ndctl_dimm_enable(struct ndctl_dimm *dimm);
+void ndctl_refresh_dimm_flags(struct ndctl_dimm *dimm);
 
 struct ndctl_cmd;
 struct ndctl_cmd *ndctl_bus_cmd_new_ars_cap(struct ndctl_bus *bus,
Signature · 1 line
2.31.1
#1 lore raw
Quoted 40 lines · Vaibhav Jain

Presently after performing a inject-smart the nvdimm flags reported are out of date as shown below where no 'smart_notify' or 'flush_fail' flags were reported even though they are set after injecting the smart error:

$ sudo inject-smart -fU nmem0
[
  {
    "dev":"nmem0",
    "health":{
      "health_state":"fatal",
      "shutdown_state":"dirty",
      "shutdown_count":0
    }
  }
]
$ sudo cat /sys/class/nd/ndctl0/device/nmem0/papr/flags
flush_fail smart_notify

This happens because nvdimm flags are only parsed once during its probe and not refreshed even after a inject-smart operation makes them out of date. To fix this the patch adds a new export from libndctl named as ndctl_refresh_dimm_flags() that can be called after inject-smart that forces a refresh of nvdimm flags. This ensures that correct nvdimm flags are reported after the inject-smart operation as shown below:

$ sudo ndctl inject-smart -fU nmem0
[
  {
    "dev":"nmem0",
    "flag_failed_flush":true,
    "flag_smart_event":true,
    "health":{
      "health_state":"fatal",
      "shutdown_state":"dirty",
      "shutdown_count":0
    }
  }
]
Quoted 8 lines

The patch refactors populate_dimm_attributes() to move the nvdimm flags parsing code to the newly introduced ndctl_refresh_dimm_flags() export. Since reading nvdimm flags requires constructing path using 'bus_prefix' which is only available during add_dimm(), the patch introduces a new member 'struct ndctl_dimm.bus_prefix' to cache its value. During ndctl_refresh_dimm_flags() the cached bus_prefix is used to read the contents of the nvdimm flag file and pass it on to the appropriate flag parsing function.

I think this can be handled without needing an explicit ndctl_refresh_dimm_flags() api. Teach all the flag retrieval apis to check if the cached value has been invalidated and re-parse the flags. Then teach the inject-smart path to invalidate the cached copy of the flags.

#2 lore raw

Thanks for looking into this patch Dan

Quoted 50 lines · Dan Williams
Quoted 40 lines · Vaibhav Jain

Presently after performing a inject-smart the nvdimm flags reported are out of date as shown below where no 'smart_notify' or 'flush_fail' flags were reported even though they are set after injecting the smart error:

$ sudo inject-smart -fU nmem0
[
  {
    "dev":"nmem0",
    "health":{
      "health_state":"fatal",
      "shutdown_state":"dirty",
      "shutdown_count":0
    }
  }
]
$ sudo cat /sys/class/nd/ndctl0/device/nmem0/papr/flags
flush_fail smart_notify

This happens because nvdimm flags are only parsed once during its probe and not refreshed even after a inject-smart operation makes them out of date. To fix this the patch adds a new export from libndctl named as ndctl_refresh_dimm_flags() that can be called after inject-smart that forces a refresh of nvdimm flags. This ensures that correct nvdimm flags are reported after the inject-smart operation as shown below:

$ sudo ndctl inject-smart -fU nmem0
[
  {
    "dev":"nmem0",
    "flag_failed_flush":true,
    "flag_smart_event":true,
    "health":{
      "health_state":"fatal",
      "shutdown_state":"dirty",
      "shutdown_count":0
    }
  }
]
Quoted 8 lines

The patch refactors populate_dimm_attributes() to move the nvdimm flags parsing code to the newly introduced ndctl_refresh_dimm_flags() export. Since reading nvdimm flags requires constructing path using 'bus_prefix' which is only available during add_dimm(), the patch introduces a new member 'struct ndctl_dimm.bus_prefix' to cache its value. During ndctl_refresh_dimm_flags() the cached bus_prefix is used to read the contents of the nvdimm flag file and pass it on to the appropriate flag parsing function.

Quoted 6 lines

I think this can be handled without needing an explicit ndctl_refresh_dimm_flags() api. Teach all the flag retrieval apis to check if the cached value has been invalidated and re-parse the flags. Then teach the inject-smart path to invalidate the cached copy of the flags.

Thanks for the suggestion. On the same line I think refreshing nvdimm flags can also be done implicitly after a successful ndctl_cmd_submit() call as that point we have interacted with the kernel_module which may have triggered a nvdimm flags change.

Such a changeset would also be smaller compared to updating all flags retrieval api's to invalidate and refresh nvdimm flags.

Signature · 2 lines
Cheers
~ Vaibhav