daxctl: add a new reconfigure-device command
Subsystems (2)
Vishal Verma
maintainerv3 cover daxctl: add a new reconfigure-device command
Changes in v3:
- In daxctl_dev_get_mode(), remove the subsystem warning, detect dax-class and simply make it return devdax
Changes in v2:
- Add examples to the documentation page (Dave Hansen)
- Clarify documentation regarding the conversion from system-ram to devdax
- Remove any references to a persistent config from the documentation - those can be added when the feature is added.
- device.c: validate option compatibility
- daxctl-list: display numa_node for device listings
- daxctl-list: display mode for device listings
- make the options more consistent by adding a '-O' short option for --attempt-offline
Add a new daxctl-reconfigure-device command that lets us reconfigure DAX devices back and forth between 'system-ram' and 'device-dax' modes. It also includes facilities to online any newly hot-plugged memory (default), and attempt to offline memory before converting away from the system-ram mode (not default, requires a --attempt-offline option).
Currently missing from this series is a way to persistently store which devices have been 'marked' for use as system-ram. This depends on a config system overhaul in ndctl, and patches for those will follow separately and are independent of this work.
Example invocations:
1. Reconfigure dax0.0 to system-ram mode, don’t online the memory
# daxctl reconfigure-device --mode=system-ram --no-online dax0.0
[
{
"chardev":"dax0.0",
"size":16777216000,
"numa_node":2,
"mode":"system-ram"
}
]2. Reconfigure dax0.0 to devdax mode, attempt to offline the memory
# daxctl reconfigure-device --human --mode=devdax --attempt-offline dax0.0
{
"chardev":"dax0.0",
"size":"15.63 GiB (16.78 GB)",
"numa_node":2,
"mode":"devdax"
}3. Reconfigure all dax devices on region0 to system-ram mode
# daxctl reconfigure-device --mode=system-ram --region=0 all
[
{
"chardev":"dax0.0",
"size":16777216000,
"numa_node":2,
"mode":"system-ram"
},
{
"chardev":"dax0.1",
"size":16777216000,
"numa_node":3,
"mode":"system-ram"
}
]These patches can also be found in the 'kmem-pending' branch on github:
https://github.com/pmem/ndctl/tree/kmem-pending
Vishal Verma (10):
libdaxctl: add interfaces in support of device modes
libdaxctl: cache 'subsystem' in daxctl_ctx
libdaxctl: add interfaces to enable/disable devices
libdaxctl: add interfaces to get/set the online state for a node daxctl/list: add numa_node for device listings
libdaxctl: add an interface to get the mode for a dax device
daxctl: add a new reconfigure-device command Documentation/daxctl: add a man page for daxctl-reconfigure-device contrib/ndctl: fix region-id completions for daxctl contrib/ndctl: add bash-completion for daxctl-reconfigure-device
Documentation/daxctl/Makefile.am | 3 +- .../daxctl/daxctl-reconfigure-device.txt | 118 ++++ contrib/ndctl | 34 +- daxctl/Makefile.am | 2 + daxctl/builtin.h | 1 + daxctl/daxctl.c | 1 + daxctl/device.c | 237 ++++++++ daxctl/lib/Makefile.am | 3 +- daxctl/lib/libdaxctl-private.h | 21 + daxctl/lib/libdaxctl.c | 552 +++++++++++++++++- daxctl/lib/libdaxctl.sym | 14 + daxctl/libdaxctl.h | 16 + util/json.c | 22 + 13 files changed, 1013 insertions(+), 11 deletions(-) create mode 100644 Documentation/daxctl/daxctl-reconfigure-device.txt create mode 100644 daxctl/device.c
Signature · 6 lines
2.20.1 _______________________________________________ Linux-nvdimm mailing list [email protected] https://lists.01.org/mailman/listinfo/linux-nvdimm
Vishal Verma
maintainerPATCH v3 1/10 libdaxctl: add interfaces in support of device modes
In preparation for libdaxctl and daxctl to grow operational modes for DAX devices, add the following supporting APIs:
daxctl_dev_get_ctx
daxctl_dev_is_enabled
--- daxctl/lib/libdaxctl.c | 30 ++++++++++++++++++++++++++++++ daxctl/lib/libdaxctl.sym | 6 ++++++ daxctl/libdaxctl.h | 2 ++ 3 files changed, 38 insertions(+)
daxctl/lib/libdaxctl.c +30 -0
@@ -559,6 +559,36 @@ static void dax_regions_init(struct daxctl_ctx *ctx) } } +static int is_enabled (const char *drvpath) +{ +struct stat st; + +if (lstat (drvpath, &st) <0 || !S_ISLNK (st.st_mode)) +return 0 ; +else +return 1 ; +} + +DAXCTL_EXPORT int daxctl_dev_is_enabled(struct daxctl_dev *dev) +{ +struct daxctl_ctx *ctx =daxctl_dev_get_ctx (dev); +char *path = dev->dev_buf; +int len = dev->buf_len; + +if (snprintf (path, len,"%s/driver" , dev->dev_path) >= len) { +err (ctx,"%s: buffer too small!\n" , +daxctl_dev_get_devname (dev)); +return 0 ; + } + +return is_enabled (path); +} + +DAXCTL_EXPORT struct daxctl_ctx *daxctl_dev_get_ctx(struct daxctl_dev *dev) +{ +return dev->region->ctx; +} + DAXCTL_EXPORT struct daxctl_dev *daxctl_dev_get_first(struct daxctl_region *region) {dax_devices_init (region);
daxctl/lib/libdaxctl.sym +6 -0
@@ -50,3 +50,9 @@ LIBDAXCTL_5 { global: daxctl_region_get_path; } LIBDAXCTL_4; + +LIBDAXCTL_6 { +global: + daxctl_dev_get_ctx; + daxctl_dev_is_enabled; +} LIBDAXCTL_5;
daxctl/libdaxctl.h +2 -0
@@ -67,6 +67,8 @@ const char *daxctl_dev_get_devname(struct daxctl_dev *dev);int daxctl_dev_get_major (struct daxctl_dev *dev);int daxctl_dev_get_minor (struct daxctl_dev *dev);unsigned long long daxctl_dev_get_size (struct daxctl_dev *dev); +struct daxctl_ctx *daxctl_dev_get_ctx (struct daxctl_dev *dev); +int daxctl_dev_is_enabled (struct daxctl_dev *dev);#define daxctl_dev_foreach (region, dev) \ for (dev = daxctl_dev_get_first(region); \
Signature · 6 lines
2.20.1 _______________________________________________ Linux-nvdimm mailing list [email protected] https://lists.01.org/mailman/listinfo/linux-nvdimm
Vishal Verma
maintainerPATCH v3 2/10 libdaxctl: cache 'subsystem' in daxctl_ctx
In reply to Vishal VermaThe 'DAX subsystem' in effect is determined at region or device init time, and dictates the sysfs base paths for all device/region operations. In preparation for adding bind/unbind functionality, cache the subsystem as determined at init time in the library context.
--- daxctl/lib/libdaxctl.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-)
daxctl/lib/libdaxctl.c +10 -2
@@ -46,6 +46,7 @@ struct daxctl_ctx {void *userdata;int regions_init;struct list_head regions; +enum dax_subsystem subsys; };/**
@@ -96,6 +97,7 @@ DAXCTL_EXPORT int daxctl_new(struct daxctl_ctx **ctx)dbg (c,"log_priority=%d\n" , c->ctx.log_priority); *ctx = c;list_head_init (&c->regions); + c->subsys =DAX_UNKNOWN ;return 0 ; }
@@ -454,14 +456,18 @@ static void dax_devices_init(struct daxctl_region *region)for (i =0 ; i <ARRAY_SIZE (dax_subsystems); i++) {char *region_path; -if (i ==DAX_BUS ) +if (i ==DAX_BUS ) { region_path = region->region_path; -else if (i ==DAX_CLASS ) { +if (ctx->subsys ==DAX_UNKNOWN ) + ctx->subsys =DAX_BUS ; + }else if (i ==DAX_CLASS ) {if (asprintf (®ion_path,"%s/dax" , region->region_path) <0 ) {dbg (ctx,"region path alloc fail\n" );continue ; } +if (ctx->subsys ==DAX_UNKNOWN ) + ctx->subsys =DAX_CLASS ; }else continue ; sysfs_device_parse(ctx, region_path, daxdev_fmt, region,
@@ -539,6 +545,8 @@ static void __dax_regions_init(struct daxctl_ctx *ctx, enum dax_subsystem subsysfree (dev_path);if (!region)err (ctx,"add_dax_region() for %s failed\n" , de->d_name); +if (ctx->subsys ==DAX_UNKNOWN ) + ctx->subsys = subsys; }closedir (dir); }
Signature · 6 lines
2.20.1 _______________________________________________ Linux-nvdimm mailing list [email protected] https://lists.01.org/mailman/listinfo/linux-nvdimm
Vishal Verma
maintainerPATCH v3 3/10 libdaxctl: add interfaces to enable/disable devices
In reply to Vishal VermaAdd new libdaxctl interfaces to disable a device_dax based devices, and to enable it into a given mode. The modes available are 'device_dax', and 'system-ram', where device_dax is the normal device DAX mode used via a character device, and 'system-ram' uses the kernel's 'kmem' facility to hotplug the device into the system's memory space, and can be used as normal system memory.
This adds the following new interfaces:
daxctl_dev_disable;
daxctl_dev_enable_devdax;
daxctl_dev_enable_ram;
--- daxctl/lib/Makefile.am | 3 +- daxctl/lib/libdaxctl-private.h | 15 ++ daxctl/lib/libdaxctl.c | 241 +++++++++++++++++++++++++++++++++ daxctl/lib/libdaxctl.sym | 3 + daxctl/libdaxctl.h | 9 ++ 5 files changed, 270 insertions(+), 1 deletion(-)
daxctl/lib/Makefile.am +2 -1
@@ -16,7 +16,8 @@ libdaxctl_la_SOURCES =\ libdaxctl.c libdaxctl_la_LIBADD =\ - $(UUID_LIBS) + $(UUID_LIBS) \ + $(KMOD_LIBS) daxctl_modprobe_data_DATA = daxctl.conf
daxctl/lib/libdaxctl-private.h +15 -0
@@ -13,6 +13,8 @@#ifndef _LIBDAXCTL_PRIVATE_H_#define _LIBDAXCTL_PRIVATE_H_ +#include <libkmod.h> +#define DAXCTL_EXPORT __attribute__ ((visibility("default"))) enum dax_subsystem {
@@ -26,6 +28,11 @@ static const char *dax_subsystems[] = { [DAX_BUS] = "/sys/bus/dax/devices", }; +static const char *dax_modules[] = { + [DAXCTL_DEV_MODE_DEVDAX ] ="device_dax" , + [DAXCTL_DEV_MODE_RAM ] ="kmem" , +}; +/** * struct daxctl_region - container for dax_devices */
@@ -53,6 +60,14 @@ struct daxctl_dev {char *dev_path;struct list_node list;unsigned long long size; +struct kmod_module *module; +struct kmod_list *kmod_list; struct daxctl_region *region; }; + +static inline int check_kmod (struct kmod_ctx *kmod_ctx) +{ +return kmod_ctx ?0 : -ENXIO ; +} +#endif /* _LIBDAXCTL_PRIVATE_H_ */
daxctl/lib/libdaxctl.c +241 -0
309 lines not on this page · raw message
daxctl/lib/libdaxctl.sym +3 -0
@@ -55,4 +55,7 @@ LIBDAXCTL_6 { global: daxctl_dev_get_ctx; daxctl_dev_is_enabled; + daxctl_dev_disable; + daxctl_dev_enable_devdax; + daxctl_dev_enable_ram; } LIBDAXCTL_5;
daxctl/libdaxctl.h +9 -0
@@ -69,6 +69,9 @@ int daxctl_dev_get_minor(struct daxctl_dev *dev);unsigned long long daxctl_dev_get_size (struct daxctl_dev *dev);struct daxctl_ctx *daxctl_dev_get_ctx (struct daxctl_dev *dev);int daxctl_dev_is_enabled (struct daxctl_dev *dev); +int daxctl_dev_disable (struct daxctl_dev *dev); +int daxctl_dev_enable_devdax (struct daxctl_dev *dev); +int daxctl_dev_enable_ram (struct daxctl_dev *dev);#define daxctl_dev_foreach (region, dev) \ for (dev = daxctl_dev_get_first(region); \
@@ -81,6 +84,12 @@ int daxctl_dev_is_enabled(struct daxctl_dev *dev); region !=NULL ; \ region = daxctl_region_get_next(region)) +enum daxctl_dev_mode { +DAXCTL_DEV_MODE_UNKNOWN , +DAXCTL_DEV_MODE_RAM , +DAXCTL_DEV_MODE_DEVDAX , +}; + #ifdef __cplusplus }/* extern "C" */ #endif
Signature · 6 lines
2.20.1 _______________________________________________ Linux-nvdimm mailing list [email protected] https://lists.01.org/mailman/listinfo/linux-nvdimm
Vishal Verma
maintainerPATCH v3 4/10 libdaxctl: add interfaces to get/set the online state for a node
In reply to Vishal VermaIn preparation for converting DAX devices for use as system-ram via the kernel's 'kmem' facility, add libndctl helpers to manipulate the sysfs interfaces to get the target_node of a DAX device, and to online, offline, and query the state of hotplugged memory sections associated with a given node.
This adds the following new interfaces:
daxctl_dev_get_target_node
daxctl_dev_online_node
daxctl_dev_offline_node
daxctl_dev_node_is_online
--- daxctl/lib/libdaxctl-private.h | 6 + daxctl/lib/libdaxctl.c | 228 +++++++++++++++++++++++++++++++++ daxctl/lib/libdaxctl.sym | 4 + daxctl/libdaxctl.h | 4 + 4 files changed, 242 insertions(+)
daxctl/lib/libdaxctl-private.h +6 -0
@@ -33,6 +33,11 @@ static const char *dax_modules[] = { [DAXCTL_DEV_MODE_RAM] = "kmem", }; +enum node_state { +NODE_OFFLINE , +NODE_ONLINE , +}; +/** * struct daxctl_region - container for dax_devices */
@@ -63,6 +68,7 @@ struct daxctl_dev {struct kmod_module *module;struct kmod_list *kmod_list;struct daxctl_region *region; +int target_node; };static inline int check_kmod (struct kmod_ctx *kmod_ctx)
daxctl/lib/libdaxctl.c +228 -0
243 lines not on this page · raw message
daxctl/lib/libdaxctl.sym +4 -0
@@ -54,8 +54,12 @@ global: LIBDAXCTL_6 { global: daxctl_dev_get_ctx; + daxctl_dev_get_target_node; daxctl_dev_is_enabled; daxctl_dev_disable; daxctl_dev_enable_devdax; daxctl_dev_enable_ram; + daxctl_dev_online_node; + daxctl_dev_offline_node; + daxctl_dev_node_is_online; } LIBDAXCTL_5;
daxctl/libdaxctl.h +4 -0
@@ -68,10 +68,14 @@ int daxctl_dev_get_major(struct daxctl_dev *dev);int daxctl_dev_get_minor (struct daxctl_dev *dev);unsigned long long daxctl_dev_get_size (struct daxctl_dev *dev);struct daxctl_ctx *daxctl_dev_get_ctx (struct daxctl_dev *dev); +int daxctl_dev_get_target_node (struct daxctl_dev *dev);int daxctl_dev_is_enabled (struct daxctl_dev *dev);int daxctl_dev_disable (struct daxctl_dev *dev);int daxctl_dev_enable_devdax (struct daxctl_dev *dev);int daxctl_dev_enable_ram (struct daxctl_dev *dev); +int daxctl_dev_online_node (struct daxctl_dev *dev); +int daxctl_dev_offline_node (struct daxctl_dev *dev); +int daxctl_dev_node_is_online (struct daxctl_dev *dev);#define daxctl_dev_foreach (region, dev) \ for (dev = daxctl_dev_get_first(region); \
Signature · 6 lines
2.20.1 _______________________________________________ Linux-nvdimm mailing list [email protected] https://lists.01.org/mailman/listinfo/linux-nvdimm
Vishal Verma
maintainerPATCH v3 5/10 daxctl/list: add numa_node for device listings
In reply to Vishal VermaThe kernel provides a 'target_node' attribute for dax devices. When converting a dax device to the system-ram mode, the memory is hotplugged into this numa node. It would be helpful to print this in device listings so that it is easy for applications to detect the numa node to which the new memory belongs.
--- util/json.c | 8 ++++++++ 1 file changed, 8 insertions(+)
util/json.c +8 -0
@@ -271,6 +271,7 @@ struct json_object *util_daxctl_dev_to_json(struct daxctl_dev *dev, {const char *devname =daxctl_dev_get_devname (dev);struct json_object *jdev, *jobj; +int node; jdev =json_object_new_object ();if (!devname || !jdev)
@@ -284,6 +285,13 @@ struct json_object *util_daxctl_dev_to_json(struct daxctl_dev *dev,if (jobj)json_object_object_add (jdev,"size" , jobj); + node =daxctl_dev_get_target_node (dev); +if (node >=0 ) { + jobj =json_object_new_int (node); +if (jobj) +json_object_object_add (jdev,"numa_node" , jobj); + } +return jdev; }
Signature · 6 lines
2.20.1 _______________________________________________ Linux-nvdimm mailing list [email protected] https://lists.01.org/mailman/listinfo/linux-nvdimm
Vishal Verma
maintainerPATCH v3 6/10 libdaxctl: add an interface to get the mode for a dax device
In reply to Vishal VermaIn preparation for a reconfigure-device command, add an interface to retrieve the 'mode' of a dax device. This will allow the reconfigure-device command (and via daxctl_dev_to_json()), also daxctl-list) to print the mode on device listings via a list command or immediately after a mode change.
--- daxctl/lib/libdaxctl.c | 41 ++++++++++++++++++++++++++++++++++++++++ daxctl/lib/libdaxctl.sym | 1 + daxctl/libdaxctl.h | 1 + util/json.c | 14 ++++++++++++++ 4 files changed, 57 insertions(+)
daxctl/lib/libdaxctl.c +41 -0
@@ -12,6 +12,8 @@*/ #include <stdio.h> #include <errno.h> +#include <limits.h> +#include <libgen.h> #include <stdlib.h> #include <dirent.h> #include <unistd.h>
@@ -840,6 +842,45 @@ DAXCTL_EXPORT int daxctl_dev_disable(struct daxctl_dev *dev)return 0 ; } +DAXCTL_EXPORT enum daxctl_dev_mode daxctl_dev_get_mode(struct daxctl_dev *dev) +{ +const char *devname =daxctl_dev_get_devname (dev); +struct daxctl_ctx *ctx =daxctl_dev_get_ctx (dev); +int rc =DAXCTL_DEV_MODE_UNKNOWN ; +char path[200 ]; +const int len =sizeof (path); +char *mod_path, *mod_base; + +if (!daxctl_dev_is_enabled (dev)) +return rc; + +/* dax-class model only has conventional devdax devices */ +if (ctx->subsys !=DAX_BUS ) +return DAXCTL_DEV_MODE_DEVDAX ; + +if (snprintf (path, len,"%s/driver/module" , dev->dev_path) >= len) { +err (ctx,"%s: buffer too small!\n" , devname); +return -ENXIO ; + } + + mod_path =realpath (path,NULL ); +if (!mod_path) { + rc = -errno; +err (ctx,"%s: unable to determine module: %s\n" , devname, +strerror (errno)); +return rc; + } + + mod_base =basename (mod_path); +if (strcmp (mod_base, dax_modules[DAXCTL_DEV_MODE_RAM ]) ==0 ) + rc =DAXCTL_DEV_MODE_RAM ; +else if (strcmp (mod_base, dax_modules[DAXCTL_DEV_MODE_DEVDAX ]) ==0 ) + rc =DAXCTL_DEV_MODE_DEVDAX ; + +free (mod_path); +return rc; +} + DAXCTL_EXPORT struct daxctl_ctx *daxctl_dev_get_ctx(struct daxctl_dev *dev) {return dev->region->ctx;
daxctl/lib/libdaxctl.sym +1 -0
@@ -62,4 +62,5 @@ global: daxctl_dev_online_node; daxctl_dev_offline_node; daxctl_dev_node_is_online; + daxctl_dev_get_mode; } LIBDAXCTL_5;
daxctl/libdaxctl.h +1 -0
@@ -76,6 +76,7 @@ int daxctl_dev_enable_ram(struct daxctl_dev *dev);int daxctl_dev_online_node (struct daxctl_dev *dev);int daxctl_dev_offline_node (struct daxctl_dev *dev);int daxctl_dev_node_is_online (struct daxctl_dev *dev); +enum daxctl_dev_mode daxctl_dev_get_mode (struct daxctl_dev *dev);#define daxctl_dev_foreach (region, dev) \ for (dev = daxctl_dev_get_first(region); \
util/json.c +14 -0
@@ -271,6 +271,7 @@ struct json_object *util_daxctl_dev_to_json(struct daxctl_dev *dev, {const char *devname =daxctl_dev_get_devname (dev);struct json_object *jdev, *jobj; +enum daxctl_dev_mode mode;int node; jdev =json_object_new_object ();
@@ -292,6 +293,19 @@ struct json_object *util_daxctl_dev_to_json(struct daxctl_dev *dev,json_object_object_add (jdev,"numa_node" , jobj); } + mode =daxctl_dev_get_mode (dev); +if (mode >0 ) { + jobj =NULL ; +if (mode ==DAXCTL_DEV_MODE_RAM ) + jobj =json_object_new_string ("system-ram" ); +else if (mode ==DAXCTL_DEV_MODE_DEVDAX ) + jobj =json_object_new_string ("devdax" ); +else + jobj =json_object_new_string ("unknown" ); +if (jobj) +json_object_object_add (jdev,"mode" , jobj); + } +return jdev; }
Signature · 6 lines
2.20.1 _______________________________________________ Linux-nvdimm mailing list [email protected] https://lists.01.org/mailman/listinfo/linux-nvdimm
Vishal Verma
maintainerPATCH v3 7/10 daxctl: add a new reconfigure-device command
In reply to Vishal VermaAdd a new command 'daxctl-reconfigure-device'. This is used to switch the mode of a dax device between regular 'device_dax' and 'system-memory'. The command also uses the memory hotplug sysfs interfaces to online the newly available memory when converting to 'system-ram', and to attempt to offline the memory when converting back to a DAX device.
--- daxctl/Makefile.am | 2 + daxctl/builtin.h | 1 + daxctl/daxctl.c | 1 + daxctl/device.c | 237 +++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 241 insertions(+) create mode 100644 daxctl/device.c
daxctl/Makefile.am +2 -0
@@ -15,10 +15,12 @@ daxctl_SOURCES =\ daxctl.c \ list.c \ migrate.c \ + device.c \ ../util/json.c daxctl_LDADD =\ lib/libdaxctl.la \ ../libutil.a \ $(UUID_LIBS) \ + $(KMOD_LIBS) \ $(JSON_LIBS)
daxctl/builtin.h +1 -0
@@ -6,4 +6,5 @@struct daxctl_ctx ;int cmd_list (int argc,const char **argv,struct daxctl_ctx *ctx);int cmd_migrate (int argc,const char **argv,struct daxctl_ctx *ctx); +int cmd_reconfig_device (int argc,const char **argv,struct daxctl_ctx *ctx);#endif /* _DAXCTL_BUILTIN_H_ */
daxctl/daxctl.c +1 -0
@@ -71,6 +71,7 @@ static struct cmd_struct commands[] = { { "list", .d_fn = cmd_list }, { "help", .d_fn = cmd_help }, { "migrate-device-model", .d_fn = cmd_migrate }, + { "reconfigure-device", .d_fn = cmd_reconfig_device }, };int main (int argc,const char **argv)
daxctl/device.c new file +237 -0
237 lines not on this page · raw message
Signature · 6 lines
2.20.1 _______________________________________________ Linux-nvdimm mailing list [email protected] https://lists.01.org/mailman/listinfo/linux-nvdimm
Vishal Verma
maintainerPATCH v3 8/10 Documentation/daxctl: add a man page for daxctl-reconfigure-device
In reply to Vishal VermaAdd a man page describing the new daxctl-reconfigure-device command.
--- Documentation/daxctl/Makefile.am | 3 +- .../daxctl/daxctl-reconfigure-device.txt | 118 ++++++++++++++++++ 2 files changed, 120 insertions(+), 1 deletion(-) create mode 100644 Documentation/daxctl/daxctl-reconfigure-device.txt
Documentation/daxctl/Makefile.am +2 -1
@@ -28,7 +28,8 @@ endif man1_MANS = \ daxctl.1 \ daxctl-list.1 \ - daxctl-migrate-device-model.1 + daxctl-migrate-device-model.1 \ + daxctl-reconfigure-device.1 CLEANFILES = $(man1_MANS)
Documentation/daxctl/daxctl-reconfigure-device.txt new file +118 -0
118 lines not on this page · raw message
Signature · 6 lines
2.20.1 _______________________________________________ Linux-nvdimm mailing list [email protected] https://lists.01.org/mailman/listinfo/linux-nvdimm
Vishal Verma
maintainerPATCH v3 9/10 contrib/ndctl: fix region-id completions for daxctl
In reply to Vishal VermaThe completion helpers for daxctl assumed the region arguments for specifying daxctl regions were the same as ndctl regions, i.e. "regionX". This is not true - daxctl region arguments are a simple numeric 'id'.
Add a new helper __daxctl_get_regions() to complete daxctl region IDs properly.
While at it, fix a useless use of 'echo' in __daxctl_get_devs() and quoting in __daxctl_comp_options()
--- contrib/ndctl | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-)
contrib/ndctl +10 -4
@@ -531,8 +531,14 @@ _ndctl() __daxctl_get_devs() { - local opts="--devices $*" - echo "$(daxctl list $opts | grep -E "^\s*\"chardev\":" | cut -d\" -f4)" + local opts=("--devices" "$*") + daxctl list "${opts[@]}" | grep -E "^\s*\"chardev\":" | cut -d'"' -f4 +} + +__daxctl_get_regions() +{ + local opts=("--regions" "$*") + daxctl list "${opts[@]}" | grep -E "^\s*\"id\":" | grep -Eo "[0-9]+" } __daxctlcomp()
@@ -561,10 +567,10 @@ __daxctl_comp_options() local cur_arg=${cur##*=} case $cur_subopt in --region) - opts=$(__ndctl_get_regions -i) + opts="$(__daxctl_get_regions -i)" ;; --dev) - opts=$(__daxctl_get_devs -i) + opts="$(__daxctl_get_devs -i)" ;; *) return
Signature · 6 lines
2.20.1 _______________________________________________ Linux-nvdimm mailing list [email protected] https://lists.01.org/mailman/listinfo/linux-nvdimm
Vishal Verma
maintainerPATCH v3 10/10 contrib/ndctl: add bash-completion for daxctl-reconfigure-device
In reply to Vishal VermaAdd bash completion helpers for the new daxctl-reconfigure-device command.
--- contrib/ndctl | 20 +++++++++++++++++--- 1 file changed, 17 insertions(+), 3 deletions(-)
contrib/ndctl +17 -3
@@ -547,7 +547,7 @@ __daxctlcomp() COMPREPLY=( $( compgen -W "$1" -- "$2" ) ) for cword in "${COMPREPLY[@]}"; do - if [[ "$cword" == @(--region|--dev) ]]; then + if [[ "$cword" == @(--region|--dev|--mode) ]]; then COMPREPLY[$i]="${cword}=" else COMPREPLY[$i]="${cword} "
@@ -572,6 +572,9 @@ __daxctl_comp_options() --dev) opts="$(__daxctl_get_devs -i)" ;; + --mode) + opts="system-ram devdax" + ;; *) return ;;
@@ -582,8 +585,19 @@ __daxctl_comp_options() __daxctl_comp_non_option_args() { - # there aren't any commands that accept non option arguments yet - return + local subcmd=$1 + local cur=$2 + local opts + + case $subcmd in + reconfigure-device) + opts="$(__daxctl_get_devs -i) all" + ;; + *) + return + ;; + esac + __daxctlcomp "$opts" "$cur" } __daxctl_main()
Signature · 6 lines
2.20.1 _______________________________________________ Linux-nvdimm mailing list [email protected] https://lists.01.org/mailman/listinfo/linux-nvdimm
Quoted 4 lines · Vishal Verma
Changes in v3:
- Indaxctl_dev_get_mode(), remove the subsystem warning, detect dax-class and simply make it return devdax
Hi Vishal,
I am still getting the same error as before:
# ndctl create-namespace --mode devdax --map mem -e namespace0.0 -f [ 141.525873] dax0.0 initialised, 524288 pages in 7ms libdaxctl: __sysfs_device_parse: dax0.0: add_dev() failed ...
I am building it via buildroot, and can share the initramfs file or anything that can help you with fixing this issue.
Thank you,
Pavel
_______________________________________________
Linux-nvdimm mailing list
[email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm
Verma, Vishal L
maintainerQuoted 11 lines · Pavel Tatashin
On Thu, May 16, 2019 at 6:40 PM Vishal Verma <[email protected]
Quoted 5 lines
wrote:
Changes in v3:
- Indaxctl_dev_get_mode(), remove the subsystem warning, detect
dax-class
and simply make it return devdaxHi Vishal,
I am still getting the same error as before:
Quoted 7 lines
# ndctl create-namespace --mode devdax --map mem -e namespace0.0 -f [ 141.525873] dax0.0 initialised, 524288 pages in 7ms libdaxctl: __sysfs_device_parse: dax0.0: add_dev() failed ...I am building it via buildroot, and can share the initramfs file or anything that can help you with fixing this issue.
Hi Pavel,
I've still not been able to hit this in my testing, is it something you hit only after applying these patches? i.e. does plain v65 work?
_______________________________________________
Linux-nvdimm mailing list
[email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm
Quoted 4 lines
Hi Pavel,
I've still not been able to hit this in my testing, is it something you hit only after applying these patches? i.e. does plain v65 work?
Yes, plain v65 works, but with these patches I see this error.
I use buildroot to build initramfs with ndctl. Here is how ndctl.mk looks like:
NDCTL_VERSION = ed17fd1NDCTL_SITE = $(call github,pmem,ndctl,$(NDCTL_VERSION)) NDCTL_LICENSE = GNU Lesser General Public License v2.1 NDCTL_LICENSE_FILES = COPYING
#NDCTL_AUTORECv65ONF = YES
define NDCTL_RUN_AUTOGEN
cd $(@D) && PATH=$(BR_PATH) ./autogen.sh
endef
NDCTL_PRE_CONFIGURE_HOOKS += NDCTL_RUN_AUTOGEN
NDCTL_LDFLAGS = $(TARGET_LDFLAGS)
NDCTL_CONF_OPTS = \
--without-bash \
--without-keyutils \
--without-systemd$(eval $(autotools-package))
This version works, but when I change:NDCTL_VERSION = 9ee82c8
I get add_dev error.
Pasha
_______________________________________________
Linux-nvdimm mailing list
[email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm
Verma, Vishal L
maintainerQuoted 8 lines · Pavel Tatashin
Quoted 5 lines
Hi Pavel,
I've still not been able to hit this in my testing, is it something you
hit only after applying these patches? i.e. does plain v65 work?Yes, plain v65 works, but with these patches I see this error.
Hm, So there are only two patches that touch the add_dax_dev function:
2bf9a8e libdaxctl: add interfaces to enable/disable devices 25be8f4 libdaxctl: add interfaces to get/set the online state for a node
And of these, the second one to get the target node doesn't return an error in any case.
The first one can fail, so it must be that condition you're hitting, but I'm not sure in what way it is failing.
The exact check is:
sprintf(path, "%s/modalias", daxdev_base);
rc = sysfs_read_attr(ctx, path, buf);
/* older kernels may be lack the modalias attribute */
if (rc < 0 && rc != -ENOENT)
goto err_read;
if (rc == 0) {
dev->kmod_list = to_module_list(ctx, buf);
if (dev->kmod_list == NULL)
goto err_read;
} else
dbg(ctx, "%s: modalias attribute missing\n", devname);Do you have the file:
/sys/bus/dax/devices/dax0.0/modalias
And what does it contain? On my system:
$ cat /sys/bus/dax/devices/dax0.0/modalias dax:t0
If this is missing, it would seem you're not correctly in 'dax-bus' mode, but that is kind of required for the kmem functionality anyway, so I'm not sure where we might be tripping.
_______________________________________________
Linux-nvdimm mailing list
[email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm
Verma, Vishal L
maintainerQuoted 4 lines · Vishal L
Quoted 3 lines · Pavel Tatashin
Quoted 3 lines
Hi Pavel,
I've still not been able to hit this in my testing, is it
something
Quoted 9 lines
Quoted 5 lines
Quoted 2 lines
you
hit only after applying these patches? i.e. does plain v65 work?Yes, plain v65 works, but with these patches I see this error.
Hm, So there are only two patches that touch the
add_dax_devfunction:2bf9a8e libdaxctl: add interfaces to enable/disable devices 25be8f4 libdaxctl: add interfaces to get/set the online state for a
node
Quoted 5 lines
And of these, the second one to get the target node doesn't return an error in any case.
The first one can fail, so it must be that condition you're hitting,
but
Quoted 4 lines
I'm not sure in what way it is failing.
The exact check is:
Quoted 9 lines
sprintf(path, "%s/modalias", daxdev_base);
rc = sysfs_read_attr(ctx, path, buf);
/* older kernels may be lack the modalias attribute */
if (rc < 0 && rc != -ENOENT)
goto err_read;
if (rc == 0) {
dev->kmod_list = to_module_list(ctx, buf);
if (dev->kmod_list == NULL)
goto err_read;Dan points out that it might actually be the kmod portion that might be failing. Is libkmod present in the buildroot setup, and has a depmod run completed successfully before this point?
In any case, this incremental patch should /at least/ delay the error until you actually try to enable a kmem device. I'll fold this into the next version of the series.
8<----
Quoted 1 line
From
2df9b6401a69833fa709ab1ad83ac27b545aeb9eMon Sep 17 00:00:00 2001
From: Vishal Verma <[email protected]>
Date: Wed, 22 May 2019 12:01:28 -0600
Subject: [ndctl PATCH] fixup! libdaxctl: add interfaces to enable/disable
devices
--- daxctl/lib/libdaxctl.c | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-)
daxctl/lib/libdaxctl.c +0 -0
@@ -393,12 +393,8 @@ static void *add_dax_dev(void *parent, int id,
const char *daxdev_base)
/* older kernels may be lack the modalias attribute */ if (rc < 0 && rc != -ENOENT)
goto err_read;
- if (rc == 0) {
+ if (rc == 0)
dev->kmod_list = to_module_list(ctx, buf);
- if (dev->kmod_list == NULL)
- goto err_read;
- } else
- dbg(ctx, "%s: modalias attribute missing\n", devname);
sprintf(path, "%s/target_node", daxdev_base); if (sysfs_read_attr(ctx, path, buf) == 0)
Signature · 7 lines
2.20.1 _______________________________________________ Linux-nvdimm mailing list [email protected] https://lists.01.org/mailman/listinfo/linux-nvdimm