daxctl: add a new reconfigure-device command
Subsystems (3)
Vishal Verma
maintainerv7 cover daxctl: add a new reconfigure-device command
Changes in v7:
- Fix a couple of checkaptch type errors in the new lines added in v6 (Dan).
- Get rid of daxctl_dev_get_mode. daxctl_dev_get_memory is sufficient to both check the mode and allocate the memory related structures on its first call. (Dan)
- Due to the above, daxctl_dev_mode is now private to libdaxctl, and not part of the API exported through libdaxctl.h
- Add a large enough buffer at init time to construct dynamic paths, and avoid asprintf() type allocations for memory blocks at runtime (Dan).
Changes in v6:
- For memory block online/offline operations, the kernel responds with an EINVAL for both 'real' errors, and if the memory was already in the requested state. Since there is a TOCTOU hole between checking the state and storing it, just perform a second check if the store results in an error. If the check shows the state to be the same as the one we're attempting, it means that another agent (usually udev) won the race, but we don't care so long as the state change happened, so don't report an error. (Fan Du)
Changes in v5:
- device.c: correctly set loglevel for daxctl_ctx for --verbose
- drop the subsys caching, its complexity started to exceed its benefit. dax-class device models will simply error out during reconfigure. (Dan)
- Add a note to the man page for the above.
- Clarify the onlining policy (online_movable) in the man page
- rename "numa_node" to "target_node" in device listings (Dan)
- When printing a device 'mode', assume devdax if !system-ram, avoiding a "mode: unknown" situation which can be confusing. (Dan)
- Add a "state: disabled" attribute to the device listing if a driver is not bound. This is more apt than the previous "mode: unknown" listing.
- add an api to get 'dev->resource' parsing /proc/iomem as a fallback for when the kernel doesn't provide the attribute (Dan)
- convert node_* apis to 'memory_* apis that act on a new daxctl_memory object (Dan)
- online only memory sections belonging to the device in question by cross referencing block indices with the dax device resource (Dan)
- Refuse to reconfigure a device that is already in the target mode. Until now, reconfiguring a system-ram device back to system-ram would result in a 'online memory may not be hot-removed' kernel warning.
- If the device was already in the system-ram mode, skip disabling/enabling, but still try to online the memory unless the --no-online option is in effect.
- In daxctl_unbind, also 'remove_id' to prevent devices automatically binding to the kmem driver on a disable + re-enable, which can be surprising (Dan).
- Rewrite the top half of daxctl/device.c to borrow elements from ndctl/namespace.c so that it can support growing additional commands that operate on devices (online-memory and offline-memory)
- Refactor the bottom half of daxctl/device.c so we only do the disabling/offlining steps if the device was enabled.
- Add new commands to online and offline memory sections (Dan) associated with a given dax device (Dan)
- Add a new test - daxctl-device.sh - to test daxctl reconfigure-device, online-memory, and offline-memory commands.
- Add an example in documentation demonstrating how to use numactl to bind a process to a node surfaced from a dax device (Andy Rudoff)
Changes in v4:
- Don't fail add_dax_dev for kmod failures. Instead fail only when the kmod list is actually used, i.e. during daxctl-reconfigure-device
Changes in v3:
- In daxctl_dev_get_mode(), remove the subsystem warning, detect dax-class and simply make it return devdax
Changes in v2:
- Add examples to the documentation page (Dave Hansen)
- Clarify documentation regarding the conversion from system-ram to devdax
- Remove any references to a persistent config from the documentation - those can be added when the feature is added.
- device.c: validate option compatibility
- daxctl-list: display numa_node for device listings
- daxctl-list: display mode for device listings
- make the options more consistent by adding a '-O' short option for --attempt-offline
Add a new daxctl-reconfigure-device command that lets us reconfigure DAX devices back and forth between 'system-ram' and 'device-dax' modes. It also includes facilities to online any newly hot-plugged memory (default), and attempt to offline memory before converting away from the system-ram mode (not default, requires a --attempt-offline option).
Currently missing from this series is a way to persistently store which devices have been 'marked' for use as system-ram. This depends on a config system overhaul in ndctl, and patches for those will follow separately and are independent of this work.
Example invocations:
1. Reconfigure dax0.0 to system-ram mode, don’t online the memory
# daxctl reconfigure-device --mode=system-ram --no-online dax0.0
[
{
"chardev":"dax0.0",
"size":16777216000,
"target_node":2,
"mode":"system-ram"
}
]2. Reconfigure dax0.0 to devdax mode, attempt to offline the memory
# daxctl reconfigure-device --human --mode=devdax --attempt-offline dax0.0
{
"chardev":"dax0.0",
"size":"15.63 GiB (16.78 GB)",
"target_node":2,
"mode":"devdax"
}3. Reconfigure all dax devices on region0 to system-ram mode
# daxctl reconfigure-device --mode=system-ram --region=0 all
[
{
"chardev":"dax0.0",
"size":16777216000,
"target_node":2,
"mode":"system-ram"
},
{
"chardev":"dax0.1",
"size":16777216000,
"target_node":3,
"mode":"system-ram"
}
]These patches can also be found in the 'kmem-pending' branch on github:
https://github.com/pmem/ndctl/tree/kmem-pending
Vishal Verma (13):
libdaxctl: add interfaces to get ctx and check device state
libdaxctl: add interfaces to enable/disable devices
libdaxctl: add an interface to retrieve the device resource
libdaxctl: add a 'daxctl_memory' object for memory based operations daxctl/list: add target_node for device listings daxctl/list: display the mode for a dax device
daxctl: add a new reconfigure-device command Documentation/daxctl: add a man page for daxctl-reconfigure-device
daxctl: add commands to online and offline memory
Documentation: Add man pages for daxctl-{on,off}line-memory contrib/ndctl: fix region-id completions for daxctl contrib/ndctl: add bash-completion for the new daxctl commands
test: Add a unit test for daxctl-reconfigure-device and friends
Documentation/daxctl/Makefile.am | 5 +- .../daxctl/daxctl-offline-memory.txt | 72 ++ Documentation/daxctl/daxctl-online-memory.txt | 80 ++ .../daxctl/daxctl-reconfigure-device.txt | 139 ++++ Makefile.am | 3 +- contrib/ndctl | 38 +- daxctl/Makefile.am | 2 + daxctl/builtin.h | 3 + daxctl/daxctl.c | 3 + daxctl/device.c | 476 ++++++++++++ daxctl/lib/Makefile.am | 5 +- daxctl/lib/libdaxctl-private.h | 38 + daxctl/lib/libdaxctl.c | 685 ++++++++++++++++++ daxctl/lib/libdaxctl.sym | 18 + daxctl/libdaxctl.h | 16 + test/Makefile.am | 3 +- test/common | 19 +- test/daxctl-devices.sh | 81 +++ util/iomem.c | 37 + util/iomem.h | 12 + util/json.c | 22 + 21 files changed, 1743 insertions(+), 14 deletions(-) create mode 100644 Documentation/daxctl/daxctl-offline-memory.txt create mode 100644 Documentation/daxctl/daxctl-online-memory.txt create mode 100644 Documentation/daxctl/daxctl-reconfigure-device.txt create mode 100644 daxctl/device.c create mode 100755 test/daxctl-devices.sh create mode 100644 util/iomem.c create mode 100644 util/iomem.h
Signature · 6 lines
2.20.1 _______________________________________________ Linux-nvdimm mailing list [email protected] https://lists.01.org/mailman/listinfo/linux-nvdimm
Vishal Verma
maintainerPATCH v7 1/13 libdaxctl: add interfaces to get ctx and check device state
In preparation for libdaxctl and daxctl to grow operational modes for DAX devices, add the following supporting APIs:
daxctl_dev_get_ctx
daxctl_dev_is_enabled
--- daxctl/lib/libdaxctl.c | 30 ++++++++++++++++++++++++++++++ daxctl/lib/libdaxctl.sym | 6 ++++++ daxctl/libdaxctl.h | 2 ++ 3 files changed, 38 insertions(+)
daxctl/lib/libdaxctl.c +30 -0
@@ -559,6 +559,36 @@ static void dax_regions_init(struct daxctl_ctx *ctx) } } +static int is_enabled (const char *drvpath) +{ +struct stat st; + +if (lstat (drvpath, &st) <0 || !S_ISLNK (st.st_mode)) +return 0 ; +else +return 1 ; +} + +DAXCTL_EXPORT int daxctl_dev_is_enabled(struct daxctl_dev *dev) +{ +struct daxctl_ctx *ctx =daxctl_dev_get_ctx (dev); +char *path = dev->dev_buf; +int len = dev->buf_len; + +if (snprintf (path, len,"%s/driver" , dev->dev_path) >= len) { +err (ctx,"%s: buffer too small!\n" , +daxctl_dev_get_devname (dev)); +return 0 ; + } + +return is_enabled (path); +} + +DAXCTL_EXPORT struct daxctl_ctx *daxctl_dev_get_ctx(struct daxctl_dev *dev) +{ +return dev->region->ctx; +} + DAXCTL_EXPORT struct daxctl_dev *daxctl_dev_get_first(struct daxctl_region *region) {dax_devices_init (region);
daxctl/lib/libdaxctl.sym +6 -0
@@ -50,3 +50,9 @@ LIBDAXCTL_5 { global: daxctl_region_get_path; } LIBDAXCTL_4; + +LIBDAXCTL_6 { +global: + daxctl_dev_get_ctx; + daxctl_dev_is_enabled; +} LIBDAXCTL_5;
daxctl/libdaxctl.h +2 -0
@@ -67,6 +67,8 @@ const char *daxctl_dev_get_devname(struct daxctl_dev *dev);int daxctl_dev_get_major (struct daxctl_dev *dev);int daxctl_dev_get_minor (struct daxctl_dev *dev);unsigned long long daxctl_dev_get_size (struct daxctl_dev *dev); +struct daxctl_ctx *daxctl_dev_get_ctx (struct daxctl_dev *dev); +int daxctl_dev_is_enabled (struct daxctl_dev *dev);#define daxctl_dev_foreach (region, dev) \ for (dev = daxctl_dev_get_first(region); \
Signature · 6 lines
2.20.1 _______________________________________________ Linux-nvdimm mailing list [email protected] https://lists.01.org/mailman/listinfo/linux-nvdimm
Vishal Verma
maintainerPATCH v7 2/13 libdaxctl: add interfaces to enable/disable devices
In reply to Vishal VermaAdd new libdaxctl interfaces to disable a device_dax based device, and to enable it into the given mode. The modes available are 'devdax', and 'system-ram', where devdax is the normal device DAX mode used via a character device, and 'system-ram' uses the kernel's 'kmem' facility to hotplug the device making it usable as normal memory.
This adds the following new interfaces:
daxctl_dev_disable;
daxctl_dev_enable_devdax;
daxctl_dev_enable_ram;
--- daxctl/lib/Makefile.am | 3 +- daxctl/lib/libdaxctl-private.h | 21 +++ daxctl/lib/libdaxctl.c | 234 +++++++++++++++++++++++++++++++++ daxctl/lib/libdaxctl.sym | 3 + daxctl/libdaxctl.h | 3 + 5 files changed, 263 insertions(+), 1 deletion(-)
daxctl/lib/Makefile.am +2 -1
@@ -16,7 +16,8 @@ libdaxctl_la_SOURCES =\ libdaxctl.c libdaxctl_la_LIBADD =\ - $(UUID_LIBS) + $(UUID_LIBS) \ + $(KMOD_LIBS) daxctl_modprobe_data_DATA = daxctl.conf
daxctl/lib/libdaxctl-private.h +21 -0
@@ -13,6 +13,8 @@#ifndef _LIBDAXCTL_PRIVATE_H_#define _LIBDAXCTL_PRIVATE_H_ +#include <libkmod.h> +#define DAXCTL_EXPORT __attribute__ ((visibility("default"))) enum dax_subsystem {
@@ -26,6 +28,17 @@ static const char *dax_subsystems[] = { [DAX_BUS] = "/sys/bus/dax/devices", }; +enum daxctl_dev_mode { +DAXCTL_DEV_MODE_DEVDAX =0 , +DAXCTL_DEV_MODE_RAM , +DAXCTL_DEV_MODE_END , +}; + +static const char *dax_modules[] = { + [DAXCTL_DEV_MODE_DEVDAX ] ="device_dax" , + [DAXCTL_DEV_MODE_RAM ] ="kmem" , +}; +/** * struct daxctl_region - container for dax_devices */
@@ -53,6 +66,14 @@ struct daxctl_dev {char *dev_path;struct list_node list;unsigned long long size; +struct kmod_module *module; +struct kmod_list *kmod_list; struct daxctl_region *region; }; + +static inline int check_kmod (struct kmod_ctx *kmod_ctx) +{ +return kmod_ctx ?0 : -ENXIO ; +} +#endif /* _LIBDAXCTL_PRIVATE_H_ */
daxctl/lib/libdaxctl.c +234 -0
302 lines not on this page · raw message
daxctl/lib/libdaxctl.sym +3 -0
@@ -55,4 +55,7 @@ LIBDAXCTL_6 { global: daxctl_dev_get_ctx; daxctl_dev_is_enabled; + daxctl_dev_disable; + daxctl_dev_enable_devdax; + daxctl_dev_enable_ram; } LIBDAXCTL_5;
daxctl/libdaxctl.h +3 -0
@@ -69,6 +69,9 @@ int daxctl_dev_get_minor(struct daxctl_dev *dev);unsigned long long daxctl_dev_get_size (struct daxctl_dev *dev);struct daxctl_ctx *daxctl_dev_get_ctx (struct daxctl_dev *dev);int daxctl_dev_is_enabled (struct daxctl_dev *dev); +int daxctl_dev_disable (struct daxctl_dev *dev); +int daxctl_dev_enable_devdax (struct daxctl_dev *dev); +int daxctl_dev_enable_ram (struct daxctl_dev *dev);#define daxctl_dev_foreach (region, dev) \ for (dev = daxctl_dev_get_first(region); \
Signature · 6 lines
2.20.1 _______________________________________________ Linux-nvdimm mailing list [email protected] https://lists.01.org/mailman/listinfo/linux-nvdimm
Vishal Verma
maintainerPATCH v7 3/13 libdaxctl: add an interface to retrieve the device resource
In reply to Vishal VermaAdd an interface to retrieve the 'resource' attribute for a dax device.
Attempt to retrieve it as usual via sysfs, but since older kernels may be missing this attribute, as a fallback, attempt to retrieve it from /proc/iomem
--- Makefile.am | 3 ++- daxctl/lib/Makefile.am | 2 ++ daxctl/lib/libdaxctl-private.h | 1 + daxctl/lib/libdaxctl.c | 12 +++++++++++ daxctl/lib/libdaxctl.sym | 1 + daxctl/libdaxctl.h | 1 + util/iomem.c | 37 ++++++++++++++++++++++++++++++++++ util/iomem.h | 12 +++++++++++ 8 files changed, 68 insertions(+), 1 deletion(-) create mode 100644 util/iomem.c create mode 100644 util/iomem.h
Makefile.am +2 -1
@@ -74,6 +74,7 @@ libutil_a_SOURCES = \ util/wrapper.c \ util/filter.c \ util/bitmap.c \ - util/abspath.c + util/abspath.c \ + util/iomem.c nobase_include_HEADERS = daxctl/libdaxctl.h
daxctl/lib/Makefile.am +2 -0
@@ -9,6 +9,8 @@ lib_LTLIBRARIES = libdaxctl.la libdaxctl_la_SOURCES =\ ../libdaxctl.h \ libdaxctl-private.h \ + ../../util/iomem.c \ + ../../util/iomem.h \ ../../util/sysfs.c \ ../../util/sysfs.h \ ../../util/log.c \
daxctl/lib/libdaxctl-private.h +1 -0
@@ -65,6 +65,7 @@ struct daxctl_dev {size_t buf_len;char *dev_path;struct list_node list; +unsigned long long resource;unsigned long long size;struct kmod_module *module;struct kmod_list *kmod_list;
daxctl/lib/libdaxctl.c +12 -0
@@ -24,6 +24,7 @@#include <util/log.h> #include <util/sysfs.h> +#include <util/iomem.h> #include <daxctl/libdaxctl.h> #include "libdaxctl-private.h"
@@ -369,6 +370,12 @@ static void *add_dax_dev(void *parent, int id, const char *daxdev_base) dev->major =major (st.st_rdev); dev->minor =minor (st.st_rdev); +sprintf (path,"%s/resource" , daxdev_base); +if (sysfs_read_attr (ctx, path, buf) ==0 ) + dev->resource =strtoull (buf,NULL ,0 ); +else + dev->resource =iomem_get_dev_resource (ctx, daxdev_base); +sprintf (path,"%s/size" , daxdev_base);if (sysfs_read_attr (ctx, path, buf) <0 ) goto err_read;
@@ -878,6 +885,11 @@ DAXCTL_EXPORT int daxctl_dev_get_minor(struct daxctl_dev *dev)return dev->minor; } +DAXCTL_EXPORT unsigned long long daxctl_dev_get_resource (struct daxctl_dev *dev) +{ +return dev->resource; +} +DAXCTL_EXPORT unsigned long long daxctl_dev_get_size (struct daxctl_dev *dev) {return dev->size;
daxctl/lib/libdaxctl.sym +1 -0
@@ -58,4 +58,5 @@ global: daxctl_dev_disable; daxctl_dev_enable_devdax; daxctl_dev_enable_ram; + daxctl_dev_get_resource; } LIBDAXCTL_5;
daxctl/libdaxctl.h +1 -0
@@ -66,6 +66,7 @@ int daxctl_dev_get_id(struct daxctl_dev *dev);const char *daxctl_dev_get_devname (struct daxctl_dev *dev);int daxctl_dev_get_major (struct daxctl_dev *dev);int daxctl_dev_get_minor (struct daxctl_dev *dev); +unsigned long long daxctl_dev_get_resource (struct daxctl_dev *dev);unsigned long long daxctl_dev_get_size (struct daxctl_dev *dev);struct daxctl_ctx *daxctl_dev_get_ctx (struct daxctl_dev *dev);int daxctl_dev_is_enabled (struct daxctl_dev *dev);
util/iomem.c new file +37 -0
@@ -0,0 +1,37 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright(c) 2019 Intel Corporation. All rights reserved. */ + +#include <stdio.h> +#include <errno.h> +#include <string.h> +#include <util/log.h> +#include <util/iomem.h> +#include <util/sysfs.h> + +unsigned long long __iomem_get_dev_resource (struct log_ctx *ctx, +const char *devpath) +{ +const char *devname =devpath_to_devname (devpath); +FILE *fp =fopen ("/proc/iomem" ,"r" ); +unsigned long long res; +char name[256 ]; + +if (fp ==NULL ) { +log_err (ctx,"%s: open /proc/iomem: %s\n" , devname, +strerror (errno)); +return 0 ; + } + +while (fscanf (fp,"%llx-%*x : %254[^\n]\n" , &res, name) ==2 ) { +if (strcmp (name, devname) ==0 ) { +log_dbg (ctx,"%s: got resource via iomem: %#llx\n" , + devname, res); +fclose (fp); +return res; + } + } + +log_dbg (ctx,"%s: not found in iomem\n" , devname); +fclose (fp); +return 0 ; +}
util/iomem.h new file +12 -0
@@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright(c) 2019 Intel Corporation. All rights reserved. */ +#ifndef _NDCTL_IOMEM_H_ +#define _NDCTL_IOMEM_H_ + +struct log_ctx ; +unsigned long long __iomem_get_dev_resource (struct log_ctx *ctx, +const char *path); + +#define iomem_get_dev_resource (c, p) __iomem_get_dev_resource(&(c)->ctx, (p)) + +#endif /* _NDCTL_IOMEM_H_ */
Signature · 6 lines
2.20.1 _______________________________________________ Linux-nvdimm mailing list [email protected] https://lists.01.org/mailman/listinfo/linux-nvdimm
Vishal Verma
maintainerPATCH v7 4/13 libdaxctl: add a 'daxctl_memory' object for memory based operations
In reply to Vishal VermaIntroduce a new 'daxctl_memory' object, which will be used for operations related to managing dax devices in 'system-memory' modes.
Add libdaxctl APIs to get the target_node of a DAX device, and to online, offline, and query the state of hotplugged memory sections associated with a given device.
This adds the following new interfaces:
daxctl_dev_get_target_node
daxctl_dev_get_memory;
daxctl_memory_get_dev;
daxctl_memory_get_node_path;
daxctl_memory_get_block_size;
daxctl_memory_set_online
daxctl_memory_set_offline
daxctl_memory_is_online
--- daxctl/lib/libdaxctl-private.h | 16 ++ daxctl/lib/libdaxctl.c | 407 +++++++++++++++++++++++++++++++++ daxctl/lib/libdaxctl.sym | 8 + daxctl/libdaxctl.h | 10 + 4 files changed, 441 insertions(+)
daxctl/lib/libdaxctl-private.h +16 -0
@@ -39,6 +39,11 @@ static const char *dax_modules[] = { [DAXCTL_DEV_MODE_RAM] = "kmem", }; +enum memory_state { +MEM_OFFLINE , +MEM_ONLINE , +}; +/** * struct daxctl_region - container for dax_devices */
@@ -70,8 +75,19 @@ struct daxctl_dev {struct kmod_module *module;struct kmod_list *kmod_list;struct daxctl_region *region; +struct daxctl_memory *mem; +int target_node; +}; + +struct daxctl_memory { +struct daxctl_dev *dev; +void *mem_buf; +size_t buf_len; +char *node_path; +unsigned long block_size; }; +static inline int check_kmod (struct kmod_ctx *kmod_ctx) {return kmod_ctx ?0 : -ENXIO ;
daxctl/lib/libdaxctl.c +407 -0
440 lines not on this page · raw message
daxctl/lib/libdaxctl.sym +8 -0
@@ -59,4 +59,12 @@ global: daxctl_dev_enable_devdax; daxctl_dev_enable_ram; daxctl_dev_get_resource; + daxctl_dev_get_target_node; + daxctl_dev_get_memory; + daxctl_memory_get_dev; + daxctl_memory_get_node_path; + daxctl_memory_get_block_size; + daxctl_memory_set_online; + daxctl_memory_set_offline; + daxctl_memory_is_online; } LIBDAXCTL_5;
daxctl/libdaxctl.h +10 -0
@@ -73,6 +73,16 @@ int daxctl_dev_is_enabled(struct daxctl_dev *dev);int daxctl_dev_disable (struct daxctl_dev *dev);int daxctl_dev_enable_devdax (struct daxctl_dev *dev);int daxctl_dev_enable_ram (struct daxctl_dev *dev); +int daxctl_dev_get_target_node (struct daxctl_dev *dev); + +struct daxctl_memory ; +struct daxctl_memory *daxctl_dev_get_memory (struct daxctl_dev *dev); +struct daxctl_dev *daxctl_memory_get_dev (struct daxctl_memory *mem); +const char *daxctl_memory_get_node_path (struct daxctl_memory *mem); +unsigned long daxctl_memory_get_block_size (struct daxctl_memory *mem); +int daxctl_memory_set_online (struct daxctl_memory *mem); +int daxctl_memory_set_offline (struct daxctl_memory *mem); +int daxctl_memory_is_online (struct daxctl_memory *mem);#define daxctl_dev_foreach (region, dev) \ for (dev = daxctl_dev_get_first(region); \
Signature · 6 lines
2.20.1 _______________________________________________ Linux-nvdimm mailing list [email protected] https://lists.01.org/mailman/listinfo/linux-nvdimm
Vishal Verma
maintainerPATCH v7 5/13 daxctl/list: add target_node for device listings
In reply to Vishal VermaThe kernel provides a 'target_node' attribute for dax devices. When converting a dax device to the system-ram mode, the memory is hotplugged into this numa node. It would be helpful to print this in device listings so that it is easy for applications to detect the node to which the new memory belongs.
--- util/json.c | 8 ++++++++ 1 file changed, 8 insertions(+)
util/json.c +8 -0
@@ -271,6 +271,7 @@ struct json_object *util_daxctl_dev_to_json(struct daxctl_dev *dev, {const char *devname =daxctl_dev_get_devname (dev);struct json_object *jdev, *jobj; +int node; jdev =json_object_new_object ();if (!devname || !jdev)
@@ -284,6 +285,13 @@ struct json_object *util_daxctl_dev_to_json(struct daxctl_dev *dev,if (jobj)json_object_object_add (jdev,"size" , jobj); + node =daxctl_dev_get_target_node (dev); +if (node >=0 ) { + jobj =json_object_new_int (node); +if (jobj) +json_object_object_add (jdev,"target_node" , jobj); + } +return jdev; }
Signature · 6 lines
2.20.1 _______________________________________________ Linux-nvdimm mailing list [email protected] https://lists.01.org/mailman/listinfo/linux-nvdimm
Vishal Verma
maintainerPATCH v7 6/13 daxctl/list: display the mode for a dax device
In reply to Vishal VermaIn preparation for a reconfigure-device command, allow JSON listings to display the 'mode' of a dax device. This will allow the reconfigure-device command (and via daxctl_dev_to_json(), also daxctl-list) to print the mode in device listings via a 'daxctl-list' command or immediately after a mode change.
Add a 'state' attribute to the json listings for devices, since a device could end up in a state where it is not bound to any driver, and hence, 'disabled'. The state attribute is only displayed for disabled devices.
--- daxctl/lib/libdaxctl.c | 2 ++ util/json.c | 14 ++++++++++++++ 2 files changed, 16 insertions(+)
daxctl/lib/libdaxctl.c +2 -0
@@ -12,6 +12,8 @@*/ #include <stdio.h> #include <errno.h> +#include <limits.h> +#include <libgen.h> #include <stdlib.h> #include <dirent.h> #include <unistd.h>
util/json.c +14 -0
@@ -269,6 +269,7 @@ struct json_object *util_dimm_to_json(struct ndctl_dimm *dimm,struct json_object *util_daxctl_dev_to_json (struct daxctl_dev *dev,unsigned long flags) { +struct daxctl_memory *mem =daxctl_dev_get_memory (dev);const char *devname =daxctl_dev_get_devname (dev);struct json_object *jdev, *jobj;int node;
@@ -292,6 +293,19 @@ struct json_object *util_daxctl_dev_to_json(struct daxctl_dev *dev,json_object_object_add (jdev,"target_node" , jobj); } +if (mem) + jobj =json_object_new_string ("system-ram" ); +else + jobj =json_object_new_string ("devdax" ); +if (jobj) +json_object_object_add (jdev,"mode" , jobj); + +if (!daxctl_dev_is_enabled (dev)) { + jobj =json_object_new_string ("disabled" ); +if (jobj) +json_object_object_add (jdev,"state" , jobj); + } +return jdev; }
Signature · 6 lines
2.20.1 _______________________________________________ Linux-nvdimm mailing list [email protected] https://lists.01.org/mailman/listinfo/linux-nvdimm
Vishal Verma
maintainerPATCH v7 7/13 daxctl: add a new reconfigure-device command
In reply to Vishal VermaAdd a new command 'daxctl-reconfigure-device'. This is used to switch the mode of a dax device between regular 'device_dax' and 'system-memory'. The command also uses the memory hotplug sysfs interfaces to online the newly available memory when converting to 'system-ram', and to attempt to offline the memory when converting back to a DAX device.
--- daxctl/Makefile.am | 2 + daxctl/builtin.h | 1 + daxctl/daxctl.c | 1 + daxctl/device.c | 347 +++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 351 insertions(+) create mode 100644 daxctl/device.c
daxctl/Makefile.am +2 -0
@@ -15,10 +15,12 @@ daxctl_SOURCES =\ daxctl.c \ list.c \ migrate.c \ + device.c \ ../util/json.c daxctl_LDADD =\ lib/libdaxctl.la \ ../libutil.a \ $(UUID_LIBS) \ + $(KMOD_LIBS) \ $(JSON_LIBS)
daxctl/builtin.h +1 -0
@@ -6,4 +6,5 @@struct daxctl_ctx ;int cmd_list (int argc,const char **argv,struct daxctl_ctx *ctx);int cmd_migrate (int argc,const char **argv,struct daxctl_ctx *ctx); +int cmd_reconfig_device (int argc,const char **argv,struct daxctl_ctx *ctx);#endif /* _DAXCTL_BUILTIN_H_ */
daxctl/daxctl.c +1 -0
@@ -71,6 +71,7 @@ static struct cmd_struct commands[] = { { "list", .d_fn = cmd_list }, { "help", .d_fn = cmd_help }, { "migrate-device-model", .d_fn = cmd_migrate }, + { "reconfigure-device", .d_fn = cmd_reconfig_device }, };int main (int argc,const char **argv)
daxctl/device.c new file +347 -0
347 lines not on this page · raw message
Signature · 6 lines
2.20.1 _______________________________________________ Linux-nvdimm mailing list [email protected] https://lists.01.org/mailman/listinfo/linux-nvdimm
Vishal Verma
maintainerPATCH v7 8/13 Documentation/daxctl: add a man page for daxctl-reconfigure-device
In reply to Vishal VermaAdd a man page describing the new daxctl-reconfigure-device command.
--- Documentation/daxctl/Makefile.am | 3 +- .../daxctl/daxctl-reconfigure-device.txt | 139 ++++++++++++++++++ 2 files changed, 141 insertions(+), 1 deletion(-) create mode 100644 Documentation/daxctl/daxctl-reconfigure-device.txt
Documentation/daxctl/Makefile.am +2 -1
@@ -28,7 +28,8 @@ endif man1_MANS = \ daxctl.1 \ daxctl-list.1 \ - daxctl-migrate-device-model.1 + daxctl-migrate-device-model.1 \ + daxctl-reconfigure-device.1 CLEANFILES = $(man1_MANS)
Documentation/daxctl/daxctl-reconfigure-device.txt new file +139 -0
139 lines not on this page · raw message
Signature · 6 lines
2.20.1 _______________________________________________ Linux-nvdimm mailing list [email protected] https://lists.01.org/mailman/listinfo/linux-nvdimm
Re: [ndctl PATCH v7 08/13] Documentation/daxctl: add a man page for daxctl-reconfigure-device
Quoted 88 lines · Vishal Verma
Add a man page describing the new daxctl-reconfigure-device command.
Cc Pavel Tatashin <[email protected]>Cc Dave Hansen <[email protected]>Cc Dan Williams <[email protected]>Signed-off-by Vishal Verma <[email protected]>--- Documentation/daxctl/Makefile.am | 3 +- .../daxctl/daxctl-reconfigure-device.txt | 139 ++++++++++++++++++ 2 files changed, 141 insertions(+), 1 deletion(-) create mode 100644 Documentation/daxctl/daxctl-reconfigure-device.txtDocumentation/daxctl/Makefile.am +2
-1@@ -28,7 +28,8 @@ endif man1_MANS = \ daxctl.1 \ daxctl-list.1 \ - daxctl-migrate-device-model.1 + daxctl-migrate-device-model.1 \ + daxctl-reconfigure-device.1 CLEANFILES = $(man1_MANS)Documentation/daxctl/daxctl-reconfigure-device.txt new file +55
-0@@ -0,0 +1,139 @@ +// SPDX-License-Identifier: GPL-2.0 + +daxctl-reconfigure-device(1) +============================ + +NAME +---- +daxctl-reconfigure-device - Reconfigure a dax device into a different mode + +SYNOPSIS +-------- +[verse] +'daxctl reconfigure-device' <dax0.0> [<dax1.0>...<daxY.Z>] [<options>] + +EXAMPLES +-------- + +* Reconfigure dax0.0 to system-ram mode, don't online the memory +---- +# daxctl reconfigure-device --mode=system-ram --no-online dax0.0 +[ + { + "chardev":"dax0.0", + "size":16777216000, + "target_node":2, + "mode":"system-ram" + } +] +---- + +* Reconfigure dax0.0 to devdax mode, attempt to offline the memory +---- +# daxctl reconfigure-device --human --mode=devdax --attempt-offline dax0.0 +{ + "chardev":"dax0.0", + "size":"15.63 GiB (16.78 GB)", + "target_node":2, + "mode":"devdax" +} +---- + +* Reconfigure all dax devices on region0 to system-ram mode +---- +# daxctl reconfigure-device --mode=system-ram --region=0 all +[ + { + "chardev":"dax0.0", + "size":16777216000, + "target_node":2, + "mode":"system-ram" + }, + { + "chardev":"dax0.1", + "size":16777216000, + "target_node":3,
Documentation/daxctl/daxctl-reconfigure-device.txt line 65
Quoted 10 lines
Documentation/daxctl/daxctl-reconfigure-device.txt new file +10
-0+ "mode":"system-ram" + } +] +---- + +* Run a process called 'some-service' using numactl to restrict its cpu +nodes to '0' and '1', and memory allocations to node 2 (determined using +daxctl_dev_get_target_node() or 'daxctl list') +---- +# daxctl reconfigure-device --mode=system-ram --no-online dax0.0
Any reason to use --no-online in this example? Presumably some-service may not start if node2 has no online memory.
Quoted 7 lines
Documentation/daxctl/daxctl-reconfigure-device.txt new file +7
-0+[ + { + "chardev":"dax0.0", + "size":16777216000, + "target_node":2, + "mode":"system-ram" + }
Documentation/daxctl/daxctl-reconfigure-device.txt line 82
Quoted 10 lines
Documentation/daxctl/daxctl-reconfigure-device.txt new file +10
-0+] + +# numactl --cpunodebind=0-1 --membind=2 -- some-service --opt1 --opt2 +---- + +DESCRIPTION +----------- + +Reconfigure the operational mode of a dax device. This can be used to convert +a regular 'devdax' mode device to the 'system-ram' mode which allows for the dax
s/allows/arranges/
Documentation/daxctl/daxctl-reconfigure-device.txt line 90
Quoted 8 lines
Documentation/daxctl/daxctl-reconfigure-device.txt new file +8
-0+range to be hot-plugged into the system as regular memory. + +NOTE: This is a destructive operation. Any data on the dax device *will* be +lost. + +NOTE: Device reconfiguration depends on the dax-bus device model. If dax-class is +in use (via the dax_pmem_compat driver), the reconfiguration will fail. See +linkdaxctl:daxctl-migrate-device-model[1] for more information.
Let's make sure that do_reconfig() bails with a common error message for the compat case and quote that message here. You can check that by comparing the device path 'subsystem' to /sys/class/dax. I.e.
# ls -l /dev/dax0.0 crw------- 1 root root 253, 4 Jul 25 12:22 /dev/dax0.0
# readlink -f /sys/dev/char/253\:4/subsystem /sys/class/dax
...it will be /sys/bus/dax otherwise.
Quoted 14 lines
Documentation/daxctl/daxctl-reconfigure-device.txt new file +14
-0+ +OPTIONS +------- +-r:: +--region=:: + Restrict the operation to devices belonging to the specified region(s). + A device-dax region is a contiguous range of memory that hosts one or + more /dev/daxX.Y devices, where X is the region id and Y is the device + instance id. + +-m:: +--mode=:: + Specify the mode to which the dax device(s) should be reconfigured. + - "system-ram": hotplug the device into system memory.
Documentation/daxctl/daxctl-reconfigure-device.txt line 114
Quoted 10 lines
Documentation/daxctl/daxctl-reconfigure-device.txt new file +10
-0+ + - "devdax": switch to the normal "device dax" mode. This requires the + kernel to support hot-unplugging 'kmem' based memory. If this is not + available, a reboot is the only way to switch back to 'devdax' mode. + +-N:: +--no-online:: + By default, memory sections provided by system-ram devices will be + brought online automatically and immediately with the 'online_movable' + policy. Use this option to disable the automatic onlining behavior.
Probably need to mention that the system might online the memory even if this is specified, or for extra credit, coordinate with that auto-online facility to arrange for it to be skipped.
Documentation/daxctl/daxctl-reconfigure-device.txt line 122
Quoted 8 lines
Documentation/daxctl/daxctl-reconfigure-device.txt new file +8
-0+ +-O:: +--attempt-offline:: + When converting from "system-ram" mode to "devdax", it is expected + that all the memory sections are first made offline. By default, + daxctl won't touch online memory. However with this option, attempt + to offline the memory on the NUMA node associated with the dax device + before converting it back to "devdax" mode.
As mentioned in patch 7, this sounds like --force to me.
_______________________________________________
Linux-nvdimm mailing list [email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm
Verma, Vishal L
maintainerRe: [ndctl PATCH v7 08/13] Documentation/daxctl: add a man page for daxctl-reconfigure-device
Quoted 13 lines · Dan Williams
Quoted 12 lines · Vishal Verma
Add a man page describing the new daxctl-reconfigure-device command.
Cc Pavel Tatashin <[email protected]>Cc Dave Hansen <[email protected]>Cc Dan Williams <[email protected]>Signed-off-by Vishal Verma <[email protected]>--- Documentation/daxctl/Makefile.am | 3 +- .../daxctl/daxctl-reconfigure-device.txt | 139 ++++++++++++++++++ 2 files changed, 141 insertions(+), 1 deletion(-) create mode 100644 Documentation/daxctl/daxctl-reconfigure-device.txt
[..]
Quoted 9 lines
Documentation/daxctl/daxctl-reconfigure-device.txt line 65
Quoted 5 lines
Documentation/daxctl/daxctl-reconfigure-device.txt new file +5
-0+* Run a process called 'some-service' using numactl to restrict its cpu +nodes to '0' and '1', and memory allocations to node 2 (determined using +daxctl_dev_get_target_node() or 'daxctl list') +---- +# daxctl reconfigure-device --mode=system-ram --no-online dax0.0Any reason to use --no-online in this example? Presumably some-service may not start if node2 has no online memory.
Yep just a copy/paste typo, removing it.
Quoted 12 lines
Quoted 12 lines
+[
Documentation/daxctl/daxctl-reconfigure-device.txt new file +10
-0+ { + "chardev":"dax0.0", + "size":16777216000, + "target_node":2, + "mode":"system-ram" + } +] + +# numactl --cpunodebind=0-1 --membind=2 -- some-service --opt1 --opt2 +----
Quoted 8 lines
Documentation/daxctl/daxctl-reconfigure-device.txt line 82
Quoted 6 lines
Documentation/daxctl/daxctl-reconfigure-device.txt new file +6
-0+ +DESCRIPTION +----------- + +Reconfigure the operational mode of a dax device. This can be used to convert +a regular 'devdax' mode device to the 'system-ram' mode which allows for the daxs/allows/arranges/
ok.
Quoted 14 lines
Documentation/daxctl/daxctl-reconfigure-device.txt line 90
Quoted 9 lines
Documentation/daxctl/daxctl-reconfigure-device.txt new file +8
-0+range to be hot-plugged into the system as regular memory. + +NOTE: This is a destructive operation. Any data on the dax device *will* be +lost. + +NOTE: Device reconfiguration depends on the dax-bus device model. If dax-class is +in use (via the dax_pmem_compat driver), the reconfiguration will fail. See +linkdaxctl:daxctl-migrate-device-model[1] for more information.Let's make sure that
do_reconfig()bails with a common error message for the compat case and quote that message here. You can check that by comparing the device path 'subsystem' to/sys/class/dax. I.e.
Quoted 7 lines
# ls -l /dev/dax0.0 crw------- 1 root root 253, 4 Jul 25 12:22 /dev/dax0.0# readlink -f /sys/dev/char/253\:4/subsystem /sys/class/dax...it will be
/sys/bus/daxotherwise.
Hm,. currently the failure is just:
libdaxctl: daxctl_dev_enable: dax3.0: failed to enable
Should we check the subsystem as above programatically, and print a better error in daxctl-reconfigure-device?
Quoted 21 lines
Quoted 21 lines
Documentation/daxctl/daxctl-reconfigure-device.txt new file +20
-0+ +OPTIONS +------- +-r:: +--region=:: + Restrict the operation to devices belonging to the specified region(s). + A device-dax region is a contiguous range of memory that hosts one or + more /dev/daxX.Y devices, where X is the region id and Y is the device + instance id. + +-m:: +--mode=:: + Specify the mode to which the dax device(s) should be reconfigured. + - "system-ram": hotplug the device into system memory. + + - "devdax": switch to the normal "device dax" mode. This requires the + kernel to support hot-unplugging 'kmem' based memory. If this is not + available, a reboot is the only way to switch back to 'devdax' mode. + +-N::
Quoted 8 lines
Documentation/daxctl/daxctl-reconfigure-device.txt line 114
Quoted 4 lines
+--no-online::
Documentation/daxctl/daxctl-reconfigure-device.txt new file +3
-0+ By default, memory sections provided by system-ram devices will be + brought online automatically and immediately with the 'online_movable' + policy. Use this option to disable the automatic onlining behavior.Probably need to mention that the system might online the memory even if this is specified, or for extra credit, coordinate with that auto-online facility to arrange for it to be skipped.
Good point, added a note regarding this.
Quoted 11 lines
Documentation/daxctl/daxctl-reconfigure-device.txt line 122
Quoted 9 lines
Documentation/daxctl/daxctl-reconfigure-device.txt new file +8
-0+ +-O:: +--attempt-offline:: + When converting from "system-ram" mode to "devdax", it is expected + that all the memory sections are first made offline. By default, + daxctl won't touch online memory. However with this option, attempt + to offline the memory on the NUMA node associated with the dax device + before converting it back to "devdax" mode.As mentioned in patch 7, this sounds like --force to me.
Done.
_______________________________________________
Linux-nvdimm mailing list
[email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm
Vishal Verma
maintainerPATCH v7 9/13 daxctl: add commands to online and offline memory
In reply to Vishal VermaAdd two new commands:
daxctl-online-memory
daxctl-offline-memory
to manage the state of hot-plugged memory from the system-ram mode for dax devices. This provides a way for the user to online/offline the memory as a separate step from the reconfiguration. Without this, a user that reconfigures a device into the system-ram mode with the --no-online option, would have no way to later online the memory, and would have to resort to shell scripting to online them manually via sysfs.
--- daxctl/builtin.h | 2 + daxctl/daxctl.c | 2 + daxctl/device.c | 131 ++++++++++++++++++++++++++++++++++++++++++++++- 3 files changed, 134 insertions(+), 1 deletion(-)
daxctl/builtin.h +2 -0
@@ -7,4 +7,6 @@ struct daxctl_ctx;int cmd_list (int argc,const char **argv,struct daxctl_ctx *ctx);int cmd_migrate (int argc,const char **argv,struct daxctl_ctx *ctx);int cmd_reconfig_device (int argc,const char **argv,struct daxctl_ctx *ctx); +int cmd_online_memory (int argc,const char **argv,struct daxctl_ctx *ctx); +int cmd_offline_memory (int argc,const char **argv,struct daxctl_ctx *ctx);#endif /* _DAXCTL_BUILTIN_H_ */
daxctl/daxctl.c +2 -0
@@ -72,6 +72,8 @@ static struct cmd_struct commands[] = { { "help", .d_fn = cmd_help }, { "migrate-device-model", .d_fn = cmd_migrate }, { "reconfigure-device", .d_fn = cmd_reconfig_device }, + { "online-memory", .d_fn = cmd_online_memory }, + { "offline-memory", .d_fn = cmd_offline_memory }, };int main (int argc,const char **argv)
daxctl/device.c +130 -1
177 lines not on this page · raw message
Signature · 6 lines
2.20.1 _______________________________________________ Linux-nvdimm mailing list [email protected] https://lists.01.org/mailman/listinfo/linux-nvdimm
Re: [ndctl PATCH v7 09/13] daxctl: add commands to online and offline memory
Quoted 96 lines · Vishal Verma
Add two new commands:
daxctl-online-memory
daxctl-offline-memoryto manage the state of hot-plugged memory from the system-ram mode for dax devices. This provides a way for the user to online/offline the memory as a separate step from the reconfiguration. Without this, a user that reconfigures a device into the system-ram mode with the --no-online option, would have no way to later online the memory, and would have to resort to shell scripting to online them manually via sysfs.
Cc Dan Williams <[email protected]>Cc Dave Hansen <[email protected]>Signed-off-by Vishal Verma <[email protected]>--- daxctl/builtin.h | 2 + daxctl/daxctl.c | 2 + daxctl/device.c | 131 ++++++++++++++++++++++++++++++++++++++++++++++- 3 files changed, 134 insertions(+), 1 deletion(-)daxctl/builtin.h +2
-0@@ -7,4 +7,6 @@ struct daxctl_ctx;int cmd_list (int argc,const char **argv,struct daxctl_ctx *ctx);int cmd_migrate (int argc,const char **argv,struct daxctl_ctx *ctx);int cmd_reconfig_device (int argc,const char **argv,struct daxctl_ctx *ctx); +int cmd_online_memory (int argc,const char **argv,struct daxctl_ctx *ctx); +int cmd_offline_memory (int argc,const char **argv,struct daxctl_ctx *ctx);#endif /* _DAXCTL_BUILTIN_H_ */ daxctl/daxctl.c +2
-0@@ -72,6 +72,8 @@ static struct cmd_struct commands[] = { { "help", .d_fn = cmd_help }, { "migrate-device-model", .d_fn = cmd_migrate }, { "reconfigure-device", .d_fn = cmd_reconfig_device }, + { "online-memory", .d_fn = cmd_online_memory }, + { "offline-memory", .d_fn = cmd_offline_memory }, };int main (int argc,const char **argv)daxctl/device.c +17
-0@@ -36,6 +36,8 @@ static unsigned long flags;enum device_action {ACTION_RECONFIG , +ACTION_ONLINE , +ACTION_OFFLINE , };#define BASE_OPTIONS () \@@ -56,6 +58,11 @@ static const struct option reconfig_options[] = { OPT_END(), }; +static const struct option memory_options[] = { +BASE_OPTIONS (), +OPT_END (), +}; +static const char *parse_device_options (int argc,const char **argv,enum device_action action,const struct option *options,const char *usage,struct daxctl_ctx *ctx)@@ -76,6 +83,12 @@ static const char *parse_device_options(int argc, const char **argv,case ACTION_RECONFIG : action_string ="reconfigure" ;break ; +case ACTION_ONLINE : + action_string ="online memory for" ; +break ; +case ACTION_OFFLINE : + action_string ="offline memory for" ; +break ;default : action_string ="<>" ;break ;@@ -124,6 +137,10 @@ static const char *parse_device_options(int argc, const char **argv, } }break ; +case ACTION_ONLINE : +case ACTION_OFFLINE : +/* nothing special */ +break ; }if (rc) {usage_with_options (u, options);
daxctl/device.c static int do_reconfig(struct daxctl_dev *dev, enum dev_mode mode, · line 312
Quoted 10 lines
daxctl/device.c +7
-0@@ -286,10 +303,75 @@ static int do_reconfig(struct daxctl_dev *dev, enum dev_mode mode,return rc; } +static int do_xline (struct daxctl_dev *dev,enum device_action action) +{ +struct daxctl_memory *mem =daxctl_dev_get_memory (dev); +const char *devname =daxctl_dev_get_devname (dev); +int rc, num_online; + +if (!daxctl_dev_is_enabled (dev)) {
Can this fail if daxctl_dev_get_memory() succeeded?
Other than that potential code reduction looks good to me.
_______________________________________________
Linux-nvdimm mailing list [email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm
Verma, Vishal L
maintainerRe: [ndctl PATCH v7 09/13] daxctl: add commands to online and offline memory
Quoted 125 lines · Dan Williams
On Wed, Jul 24, 2019 at 2:58 PM Vishal Verma <[email protected]
Quoted 111 lines
wrote:
Add two new commands:daxctl-online-memory
daxctl-offline-memoryto manage the state of hot-plugged memory from the system-ram mode for
dax devices. This provides a way for the user to online/offline the memory as a separate step from the reconfiguration. Without this, a user
that reconfigures a device into the system-ram mode with the --no- online
option, would have no way to later online the memory, and would have to
resort to shell scripting to online them manually via sysfs.Cc Dan Williams <[email protected]>Cc Dave Hansen <[email protected]>Signed-off-by Vishal Verma <[email protected]>--- daxctl/builtin.h | 2 + daxctl/daxctl.c | 2 + daxctl/device.c | 131 ++++++++++++++++++++++++++++++++++++++++++++++- 3 files changed, 134 insertions(+), 1 deletion(-)daxctl/builtin.h +0
-0@@ -7,4 +7,6 @@ struct daxctl_ctx;int cmd_list (int argc,const char **argv,struct daxctl_ctx *ctx);int cmd_migrate (int argc,const char **argv,struct daxctl_ctx *ctx); int cmd_reconfig_device(int argc, const char **argv, struct
daxctl_ctx*ctx);
+intcmd_online_memory(int argc, const char **argv, struct daxctl_ctx *ctx);
+intcmd_offline_memory(int argc, const char **argv, struct daxctl_ctx *ctx);
#endif /* _DAXCTL_BUILTIN_H_ */daxctl/daxctl.c +2
-0@@ -72,6 +72,8 @@ static struct cmd_struct commands[] = { { "help", .d_fn = cmd_help }, { "migrate-device-model", .d_fn = cmd_migrate }, { "reconfigure-device", .d_fn = cmd_reconfig_device }, + { "online-memory", .d_fn = cmd_online_memory }, + { "offline-memory", .d_fn = cmd_offline_memory }, };int main (int argc,const char **argv)daxctl/device.c +7
-0@@ -36,6 +36,8 @@ static unsigned long flags;enum device_action {ACTION_RECONFIG , +ACTION_ONLINE , +ACTION_OFFLINE , };#define BASE_OPTIONS () \@@ -56,6 +58,11 @@ static const struct option reconfig_options[] = { OPT_END(), }; +static const struct option memory_options[] = { +BASE_OPTIONS (), +OPT_END (), +}; +static const char *parse_device_options (int argc,const char **argv, enum device_action action, const struct option*options,
const char *usage, structdaxctl_ctx*ctx)daxctl/device.c +6
-0@@ -76,6 +83,12 @@ static const char *parse_device_options(int argc,const char **argv, case ACTION_RECONFIG: action_string ="reconfigure" ;break ; +case ACTION_ONLINE : + action_string ="online memory for" ; +break ; +case ACTION_OFFLINE : + action_string ="offline memory for" ; +break ;default : action_string ="<>" ;break;daxctl/device.c +4
-0@@ -124,6 +137,10 @@ static const char *parse_device_options(int argc, const char **argv, } }break ; +case ACTION_ONLINE : +case ACTION_OFFLINE : +/* nothing special */ +break ; }if (rc) {usage_with_options(u, options);daxctl/device.c +0
-0@@ -286,10 +303,75 @@ static int do_reconfig(struct daxctl_dev *dev,enum dev_mode mode, return rc;daxctl/device.c static int do_reconfig(struct daxctl_dev *dev, · line 314
Quoted 10 lines
daxctl/device.c +7
-0} +static int do_xline (struct daxctl_dev *dev,enum device_action action) +{ +struct daxctl_memory *mem =daxctl_dev_get_memory (dev); +const char *devname =daxctl_dev_get_devname (dev); +int rc, num_online; + +if (!daxctl_dev_is_enabled (dev)) {Can this fail if
daxctl_dev_get_memory()succeeded?
Quoted 1 line
Other than that potential code reduction looks good to me.
Nope, not needed. Removed, and got even more code reduction/consolidation as a result of adding the section count interface.
_______________________________________________
Linux-nvdimm mailing list
[email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm
Vishal Verma
maintainerPATCH v7 10/13 Documentation: Add man pages for daxctl-{on, off}line-memory
In reply to Vishal VermaAdd man pages for the two new commands: daxctl-online-memory, and daxctl-offline-memory.
--- Documentation/daxctl/Makefile.am | 4 +- .../daxctl/daxctl-offline-memory.txt | 72 +++++++++++++++++ Documentation/daxctl/daxctl-online-memory.txt | 80 +++++++++++++++++++ 3 files changed, 155 insertions(+), 1 deletion(-) create mode 100644 Documentation/daxctl/daxctl-offline-memory.txt create mode 100644 Documentation/daxctl/daxctl-online-memory.txt
Documentation/daxctl/Makefile.am +3 -1
@@ -29,7 +29,9 @@ man1_MANS = \ daxctl.1 \ daxctl-list.1 \ daxctl-migrate-device-model.1 \ - daxctl-reconfigure-device.1 + daxctl-reconfigure-device.1 \ + daxctl-online-memory.1 \ + daxctl-offline-memory.1 CLEANFILES = $(man1_MANS)
Documentation/daxctl/daxctl-offline-memory.txt new file +72 -0
72 lines not on this page · raw message
Documentation/daxctl/daxctl-online-memory.txt new file +80 -0
80 lines not on this page · raw message
Signature · 6 lines
2.20.1 _______________________________________________ Linux-nvdimm mailing list [email protected] https://lists.01.org/mailman/listinfo/linux-nvdimm
Re: [ndctl PATCH v7 10/13] Documentation: Add man pages for daxctl-{on, off}line-memory
Quoted 7 lines · Vishal Verma
Add man pages for the two new commands: daxctl-online-memory, and daxctl-offline-memory.
Cc Dan Williams <[email protected]>Cc Dave Hansen <[email protected]>Signed-off-by Vishal Verma <[email protected]>
Looks good:
_______________________________________________
Linux-nvdimm mailing list
[email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm
Vishal Verma
maintainerPATCH v7 11/13 contrib/ndctl: fix region-id completions for daxctl
In reply to Vishal VermaThe completion helpers for daxctl assumed the region arguments for specifying daxctl regions were the same as ndctl regions, i.e. "regionX". This is not true - daxctl region arguments are a simple numeric 'id'.
Add a new helper __daxctl_get_regions() to complete daxctl region IDs properly.
While at it, fix a useless use of 'echo' in __daxctl_get_devs() and quoting in __daxctl_comp_options()
--- contrib/ndctl | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-)
contrib/ndctl +10 -4
@@ -531,8 +531,14 @@ _ndctl() __daxctl_get_devs() { - local opts="--devices $*" - echo "$(daxctl list $opts | grep -E "^\s*\"chardev\":" | cut -d\" -f4)" + local opts=("--devices" "$*") + daxctl list "${opts[@]}" | grep -E "^\s*\"chardev\":" | cut -d'"' -f4 +} + +__daxctl_get_regions() +{ + local opts=("--regions" "$*") + daxctl list "${opts[@]}" | grep -E "^\s*\"id\":" | grep -Eo "[0-9]+" } __daxctlcomp()
@@ -561,10 +567,10 @@ __daxctl_comp_options() local cur_arg=${cur##*=} case $cur_subopt in --region) - opts=$(__ndctl_get_regions -i) + opts="$(__daxctl_get_regions -i)" ;; --dev) - opts=$(__daxctl_get_devs -i) + opts="$(__daxctl_get_devs -i)" ;; *) return
Signature · 6 lines
2.20.1 _______________________________________________ Linux-nvdimm mailing list [email protected] https://lists.01.org/mailman/listinfo/linux-nvdimm
Re: [ndctl PATCH v7 11/13] contrib/ndctl: fix region-id completions for daxctl
Quoted 5 lines · Vishal Verma
The completion helpers for daxctl assumed the region arguments for specifying daxctl regions were the same as ndctl regions, i.e. "regionX". This is not true - daxctl region arguments are a simple numeric 'id'.
Oh, that's an unfortunate difference, but too late to change now I think, good find.
_______________________________________________
Linux-nvdimm mailing list
[email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm
Verma, Vishal L
maintainerRe: [ndctl PATCH v7 11/13] contrib/ndctl: fix region-id completions for daxctl
Quoted 11 lines · Dan Williams
On Wed, Jul 24, 2019 at 2:58 PM Vishal Verma <[email protected]
Quoted 5 lines
wrote:
The completion helpers for daxctl assumed the region arguments for specifying daxctl regions were the same as ndctl regions, i.e. "regionX". This is not true - daxctl region arguments are a simple numeric 'id'.Oh, that's an unfortunate difference, but too late to change now I think, good find.
Reviewed-by Dan Williams <[email protected]>
Yep - though I think I like --region=0 better than --region=region0, less redundancy. But agreed, probably too late to change.
Thanks for reviewing these!
_______________________________________________
Linux-nvdimm mailing list
[email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm
Re: [ndctl PATCH v7 11/13] contrib/ndctl: fix region-id completions for daxctl
Quoted 10 lines · Vishal L
Quoted 7 lines · Dan Williams
On Wed, Jul 24, 2019 at 2:58 PM Vishal Verma <[email protected]
Quoted 5 lines
wrote:
The completion helpers for daxctl assumed the region arguments for specifying daxctl regions were the same as ndctl regions, i.e. "regionX". This is not true - daxctl region arguments are a simple numeric 'id'.
Quoted 7 lines
Quoted 4 lines
Oh, that's an unfortunate difference, but too late to change now I think, good find.
Reviewed-by Dan Williams <[email protected]>Yep - though I think I like --region=0 better than --region=region0, less redundancy. But agreed, probably too late to change.
We could teach it to allow both. Something for the backlog, but at least the kernel is using a common number allocator for pmem regions and hmem regions.
_______________________________________________
Linux-nvdimm mailing list
[email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm
Vishal Verma
maintainerPATCH v7 12/13 contrib/ndctl: add bash-completion for the new daxctl commands
In reply to Vishal VermaAdd bash completion helpers for the new daxctl-reconfigure-device, daxctl-online-memory, and daxctl-offline-memory commands.
--- contrib/ndctl | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-)
contrib/ndctl +21 -3
@@ -547,7 +547,7 @@ __daxctlcomp() COMPREPLY=( $( compgen -W "$1" -- "$2" ) ) for cword in "${COMPREPLY[@]}"; do - if [[ "$cword" == @(--region|--dev) ]]; then + if [[ "$cword" == @(--region|--dev|--mode) ]]; then COMPREPLY[$i]="${cword}=" else COMPREPLY[$i]="${cword} "
@@ -572,6 +572,9 @@ __daxctl_comp_options() --dev) opts="$(__daxctl_get_devs -i)" ;; + --mode) + opts="system-ram devdax" + ;; *) return ;;
@@ -582,8 +585,23 @@ __daxctl_comp_options() __daxctl_comp_non_option_args() { - # there aren't any commands that accept non option arguments yet - return + local subcmd=$1 + local cur=$2 + local opts + + case $subcmd in + reconfigure-device) + ;& + online-memory) + ;& + offline-memory) + opts="$(__daxctl_get_devs -i) all" + ;; + *) + return + ;; + esac + __daxctlcomp "$opts" "$cur" } __daxctl_main()
Signature · 6 lines
2.20.1 _______________________________________________ Linux-nvdimm mailing list [email protected] https://lists.01.org/mailman/listinfo/linux-nvdimm
Re: [ndctl PATCH v7 12/13] contrib/ndctl: add bash-completion for the new daxctl commands
Quoted 4 lines · Vishal Verma
Add bash completion helpers for the new daxctl-reconfigure-device, daxctl-online-memory, and daxctl-offline-memory commands.
Looks good,
_______________________________________________
Linux-nvdimm mailing list
[email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm
Vishal Verma
maintainerPATCH v7 13/13 test: Add a unit test for daxctl-reconfigure-device and friends
In reply to Vishal VermaAdd a new unit test to test dax device reconfiguration and memory operations. This teaches test/common about daxctl, and adds an ACPI.NFIT bus variable. Since we have to operate on the ACPI.NFIT bus, the test is marked as destructive.
--- test/Makefile.am | 3 +- test/common | 19 ++++++++-- test/daxctl-devices.sh | 81 ++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 99 insertions(+), 4 deletions(-) create mode 100755 test/daxctl-devices.sh
test/Makefile.am +2 -1
@@ -49,7 +49,8 @@ TESTS +=\ dax.sh \ device-dax \ device-dax-fio.sh \ - mmap.sh + mmap.sh \ + daxctl-devices.sh if ENABLE_KEYUTILS TESTS += security.sh
test/common +16 -3
@@ -15,12 +15,25 @@ else exit 1 fi -# NFIT_TEST_BUS[01] +# DAXCTL # -NFIT_TEST_BUS0=nfit_test.0 -NFIT_TEST_BUS1=nfit_test.1 +if [ -f "../daxctl/daxctl" ] && [ -x "../daxctl/daxctl" ]; then + export DAXCTL=../daxctl/daxctl +elif [ -f "./daxctl/daxctl" ] && [ -x "./daxctl/daxctl" ]; then + export DAXCTL=./daxctl/daxctl +else + echo "Couldn't find an daxctl binary" + exit 1 +fi +# NFIT_TEST_BUS[01] +# +NFIT_TEST_BUS0="nfit_test.0" +NFIT_TEST_BUS1="nfit_test.1" +ACPI_BUS="ACPI.NFIT" +E820_BUS="e820" + # Functions # err
test/daxctl-devices.sh new file +81 -0
81 lines not on this page · raw message
Signature · 6 lines
2.20.1 _______________________________________________ Linux-nvdimm mailing list [email protected] https://lists.01.org/mailman/listinfo/linux-nvdimm
Re: [ndctl PATCH v7 13/13] test: Add a unit test for daxctl-reconfigure-device and friends
Quoted 6 lines · Vishal Verma
Add a new unit test to test dax device reconfiguration and memory operations. This teaches test/common about daxctl, and adds an ACPI.NFIT bus variable. Since we have to operate on the ACPI.NFIT bus, the test is marked as destructive.
Looks good.
_______________________________________________
Linux-nvdimm mailing list
[email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm