daxctl: add a new reconfigure-device command
Subsystems (3)
Vishal Verma
maintainerv9 cover daxctl: add a new reconfigure-device command
Changes in v9:
- Move the device model checking into the library. This way, daxctl-list can correctly determine 'state' which only applies to the dax-bus model.
Changes in v8:
- rename the --attempt-offline option to --force (Dan)
- clarify the messages when device is already in the requested state (Dan)
- s/unable/failed/ in device.c error messages (Dan)
- daxctl_memory_{on,off}line() instead of daxctl_memory_set_{on,off}line (Dan)
- Add an interface to get a count of the memory sections associated with a device (Dan)
- As a result, refactor the readdir loop into a common memory_op function that can set the state, get the online state, and get a count of all blocks.
- Update the onlining/offlining routines used in both the reconfigure-device and {on,off}line-memory commands to use the new daxctl_memory_num_sections() interface to validate the number of sections for which we changed the state.
- Add some small clarifications in the daxctl-reconfigure-device man page (Dan)
- In device.c add a verify_dax_bus_model() helper to check for the dax-bus subsystem (Dan).
Changes in v7:
- Fix a couple of checkaptch type errors in the new lines added in v6 (Dan).
- Get rid of daxctl_dev_get_mode. daxctl_dev_get_memory is sufficient to both check the mode and allocate the memory related structures on its first call. (Dan)
- Due to the above, daxctl_dev_mode is now private to libdaxctl, and not part of the API exported through libdaxctl.h
- Add a large enough buffer at init time to construct dynamic paths, and avoid asprintf() type allocations for memory blocks at runtime (Dan).
Changes in v6:
- For memory block online/offline operations, the kernel responds with an EINVAL for both 'real' errors, and if the memory was already in the requested state. Since there is a TOCTOU hole between checking the state and storing it, just perform a second check if the store results in an error. If the check shows the state to be the same as the one we're attempting, it means that another agent (usually udev) won the race, but we don't care so long as the state change happened, so don't report an error. (Fan Du)
Changes in v5:
- device.c: correctly set loglevel for daxctl_ctx for --verbose
- drop the subsys caching, its complexity started to exceed its benefit. dax-class device models will simply error out during reconfigure. (Dan)
- Add a note to the man page for the above.
- Clarify the onlining policy (online_movable) in the man page
- rename "numa_node" to "target_node" in device listings (Dan)
- When printing a device 'mode', assume devdax if !system-ram, avoiding a "mode: unknown" situation which can be confusing. (Dan)
- Add a "state: disabled" attribute to the device listing if a driver is not bound. This is more apt than the previous "mode: unknown" listing.
- add an api to get 'dev->resource' parsing /proc/iomem as a fallback for when the kernel doesn't provide the attribute (Dan)
- convert node_* apis to 'memory_* apis that act on a new daxctl_memory object (Dan)
- online only memory sections belonging to the device in question by cross referencing block indices with the dax device resource (Dan)
- Refuse to reconfigure a device that is already in the target mode. Until now, reconfiguring a system-ram device back to system-ram would result in a 'online memory may not be hot-removed' kernel warning.
- If the device was already in the system-ram mode, skip disabling/enabling, but still try to online the memory unless the --no-online option is in effect.
- In daxctl_unbind, also 'remove_id' to prevent devices automatically binding to the kmem driver on a disable + re-enable, which can be surprising (Dan).
- Rewrite the top half of daxctl/device.c to borrow elements from ndctl/namespace.c so that it can support growing additional commands that operate on devices (online-memory and offline-memory)
- Refactor the bottom half of daxctl/device.c so we only do the disabling/offlining steps if the device was enabled.
- Add new commands to online and offline memory sections (Dan) associated with a given dax device (Dan)
- Add a new test - daxctl-device.sh - to test daxctl reconfigure-device, online-memory, and offline-memory commands.
- Add an example in documentation demonstrating how to use numactl to bind a process to a node surfaced from a dax device (Andy Rudoff)
Changes in v4:
- Don't fail add_dax_dev for kmod failures. Instead fail only when the kmod list is actually used, i.e. during daxctl-reconfigure-device
Changes in v3:
- In daxctl_dev_get_mode(), remove the subsystem warning, detect dax-class and simply make it return devdax
Changes in v2:
- Add examples to the documentation page (Dave Hansen)
- Clarify documentation regarding the conversion from system-ram to devdax
- Remove any references to a persistent config from the documentation - those can be added when the feature is added.
- device.c: validate option compatibility
- daxctl-list: display numa_node for device listings
- daxctl-list: display mode for device listings
- make the options more consistent by adding a '-O' short option for --attempt-offline
Add a new daxctl-reconfigure-device command that lets us reconfigure DAX devices back and forth between 'system-ram' and 'device-dax' modes. It also includes facilities to online any newly hot-plugged memory (default), and attempt to offline memory before converting away from the system-ram mode (not default, requires a --attempt-offline option).
Currently missing from this series is a way to persistently store which devices have been 'marked' for use as system-ram. This depends on a config system overhaul in ndctl, and patches for those will follow separately and are independent of this work.
Example invocations:
1. Reconfigure dax0.0 to system-ram mode, don’t online the memory
# daxctl reconfigure-device --mode=system-ram --no-online dax0.0
[
{
"chardev":"dax0.0",
"size":16777216000,
"target_node":2,
"mode":"system-ram"
}
]2. Reconfigure dax0.0 to devdax mode, attempt to offline the memory
# daxctl reconfigure-device --human --mode=devdax --attempt-offline dax0.0
{
"chardev":"dax0.0",
"size":"15.63 GiB (16.78 GB)",
"target_node":2,
"mode":"devdax"
}3. Reconfigure all dax devices on region0 to system-ram mode
# daxctl reconfigure-device --mode=system-ram --region=0 all
[
{
"chardev":"dax0.0",
"size":16777216000,
"target_node":2,
"mode":"system-ram"
},
{
"chardev":"dax0.1",
"size":16777216000,
"target_node":3,
"mode":"system-ram"
}
]These patches can also be found in the 'kmem-pending' branch on github:
https://github.com/pmem/ndctl/tree/kmem-pending
Vishal Verma (13):
libdaxctl: add interfaces to get ctx and check device state
libdaxctl: add interfaces to enable/disable devices
libdaxctl: add an interface to retrieve the device resource
libdaxctl: add a 'daxctl_memory' object for memory based operations daxctl/list: add target_node for device listings daxctl/list: display the mode for a dax device
daxctl: add a new reconfigure-device command Documentation/daxctl: add a man page for daxctl-reconfigure-device
daxctl: add commands to online and offline memory
Documentation: Add man pages for daxctl-{on,off}line-memory contrib/ndctl: fix region-id completions for daxctl contrib/ndctl: add bash-completion for the new daxctl commands
test: Add a unit test for daxctl-reconfigure-device and friends
Documentation/daxctl/Makefile.am | 5 +- .../daxctl/daxctl-offline-memory.txt | 72 ++ Documentation/daxctl/daxctl-online-memory.txt | 80 ++ .../daxctl/daxctl-reconfigure-device.txt | 157 ++++ Makefile.am | 3 +- contrib/ndctl | 38 +- daxctl/Makefile.am | 2 + daxctl/builtin.h | 3 + daxctl/daxctl.c | 3 + daxctl/device.c | 543 +++++++++++++ daxctl/lib/Makefile.am | 5 +- daxctl/lib/libdaxctl-private.h | 40 + daxctl/lib/libdaxctl.c | 712 ++++++++++++++++++ daxctl/lib/libdaxctl.sym | 19 + daxctl/libdaxctl.h | 17 + test/Makefile.am | 3 +- test/common | 19 +- test/daxctl-devices.sh | 81 ++ util/iomem.c | 37 + util/iomem.h | 12 + util/json.c | 22 + 21 files changed, 1859 insertions(+), 14 deletions(-) create mode 100644 Documentation/daxctl/daxctl-offline-memory.txt create mode 100644 Documentation/daxctl/daxctl-online-memory.txt create mode 100644 Documentation/daxctl/daxctl-reconfigure-device.txt create mode 100644 daxctl/device.c create mode 100755 test/daxctl-devices.sh create mode 100644 util/iomem.c create mode 100644 util/iomem.h
Signature · 6 lines
2.20.1 _______________________________________________ Linux-nvdimm mailing list [email protected] https://lists.01.org/mailman/listinfo/linux-nvdimm
Vishal Verma
maintainerPATCH v9 1/13 libdaxctl: add interfaces to get ctx and check device state
In preparation for libdaxctl and daxctl to grow operational modes for DAX devices, add the following supporting APIs:
daxctl_dev_get_ctx
daxctl_dev_is_enabled
It also adds and uses a helper to verify the device model for the _is_enabled API, since enable/disable only make sense for the dax-bus model.
--- daxctl/lib/libdaxctl.c | 70 ++++++++++++++++++++++++++++++++++++++++ daxctl/lib/libdaxctl.sym | 6 ++++ daxctl/libdaxctl.h | 2 ++ 3 files changed, 78 insertions(+)
daxctl/lib/libdaxctl.c +70 -0
@@ -306,6 +306,43 @@ DAXCTL_EXPORT struct daxctl_region *daxctl_new_region(struct daxctl_ctx *ctx,return region; } +static bool device_model_is_dax_bus (struct daxctl_dev *dev) +{ +const char *devname =daxctl_dev_get_devname (dev); +struct daxctl_ctx *ctx =daxctl_dev_get_ctx (dev); +char *path = dev->dev_buf, *resolved; +size_t len = dev->buf_len; +struct stat sb; + +if (snprintf (path, len,"/dev/%s" , devname) <0 ) +return false; + +if (lstat (path, &sb) <0 ) { +err (ctx,"%s: stat for %s failed: %s\n" , + devname, path,strerror (errno)); +return false; + } + +if (snprintf (path, len,"/sys/dev/char/%d:%d/subsystem" , +major (sb.st_rdev),minor (sb.st_rdev)) <0 ) +return false; + + resolved =realpath (path,NULL ); +if (!resolved) { +err (ctx,"%s: unable to determine subsys: %s\n" , + devname,strerror (errno)); +return false; + } + +if (strcmp (resolved,"/sys/bus/dax" ) ==0 ) { +free (resolved); +return true; + } + +free (resolved); +return false; +} +static void *add_dax_dev (void *parent,int id,const char *daxdev_base) {const char *devname =devpath_to_devname (daxdev_base);
@@ -559,6 +596,39 @@ static void dax_regions_init(struct daxctl_ctx *ctx) } } +static int is_enabled (const char *drvpath) +{ +struct stat st; + +if (lstat (drvpath, &st) <0 || !S_ISLNK (st.st_mode)) +return 0 ; +else +return 1 ; +} + +DAXCTL_EXPORT int daxctl_dev_is_enabled(struct daxctl_dev *dev) +{ +struct daxctl_ctx *ctx =daxctl_dev_get_ctx (dev); +char *path = dev->dev_buf; +int len = dev->buf_len; + +if (!device_model_is_dax_bus (dev)) +return 1 ; + +if (snprintf (path, len,"%s/driver" , dev->dev_path) >= len) { +err (ctx,"%s: buffer too small!\n" , +daxctl_dev_get_devname (dev)); +return 0 ; + } + +return is_enabled (path); +} + +DAXCTL_EXPORT struct daxctl_ctx *daxctl_dev_get_ctx(struct daxctl_dev *dev) +{ +return dev->region->ctx; +} + DAXCTL_EXPORT struct daxctl_dev *daxctl_dev_get_first(struct daxctl_region *region) {dax_devices_init (region);
daxctl/lib/libdaxctl.sym +6 -0
@@ -50,3 +50,9 @@ LIBDAXCTL_5 { global: daxctl_region_get_path; } LIBDAXCTL_4; + +LIBDAXCTL_6 { +global: + daxctl_dev_get_ctx; + daxctl_dev_is_enabled; +} LIBDAXCTL_5;
daxctl/libdaxctl.h +2 -0
@@ -67,6 +67,8 @@ const char *daxctl_dev_get_devname(struct daxctl_dev *dev);int daxctl_dev_get_major (struct daxctl_dev *dev);int daxctl_dev_get_minor (struct daxctl_dev *dev);unsigned long long daxctl_dev_get_size (struct daxctl_dev *dev); +struct daxctl_ctx *daxctl_dev_get_ctx (struct daxctl_dev *dev); +int daxctl_dev_is_enabled (struct daxctl_dev *dev);#define daxctl_dev_foreach (region, dev) \ for (dev = daxctl_dev_get_first(region); \
Signature · 6 lines
2.20.1 _______________________________________________ Linux-nvdimm mailing list [email protected] https://lists.01.org/mailman/listinfo/linux-nvdimm
Vishal Verma
maintainerPATCH v9 2/13 libdaxctl: add interfaces to enable/disable devices
In reply to Vishal VermaAdd new libdaxctl interfaces to disable a device_dax based device, and to enable it into the given mode. The modes available are 'devdax', and 'system-ram', where devdax is the normal device DAX mode used via a character device, and 'system-ram' uses the kernel's 'kmem' facility to hotplug the device making it usable as normal memory.
This adds the following new interfaces:
daxctl_dev_disable;
daxctl_dev_enable_devdax;
daxctl_dev_enable_ram;
--- daxctl/lib/Makefile.am | 3 +- daxctl/lib/libdaxctl-private.h | 21 +++ daxctl/lib/libdaxctl.c | 244 +++++++++++++++++++++++++++++++++ daxctl/lib/libdaxctl.sym | 3 + daxctl/libdaxctl.h | 3 + 5 files changed, 273 insertions(+), 1 deletion(-)
daxctl/lib/Makefile.am +2 -1
@@ -16,7 +16,8 @@ libdaxctl_la_SOURCES =\ libdaxctl.c libdaxctl_la_LIBADD =\ - $(UUID_LIBS) + $(UUID_LIBS) \ + $(KMOD_LIBS) daxctl_modprobe_data_DATA = daxctl.conf
daxctl/lib/libdaxctl-private.h +21 -0
@@ -13,6 +13,8 @@#ifndef _LIBDAXCTL_PRIVATE_H_#define _LIBDAXCTL_PRIVATE_H_ +#include <libkmod.h> +#define DAXCTL_EXPORT __attribute__ ((visibility("default"))) enum dax_subsystem {
@@ -26,6 +28,17 @@ static const char *dax_subsystems[] = { [DAX_BUS] = "/sys/bus/dax/devices", }; +enum daxctl_dev_mode { +DAXCTL_DEV_MODE_DEVDAX =0 , +DAXCTL_DEV_MODE_RAM , +DAXCTL_DEV_MODE_END , +}; + +static const char *dax_modules[] = { + [DAXCTL_DEV_MODE_DEVDAX ] ="device_dax" , + [DAXCTL_DEV_MODE_RAM ] ="kmem" , +}; +/** * struct daxctl_region - container for dax_devices */
@@ -53,6 +66,14 @@ struct daxctl_dev {char *dev_path;struct list_node list;unsigned long long size; +struct kmod_module *module; +struct kmod_list *kmod_list; struct daxctl_region *region; }; + +static inline int check_kmod (struct kmod_ctx *kmod_ctx) +{ +return kmod_ctx ?0 : -ENXIO ; +} +#endif /* _LIBDAXCTL_PRIVATE_H_ */
daxctl/lib/libdaxctl.c +244 -0
312 lines not on this page · raw message
daxctl/lib/libdaxctl.sym +3 -0
@@ -55,4 +55,7 @@ LIBDAXCTL_6 { global: daxctl_dev_get_ctx; daxctl_dev_is_enabled; + daxctl_dev_disable; + daxctl_dev_enable_devdax; + daxctl_dev_enable_ram; } LIBDAXCTL_5;
daxctl/libdaxctl.h +3 -0
@@ -69,6 +69,9 @@ int daxctl_dev_get_minor(struct daxctl_dev *dev);unsigned long long daxctl_dev_get_size (struct daxctl_dev *dev);struct daxctl_ctx *daxctl_dev_get_ctx (struct daxctl_dev *dev);int daxctl_dev_is_enabled (struct daxctl_dev *dev); +int daxctl_dev_disable (struct daxctl_dev *dev); +int daxctl_dev_enable_devdax (struct daxctl_dev *dev); +int daxctl_dev_enable_ram (struct daxctl_dev *dev);#define daxctl_dev_foreach (region, dev) \ for (dev = daxctl_dev_get_first(region); \
Signature · 6 lines
2.20.1 _______________________________________________ Linux-nvdimm mailing list [email protected] https://lists.01.org/mailman/listinfo/linux-nvdimm
Vishal Verma
maintainerPATCH v9 3/13 libdaxctl: add an interface to retrieve the device resource
In reply to Vishal VermaAdd an interface to retrieve the 'resource' attribute for a dax device.
Attempt to retrieve it as usual via sysfs, but since older kernels may be missing this attribute, as a fallback, attempt to retrieve it from /proc/iomem
--- Makefile.am | 3 ++- daxctl/lib/Makefile.am | 2 ++ daxctl/lib/libdaxctl-private.h | 1 + daxctl/lib/libdaxctl.c | 12 +++++++++++ daxctl/lib/libdaxctl.sym | 1 + daxctl/libdaxctl.h | 1 + util/iomem.c | 37 ++++++++++++++++++++++++++++++++++ util/iomem.h | 12 +++++++++++ 8 files changed, 68 insertions(+), 1 deletion(-) create mode 100644 util/iomem.c create mode 100644 util/iomem.h
Makefile.am +2 -1
@@ -74,6 +74,7 @@ libutil_a_SOURCES = \ util/wrapper.c \ util/filter.c \ util/bitmap.c \ - util/abspath.c + util/abspath.c \ + util/iomem.c nobase_include_HEADERS = daxctl/libdaxctl.h
daxctl/lib/Makefile.am +2 -0
@@ -9,6 +9,8 @@ lib_LTLIBRARIES = libdaxctl.la libdaxctl_la_SOURCES =\ ../libdaxctl.h \ libdaxctl-private.h \ + ../../util/iomem.c \ + ../../util/iomem.h \ ../../util/sysfs.c \ ../../util/sysfs.h \ ../../util/log.c \
daxctl/lib/libdaxctl-private.h +1 -0
@@ -65,6 +65,7 @@ struct daxctl_dev {size_t buf_len;char *dev_path;struct list_node list; +unsigned long long resource;unsigned long long size;struct kmod_module *module;struct kmod_list *kmod_list;
daxctl/lib/libdaxctl.c +12 -0
@@ -24,6 +24,7 @@#include <util/log.h> #include <util/sysfs.h> +#include <util/iomem.h> #include <daxctl/libdaxctl.h> #include "libdaxctl-private.h"
@@ -406,6 +407,12 @@ static void *add_dax_dev(void *parent, int id, const char *daxdev_base) dev->major =major (st.st_rdev); dev->minor =minor (st.st_rdev); +sprintf (path,"%s/resource" , daxdev_base); +if (sysfs_read_attr (ctx, path, buf) ==0 ) + dev->resource =strtoull (buf,NULL ,0 ); +else + dev->resource =iomem_get_dev_resource (ctx, daxdev_base); +sprintf (path,"%s/size" , daxdev_base);if (sysfs_read_attr (ctx, path, buf) <0 ) goto err_read;
@@ -928,6 +935,11 @@ DAXCTL_EXPORT int daxctl_dev_get_minor(struct daxctl_dev *dev)return dev->minor; } +DAXCTL_EXPORT unsigned long long daxctl_dev_get_resource (struct daxctl_dev *dev) +{ +return dev->resource; +} +DAXCTL_EXPORT unsigned long long daxctl_dev_get_size (struct daxctl_dev *dev) {return dev->size;
daxctl/lib/libdaxctl.sym +1 -0
@@ -58,4 +58,5 @@ global: daxctl_dev_disable; daxctl_dev_enable_devdax; daxctl_dev_enable_ram; + daxctl_dev_get_resource; } LIBDAXCTL_5;
daxctl/libdaxctl.h +1 -0
@@ -66,6 +66,7 @@ int daxctl_dev_get_id(struct daxctl_dev *dev);const char *daxctl_dev_get_devname (struct daxctl_dev *dev);int daxctl_dev_get_major (struct daxctl_dev *dev);int daxctl_dev_get_minor (struct daxctl_dev *dev); +unsigned long long daxctl_dev_get_resource (struct daxctl_dev *dev);unsigned long long daxctl_dev_get_size (struct daxctl_dev *dev);struct daxctl_ctx *daxctl_dev_get_ctx (struct daxctl_dev *dev);int daxctl_dev_is_enabled (struct daxctl_dev *dev);
util/iomem.c new file +37 -0
@@ -0,0 +1,37 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright(c) 2019 Intel Corporation. All rights reserved. */ + +#include <stdio.h> +#include <errno.h> +#include <string.h> +#include <util/log.h> +#include <util/iomem.h> +#include <util/sysfs.h> + +unsigned long long __iomem_get_dev_resource (struct log_ctx *ctx, +const char *devpath) +{ +const char *devname =devpath_to_devname (devpath); +FILE *fp =fopen ("/proc/iomem" ,"r" ); +unsigned long long res; +char name[256 ]; + +if (fp ==NULL ) { +log_err (ctx,"%s: open /proc/iomem: %s\n" , devname, +strerror (errno)); +return 0 ; + } + +while (fscanf (fp,"%llx-%*x : %254[^\n]\n" , &res, name) ==2 ) { +if (strcmp (name, devname) ==0 ) { +log_dbg (ctx,"%s: got resource via iomem: %#llx\n" , + devname, res); +fclose (fp); +return res; + } + } + +log_dbg (ctx,"%s: not found in iomem\n" , devname); +fclose (fp); +return 0 ; +}
util/iomem.h new file +12 -0
@@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright(c) 2019 Intel Corporation. All rights reserved. */ +#ifndef _NDCTL_IOMEM_H_ +#define _NDCTL_IOMEM_H_ + +struct log_ctx ; +unsigned long long __iomem_get_dev_resource (struct log_ctx *ctx, +const char *path); + +#define iomem_get_dev_resource (c, p) __iomem_get_dev_resource(&(c)->ctx, (p)) + +#endif /* _NDCTL_IOMEM_H_ */
Signature · 6 lines
2.20.1 _______________________________________________ Linux-nvdimm mailing list [email protected] https://lists.01.org/mailman/listinfo/linux-nvdimm
Vishal Verma
maintainerPATCH v9 4/13 libdaxctl: add a 'daxctl_memory' object for memory based operations
In reply to Vishal VermaIntroduce a new 'daxctl_memory' object, which will be used for operations related to managing dax devices in 'system-memory' modes.
Add libdaxctl APIs to get the target_node of a DAX device, and to online, offline, and query the state of hotplugged memory sections associated with a given device.
This adds the following new interfaces:
daxctl_dev_get_target_node
daxctl_dev_get_memory
daxctl_memory_get_dev
daxctl_memory_get_node_path
daxctl_memory_get_block_size
daxctl_memory_online
daxctl_memory_offline
daxctl_memory_is_online
daxctl_memory_num_sections
--- daxctl/lib/libdaxctl-private.h | 18 ++ daxctl/lib/libdaxctl.c | 384 +++++++++++++++++++++++++++++++++ daxctl/lib/libdaxctl.sym | 9 + daxctl/libdaxctl.h | 11 + 4 files changed, 422 insertions(+)
daxctl/lib/libdaxctl-private.h +18 -0
@@ -39,6 +39,13 @@ static const char *dax_modules[] = { [DAXCTL_DEV_MODE_RAM] = "kmem", }; +enum memory_op { +MEM_SET_OFFLINE , +MEM_SET_ONLINE , +MEM_IS_ONLINE , +MEM_COUNT , +}; +/** * struct daxctl_region - container for dax_devices */
@@ -70,8 +77,19 @@ struct daxctl_dev {struct kmod_module *module;struct kmod_list *kmod_list;struct daxctl_region *region; +struct daxctl_memory *mem; +int target_node; +}; + +struct daxctl_memory { +struct daxctl_dev *dev; +void *mem_buf; +size_t buf_len; +char *node_path; +unsigned long block_size; }; +static inline int check_kmod (struct kmod_ctx *kmod_ctx) {return kmod_ctx ?0 : -ENXIO ;
daxctl/lib/libdaxctl.c +384 -0
417 lines not on this page · raw message
daxctl/lib/libdaxctl.sym +9 -0
@@ -59,4 +59,13 @@ global: daxctl_dev_enable_devdax; daxctl_dev_enable_ram; daxctl_dev_get_resource; + daxctl_dev_get_target_node; + daxctl_dev_get_memory; + daxctl_memory_get_dev; + daxctl_memory_get_node_path; + daxctl_memory_get_block_size; + daxctl_memory_online; + daxctl_memory_offline; + daxctl_memory_is_online; + daxctl_memory_num_sections; } LIBDAXCTL_5;
daxctl/libdaxctl.h +11 -0
@@ -73,6 +73,17 @@ int daxctl_dev_is_enabled(struct daxctl_dev *dev);int daxctl_dev_disable (struct daxctl_dev *dev);int daxctl_dev_enable_devdax (struct daxctl_dev *dev);int daxctl_dev_enable_ram (struct daxctl_dev *dev); +int daxctl_dev_get_target_node (struct daxctl_dev *dev); + +struct daxctl_memory ; +struct daxctl_memory *daxctl_dev_get_memory (struct daxctl_dev *dev); +struct daxctl_dev *daxctl_memory_get_dev (struct daxctl_memory *mem); +const char *daxctl_memory_get_node_path (struct daxctl_memory *mem); +unsigned long daxctl_memory_get_block_size (struct daxctl_memory *mem); +int daxctl_memory_online (struct daxctl_memory *mem); +int daxctl_memory_offline (struct daxctl_memory *mem); +int daxctl_memory_is_online (struct daxctl_memory *mem); +int daxctl_memory_num_sections (struct daxctl_memory *mem);#define daxctl_dev_foreach (region, dev) \ for (dev = daxctl_dev_get_first(region); \
Signature · 6 lines
2.20.1 _______________________________________________ Linux-nvdimm mailing list [email protected] https://lists.01.org/mailman/listinfo/linux-nvdimm
Re: [ndctl PATCH v9 04/13] libdaxctl: add a 'daxctl_memory' object for memory based operations
Quoted 26 lines · Vishal Verma
Introduce a new '
daxctl_memory' object, which will be used for operations related to managing dax devices in 'system-memory' modes.Add libdaxctl APIs to get the
target_nodeof a DAX device, and to online, offline, and query the state of hotplugged memory sections associated with a given device.This adds the following new interfaces:
daxctl_dev_get_target_node
daxctl_dev_get_memory
daxctl_memory_get_dev
daxctl_memory_get_node_path
daxctl_memory_get_block_size
daxctl_memory_online
daxctl_memory_offline
daxctl_memory_is_online
daxctl_memory_num_sectionsCc Pavel Tatashin <[email protected]>Cc Dave Hansen <[email protected]>Cc Dan Williams <[email protected]>[for the memblock-already-online TOCTOU hole]Reported-by Fan Du <[email protected]>Tested-by Fan Du <[email protected]>Signed-off-by Vishal Verma <[email protected]>
Quoted 3 lines
--- daxctl/lib/libdaxctl-private.h | 18 ++ daxctl/lib/libdaxctl.c | 384
+++++++++++++++++++++++++++++++++
Quoted 4 lines
daxctl/lib/libdaxctl.sym | 9 + daxctl/libdaxctl.h | 11 + 4 files changed, 422 insertions(+)
[..]
Review
Quoted 2 lines
(quoted diff) +2
-0+ +static bool memblock_in_dev(struct daxctl_dev *dev, const char
*memblock)
Review
Quoted 3 lines
(quoted diff) +2
-0+{ + struct daxctl_memory *mem = daxctl_dev_get_memory(dev);
Static analysis complains that this can potentially cause a NULL dereference. Fix it by passing the mem object to memblock_in_dev(), since it has already been validated by that time.
3<----
Quoted 1 line
From
e8bf803e359b784259f645d1ff68e964b2c8618fMon Sep 17 00:00:00 2001
From: Vishal Verma <[email protected]>
Date: Fri, 3 May 2019 13:27:35 -0600
Subject: [ndctl PATCH] libdaxctl: add a 'daxctl_memory' object for memory
based operations
Introduce a new 'daxctl_memory' object, which will be used for operations related to managing dax devices in 'system-memory' modes.
Add libdaxctl APIs to get the target_node of a DAX device, and to online, offline, and query the state of hotplugged memory sections associated with a given device.
This adds the following new interfaces:
daxctl_dev_get_target_node
daxctl_dev_get_memory
daxctl_memory_get_dev
daxctl_memory_get_node_path
daxctl_memory_get_block_size
daxctl_memory_online
daxctl_memory_offline
daxctl_memory_is_online
daxctl_memory_num_sections
--- daxctl/lib/libdaxctl-private.h | 18 ++ daxctl/lib/libdaxctl.c | 384 +++++++++++++++++++++++++++++++++ daxctl/lib/libdaxctl.sym | 9 + daxctl/libdaxctl.h | 11 + 4 files changed, 422 insertions(+)
daxctl/lib/libdaxctl- renamed from daxctl/lib/libdaxctl-private.h +0 -0
private.h
index fee67d1..01091de 100644
daxctl/lib/libdaxctl-private.h +18 -0
@@ -39,6 +39,13 @@ static const char *dax_modules[] = { [DAXCTL_DEV_MODE_RAM] = "kmem", }; +enum memory_op { +MEM_SET_OFFLINE , +MEM_SET_ONLINE , +MEM_IS_ONLINE , +MEM_COUNT , +}; +/** * struct daxctl_region - container for dax_devices */
@@ -70,8 +77,19 @@ struct daxctl_dev {struct kmod_module *module;struct kmod_list *kmod_list;struct daxctl_region *region; +struct daxctl_memory *mem; +int target_node; +}; + +struct daxctl_memory { +struct daxctl_dev *dev; +void *mem_buf; +size_t buf_len; +char *node_path; +unsigned long block_size; }; +static inline int check_kmod (struct kmod_ctx *kmod_ctx) {return kmod_ctx ?0 : -ENXIO ;
daxctl/lib/libdaxctl.c +0 -0
@@ -200,6 +200,15 @@ DAXCTL_EXPORT void daxctl_region_get_uuid(struct
daxctl_region *region, uuid_t u
uuid_copy(uu, region->uuid);
}
+static void free_mem(struct daxctl_dev *dev)
+{
+ if (dev && dev->mem) {
+ free(dev->mem->node_path);
+ free(dev->mem);
+ dev->mem = NULL;
+ }
+}
+
static void free_dev(struct daxctl_dev *dev, struct list_head *head) {
if (head)
@@ -207,6 +216,7 @@ static void free_dev(struct daxctl_dev *dev, struct list_head *head)
kmod_module_unref_list(dev->kmod_list); free(dev->dev_buf);
free(dev->dev_path);
+ free_mem(dev);
free(dev);
}
@@ -380,6 +390,94 @@ static struct kmod_list *to_module_list(struct daxctl_ctx *ctx,
return list;
}
+static int dev_is_system_ram_capable(struct daxctl_dev *dev) +{
+ const char *devname = daxctl_dev_get_devname(dev);
+ struct daxctl_ctx *ctx = daxctl_dev_get_ctx(dev);
+ char *mod_path, *mod_base;
+ char path[200];
+ const int len = sizeof(path);
+
+ if (!device_model_is_dax_bus(dev))
+ return false;
+
+ if (!daxctl_dev_is_enabled(dev))
+ return false;
+
+ if (snprintf(path, len, "%s/driver/module", dev->dev_path) >=
len) {
+ err(ctx, "%s: buffer too small!\n", devname);
+ return false;
+ }
+
+ mod_path = realpath(path, NULL);
+ if (!mod_path)
+ return false;
+
+ mod_base = basename(mod_path);
+ if (strcmp(mod_base, dax_modules[DAXCTL_DEV_MODE_RAM]) == 0) {
+ free(mod_path);
+ return true;
+ }
+
+ free(mod_path);
+ return false;
+}
+
+/*
+ * This checks for the device to be in system-ram mode, so calling
+ * daxctl_dev_get_memory() on a devdax mode device will always return NULL.
+ */
+static struct daxctl_memory *daxctl_dev_alloc_mem(struct daxctl_dev *dev)
+{
+ const char *size_path =
"/sys/devices/system/memory/block_size_bytes";
+ const char *node_base = "/sys/devices/system/node/node";
+ const char *devname = daxctl_dev_get_devname(dev);
+ struct daxctl_ctx *ctx = daxctl_dev_get_ctx(dev);
+ struct daxctl_memory *mem;
+ char buf[SYSFS_ATTR_SIZE];
+ int node_num;
+
+ if (!dev_is_system_ram_capable(dev))
+ return NULL;
+
+ mem = calloc(1, sizeof(*mem));
+ if (!mem)
+ return NULL;
+
+ mem->dev = dev;
+
+ if (sysfs_read_attr(ctx, size_path, buf) == 0) {
+ mem->block_size = strtoul(buf, NULL, 16);
+ if (mem->block_size == 0 || mem->block_size ==ULONG_MAX) {
+ err(ctx, "%s: Unable to determine memblock size:
%s\n",
+ devname, strerror(errno));
+ mem->block_size = 0;
+ }
+ }
+
+ node_num = daxctl_dev_get_target_node(dev);
+ if (node_num >= 0) {
+ if (asprintf(&mem->node_path, "%s%d", node_base,
+ node_num) < 0) {
+ err(ctx, "%s: Unable to set node_path\n",
devname);
+ goto err_mem;
+ }
+ }
+
+ mem->mem_buf = calloc(1, strlen(node_base) + 256);
+ if (!mem->mem_buf)
+ goto err_node;
+ mem->buf_len = strlen(node_base) + 256;
+
+ return mem;
+
+err_node:
+ free(mem->node_path);
+err_mem:
+ free(mem);
+ return NULL;
+}
+
static void *add_dax_dev(void *parent, int id, const char *daxdev_base) {
const char *devname = devpath_to_devname(daxdev_base);
@@ -435,6 +533,12 @@ static void *add_dax_dev(void *parent, int id, const char *daxdev_base)
if (rc == 0)
dev->kmod_list = to_module_list(ctx, buf);
+ sprintf(path, "%s/target_node", daxdev_base);
+ if (sysfs_read_attr(ctx, path, buf) == 0)
+ dev->target_node = strtol(buf, NULL, 0);
+ else
+ dev->target_node = -1;
+
daxctl_dev_foreach(region, dev_dup)
if (dev_dup->id == dev->id) {
free_dev(dev, NULL);
@@ -862,6 +966,9 @@ DAXCTL_EXPORT int daxctl_dev_disable(struct daxctl_dev *dev)
if (!daxctl_dev_is_enabled(dev))
return 0;
+ /* If there is a memory object, first free that */
+ free_mem(dev);
+
daxctl_unbind(ctx, dev->dev_path);
if (daxctl_dev_is_enabled(dev)) {
@@ -944,3 +1051,280 @@ DAXCTL_EXPORT unsigned long long daxctl_dev_get_size(struct daxctl_dev *dev)
{
return dev->size;
}
+
+DAXCTL_EXPORT int daxctl_dev_get_target_node(struct daxctl_dev *dev) +{
+ return dev->target_node;
+}
+
+DAXCTL_EXPORT struct daxctl_memory *daxctl_dev_get_memory(struct daxctl_dev *dev)
+{
+ if (dev->mem)
+ return dev->mem;
+ else
+ return daxctl_dev_alloc_mem(dev);
+}
+
+DAXCTL_EXPORT struct daxctl_dev *daxctl_memory_get_dev(struct daxctl_memory *mem)
+{
+ return mem->dev;
+}
+
+DAXCTL_EXPORT const char *daxctl_memory_get_node_path(struct daxctl_memory *mem)
+{
+ return mem->node_path;
+}
+
+DAXCTL_EXPORT unsigned long daxctl_memory_get_block_size(struct daxctl_memory *mem)
+{
+ return mem->block_size;
+}
+
+static int online_one_memblock(struct daxctl_dev *dev, char *path) +{
+ const char *devname = daxctl_dev_get_devname(dev);
+ struct daxctl_ctx *ctx = daxctl_dev_get_ctx(dev);
+ const char *mode = "online_movable";
+ char buf[SYSFS_ATTR_SIZE];
+ int rc;
+
+ rc = sysfs_read_attr(ctx, path, buf);
+ if (rc) {
+ err(ctx, "%s: Failed to read %s: %s\n",
+ devname, path, strerror(-rc));
+ return rc;
+ }
+
+ /*
+ * if already online, possibly due to kernel config or a udev
rule,
+ * there is nothing to do and we can skip over the memblock
+ */
+ if (strncmp(buf, "online", 6) == 0)
+ return 1;
+
+ rc = sysfs_write_attr_quiet(ctx, path, mode);
+ if (rc) {
+ /*
+ * While we performed an already-online check above,
there
+ * is still a TOCTOU hole where someone (such as a udev
rule)
+ * may have raced to online the memory. In such a case,
+ * the sysfs store will fail, however we can check for
this
+ * by simply reading the state again. If it changed to
the
+ * desired state, then we don't have to error out.
+ */
+ if (sysfs_read_attr(ctx, path, buf) == 0) {
+ if (strncmp(buf, "online", 6) == 0)
+ return 1;
+ }
+ err(ctx, "%s: Failed to online %s: %s\n",
+ devname, path, strerror(-rc));
+ }
+ return rc;
+}
+
+static int offline_one_memblock(struct daxctl_dev *dev, char *path) +{
+ const char *devname = daxctl_dev_get_devname(dev);
+ struct daxctl_ctx *ctx = daxctl_dev_get_ctx(dev);
+ const char *mode = "offline";
+ char buf[SYSFS_ATTR_SIZE];
+ int rc;
+
+ rc = sysfs_read_attr(ctx, path, buf);
+ if (rc) {
+ err(ctx, "%s: Failed to read %s: %s\n",
+ devname, path, strerror(-rc));
+ return rc;
+ }
+
+ /* if already offline, there is nothing to do */
+ if (strncmp(buf, "offline", 7) == 0)
+ return 1;
+
+ rc = sysfs_write_attr_quiet(ctx, path, mode);
+ if (rc) {
+ /* Close the TOCTOU hole like in online_one_memblock()
above */
+ if (sysfs_read_attr(ctx, path, buf) == 0) {
+ if (strncmp(buf, "offline", 7) == 0)
+ return 1;
+ }
+ err(ctx, "%s: Failed to offline %s: %s\n",
+ devname, path, strerror(-rc));
+ }
+ return rc;
+}
+
+static int memblock_is_online(struct daxctl_dev *dev, char *path) +{
+ const char *devname = daxctl_dev_get_devname(dev);
+ struct daxctl_ctx *ctx = daxctl_dev_get_ctx(dev);
+ char buf[SYSFS_ATTR_SIZE];
+ int rc;
+
+ rc = sysfs_read_attr(ctx, path, buf);
+ if (rc) {
+ err(ctx, "%s: Failed to read %s: %s\n",
+ devname, path, strerror(-rc));
+ return rc;
+ }
+
+ if (strncmp(buf, "online", 6) == 0)
+ return 1;
+
+ /* offline */
+ return 0;
+}
+
+static bool memblock_in_dev(struct daxctl_memory *mem, const char *memblock)
+{
+ const char *mem_base = "/sys/devices/system/memory/";
+ struct daxctl_dev *dev = daxctl_memory_get_dev(mem);
+ unsigned long long memblock_res, dev_start, dev_end;
+ const char *devname = daxctl_dev_get_devname(dev);
+ struct daxctl_ctx *ctx = daxctl_dev_get_ctx(dev);
+ unsigned long memblock_size;
+ int path_len = mem->buf_len;
+ char buf[SYSFS_ATTR_SIZE];
+ unsigned long phys_index;
+ char *path = mem->mem_buf;
+
+ if (snprintf(path, path_len, "%s/%s/phys_index",
+ mem_base, memblock) < 0)
+ return false;
+
+ if (sysfs_read_attr(ctx, path, buf) == 0) {
+ phys_index = strtoul(buf, NULL, 16);
+ if (phys_index == 0 || phys_index == ULONG_MAX) {
+ err(ctx, "%s: %s: Unable to determinephys_index: %s\n",
+ devname, memblock, strerror(errno));
+ return false;
+ }
+ } else {
+ err(ctx, "%s: %s: Unable to determine phys_index: %s\n",
+ devname, memblock, strerror(errno));
+ return false;
+ }
+
+ dev_start = daxctl_dev_get_resource(dev);
+ if (!dev_start) {
+ err(ctx, "%s: Unable to determine resource\n", devname);
+ return false;
+ }
+ dev_end = dev_start + daxctl_dev_get_size(dev);
+
+ memblock_size = daxctl_memory_get_block_size(mem);
+ if (!memblock_size) {
+ err(ctx, "%s: Unable to determine memory block size\n",
+ devname);
+ return false;
+ }
+ memblock_res = phys_index * memblock_size;
+
+ if (memblock_res >= dev_start && memblock_res <= dev_end)
+ return true;
+
+ return false;
+}
+
+static int op_for_one_memblock(struct daxctl_memory *mem, char *path,
+ enum memory_op op)
+{
+ struct daxctl_dev *dev = daxctl_memory_get_dev(mem);
+ const char *devname = daxctl_dev_get_devname(dev);
+ struct daxctl_ctx *ctx = daxctl_dev_get_ctx(dev);
+ int rc;
+
+ switch (op) {
+ case MEM_SET_ONLINE:
+ return online_one_memblock(dev, path);
+ case MEM_SET_OFFLINE:
+ return offline_one_memblock(dev, path);
+ case MEM_IS_ONLINE:
+ rc = memblock_is_online(dev, path);
+ if (rc < 0)
+ return rc;
+ /*
+ * Retain the 'normal' semantics for if
(memblock_is_online()),
+ * but since count needs rc == 0, we'll just flip rc for
this op
+ */
+ return !rc;
+ case MEM_COUNT:
+ return 0;
+ }
+
+ err(ctx, "%s: BUG: unknown op: %d\n", devname, op);
+ return -EINVAL;
+}
+
+static int daxctl_memory_op(struct daxctl_memory *mem, enum memory_op op)
+{
+ struct daxctl_dev *dev = daxctl_memory_get_dev(mem);
+ const char *devname = daxctl_dev_get_devname(dev);
+ struct daxctl_ctx *ctx = daxctl_dev_get_ctx(dev);
+ const char *node_path;
+ int rc, count = 0;
+ struct dirent *de;
+ DIR *node_dir;
+
+ node_path = daxctl_memory_get_node_path(mem);
+ if (!node_path) {
+ err(ctx, "%s: Failed to get node_path\n", devname);
+ return -ENXIO;
+ }
+
+ node_dir = opendir(node_path);
+ if (!node_dir)
+ return -errno;
+
+ errno = 0;
+ while ((de = readdir(node_dir)) != NULL) {
+ char *path = mem->mem_buf;
+ int len = mem->buf_len;
+
+ if (strncmp(de->d_name, "memory", 6) == 0) {
+ if (!memblock_in_dev(mem, de->d_name))
+ continue;
+ rc = snprintf(path, len, "%s/%s/state",
+ node_path, de->d_name);
+ if (rc < 0) {
+ rc = -ENOMEM;
+ goto out_dir;
+ }
+ rc = op_for_one_memblock(mem, path, op);
+ if (rc < 0)
+ goto out_dir;
+ if (rc == 0)
+ count++;
+ }
+ errno = 0;
+ }
+ if (errno) {
+ rc = -errno;
+ goto out_dir;
+ }
+ rc = count;
+
+out_dir:
+ closedir(node_dir);
+ return rc;
+}
+
+DAXCTL_EXPORT int daxctl_memory_online(struct daxctl_memory *mem) +{
+ return daxctl_memory_op(mem, MEM_SET_ONLINE);
+}
+
+DAXCTL_EXPORT int daxctl_memory_offline(struct daxctl_memory *mem) +{
+ return daxctl_memory_op(mem, MEM_SET_OFFLINE);
+}
+
+DAXCTL_EXPORT int daxctl_memory_is_online(struct daxctl_memory *mem) +{
+ return daxctl_memory_op(mem, MEM_IS_ONLINE);
+}
+
+DAXCTL_EXPORT int daxctl_memory_num_sections(struct daxctl_memory *mem) +{
+ return daxctl_memory_op(mem, MEM_COUNT);
+}
daxctl/lib/libdaxctl.sym +9 -0
@@ -59,4 +59,13 @@ global: daxctl_dev_enable_devdax; daxctl_dev_enable_ram; daxctl_dev_get_resource; + daxctl_dev_get_target_node; + daxctl_dev_get_memory; + daxctl_memory_get_dev; + daxctl_memory_get_node_path; + daxctl_memory_get_block_size; + daxctl_memory_online; + daxctl_memory_offline; + daxctl_memory_is_online; + daxctl_memory_num_sections; } LIBDAXCTL_5;
daxctl/libdaxctl.h +11 -0
@@ -73,6 +73,17 @@ int daxctl_dev_is_enabled(struct daxctl_dev *dev);int daxctl_dev_disable (struct daxctl_dev *dev);int daxctl_dev_enable_devdax (struct daxctl_dev *dev);int daxctl_dev_enable_ram (struct daxctl_dev *dev); +int daxctl_dev_get_target_node (struct daxctl_dev *dev); + +struct daxctl_memory ; +struct daxctl_memory *daxctl_dev_get_memory (struct daxctl_dev *dev); +struct daxctl_dev *daxctl_memory_get_dev (struct daxctl_memory *mem); +const char *daxctl_memory_get_node_path (struct daxctl_memory *mem); +unsigned long daxctl_memory_get_block_size (struct daxctl_memory *mem); +int daxctl_memory_online (struct daxctl_memory *mem); +int daxctl_memory_offline (struct daxctl_memory *mem); +int daxctl_memory_is_online (struct daxctl_memory *mem); +int daxctl_memory_num_sections (struct daxctl_memory *mem);#define daxctl_dev_foreach (region, dev) \ for (dev = daxctl_dev_get_first(region); \
Signature · 7 lines
2.20.1 _______________________________________________ Linux-nvdimm mailing list [email protected] https://lists.01.org/mailman/listinfo/linux-nvdimm
Vishal Verma
maintainerPATCH v9 5/13 daxctl/list: add target_node for device listings
In reply to Vishal VermaThe kernel provides a 'target_node' attribute for dax devices. When converting a dax device to the system-ram mode, the memory is hotplugged into this numa node. It would be helpful to print this in device listings so that it is easy for applications to detect the node to which the new memory belongs.
--- util/json.c | 8 ++++++++ 1 file changed, 8 insertions(+)
util/json.c +8 -0
@@ -271,6 +271,7 @@ struct json_object *util_daxctl_dev_to_json(struct daxctl_dev *dev, {const char *devname =daxctl_dev_get_devname (dev);struct json_object *jdev, *jobj; +int node; jdev =json_object_new_object ();if (!devname || !jdev)
@@ -284,6 +285,13 @@ struct json_object *util_daxctl_dev_to_json(struct daxctl_dev *dev,if (jobj)json_object_object_add (jdev,"size" , jobj); + node =daxctl_dev_get_target_node (dev); +if (node >=0 ) { + jobj =json_object_new_int (node); +if (jobj) +json_object_object_add (jdev,"target_node" , jobj); + } +return jdev; }
Signature · 6 lines
2.20.1 _______________________________________________ Linux-nvdimm mailing list [email protected] https://lists.01.org/mailman/listinfo/linux-nvdimm
Vishal Verma
maintainerPATCH v9 6/13 daxctl/list: display the mode for a dax device
In reply to Vishal VermaIn preparation for a reconfigure-device command, allow JSON listings to display the 'mode' of a dax device. This will allow the reconfigure-device command (and via daxctl_dev_to_json(), also daxctl-list) to print the mode in device listings via a 'daxctl-list' command or immediately after a mode change.
Add a 'state' attribute to the json listings for devices, since a device could end up in a state where it is not bound to any driver, and hence, 'disabled'. The state attribute is only displayed for disabled devices.
--- daxctl/lib/libdaxctl.c | 2 ++ util/json.c | 14 ++++++++++++++ 2 files changed, 16 insertions(+)
daxctl/lib/libdaxctl.c +2 -0
@@ -12,6 +12,8 @@*/ #include <stdio.h> #include <errno.h> +#include <limits.h> +#include <libgen.h> #include <stdlib.h> #include <dirent.h> #include <unistd.h>
util/json.c +14 -0
@@ -269,6 +269,7 @@ struct json_object *util_dimm_to_json(struct ndctl_dimm *dimm,struct json_object *util_daxctl_dev_to_json (struct daxctl_dev *dev,unsigned long flags) { +struct daxctl_memory *mem =daxctl_dev_get_memory (dev);const char *devname =daxctl_dev_get_devname (dev);struct json_object *jdev, *jobj;int node;
@@ -292,6 +293,19 @@ struct json_object *util_daxctl_dev_to_json(struct daxctl_dev *dev,json_object_object_add (jdev,"target_node" , jobj); } +if (mem) + jobj =json_object_new_string ("system-ram" ); +else + jobj =json_object_new_string ("devdax" ); +if (jobj) +json_object_object_add (jdev,"mode" , jobj); + +if (!daxctl_dev_is_enabled (dev)) { + jobj =json_object_new_string ("disabled" ); +if (jobj) +json_object_object_add (jdev,"state" , jobj); + } +return jdev; }
Signature · 6 lines
2.20.1 _______________________________________________ Linux-nvdimm mailing list [email protected] https://lists.01.org/mailman/listinfo/linux-nvdimm
Vishal Verma
maintainerPATCH v9 7/13 daxctl: add a new reconfigure-device command
In reply to Vishal VermaAdd a new command 'daxctl-reconfigure-device'. This is used to switch the mode of a dax device between regular 'device_dax' and 'system-memory'. The command also uses the memory hotplug sysfs interfaces to online the newly available memory when converting to 'system-ram', and to attempt to offline the memory when converting back to a DAX device.
--- daxctl/Makefile.am | 2 + daxctl/builtin.h | 1 + daxctl/daxctl.c | 1 + daxctl/device.c | 455 +++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 459 insertions(+) create mode 100644 daxctl/device.c
daxctl/Makefile.am +2 -0
@@ -15,10 +15,12 @@ daxctl_SOURCES =\ daxctl.c \ list.c \ migrate.c \ + device.c \ ../util/json.c daxctl_LDADD =\ lib/libdaxctl.la \ ../libutil.a \ $(UUID_LIBS) \ + $(KMOD_LIBS) \ $(JSON_LIBS)
daxctl/builtin.h +1 -0
@@ -6,4 +6,5 @@struct daxctl_ctx ;int cmd_list (int argc,const char **argv,struct daxctl_ctx *ctx);int cmd_migrate (int argc,const char **argv,struct daxctl_ctx *ctx); +int cmd_reconfig_device (int argc,const char **argv,struct daxctl_ctx *ctx);#endif /* _DAXCTL_BUILTIN_H_ */
daxctl/daxctl.c +1 -0
@@ -71,6 +71,7 @@ static struct cmd_struct commands[] = { { "list", .d_fn = cmd_list }, { "help", .d_fn = cmd_help }, { "migrate-device-model", .d_fn = cmd_migrate }, + { "reconfigure-device", .d_fn = cmd_reconfig_device }, };int main (int argc,const char **argv)
daxctl/device.c new file +455 -0
455 lines not on this page · raw message
Signature · 6 lines
2.20.1 _______________________________________________ Linux-nvdimm mailing list [email protected] https://lists.01.org/mailman/listinfo/linux-nvdimm
Vishal Verma
maintainerPATCH v9 8/13 Documentation/daxctl: add a man page for daxctl-reconfigure-device
In reply to Vishal VermaAdd a man page describing the new daxctl-reconfigure-device command.
--- Documentation/daxctl/Makefile.am | 3 +- .../daxctl/daxctl-reconfigure-device.txt | 157 ++++++++++++++++++ 2 files changed, 159 insertions(+), 1 deletion(-) create mode 100644 Documentation/daxctl/daxctl-reconfigure-device.txt
Documentation/daxctl/Makefile.am +2 -1
@@ -28,7 +28,8 @@ endif man1_MANS = \ daxctl.1 \ daxctl-list.1 \ - daxctl-migrate-device-model.1 + daxctl-migrate-device-model.1 \ + daxctl-reconfigure-device.1 CLEANFILES = $(man1_MANS)
Documentation/daxctl/daxctl-reconfigure-device.txt new file +157 -0
157 lines not on this page · raw message
Signature · 6 lines
2.20.1 _______________________________________________ Linux-nvdimm mailing list [email protected] https://lists.01.org/mailman/listinfo/linux-nvdimm
Vishal Verma
maintainerPATCH v9 9/13 daxctl: add commands to online and offline memory
In reply to Vishal VermaAdd two new commands:
daxctl-online-memory
daxctl-offline-memory
to manage the state of hot-plugged memory from the system-ram mode for dax devices. This provides a way for the user to online/offline the memory as a separate step from the reconfiguration. Without this, a user that reconfigures a device into the system-ram mode with the --no-online option, would have no way to later online the memory, and would have to resort to shell scripting to online them manually via sysfs.
--- daxctl/builtin.h | 2 ++ daxctl/daxctl.c | 2 ++ daxctl/device.c | 88 ++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 92 insertions(+)
daxctl/builtin.h +2 -0
@@ -7,4 +7,6 @@ struct daxctl_ctx;int cmd_list (int argc,const char **argv,struct daxctl_ctx *ctx);int cmd_migrate (int argc,const char **argv,struct daxctl_ctx *ctx);int cmd_reconfig_device (int argc,const char **argv,struct daxctl_ctx *ctx); +int cmd_online_memory (int argc,const char **argv,struct daxctl_ctx *ctx); +int cmd_offline_memory (int argc,const char **argv,struct daxctl_ctx *ctx);#endif /* _DAXCTL_BUILTIN_H_ */
daxctl/daxctl.c +2 -0
@@ -72,6 +72,8 @@ static struct cmd_struct commands[] = { { "help", .d_fn = cmd_help }, { "migrate-device-model", .d_fn = cmd_migrate }, { "reconfigure-device", .d_fn = cmd_reconfig_device }, + { "online-memory", .d_fn = cmd_online_memory }, + { "offline-memory", .d_fn = cmd_offline_memory }, };int main (int argc,const char **argv)
daxctl/device.c +88 -0
127 lines not on this page · raw message
Signature · 6 lines
2.20.1 _______________________________________________ Linux-nvdimm mailing list [email protected] https://lists.01.org/mailman/listinfo/linux-nvdimm
Vishal Verma
maintainerPATCH v9 10/13 Documentation: Add man pages for daxctl-{on, off}line-memory
In reply to Vishal VermaAdd man pages for the two new commands: daxctl-online-memory, and daxctl-offline-memory.
--- Documentation/daxctl/Makefile.am | 4 +- .../daxctl/daxctl-offline-memory.txt | 72 +++++++++++++++++ Documentation/daxctl/daxctl-online-memory.txt | 80 +++++++++++++++++++ 3 files changed, 155 insertions(+), 1 deletion(-) create mode 100644 Documentation/daxctl/daxctl-offline-memory.txt create mode 100644 Documentation/daxctl/daxctl-online-memory.txt
Documentation/daxctl/Makefile.am +3 -1
@@ -29,7 +29,9 @@ man1_MANS = \ daxctl.1 \ daxctl-list.1 \ daxctl-migrate-device-model.1 \ - daxctl-reconfigure-device.1 + daxctl-reconfigure-device.1 \ + daxctl-online-memory.1 \ + daxctl-offline-memory.1 CLEANFILES = $(man1_MANS)
Documentation/daxctl/daxctl-offline-memory.txt new file +72 -0
72 lines not on this page · raw message
Documentation/daxctl/daxctl-online-memory.txt new file +80 -0
80 lines not on this page · raw message
Signature · 6 lines
2.20.1 _______________________________________________ Linux-nvdimm mailing list [email protected] https://lists.01.org/mailman/listinfo/linux-nvdimm
Vishal Verma
maintainerPATCH v9 11/13 contrib/ndctl: fix region-id completions for daxctl
In reply to Vishal VermaThe completion helpers for daxctl assumed the region arguments for specifying daxctl regions were the same as ndctl regions, i.e. "regionX". This is not true - daxctl region arguments are a simple numeric 'id'.
Add a new helper __daxctl_get_regions() to complete daxctl region IDs properly.
While at it, fix a useless use of 'echo' in __daxctl_get_devs() and quoting in __daxctl_comp_options()
--- contrib/ndctl | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-)
contrib/ndctl +10 -4
@@ -531,8 +531,14 @@ _ndctl() __daxctl_get_devs() { - local opts="--devices $*" - echo "$(daxctl list $opts | grep -E "^\s*\"chardev\":" | cut -d\" -f4)" + local opts=("--devices" "$*") + daxctl list "${opts[@]}" | grep -E "^\s*\"chardev\":" | cut -d'"' -f4 +} + +__daxctl_get_regions() +{ + local opts=("--regions" "$*") + daxctl list "${opts[@]}" | grep -E "^\s*\"id\":" | grep -Eo "[0-9]+" } __daxctlcomp()
@@ -561,10 +567,10 @@ __daxctl_comp_options() local cur_arg=${cur##*=} case $cur_subopt in --region) - opts=$(__ndctl_get_regions -i) + opts="$(__daxctl_get_regions -i)" ;; --dev) - opts=$(__daxctl_get_devs -i) + opts="$(__daxctl_get_devs -i)" ;; *) return
Signature · 6 lines
2.20.1 _______________________________________________ Linux-nvdimm mailing list [email protected] https://lists.01.org/mailman/listinfo/linux-nvdimm
Vishal Verma
maintainerPATCH v9 12/13 contrib/ndctl: add bash-completion for the new daxctl commands
In reply to Vishal VermaAdd bash completion helpers for the new daxctl-reconfigure-device, daxctl-online-memory, and daxctl-offline-memory commands.
--- contrib/ndctl | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-)
contrib/ndctl +21 -3
@@ -547,7 +547,7 @@ __daxctlcomp() COMPREPLY=( $( compgen -W "$1" -- "$2" ) ) for cword in "${COMPREPLY[@]}"; do - if [[ "$cword" == @(--region|--dev) ]]; then + if [[ "$cword" == @(--region|--dev|--mode) ]]; then COMPREPLY[$i]="${cword}=" else COMPREPLY[$i]="${cword} "
@@ -572,6 +572,9 @@ __daxctl_comp_options() --dev) opts="$(__daxctl_get_devs -i)" ;; + --mode) + opts="system-ram devdax" + ;; *) return ;;
@@ -582,8 +585,23 @@ __daxctl_comp_options() __daxctl_comp_non_option_args() { - # there aren't any commands that accept non option arguments yet - return + local subcmd=$1 + local cur=$2 + local opts + + case $subcmd in + reconfigure-device) + ;& + online-memory) + ;& + offline-memory) + opts="$(__daxctl_get_devs -i) all" + ;; + *) + return + ;; + esac + __daxctlcomp "$opts" "$cur" } __daxctl_main()
Signature · 6 lines
2.20.1 _______________________________________________ Linux-nvdimm mailing list [email protected] https://lists.01.org/mailman/listinfo/linux-nvdimm
Vishal Verma
maintainerPATCH v9 13/13 test: Add a unit test for daxctl-reconfigure-device and friends
In reply to Vishal VermaAdd a new unit test to test dax device reconfiguration and memory operations. This teaches test/common about daxctl, and adds an ACPI.NFIT bus variable. Since we have to operate on the ACPI.NFIT bus, the test is marked as destructive.
--- test/Makefile.am | 3 +- test/common | 19 ++++++++-- test/daxctl-devices.sh | 81 ++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 99 insertions(+), 4 deletions(-) create mode 100755 test/daxctl-devices.sh
test/Makefile.am +2 -1
@@ -49,7 +49,8 @@ TESTS +=\ dax.sh \ device-dax \ device-dax-fio.sh \ - mmap.sh + mmap.sh \ + daxctl-devices.sh if ENABLE_KEYUTILS TESTS += security.sh
test/common +16 -3
@@ -15,12 +15,25 @@ else exit 1 fi -# NFIT_TEST_BUS[01] +# DAXCTL # -NFIT_TEST_BUS0=nfit_test.0 -NFIT_TEST_BUS1=nfit_test.1 +if [ -f "../daxctl/daxctl" ] && [ -x "../daxctl/daxctl" ]; then + export DAXCTL=../daxctl/daxctl +elif [ -f "./daxctl/daxctl" ] && [ -x "./daxctl/daxctl" ]; then + export DAXCTL=./daxctl/daxctl +else + echo "Couldn't find an daxctl binary" + exit 1 +fi +# NFIT_TEST_BUS[01] +# +NFIT_TEST_BUS0="nfit_test.0" +NFIT_TEST_BUS1="nfit_test.1" +ACPI_BUS="ACPI.NFIT" +E820_BUS="e820" + # Functions # err
test/daxctl-devices.sh new file +81 -0
81 lines not on this page · raw message
Signature · 6 lines
2.20.1 _______________________________________________ Linux-nvdimm mailing list [email protected] https://lists.01.org/mailman/listinfo/linux-nvdimm