fetch the alignment of device dax
v2 cover fetch the alignment of device dax
This series adds libdaxctl support and fetchs the alignment of device dax through libdaxctl [1] APIs.
QEMU uses mmap(2) to maps vNVDIMM backends and aligns the mapping address to the page size (getpagesize(2)) by default. However, some types of backends may require an alignment different than the page size. The 'align' option is provided to memory-backend-file to allow users to specify the proper alignment.
For device dax (e.g., /dev/dax0.0), the 'align' option needs to match the alignment requirement of the device dax, which can be fetched through the libdaxctl APIs.
[1] Libdaxctl is a part of ndctl project.
The project's repository is: https://github.com/pmem/ndctl
Changelog:
v2: Per Paolo and Dan suggestions, fetch the alignment of device dax
through libdaxctl APIs.
v1: The initial version.
Fetch the alignment through "/sys/dev/char/%d:%d/device/align".
Jingqi Liu (3):
exec: fetch the alignment of Linux devdax pmem character device nodes docs/nvdimm: add description of alignment requirement of device dax
configure: add libdaxctl support
configure | 30 +++++++++++++++++++++++++++ docs/nvdimm.txt | 9 +++++++++ exec.c | 54 ++++++++++++++++++++++++++++++++++++++++++++++++- 3 files changed, 92 insertions(+), 1 deletion(-)
Signature · 1 line
2.17.1
PATCH v2 1/3 exec: fetch the alignment of Linux devdax pmem character device nodes
If the backend file is devdax pmem character device, the alignment specified by the option 'align=NUM' in the '-object memory-backend-file' needs to match the alignment requirement of the devdax pmem character device.
This patch uses the interfaces of libdaxctl to fetch the devdax pmem file 'align', so that we can compare it with the NUM of 'align=NUM'. The NUM needs to be larger than or equal to the devdax pmem file 'align'.
It also fixes the problem that mmap() returns failure in qemu_ram_mmap() when the NUM of 'align=NUM' is less than the devdax pmem file 'align'.
--- exec.c | 54 +++++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 53 insertions(+), 1 deletion(-)
exec.c +53 -1
@@ -77,6 +77,10 @@#include "monitor/monitor.h" +#ifdef CONFIG_LIBDAXCTL +#include <daxctl/libdaxctl.h> +#endif +//#define DEBUG_SUBPAGE #if !defined(CONFIG_USER_ONLY )
@@ -1736,6 +1740,46 @@ static int64_t get_file_size(int fd)return size; } +static int64_t get_file_align (int fd) +{ +int64_t align =-1 ; +#if defined(__linux__) && defined(CONFIG_LIBDAXCTL ) +struct stat st; + +if (fstat (fd, &st) <0 ) { +return -errno; + } + +/* Special handling for devdax character devices */ +if (S_ISCHR (st.st_mode)) { + g_autofree char *path = NULL; + g_autofree char *rpath = NULL; +struct daxctl_ctx *ctx; +struct daxctl_region *region; +int rc =0 ; + + path =g_strdup_printf ("/sys/dev/char/%d:%d" , +major (st.st_rdev),minor (st.st_rdev)); + rpath =realpath (path,NULL ); + + rc =daxctl_new (&ctx); +if (rc) { +return -1 ; + } + + daxctl_region_foreach(ctx, region) { +if (strstr (rpath,daxctl_region_get_path (region))) { + align =daxctl_region_get_align (region); +break ; + } + } +daxctl_unref (ctx); + } +#endif /* defined(__linux__) && defined(CONFIG_LIBDAXCTL) */ + +return align; +} + static int file_ram_open(const char *path, const char *region_name, bool *created,
@@ -2275,7 +2319,7 @@ RAMBlock *qemu_ram_alloc_from_fd(ram_addr_t size, MemoryRegion *mr, {RAMBlock *new_block;Error *local_err =NULL ; -int64_t file_size; +int64_t file_size, file_align;/* Just support these ram flags by now. */ assert ((ram_flags & ~(RAM_SHARED |RAM_PMEM )) ==0 );
@@ -2311,6 +2355,14 @@ RAMBlock *qemu_ram_alloc_from_fd(ram_addr_t size, MemoryRegion *mr,return NULL ; } + file_align =get_file_align (fd); +if (file_align >0 && mr && file_align > mr->align) { +error_setg (errp,"backing store align 0x%" PRIx64 +" is larger than 'align' option 0x" RAM_ADDR_FMT , + file_align, mr->align); +return NULL ; + } + new_block =g_malloc0 (sizeof (*new_block)); new_block->mr = mr; new_block->used_length = size;
Signature · 1 line
2.17.1
Re: [PATCH v2 1/3] exec: fetch the alignment of Linux devdax pmem character device nodes
Quoted 8 lines · Jingqi Liu
If the backend file is devdax pmem character device, the alignment specified by the option 'align=NUM' in the '-object memory-backend-file' needs to match the alignment requirement of the devdax pmem character device.
This patch uses the interfaces of libdaxctl to fetch the devdax pmem file 'align', so that we can compare it with the NUM of 'align=NUM'. The NUM needs to be larger than or equal to the devdax pmem file 'align'.
Quoted 5 lines
It also fixes the problem that
mmap()returns failure inqemu_ram_mmap()when the NUM of 'align=NUM' is less than the devdax pmem file 'align'.Suggested-by Dan Williams <[email protected]>Signed-off-by Jingqi Liu <[email protected]>
I also no longer see the previous problem we talked in v1 wrt to ABI differences between dax_pmem_compat (your case) vs dax_pmem|dax_hmem (my case). The @align validation also occurs as expected.
Quoted 91 lines
--- exec.c | 54 +++++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 53 insertions(+), 1 deletion(-)exec.c +53
-1@@ -77,6 +77,10 @@#include "monitor/monitor.h" +#ifdef CONFIG_LIBDAXCTL +#include <daxctl/libdaxctl.h> +#endif +//#define DEBUG_SUBPAGE #if !defined(CONFIG_USER_ONLY )@@ -1736,6 +1740,46 @@ static int64_t get_file_size(int fd)return size; } +static int64_t get_file_align (int fd) +{ +int64_t align =-1 ; +#if defined(__linux__) && defined(CONFIG_LIBDAXCTL ) +struct stat st; + +if (fstat (fd, &st) <0 ) { +return -errno; + } + +/* Special handling for devdax character devices */ +if (S_ISCHR (st.st_mode)) { + g_autofree char *path = NULL; + g_autofree char *rpath = NULL; +struct daxctl_ctx *ctx; +struct daxctl_region *region; +int rc =0 ; + + path =g_strdup_printf ("/sys/dev/char/%d:%d" , +major (st.st_rdev),minor (st.st_rdev)); + rpath =realpath (path,NULL ); + + rc =daxctl_new (&ctx); +if (rc) { +return -1 ; + } + + daxctl_region_foreach(ctx, region) { +if (strstr (rpath,daxctl_region_get_path (region))) { + align =daxctl_region_get_align (region); +break ; + } + } +daxctl_unref (ctx); + } +#endif /* defined(__linux__) && defined(CONFIG_LIBDAXCTL) */ + +return align; +} + static int file_ram_open(const char *path, const char *region_name, bool *created,@@ -2275,7 +2319,7 @@ RAMBlock *qemu_ram_alloc_from_fd(ram_addr_t size, MemoryRegion *mr, {RAMBlock *new_block;Error *local_err =NULL ; -int64_t file_size; +int64_t file_size, file_align;/* Just support these ram flags by now. */ assert ((ram_flags & ~(RAM_SHARED |RAM_PMEM )) ==0 );@@ -2311,6 +2355,14 @@ RAMBlock *qemu_ram_alloc_from_fd(ram_addr_t size, MemoryRegion *mr,return NULL ; } + file_align =get_file_align (fd); +if (file_align >0 && mr && file_align > mr->align) { +error_setg (errp,"backing store align 0x%" PRIx64 +" is larger than 'align' option 0x" RAM_ADDR_FMT , + file_align, mr->align); +return NULL ; + } + new_block =g_malloc0 (sizeof (*new_block)); new_block->mr = mr; new_block->used_length = size;
PATCH v2 2/3 docs/nvdimm: add description of alignment requirement of device dax
In reply to Jingqi LiuFor device dax (e.g., /dev/dax0.0), the NUM of 'align=NUM' option needs to match the alignment requirement of the device dax. It must be larger than or equal to the 'align' of device dax.
--- docs/nvdimm.txt | 9 +++++++++ 1 file changed, 9 insertions(+)
docs/nvdimm.txt +9 -0
@@ -132,6 +132,15 @@ address to the page size (getpagesize(2)) by default. However, some types of backends may require an alignment different than the page size. In that case, QEMU v2.12.0 and later provide 'align' option to memory-backend-file to allow users to specify the proper alignment. +For device dax (e.g., /dev/dax0.0), this alignment needs to match the +alignment requirement of the device dax. The NUM of 'align=NUM' option +must be larger than or equal to the 'align' of device dax. +We can use the following command to show the 'align' of device dax. + + ndctl list -X + +In order to get the proper 'align' of device dax, you need to install +the library 'libdaxctl'. For example, device dax require the 2 MB alignment, so we can use following QEMU command line options to use it (/dev/dax0.0) as the
Signature · 1 line
2.17.1
Re: [PATCH v2 2/3] docs/nvdimm: add description of alignment requirement of device dax
Quoted 20 lines · Jingqi Liu
For device dax (e.g.,
/dev/dax0.0), the NUM of 'align=NUM' option needs to match the alignment requirement of the device dax. It must be larger than or equal to the 'align' of device dax.Signed-off-by Jingqi Liu <[email protected]>--- docs/nvdimm.txt | 9 +++++++++ 1 file changed, 9 insertions(+)docs/nvdimm.txt +2
-0@@ -132,6 +132,15 @@ address to the page size (getpagesize(2)) by default. However, some types of backends may require an alignment different than the page size. In that case, QEMU v2.12.0 and later provide 'align' option to memory-backend-file to allow users to specify the proper alignment. +For device dax (e.g., /dev/dax0.0), this alignment needs to match the +alignment requirement of the device dax. The NUM of 'align=NUM' option
docs/nvdimm.txt address to the page size (getpagesize(2)) by default. However, some · line 146
Quoted 10 lines
docs/nvdimm.txt +7
-0+must be larger than or equal to the 'align' of device dax. +We can use the following command to show the 'align' of device dax. + + ndctl list -X + +In order to get the proper 'align' of device dax, you need to install +the library 'libdaxctl'. For example, device dax require the 2 MB alignment, so we can use following QEMU command line options to use it (/dev/dax0.0) as the
Given that this series introduces support for libdaxctl, perhaps suggesting in the docs how you can also query the @align with daxctl tool? e.g.
"We can one of the following commands to show the 'align' of device dax:
ndctl list -X
daxctl list -R"
The unlikely reason being that users may only install daxctl-{,libs,devel} and not ndctl-*. It also covers other users like ACPI HMAT/hmem which are not instrumented with ndctl.
With that:
Joao
Re: [PATCH v2 2/3] docs/nvdimm: add description of alignment requirement of device dax
Quoted 34 lines · Joao Martins
Quoted 20 lines · Jingqi Liu
For device dax (e.g.,
/dev/dax0.0), the NUM of 'align=NUM' option needs to match the alignment requirement of the device dax. It must be larger than or equal to the 'align' of device dax.Signed-off-by Jingqi Liu <[email protected]>--- docs/nvdimm.txt | 9 +++++++++ 1 file changed, 9 insertions(+)docs/nvdimm.txt +2
-0@@ -132,6 +132,15 @@ address to the page size (getpagesize(2)) by default. However, some types of backends may require an alignment different than the page size. In that case, QEMU v2.12.0 and later provide 'align' option to memory-backend-file to allow users to specify the proper alignment. +For device dax (e.g., /dev/dax0.0), this alignment needs to match the +alignment requirement of the device dax. The NUM of 'align=NUM' optiondocs/nvdimm.txt address to the page size (getpagesize(2)) by default. However, some · line 146
Quoted 10 lines
docs/nvdimm.txt +7
-0+must be larger than or equal to the 'align' of device dax. +We can use the following command to show the 'align' of device dax. + + ndctl list -X + +In order to get the proper 'align' of device dax, you need to install +the library 'libdaxctl'. For example, device dax require the 2 MB alignment, so we can use following QEMU command line options to use it (/dev/dax0.0) as theGiven that this series introduces support for libdaxctl, perhaps suggesting in the docs how you can also query the @align with daxctl tool? e.g.
Quoted 4 lines
"We can one of the following commands to show the 'align' of device dax:
ndctl list -X
daxctl list -R"
Thanks for your comments.
I'll add "daxctl list -R" in this doc.
Jingqi
Quoted 9 lines
The unlikely reason being that users may only install daxctl-{,libs,devel} and not ndctl-*. It also covers other users like ACPI HMAT/hmem which are not instrumented with ndctl.
With that:
Reviewed-by Joao Martins <[email protected]>Joao
Thanks.
PATCH v2 3/3 configure: add libdaxctl support
In reply to Jingqi LiuAdd a pair of configure options --{enable,disable}-libdaxctl to control whether QEMU is compiled with libdaxctl [1]. Libdaxctl is a utility library for managing the device dax subsystem.
QEMU uses mmap(2) to maps vNVDIMM backends and aligns the mapping address to the page size (getpagesize(2)) by default. However, some types of backends may require an alignment different than the page size. The 'align' option is provided to memory-backend-file to allow users to specify the proper alignment.
For device dax (e.g., /dev/dax0.0), the 'align' option needs to match the alignment requirement of the device dax, which can be fetched through the libdaxctl APIs.
[1] Libdaxctl is a part of ndctl project.
The project's repository is: https://github.com/pmem/ndctl
For more information about libdaxctl APIs, you can refer to the comments in source code of: pmem/ndctl/daxctl/lib/libdaxctl.c.
--- configure | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+)
configure +30 -0
@@ -509,6 +509,7 @@ libpmem="" default_devices="yes" plugins="no" fuzzing="no" +libdaxctl="" supported_cpu="no" supported_os="no"
@@ -1601,6 +1602,10 @@ for opt do ;; --gdb=*) gdb_bin="$optarg" ;; + --enable-libdaxctl) libdaxctl=yes + ;; + --disable-libdaxctl) libdaxctl=no + ;; *) echo "ERROR: unknown option $opt" echo "Try '$0 --help' for more information"
@@ -1894,6 +1899,7 @@ disabled with --disable-FEATURE, default is enabled if available: debug-mutex mutex debugging support libpmem libpmem support xkbcommon xkbcommon support + libdaxctl libdaxctl support NOTE: The object files are built at the place where configure is launched EOF
@@ -6190,6 +6196,25 @@ if test "$libpmem" != "no"; then fi fi +########################################## +# check for libdaxctl + +if test "$libdaxctl" != "no"; then + if $pkg_config --exists "libdaxctl"; then + libdaxctl="yes" + libdaxctl_libs=$($pkg_config --libs libdaxctl) + libdaxctl_cflags=$($pkg_config --cflags libdaxctl) + libs_softmmu="$libs_softmmu $libdaxctl_libs" + QEMU_CFLAGS="$QEMU_CFLAGS $libdaxctl_cflags" + else + if test "$libdaxctl" = "yes" ; then + feature_not_found "libdaxctl" "Install libdaxctl" + fi + libdaxctl="no" + fi +fi + + ########################################## # check for slirp
@@ -6767,6 +6792,7 @@ echo "parallels support $parallels" echo "sheepdog support $sheepdog" echo "capstone $capstone" echo "libpmem support $libpmem" +echo "libdaxctl support $libdaxctl" echo "libudev $libudev" echo "default devices $default_devices" echo "plugin support $plugins"
@@ -7590,6 +7616,10 @@ if test "$libpmem" = "yes" ; then echo "CONFIG_LIBPMEM=y" >> $config_host_mak fi +if test "$libdaxctl" = "yes" ; then + echo "CONFIG_LIBDAXCTL=y" >> $config_host_mak +fi + if test "$bochs" = "yes" ; then echo "CONFIG_BOCHS=y" >> $config_host_mak fi
Signature · 1 line
2.17.1
Re: [PATCH v2 3/3] configure: add libdaxctl support
Quoted 64 lines · Jingqi Liu
Add a pair of configure options --{enable,disable}-libdaxctl to control whether QEMU is compiled with libdaxctl [1]. Libdaxctl is a utility library for managing the device dax subsystem.
QEMU uses mmap(2) to maps vNVDIMM backends and aligns the mapping address to the page size (getpagesize(2)) by default. However, some types of backends may require an alignment different than the page size. The 'align' option is provided to memory-backend-file to allow users to specify the proper alignment.
For device dax (e.g.,
/dev/dax0.0), the 'align' option needs to match the alignment requirement of the device dax, which can be fetched through the libdaxctl APIs.[1] Libdaxctl is a part of ndctl project.
The project's repository is: https://github.com/pmem/ndctlFor more information about libdaxctl APIs, you can refer to the comments in source code of:
pmem/ndctl/daxctl/lib/libdaxctl.c.Signed-off-by Jingqi Liu <[email protected]>--- configure | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+)configure +10
-0@@ -509,6 +509,7 @@ libpmem="" default_devices="yes" plugins="no" fuzzing="no" +libdaxctl="" supported_cpu="no" supported_os="no"@@ -1601,6 +1602,10 @@ for opt do ;; --gdb=*) gdb_bin="$optarg" ;; + --enable-libdaxctl) libdaxctl=yes + ;; + --disable-libdaxctl) libdaxctl=no + ;; *) echo "ERROR: unknown option $opt" echo "Try '$0 --help' for more information"@@ -1894,6 +1899,7 @@ disabled with --disable-FEATURE, default is enabled if available: debug-mutex mutex debugging support libpmem libpmem support xkbcommon xkbcommon support + libdaxctl libdaxctl support NOTE: The object files are built at the place where configure is launched EOF@@ -6190,6 +6196,25 @@ if test "$libpmem" != "no"; then fi fi +########################################## +# check for libdaxctl + +if test "$libdaxctl" != "no"; then
configure if test "$libpmem" != "no"; then · line 6212
Quoted 10 lines
configure +10
-0+ if $pkg_config --exists "libdaxctl"; then + libdaxctl="yes" + libdaxctl_libs=$($pkg_config --libs libdaxctl) + libdaxctl_cflags=$($pkg_config --cflags libdaxctl) + libs_softmmu="$libs_softmmu $libdaxctl_libs" + QEMU_CFLAGS="$QEMU_CFLAGS $libdaxctl_cflags" + else + if test "$libdaxctl" = "yes" ; then + feature_not_found "libdaxctl" "Install libdaxctl" + fi
Region iteration APIs, align and path getter routines are only available since libdaxctl v56/v57 (the latest is v68).
Not sure how likely this happens in today's distros but if we care about systems with < v57 we should probably check that daxctl_region_foreach/daxctl_region_get_align/daxctl_region_get_path symbols exist? Or alternatively requiring v57 or up which serves as a bandage, but more long term ... any usage of newer daxctl APIs will require the former.
Quoted 28 lines
configure +10
-0+ libdaxctl="no" + fi +fi + + ########################################## # check for slirp@@ -6767,6 +6792,7 @@ echo "parallels support $parallels" echo "sheepdog support $sheepdog" echo "capstone $capstone" echo "libpmem support $libpmem" +echo "libdaxctl support $libdaxctl" echo "libudev $libudev" echo "default devices $default_devices" echo "plugin support $plugins"@@ -7590,6 +7616,10 @@ if test "$libpmem" = "yes" ; then echo "CONFIG_LIBPMEM=y" >> $config_host_mak fi +if test "$libdaxctl" = "yes" ; then + echo "CONFIG_LIBDAXCTL=y" >> $config_host_mak +fi + if test "$bochs" = "yes" ; then echo "CONFIG_BOCHS=y" >> $config_host_mak fi
Re: [PATCH v2 3/3] configure: add libdaxctl support
Quoted 78 lines · Joao Martins
Quoted 64 lines · Jingqi Liu
Add a pair of configure options --{enable,disable}-libdaxctl to control whether QEMU is compiled with libdaxctl [1]. Libdaxctl is a utility library for managing the device dax subsystem.
QEMU uses mmap(2) to maps vNVDIMM backends and aligns the mapping address to the page size (getpagesize(2)) by default. However, some types of backends may require an alignment different than the page size. The 'align' option is provided to memory-backend-file to allow users to specify the proper alignment.
For device dax (e.g.,
/dev/dax0.0), the 'align' option needs to match the alignment requirement of the device dax, which can be fetched through the libdaxctl APIs.[1] Libdaxctl is a part of ndctl project.
The project's repository is: https://github.com/pmem/ndctlFor more information about libdaxctl APIs, you can refer to the comments in source code of:
pmem/ndctl/daxctl/lib/libdaxctl.c.Signed-off-by Jingqi Liu <[email protected]>--- configure | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+)configure +10
-0@@ -509,6 +509,7 @@ libpmem="" default_devices="yes" plugins="no" fuzzing="no" +libdaxctl="" supported_cpu="no" supported_os="no"@@ -1601,6 +1602,10 @@ for opt do ;; --gdb=*) gdb_bin="$optarg" ;; + --enable-libdaxctl) libdaxctl=yes + ;; + --disable-libdaxctl) libdaxctl=no + ;; *) echo "ERROR: unknown option $opt" echo "Try '$0 --help' for more information"@@ -1894,6 +1899,7 @@ disabled with --disable-FEATURE, default is enabled if available: debug-mutex mutex debugging support libpmem libpmem support xkbcommon xkbcommon support + libdaxctl libdaxctl support NOTE: The object files are built at the place where configure is launched EOF@@ -6190,6 +6196,25 @@ if test "$libpmem" != "no"; then fi fi +########################################## +# check for libdaxctl + +if test "$libdaxctl" != "no"; thenconfigure if test "$libpmem" != "no"; then · line 6212
Quoted 10 lines
configure +10
-0+ if $pkg_config --exists "libdaxctl"; then + libdaxctl="yes" + libdaxctl_libs=$($pkg_config --libs libdaxctl) + libdaxctl_cflags=$($pkg_config --cflags libdaxctl) + libs_softmmu="$libs_softmmu $libdaxctl_libs" + QEMU_CFLAGS="$QEMU_CFLAGS $libdaxctl_cflags" + else + if test "$libdaxctl" = "yes" ; then + feature_not_found "libdaxctl" "Install libdaxctl" + fiRegion iteration APIs, align and path getter routines are only available since libdaxctl v56/v57 (the latest is v68).
Quoted 5 lines
Not sure how likely this happens in today's distros but if we care about systems with < v57 we should probably check that daxctl_region_foreach/daxctl_region_get_align/daxctl_region_get_path symbols exist? Or alternatively requiring v57 or up which serves as a bandage, but more long term ... any usage of newer daxctl APIs will require the former.
Thanks for your suggestion.
Better to require v57 or up.
How about adding the following check:
+ if $pkg_config --atleast-version=57 "libdaxctl"; then
Jingqi
Ping for comments.
Hi Paolo, Dan, Joao,
Any comments for this series.
Thanks,
Jingqi
Quoted 33 lines · Jingqi
This series adds libdaxctl support and fetchs the alignment of device dax through libdaxctl [1] APIs.
QEMU uses mmap(2) to maps vNVDIMM backends and aligns the mapping address to the page size (getpagesize(2)) by default. However, some types of backends may require an alignment different than the page size. The 'align' option is provided to memory-backend-file to allow users to specify the proper alignment.
For device dax (e.g.,
/dev/dax0.0), the 'align' option needs to match the alignment requirement of the device dax, which can be fetched through the libdaxctl APIs.[1] Libdaxctl is a part of ndctl project.
The project's repository is: https://github.com/pmem/ndctlChangelog:
v2: Per Paolo and Dan suggestions, fetch the alignment of device dax
through libdaxctl APIs.v1: The initial version.
Fetch the alignment through "/sys/dev/char/%d:%d/device/align".Jingqi Liu (3):
exec: fetch the alignment of Linux devdax pmem character device nodes docs/nvdimm: add description of alignment requirement of device dax
configure: add libdaxctl supportconfigure | 30 +++++++++++++++++++++++++++ docs/nvdimm.txt | 9 +++++++++ exec.c | 54 ++++++++++++++++++++++++++++++++++++++++++++++++- 3 files changed, 92 insertions(+), 1 deletion(-)