nvdimm: support MAP_SYNC for memory-backend-file
v5 cover nvdimm: support MAP_SYNC for memory-backend-file
Linux 4.15 introduces a new mmap flag MAP_SYNC, which can be used to guarantee the write persistence to mmap'ed files supporting DAX (e.g., files on ext4/xfs file system mounted with '-o dax').
A description of MAP_SYNC and MAP_SHARED_VALIDATE can be found at
https://patchwork.kernel.org/patch/10028151/
In order to make sure that the file metadata is in sync after a fault while we are writing a shared DAX supporting backend files, this patch-set enables QEMU to use MAP_SYNC flag for memory-backend-dax-file.
As the DAX vs DMA truncated issue was solved, we refined the code and send out this feature for the v5 version.
A new auto on/off option 'sync' is added to memory-backend-file:
- on: try to pass MAP_SYNC to mmap(2); if MAP_SYNC is not supported or 'share=off', QEMU will abort
- off: never pass MAP_SYNC to mmap(2)
- auto (default): if MAP_SYNC is supported and 'share=on', work as if 'sync=on'; otherwise, work as if 'sync=off'
Changes in v5:
* Add patch 1 to fix a memory leak issue.
* Refine the patch 4-6
* Remove the patch 3 as we already change the parameter from "shared" to "flags"
Changes in v4:
* Add patch 1-3 to switch some functions to a single 'flags' parameters. (Michael S. Tsirkin)
* v3 patch 1-3 become v4 patch 4-6.
* Patch 4: move definitions of MAP_SYNC and MAP_SHARED_VALIDATE to a new header file under include/standard-headers/linux/. (Michael S. Tsirkin)
* Patch 6: refine the description of the 'sync' option. (Michael S. Tsirkin)
Changes in v3:
* Patch 1: add MAP_SHARED_VALIDATE in both sync=on and sync=auto cases, and add back the retry mechanism. MAP_SYNC will be ignored by Linux kernel 4.15 if MAP_SHARED_VALIDATE is missed.
* Patch 1: define MAP_SYNC and MAP_SHARED_VALIDATE as 0 on non-Linux platforms in order to make qemu_ram_mmap() compile on those platforms.
* Patch 2&3: include more information in error messages of memory-backend in hope to help user to identify the error. (Dr. David Alan Gilbert)
* Patch 3: fix typo in the commit message. (Dr. David Alan Gilbert)
Changes in v2:
* Add 'sync' option to control the use of MAP_SYNC. (Eduardo Habkost)
* Remove the unnecessary set of MAP_SHARED_VALIDATE in some cases and the retry mechanism in qemu_ram_mmap(). (Michael S. Tsirkin)
* Move OS dependent definitions of MAP_SYNC and MAP_SHARED_VALIDATE to osdep.h. (Michael S. Tsirkin)
Zhang Yi (7):
numa: Fixed the memory leak of numa error message util/mmap-alloc: switch qemu_ram_mmap() to 'flags' parameter
exec: switch qemu_ram_alloc_from_{file, fd} to the 'flags' parameter util/mmap-alloc: support MAP_SYNC in qemu_ram_mmap() util/mmap-alloc: Switch the RAM_SYNC flags to OnOffAuto
hostmem: add more information in error messages
hostmem-file: add 'sync' option
backends/hostmem-file.c | 45 +++++++++++++++++++++++++++++++++-- backends/hostmem.c | 8 ++++--- docs/nvdimm.txt | 20 +++++++++++++++- exec.c | 9 +++---- include/exec/memory.h | 18 ++++++++++++++ include/exec/ram_addr.h | 1 + include/qemu/mmap-alloc.h | 20 +++++++++++++++- include/standard-headers/linux/mman.h | 44 ++++++++++++++++++++++++++++++++++ numa.c | 1 + qemu-options.hx | 22 ++++++++++++++++- util/mmap-alloc.c | 26 ++++++++++++++++---- util/oslib-posix.c | 4 +++- 12 files changed, 200 insertions(+), 18 deletions(-) create mode 100644 include/standard-headers/linux/mman.h
Signature · 1 line
2.7.4
PATCH v5 1/7 numa: Fixed the memory leak of numa error message
object_get_canonical_path_component() returns a string which must be freed using g_free().
--- numa.c | 1 + 1 file changed, 1 insertion(+)
numa.c +1 -0
@@ -533,6 +533,7 @@ void memory_region_allocate_system_memory(MemoryRegion *mr, Object *owner,error_report ("memory backend %s is used multiple times. Each " "-numa option must use a different memdev value." , path); +g_free (path);exit (1 ); }
Signature · 1 line
2.7.4
PATCH v5 2/7 util/mmap-alloc: switch qemu_ram_mmap() to 'flags' parameter
In reply to Zhang YiAs more flag parameters besides the existing 'shared' are going to be added to qemu_ram_mmap(), let's switch 'shared' to a 'flags' parameter in advance, so as to ease the further additions.
--- exec.c | 3 +-- include/qemu/mmap-alloc.h | 19 ++++++++++++++++++- util/mmap-alloc.c | 8 +++++--- util/oslib-posix.c | 4 +++- 4 files changed, 27 insertions(+), 7 deletions(-)
exec.c +1 -2
@@ -1859,8 +1859,7 @@ static void *file_ram_alloc(RAMBlock *block,perror ("ftruncate" ); } - area =qemu_ram_mmap (fd, memory, block->mr->align, - block->flags &RAM_SHARED ); + area =qemu_ram_mmap (fd, memory, block->mr->align, block->flags);if (area ==MAP_FAILED ) {error_setg_errno (errp, errno,"unable to map backing store for guest RAM" );
include/qemu/mmap-alloc.h +18 -1
@@ -7,7 +7,24 @@ size_t qemu_fd_getpagesize(int fd);size_t qemu_mempath_getpagesize (const char *mem_path); -void *qemu_ram_mmap (int fd,size_t size,size_t align,bool shared); +/** +* qemu_ram_mmap: mmap the specified file or device. +* +* Parameters: +* @fd: the file or the device to mmap +* @size: the number of bytes to be mmaped +* @align: if not zero, specify the alignment of the starting mapping address; +* otherwise, the alignment in use will be determined by QEMU. +* @flags: specifies additional properties of the mapping, which can be one or +* bit-or of following values +* - RAM_SHARED: mmap with MAP_SHARED flag +* Other bits are ignored. +* +* Return: +* On success, return a pointer to the mapped area. +* On failure, return MAP_FAILED. +*/ +void *qemu_ram_mmap (int fd,size_t size,size_t align,uint32_t flags);void qemu_ram_munmap (void *ptr,size_t size);
util/mmap-alloc.c +5 -3
@@ -13,6 +13,7 @@#include "qemu/osdep.h" #include "qemu/mmap-alloc.h" #include "qemu/host-utils.h" +#include "exec/memory.h" #define HUGETLBFS_MAGIC 0x958458f6
@@ -75,7 +76,7 @@ size_t qemu_mempath_getpagesize(const char *mem_path)return getpagesize (); } -void *qemu_ram_mmap (int fd,size_t size,size_t align,bool shared) +void *qemu_ram_mmap (int fd,size_t size,size_t align,uint32_t flags) {/* * Note: this always allocates at least one extra page of virtual address
@@ -92,11 +93,12 @@ void *qemu_ram_mmap(int fd, size_t size, size_t align, bool shared)* anonymous memory is OK. */ int anonfd = fd ==-1 ||qemu_fd_getpagesize (fd) ==getpagesize () ?-1 : fd; -int flags = anonfd ==-1 ?MAP_ANONYMOUS :MAP_NORESERVE ; -void *ptr =mmap (0 , total,PROT_NONE , flags |MAP_PRIVATE , anonfd,0 ); +int mmap_flags = anonfd ==-1 ?MAP_ANONYMOUS :MAP_NORESERVE ; +void *ptr =mmap (0 , total,PROT_NONE , mmap_flags |MAP_PRIVATE , anonfd,0 );#else void *ptr =mmap (0 , total,PROT_NONE ,MAP_ANONYMOUS |MAP_PRIVATE ,-1 ,0 );#endif +bool shared = flags &RAM_SHARED ;size_t offset;void *ptr1;
util/oslib-posix.c +3 -1
@@ -203,7 +203,9 @@ void *qemu_memalign(size_t alignment, size_t size)void *qemu_anon_ram_alloc (size_t size,uint64_t *alignment,bool shared) {size_t align =QEMU_VMALLOC_ALIGN ; -void *ptr =qemu_ram_mmap (-1 , size, align, shared); +uint32_t flags =0 ; + flags |= shared; +void *ptr =qemu_ram_mmap (-1 , size, align, flags);if (ptr ==MAP_FAILED ) {return NULL ;
Signature · 1 line
2.7.4
PATCH v5 3/7 exec: switch qemu_ram_alloc_from_{file, fd} to the 'flags' parameter
In reply to Zhang YiAs more flag parameters besides the existing 'share' are going to be added to qemu_ram_alloc_from_{file,fd}(), let's swith 'share' to a 'flags' parameters in advance, so as to ease the further additions.
--- exec.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
exec.c +4 -2
@@ -1810,6 +1810,7 @@ static void *file_ram_alloc(RAMBlock *block,ram_addr_t memory, int fd, bool truncate, + uint32_t flags, Error **errp) { void *area;
@@ -1859,7 +1860,7 @@ static void *file_ram_alloc(RAMBlock *block,perror ("ftruncate" ); } - area =qemu_ram_mmap (fd, memory, block->mr->align, block->flags); + area =qemu_ram_mmap (fd, memory, block->mr->align, flags);if (area ==MAP_FAILED ) {error_setg_errno (errp, errno,"unable to map backing store for guest RAM" );
@@ -2278,7 +2279,8 @@ RAMBlock *qemu_ram_alloc_from_fd(ram_addr_t size, MemoryRegion *mr, new_block->used_length = size; new_block->max_length = size; new_block->flags = ram_flags; - new_block->host =file_ram_alloc (new_block, size, fd, !file_size, errp); + new_block->host =file_ram_alloc (new_block, size, fd, !file_size, + ram_flags, errp);if (!new_block->host) {g_free (new_block);return NULL ;
Signature · 1 line
2.7.4
PATCH v5 4/7 util/mmap-alloc: support MAP_SYNC in qemu_ram_mmap()
In reply to Zhang YiWhen a file supporting DAX is used as vNVDIMM backend, mmap it with MAP_SYNC flag in addition can guarantee the persistence of guest write to the backend file without other QEMU actions (e.g., periodic fsync() by QEMU).
A set of RAM_SYNC flags are added to qemu_ram_mmap():
--- exec.c | 2 +- include/exec/memory.h | 3 +++ include/exec/ram_addr.h | 1 + include/qemu/mmap-alloc.h | 1 + include/standard-headers/linux/mman.h | 44 +++++++++++++++++++++++++++++++++++ util/mmap-alloc.c | 14 +++++++---- 6 files changed, 60 insertions(+), 5 deletions(-) create mode 100644 include/standard-headers/linux/mman.h
exec.c +1 -1
@@ -2241,7 +2241,7 @@ RAMBlock *qemu_ram_alloc_from_fd(ram_addr_t size, MemoryRegion *mr,int64_t file_size;/* Just support these ram flags by now. */ -assert ((ram_flags & ~(RAM_SHARED |RAM_PMEM )) ==0 ); +assert ((ram_flags & ~(RAM_SHARED |RAM_PMEM |RAM_SYNC )) ==0 );if (xen_enabled ()) {error_setg (errp,"-mem-path not supported with Xen" );
include/exec/memory.h +3 -0
@@ -126,6 +126,9 @@ typedef struct IOMMUNotifier IOMMUNotifier;/* RAM is a persistent kind memory */ #define RAM_PMEM (1 << 5) +/* RAM can be mmap by a MAP_SYNC flag */ +#define RAM_SYNC (1 << 6) + static inline void iommu_notifier_init(IOMMUNotifier *n, IOMMUNotify fn, IOMMUNotifierFlag flags, hwaddr start, hwaddr end,
include/exec/ram_addr.h +1 -0
@@ -87,6 +87,7 @@ long qemu_getrampagesize(void);* or bit-or of following values * - RAM_SHARED: mmap the backing file or device with MAP_SHARED * - RAM_PMEM: the backend @mem_path or @fd is persistent memory +* - RAM_SYNC: mmap with MAP_SYNC flag * Other bits are ignored. * @mem_path or @fd: specify the backing file or device * @errp: pointer to Error*, to store an error if it happens
include/qemu/mmap-alloc.h +1 -0
@@ -18,6 +18,7 @@ size_t qemu_mempath_getpagesize(const char *mem_path);* @flags: specifies additional properties of the mapping, which can be one or * bit-or of following values * - RAM_SHARED: mmap with MAP_SHARED flag +* - RAM_SYNC: mmap with MAP_SYNC flag * Other bits are ignored. * * Return:
include/standard-headers/linux/mman.h new file +44 -0
44 lines not on this page · raw message
util/mmap-alloc.c +10 -4
@@ -14,6 +14,7 @@#include "qemu/mmap-alloc.h" #include "qemu/host-utils.h" #include "exec/memory.h" +#include "standard-headers/linux/mman.h" #define HUGETLBFS_MAGIC 0x958458f6
@@ -99,6 +100,7 @@ void *qemu_ram_mmap(int fd, size_t size, size_t align, uint32_t flags)void *ptr =mmap (0 , total,PROT_NONE ,MAP_ANONYMOUS |MAP_PRIVATE ,-1 ,0 );#endif bool shared = flags &RAM_SHARED ; +int mmap_xflags =0 ;size_t offset;void *ptr1;
@@ -109,16 +111,20 @@ void *qemu_ram_mmap(int fd, size_t size, size_t align, uint32_t flags)assert (is_power_of_2 (align));/* Always align to host page size */ assert (align >=getpagesize ()); +if ((flags &RAM_SYNC ) && shared) { + mmap_xflags |=MAP_SYNC_FLAGS ; + } offset =QEMU_ALIGN_UP ((uintptr_t )ptr, align) - (uintptr_t )ptr; + retry_mmap_fd: ptr1 =mmap (ptr + offset, size,PROT_READ |PROT_WRITE ,MAP_FIXED | (fd ==-1 ?MAP_ANONYMOUS :0 ) | - (shared ?MAP_SHARED :MAP_PRIVATE ), + (shared ?MAP_SHARED :MAP_PRIVATE ) | mmap_xflags, fd,0 ); -if (ptr1 ==MAP_FAILED ) { -munmap (ptr, total); -return MAP_FAILED ; +if ((ptr1 ==MAP_FAILED ) && (mmap_xflags &MAP_SYNC_FLAGS )) { + mmap_xflags &= ~MAP_SYNC_FLAGS ; + goto retry_mmap_fd; }if (offset >0 ) {
Signature · 1 line
2.7.4
PATCH v5 5/7 util/mmap-alloc: Switch the RAM_SYNC flags to OnOffAuto
In reply to Zhang YiA set of RAM_SYNC_ON_OFF_AUTO{AUTO,ON,OFF} flags are added to qemu_ram_mmap():
- If RAM_SYNC_ON_OFF_AUTO_ON is present, qemu_ram_mmap() will try to pass MAP_SYNC to mmap(). It will then fail if the host OS or the backend file do not support MAP_SYNC, or MAP_SYNC is conflict with other flags.
- If RAM_SYNC_ON_OFF_AUTO_OFF is present, qemu_ram_mmap() will never pass MAP_SYNC to mmap().
- If RAM_SYNC_ON_OFF_AUTO_AUTO is present, and
* if the host OS and the backend file support MAP_SYNC, and MAP_SYNC is not conflict with other flags, qemu_ram_mmap() will work as if RAM_SYNC_ON_OFF_AUTO_ON is present;
* otherwise, qemu_ram_mmap() will work as if RAM_SYNC_ON_OFF_AUTO_OFF is present.
--- include/exec/memory.h | 9 ++++++++- util/mmap-alloc.c | 12 ++++++++++-- 2 files changed, 18 insertions(+), 3 deletions(-)
include/exec/memory.h +8 -1
@@ -127,7 +127,14 @@ typedef struct IOMMUNotifier IOMMUNotifier;#define RAM_PMEM (1 << 5)/* RAM can be mmap by a MAP_SYNC flag */ -#define RAM_SYNC (1 << 6) +#define RAM_SYNC_SHIFT 6 +#define RAM_SYNC_SHIFT_AUTO 7 + +#define RAM_SYNC_ON_OFF_AUTO_ON (1UL << RAM_SYNC_SHIFT) +#define RAM_SYNC_ON_OFF_AUTO_OFF (0UL << RAM_SYNC_SHIFT) +#define RAM_SYNC_ON_OFF_AUTO_AUTO (1UL << RAM_SYNC_SHIFT_AUTO) + +#define RAM_SYNC (RAM_SYNC_ON_OFF_AUTO_ON | RAM_SYNC_ON_OFF_AUTO_AUTO) static inline void iommu_notifier_init(IOMMUNotifier *n, IOMMUNotify fn, IOMMUNotifierFlag flags,
util/mmap-alloc.c +10 -2
@@ -111,6 +111,10 @@ void *qemu_ram_mmap(int fd, size_t size, size_t align, uint32_t flags)assert (is_power_of_2 (align));/* Always align to host page size */ assert (align >=getpagesize ()); +if ((flags &RAM_SYNC_ON_OFF_AUTO_ON ) && + (!shared || !MAP_SYNC_FLAGS )) { +return MAP_FAILED ; + }if ((flags &RAM_SYNC ) && shared) { mmap_xflags |=MAP_SYNC_FLAGS ; }
@@ -123,8 +127,12 @@ void *qemu_ram_mmap(int fd, size_t size, size_t align, uint32_t flags) (shared ?MAP_SHARED :MAP_PRIVATE ) | mmap_xflags, fd, 0);if ((ptr1 ==MAP_FAILED ) && (mmap_xflags &MAP_SYNC_FLAGS )) { - mmap_xflags &= ~MAP_SYNC_FLAGS ; - goto retry_mmap_fd; +if (flags &RAM_SYNC_ON_OFF_AUTO_AUTO ) { + mmap_xflags &= ~MAP_SYNC_FLAGS ; + goto retry_mmap_fd; + } +munmap (ptr, total); +return MAP_FAILED ; }if (offset >0 ) {
Signature · 1 line
2.7.4
PATCH v5 6/7 hostmem: add more information in error messages
In reply to Zhang YiWhen there are multiple memory backends in use, including the object type name, ID and the property name in the error message can help users to locate the error.
--- backends/hostmem-file.c | 6 ++++-- backends/hostmem.c | 8 +++++--- 2 files changed, 9 insertions(+), 5 deletions(-)
backends/hostmem-file.c +4 -2
@@ -82,7 +82,8 @@ static void set_mem_path(Object *o, const char *str, Error **errp)HostMemoryBackendFile *fb =MEMORY_BACKEND_FILE (o);if (host_memory_backend_mr_inited (backend)) { -error_setg (errp,"cannot change property value" ); +error_setg (errp,"cannot change property 'mem-path' of %s" , +object_get_typename (o));return ; }g_free (fb->mem_path);
@@ -120,7 +121,8 @@ static void file_memory_backend_set_align(Object *o, Visitor *v,uint64_t val;if (host_memory_backend_mr_inited (backend)) { -error_setg (&local_err,"cannot change property value" ); +error_setg (&local_err,"cannot change property '%s' of %s" , + name,object_get_typename (o)); goto out; }
backends/hostmem.c +5 -3
@@ -47,7 +47,8 @@ host_memory_backend_set_size(Object *obj, Visitor *v, const char *name,uint64_t value;if (host_memory_backend_mr_inited (backend)) { -error_setg (&local_err,"cannot change property value" ); +error_setg (&local_err,"cannot change property %s of %s " , + name,object_get_typename (obj)); goto out; }
@@ -56,8 +57,9 @@ host_memory_backend_set_size(Object *obj, Visitor *v, const char *name, goto out; }if (!value) { -error_setg (&local_err,"Property '%s.%s' doesn't take value '%" - PRIu64"'" ,object_get_typename (obj), name, value); +error_setg (&local_err, +"property '%s' of %s doesn't take value '%" PRIu64"'" , + name,object_get_typename (obj), value); goto out; } backend->size = value;
Signature · 1 line
2.7.4
PATCH v5 7/7 hostmem-file: add 'sync' option
In reply to Zhang YiThis option controls whether QEMU mmap(2) the memory backend file with MAP_SYNC flag, which can fully guarantee the guest write persistence to the backend, if MAP_SYNC flag is supported by the host kernel (Linux kernel 4.15 and later) and the backend is a file supporting DAX (e.g., file on ext4/xfs file system mounted with '-o dax').
It can take one of following values:
- on: try to pass MAP_SYNC to mmap(2); if MAP_SYNC is not supported or 'share=off', QEMU will abort
- off: never pass MAP_SYNC to mmap(2)
- auto (default): if MAP_SYNC is supported and 'share=on', work as if 'sync=on'; otherwise, work as if 'sync=off'
--- backends/hostmem-file.c | 39 +++++++++++++++++++++++++++++++++++++++ docs/nvdimm.txt | 20 +++++++++++++++++++- include/exec/memory.h | 8 ++++++++ qemu-options.hx | 22 +++++++++++++++++++++- 4 files changed, 87 insertions(+), 2 deletions(-)
backends/hostmem-file.c +39 -0
@@ -16,6 +16,7 @@#include "sysemu/hostmem.h" #include "sysemu/sysemu.h" #include "qom/object_interfaces.h" +#include "qapi/qapi-visit.h" /* hostmem-file.c */ /**
@@ -36,6 +37,7 @@ struct HostMemoryBackendFile {uint64_t align;bool discard_data;bool is_pmem; +OnOffAuto sync; }; static void
@@ -62,6 +64,7 @@ file_backend_memory_alloc(HostMemoryBackend *backend, Error **errp) path, backend->size, fb->align, (backend->share ?RAM_SHARED :0 ) | +qemu_ram_sync_flags (fb->sync) | (fb->is_pmem ?RAM_PMEM :0 ), fb->mem_path, errp);g_free (path);
@@ -136,6 +139,39 @@ static void file_memory_backend_set_align(Object *o, Visitor *v,error_propagate (errp, local_err); } +static void file_memory_backend_get_sync ( +Object *obj,Visitor *v,const char *name,void *opaque,Error **errp) +{ +HostMemoryBackendFile *fb =MEMORY_BACKEND_FILE (obj); +OnOffAuto value = fb->sync; + +visit_type_OnOffAuto (v, name, &value, errp); +} + +static void file_memory_backend_set_sync ( +Object *obj,Visitor *v,const char *name,void *opaque,Error **errp) +{ +HostMemoryBackend *backend =MEMORY_BACKEND (obj); +HostMemoryBackendFile *fb =MEMORY_BACKEND_FILE (obj); +Error *local_err =NULL ; +OnOffAuto value; + +if (host_memory_backend_mr_inited (backend)) { +error_setg (&local_err,"cannot change property '%s' of %s" , + name,object_get_typename (obj)); + goto out; + } + +visit_type_OnOffAuto (v, name, &value, &local_err); +if (local_err) { + goto out; + } + fb->sync = value; + + out: +error_propagate (errp, local_err); +} +static bool file_memory_backend_get_pmem (Object *o,Error **errp) {return MEMORY_BACKEND_FILE (o)->is_pmem;
@@ -203,6 +239,9 @@ file_backend_class_init(ObjectClass *oc, void *data)object_class_property_add_bool (oc,"pmem" , file_memory_backend_get_pmem, file_memory_backend_set_pmem, &error_abort); +object_class_property_add (oc,"sync" ,"OnOffAuto" , + file_memory_backend_get_sync, file_memory_backend_set_sync, +NULL ,NULL , &error_abort); }static void file_backend_instance_finalize (Object *o)
docs/nvdimm.txt +19 -1
@@ -142,11 +142,29 @@ backend of vNVDIMM: Guest Data Persistence ---------------------- +vNVDIMM is designed and implemented to guarantee the guest data +persistence on the backends even on the host crash and power +failures. However, there are still some requirements and limitations +as explained below. + Though QEMU supports multiple types of vNVDIMM backends on Linux, -currently the only one that can guarantee the guest write persistence +if MAP_SYNC is not supported by the host kernel and the backends, +the only backend that can guarantee the guest write persistence is the device DAX on the real NVDIMM device (e.g., /dev/dax0.0), to which all guest access do not involve any host-side kernel cache. +mmap(2) flag MAP_SYNC is added since Linux kernel 4.15. On such +systems, QEMU can mmap(2) the backend with MAP_SYNC, which can +guarantee the guest write persistence to vNVDIMM. Besides the host +kernel support, enabling MAP_SYNC in QEMU also requires: + + - the backend is a file supporting DAX, e.g., a file on an ext4 or + xfs file system mounted with '-o dax', + + - 'sync' option of memory-backend-file is not 'off', and + + - 'share' option of memory-backend-file is 'on'. + When using other types of backends, it's suggested to set 'unarmed' option of '-device nvdimm' to 'on', which sets the unarmed flag of the guest NVDIMM region mapping structure. This unarmed flag indicates
include/exec/memory.h +8 -0
@@ -26,6 +26,7 @@#include "qom/object.h" #include "qemu/rcu.h" #include "hw/qdev-core.h" +#include "qapi/error.h" #define RAM_ADDR_INVALID (~(ram_addr_t)0)
@@ -136,6 +137,13 @@ typedef struct IOMMUNotifier IOMMUNotifier;#define RAM_SYNC (RAM_SYNC_ON_OFF_AUTO_ON | RAM_SYNC_ON_OFF_AUTO_AUTO) +static inline uint64_t qemu_ram_sync_flags (OnOffAuto v) +{ +return v ==ON_OFF_AUTO_OFF ?RAM_SYNC_ON_OFF_AUTO_OFF : + v ==ON_OFF_AUTO_ON ?RAM_SYNC_ON_OFF_AUTO_ON : +RAM_SYNC_ON_OFF_AUTO_AUTO ; +} + static inline void iommu_notifier_init(IOMMUNotifier *n, IOMMUNotify fn, IOMMUNotifierFlag flags, hwaddr start, hwaddr end,
qemu-options.hx +21 -1
@@ -3928,7 +3928,7 @@ property must be set. These objects are placed in the @table @option -@item -object memory-backend-file,id=@var{id},size=@var{size},mem-path=@var{dir},share=@var{on|off},discard-data=@var{on|off},merge=@var{on|off},dump=@var{on|off},prealloc=@var{on|off},host-nodes=@var{host-nodes},policy=@var{default|preferred|bind|interleave},align=@var{align} +@item -object memory-backend-file,id=@var{id},size=@var{size},mem-path=@var{dir},share=@var{on|off},discard-data=@var{on|off},merge=@var{on|off},dump=@var{on|off},prealloc=@var{on|off},host-nodes=@var{host-nodes},policy=@var{default|preferred|bind|interleave},align=@var{align},sync=@var{on|off|auto} Creates a memory file backend object, which can be used to back the guest RAM with huge pages.
@@ -4003,6 +4003,26 @@ If @option{pmem} is set to 'on', QEMU will take necessary operations to guarantee the persistence of its own writes to @option{mem-path} (e.g. in vNVDIMM label emulation and live migration). +The @option{sync} option specifies whether QEMU mmap(2) @option{mem-path} +with MAP_SYNC flag, which can guarantee the guest write persistence to +@option{mem-path} even on the host crash and power failures. MAP_SYNC +requires supports from both the host kernel (since Linux kernel 4.15) +and @option{mem-path} (only files supporting DAX). It can take one of +following values: + +@table @option +@item @var{on} +try to pass MAP_SYNC to mmap(2); if MAP_SYNC is not supported or +@option{share}=@var{off}, QEMU will abort + +@item @var{off} +never pass MAP_SYNC to mmap(2) + +@item @var{auto} (default) +if MAP_SYNC is supported and @option{share}=@var{on}, work as if +@option{sync}=@var{on}; otherwise, work as if @option{sync}=@var{off} +@end table + @item -object memory-backend-ram,id=@var{id},merge=@var{on|off},dump=@var{on|off},share=@var{on|off},prealloc=@var{on|off},size=@var{size},host-nodes=@var{host-nodes},policy=@var{default|preferred|bind|interleave} Creates a memory backend object, which can be used to back the guest RAM.
Signature · 1 line
2.7.4
Re: [Qemu-devel] [PATCH v5 2/7] util/mmap-alloc: switch qemu_ram_mmap() to 'flags' parameter
In reply to Zhang Yiutil/oslib-posix.c void *qemu_memalign(size_t alignment, size_t size) · line 207
Quoted 11 lines · Zhang Yi
util/oslib-posix.c +2
-1@@ -203,7 +203,9 @@ void *qemu_memalign(size_t alignment, size_t size)void *qemu_anon_ram_alloc (size_t size,uint64_t *alignment,bool shared) {size_t align =QEMU_VMALLOC_ALIGN ; -void *ptr =qemu_ram_mmap (-1 , size, align, shared); +uint32_t flags =0 ; + flags |= shared;
O_o .o(???)
shared is bool, what is going on here? Did you mean:
uint32_t flags = 0;
void *ptr;
if (shared) {
flags |= MAP_SHARED;
}
ptr = qemu_ram_mmap(-1, size, align, flags);Re: [Qemu-devel] [PATCH v5 4/7] util/mmap-alloc: support MAP_SYNC in qemu_ram_mmap()
In reply to Zhang Yiinclude/standard-headers/linux/mman.h
Quoted 5 lines · Zhang Yi
include/standard-headers/linux/mman.h new file +0
-0
include/standard-headers/ contains mechanically-translated Linux kernel headers. You wrote your own header file from scratch and scripts/update-linux-headers.h doesn't know about it.
Please put your changes in include/qemu/osdep.h instead of include/standard-headers/.
Re: [Qemu-devel] [PATCH v5 5/7] util/mmap-alloc: Switch the RAM_SYNC flags to OnOffAuto
In reply to Zhang Yiutil/mmap-alloc.c void *qemu_ram_mmap(int fd, size_t size, size_t align, uint32_t flags) · line 117
Quoted 12 lines · Zhang Yi
util/mmap-alloc.c +4
-0@@ -111,6 +111,10 @@ void *qemu_ram_mmap(int fd, size_t size, size_t align, uint32_t flags)assert (is_power_of_2 (align));/* Always align to host page size */ assert (align >=getpagesize ()); +if ((flags &RAM_SYNC_ON_OFF_AUTO_ON ) && + (!shared || !MAP_SYNC_FLAGS )) { +return MAP_FAILED ; + }
Missing munmap(ptr, total)?
Quoted 4 lines · Zhang Yi
Linux 4.15 introduces a new mmap flag
MAP_SYNC, which can be used to guarantee the write persistence to mmap'ed files supporting DAX (e.g., files on ext4/xfs file system mounted with '-o dax').
Hi,
There are a lot of people on CC so probably no one felt responsible to review this series quickly. Feel free to send a "ping" email if your patches aren't receiving attention.
Overall this feature looks fine. I left a few small comments.
Thanks,
Stefan
Quoted 77 lines
A description of
MAP_SYNCandMAP_SHARED_VALIDATEcan be found athttps://patchwork.kernel.org/patch/10028151/In order to make sure that the file metadata is in sync after a fault while we are writing a shared DAX supporting backend files, this patch-set enables QEMU to use
MAP_SYNCflag for memory-backend-dax-file.As the DAX vs DMA truncated issue was solved, we refined the code and send out this feature for the v5 version.
A new auto on/off option 'sync' is added to memory-backend-file:
- on: try to passMAP_SYNCto mmap(2); ifMAP_SYNCis not supported or 'share=off', QEMU will abort
- off: never passMAP_SYNCto mmap(2)
- auto (default): ifMAP_SYNCis supported and 'share=on', work as if 'sync=on'; otherwise, work as if 'sync=off'Changes in v5:
* Add patch 1 to fix a memory leak issue.
* Refine the patch 4-6
* Remove the patch 3 as we already change the parameter from "shared" to "flags"Changes in v4:
* Add patch 1-3 to switch some functions to a single 'flags' parameters. (Michael S. Tsirkin)
* v3 patch 1-3 become v4 patch 4-6.
* Patch 4: move definitions ofMAP_SYNCandMAP_SHARED_VALIDATEto a new header file underinclude/standard-headers/linux/. (Michael S. Tsirkin)
* Patch 6: refine the description of the 'sync' option. (Michael S. Tsirkin)Changes in v3:
* Patch 1: addMAP_SHARED_VALIDATEin both sync=on and sync=auto cases, and add back the retry mechanism.MAP_SYNCwill be ignored by Linux kernel 4.15 ifMAP_SHARED_VALIDATEis missed.
* Patch 1: defineMAP_SYNCandMAP_SHARED_VALIDATEas 0 on non-Linux platforms in order to makeqemu_ram_mmap()compile on those platforms.
* Patch 2&3: include more information in error messages of memory-backend in hope to help user to identify the error. (Dr. David Alan Gilbert)
* Patch 3: fix typo in the commit message. (Dr. David Alan Gilbert)Changes in v2:
* Add 'sync' option to control the use ofMAP_SYNC. (Eduardo Habkost)
* Remove the unnecessary set ofMAP_SHARED_VALIDATEin some cases and the retry mechanism inqemu_ram_mmap(). (Michael S. Tsirkin)
* Move OS dependent definitions ofMAP_SYNCandMAP_SHARED_VALIDATEtoosdep.h. (Michael S. Tsirkin)Zhang Yi (7):
numa: Fixed the memory leak of numa error message util/mmap-alloc: switchqemu_ram_mmap()to 'flags' parameter
exec: switch qemu_ram_alloc_from_{file, fd} to the 'flags' parameter util/mmap-alloc: supportMAP_SYNCinqemu_ram_mmap()util/mmap-alloc: Switch theRAM_SYNCflags to OnOffAuto
hostmem: add more information in error messages
hostmem-file: add 'sync' optionbackends/hostmem-file.c | 45 +++++++++++++++++++++++++++++++++-- backends/hostmem.c | 8 ++++--- docs/nvdimm.txt | 20 +++++++++++++++- exec.c | 9 +++---- include/exec/memory.h | 18 ++++++++++++++ include/exec/ram_addr.h | 1 + include/qemu/mmap-alloc.h | 20 +++++++++++++++- include/standard-headers/linux/mman.h | 44 ++++++++++++++++++++++++++++++++++ numa.c | 1 + qemu-options.hx | 22 ++++++++++++++++- util/mmap-alloc.c | 26 ++++++++++++++++---- util/oslib-posix.c | 4 +++- 12 files changed, 200 insertions(+), 18 deletions(-) create mode 100644 include/standard-headers/linux/mman.hSignature · 1 line
2.7.4
Quoted 11 lines · Stefan Hajnoczi
Quoted 4 lines · Zhang Yi
Linux 4.15 introduces a new mmap flag
MAP_SYNC, which can be used to guarantee the write persistence to mmap'ed files supporting DAX (e.g., files on ext4/xfs file system mounted with '-o dax').Hi,
There are a lot of people on CC so probably no one felt responsible to review this series quickly. Feel free to send a "ping" email if your patches aren't receiving attention.
Quoted 4 lines
Overall this feature looks fine. I left a few small comments.
Thanks,
Stefan
Thanks your review, Stefan, I will post the next version soon.
Quoted 78 lines
Quoted 78 lines
A description of
MAP_SYNCandMAP_SHARED_VALIDATEcan be found athttps://patchwork.kernel.org/patch/10028151/In order to make sure that the file metadata is in sync after a fault while we are writing a shared DAX supporting backend files, this patch-set enables QEMU to use
MAP_SYNCflag for memory-backend-dax-file.As the DAX vs DMA truncated issue was solved, we refined the code and send out this feature for the v5 version.
A new auto on/off option 'sync' is added to memory-backend-file:
- on: try to passMAP_SYNCto mmap(2); ifMAP_SYNCis not supported or 'share=off', QEMU will abort
- off: never passMAP_SYNCto mmap(2)
- auto (default): ifMAP_SYNCis supported and 'share=on', work as if 'sync=on'; otherwise, work as if 'sync=off'Changes in v5:
* Add patch 1 to fix a memory leak issue.
* Refine the patch 4-6
* Remove the patch 3 as we already change the parameter from "shared" to "flags"Changes in v4:
* Add patch 1-3 to switch some functions to a single 'flags' parameters. (Michael S. Tsirkin)
* v3 patch 1-3 become v4 patch 4-6.
* Patch 4: move definitions ofMAP_SYNCandMAP_SHARED_VALIDATEto a new header file underinclude/standard-headers/linux/. (Michael S. Tsirkin)
* Patch 6: refine the description of the 'sync' option. (Michael S. Tsirkin)Changes in v3:
* Patch 1: addMAP_SHARED_VALIDATEin both sync=on and sync=auto cases, and add back the retry mechanism.MAP_SYNCwill be ignored by Linux kernel 4.15 ifMAP_SHARED_VALIDATEis missed.
* Patch 1: defineMAP_SYNCandMAP_SHARED_VALIDATEas 0 on non-Linux platforms in order to makeqemu_ram_mmap()compile on those platforms.
* Patch 2&3: include more information in error messages of memory-backend in hope to help user to identify the error. (Dr. David Alan Gilbert)
* Patch 3: fix typo in the commit message. (Dr. David Alan Gilbert)Changes in v2:
* Add 'sync' option to control the use ofMAP_SYNC. (Eduardo Habkost)
* Remove the unnecessary set ofMAP_SHARED_VALIDATEin some cases and the retry mechanism inqemu_ram_mmap(). (Michael S. Tsirkin)
* Move OS dependent definitions ofMAP_SYNCandMAP_SHARED_VALIDATEtoosdep.h. (Michael S. Tsirkin)Zhang Yi (7):
numa: Fixed the memory leak of numa error message util/mmap-alloc: switchqemu_ram_mmap()to 'flags' parameter
exec: switch qemu_ram_alloc_from_{file, fd} to the 'flags' parameter util/mmap-alloc: supportMAP_SYNCinqemu_ram_mmap()util/mmap-alloc: Switch theRAM_SYNCflags to OnOffAuto
hostmem: add more information in error messages
hostmem-file: add 'sync' optionbackends/hostmem-file.c | 45 +++++++++++++++++++++++++++++++++-- backends/hostmem.c | 8 ++++--- docs/nvdimm.txt | 20 +++++++++++++++- exec.c | 9 +++---- include/exec/memory.h | 18 ++++++++++++++ include/exec/ram_addr.h | 1 + include/qemu/mmap-alloc.h | 20 +++++++++++++++- include/standard-headers/linux/mman.h | 44 ++++++++++++++++++++++++++++++++++ numa.c | 1 + qemu-options.hx | 22 ++++++++++++++++- util/mmap-alloc.c | 26 ++++++++++++++++---- util/oslib-posix.c | 4 +++- 12 files changed, 200 insertions(+), 18 deletions(-) create mode 100644 include/standard-headers/linux/mman.hSignature · 1 line
2.7.4