support MAP_SYNC for memory-backend-file
v13 cover support MAP_SYNC for memory-backend-file
Linux 4.15 introduces a new mmap flag MAP_SYNC, which can be used to guarantee the write persistence to mmap'ed files supporting DAX (e.g., files on ext4/xfs file system mounted with '-o dax').
A description of MAP_SYNC and MAP_SHARED_VALIDATE can be found at
https://patchwork.kernel.org/patch/10028151/
In order to make sure that the file metadata is in sync after a fault while we are writing a shared DAX supporting backend files, this patch-set enables QEMU to use MAP_SYNC flag for memory-backend-dax-file.
As the DAX vs DMA truncated issue was solved, we refined the code and send out this feature for the v5 version.
We will pass MAP_SYNC to mmap(2); if MAP_SYNC is supported and 'share=on' & 'pmem=on'.
Or QEMU will not pass this flag to mmap(2)
Test with below cases:
1. pmem=on is set, shared=on is set, MAP_SYNC supported:
a: backend is a dax supporting file.
1) start VM1 with options:
-object memory-backend-file,id=nv_be4,share,mem-path=${DAX_FILE_1},size=${DAX_FILE_SIZE_1},align=128M,pmem=on,share=on -device nvdimm,id=nv4,memdev=nv_be4,label-size=2M.
2) start VM2 with options:
-object memory-backend-file,id=nv_be4,share,mem-path=${DAX_FILE_2,size=${DAX_FILE_SIZE_2},align=128M,pmem=on,share=on -device nvdimm,id=nv4,memdev=nv_be4,label-size=2M.
3) live migrate from VM1 to VM2.
4) Suddly let Host crash or power failure.
5) check DAX_FILE_1 and DAX_FILE_2, no corrupt.
b: backend is a regular file.
1) start with options
-object memory-backend-file,id=nv_be4,share,mem-path=${REG_FILE},size=${REG_FILE_SIZE},align=128M,pmem=on,share=on -device nvdimm,id=nv4,memdev=nv_be4,label-size=2M.
will warning "failed to validate with mapping flags: Operation not supported" FILE_1 and FILE_2 random corrupt.
2. Other cases:
FILE_1 and FILE_2 random corrupt.
Changes in V13:
* 4/5 Micheal: move the inlcude to mmap_alloc.c.
* 4/5 Micheal: refine the warning message.
* 5/5 Micheal: refine the Documentations.
Changes in V12:
* 2/5: Micheal: Update update-linux-headers.sh
* 3/5: Micheal: Use script update add linux/mman.h
* 4/5: Pankaj,Micheal: 1) fallback to mmap without MAP_SYNC & MAP_SHARED_VALIDATE if sync not supported or failed
2) Replace the include with 3/5 added linux/mman.h
* 5/5: Micheal: Refine the Documentations.
Changes in V11:
* 1/3: Micheal: Change to just add a bool is_pmem in qemu_ram_mmap.
* 2/3: Micheal: Fix the compatibility for old kernel.
* 2/3&3/3: Micheal&Eduardo :Update the behavior below:
Waning at no-dax and continue without MAP_SYNC.
Test if fails again for compatibility, then remove the MAP_VALIDATE and silently proceed.
Changes in V10:
* 4/4: refine the document.
* 3/4: Reviewed-by: Stefano Garzarella <[email protected]>
* 2/4: refine the commit message, Added MAP_SHARED_VALIDATE.
* 2/4: Fix the wrong include header
Changes in V9:
* 1/6: Reviewed-by: Eduardo Habkost <[email protected]>
* 2/6: New Added: Micheal: use sparse feature define RAM_FLAG. since I don't have much knowledge about the sparse feature, @Micheal Could you add some documentation/commit message on this patch? Thank you very much.
* 3/6: from 2/5: Eduardo: updated the commit message.
* 4/6: from 3/5: Micheal: don't ignore MAP_SYNC failures silently.
* 5/6: from 4/5: Eduardo: updated the commit message.
* 6/6: from 5/5: Micheal: Drop the sync option, document the MAP_SYNC.
Changes in v8:
* Micheal: 3/5, remove the duplicated define in the os_dep.h
* Micheal: 2/5, make type define safety.
* Micheal: 2/5, fixed the incorrect define MAP_SHARE on qemu_anon_ram_alloc.
* 4/6 removed, we remove the on/off/auto define of sync, as by now, MAP_SYNC only worked with pmem=on.
* @Micheal, I still reuse the RAM_SYNC flag, it is much straightforward to parse all the flags in one parameter.
Changes in v7:
* Micheal: [3,4,6]/6 limited the "sync" flag only on a nvdimm backend.(pmem=on)
Changes in v6:
* Pankaj: 3/7 are squashed with 2/7
* Pankaj: 7/7 update comments to "consistent filesystem metadata".
* Pankaj, Igor: 1/7 Added Reviewed-by in patch-1/7
* Stefan, 4/7 move the include header from "/linux/mman.h" to "osdep.h"
* Stefan, 5/7 Add missing "munmap"
* Stefan, 2/7 refine the shared/flag.
Changes in v5:
* Add patch 1 to fix a memory leak issue.
* Refine the patch 4-6
* Remove the patch 3 as we already change the parameter from "shared" to "flags"
Changes in v4:
* Add patch 1-3 to switch some functions to a single 'flags' parameters. (Michael S. Tsirkin)
* v3 patch 1-3 become v4 patch 4-6.
* Patch 4: move definitions of MAP_SYNC and MAP_SHARED_VALIDATE to a new header file under include/standard-headers/linux/. (Michael S. Tsirkin)
* Patch 6: refine the description of the 'sync' option. (Michael S. Tsirkin)
Changes in v3:
* Patch 1: add MAP_SHARED_VALIDATE in both sync=on and sync=auto cases, and add back the retry mechanism. MAP_SYNC will be ignored by Linux kernel 4.15 if MAP_SHARED_VALIDATE is missed.
* Patch 1: define MAP_SYNC and MAP_SHARED_VALIDATE as 0 on non-Linux platforms in order to make qemu_ram_mmap() compile on those platforms.
* Patch 2&3: include more information in error messages of memory-backend in hope to help user to identify the error. (Dr. David Alan Gilbert)
* Patch 3: fix typo in the commit message. (Dr. David Alan Gilbert)
Changes in v2:
* Add 'sync' option to control the use of MAP_SYNC. (Eduardo Habkost)
* Remove the unnecessary set of MAP_SHARED_VALIDATE in some cases and the retry mechanism in qemu_ram_mmap(). (Michael S. Tsirkin)
* Move OS dependent definitions of MAP_SYNC and MAP_SHARED_VALIDATE to osdep.h. (Michael S. Tsirkin)
Zhang Yi (5):
util/mmap-alloc: Add a 'is_pmem' parameter to qemu_ram_mmap scripts/update-linux-headers: add linux/mman.h
linux-headers: add linux/mman.h.
util/mmap-alloc: support MAP_SYNC in qemu_ram_mmap()
docs: Added MAP_SYNC documentation
Zhang Yi (5):
util/mmap-alloc: Add a 'is_pmem' parameter to qemu_ram_mmap scripts/update-linux-headers: add linux/mman.h
linux-headers: add linux/mman.h.
util/mmap-alloc: support MAP_SYNC in qemu_ram_mmap()
docs: Added MAP_SYNC documentation
docs/nvdimm.txt | 25 ++++++- exec.c | 2 +- include/qemu/mmap-alloc.h | 21 +++++- linux-headers/asm-arm/mman.h | 4 ++ linux-headers/asm-arm64/mman.h | 1 + linux-headers/asm-generic/hugetlb_encode.h | 36 ++++++++++ linux-headers/asm-generic/mman-common.h | 77 ++++++++++++++++++++ linux-headers/asm-generic/mman.h | 24 +++++++ linux-headers/asm-mips/mman.h | 108 +++++++++++++++++++++++++++++ linux-headers/asm-powerpc/mman.h | 39 +++++++++++ linux-headers/asm-s390/mman.h | 1 + linux-headers/asm-x86/mman.h | 31 +++++++++ linux-headers/linux/mman.h | 38 ++++++++++ qemu-options.hx | 5 ++ scripts/update-linux-headers.sh | 6 +- util/mmap-alloc.c | 51 +++++++++++++- util/oslib-posix.c | 2 +- 17 files changed, 460 insertions(+), 11 deletions(-) create mode 100644 linux-headers/asm-arm/mman.h create mode 100644 linux-headers/asm-arm64/mman.h create mode 100644 linux-headers/asm-generic/hugetlb_encode.h create mode 100644 linux-headers/asm-generic/mman-common.h create mode 100644 linux-headers/asm-generic/mman.h create mode 100644 linux-headers/asm-mips/mman.h create mode 100644 linux-headers/asm-powerpc/mman.h create mode 100644 linux-headers/asm-s390/mman.h create mode 100644 linux-headers/asm-x86/mman.h create mode 100644 linux-headers/linux/mman.h
Signature · 1 line
2.7.4
PATCH v13 1/5 util/mmap-alloc: Add a 'is_pmem' parameter to qemu_ram_mmap
From: Zhang Yi <[email protected]>
besides the existing 'shared' flags, we are going to add 'is_pmem' to qemu_ram_mmap(), which indicated the memory backend file is a persist memory.
--- exec.c | 2 +- include/qemu/mmap-alloc.h | 21 ++++++++++++++++++++- util/mmap-alloc.c | 6 +++++- util/oslib-posix.c | 2 +- 4 files changed, 27 insertions(+), 4 deletions(-)
exec.c +1 -1
@@ -1860,7 +1860,7 @@ static void *file_ram_alloc(RAMBlock *block, } area =qemu_ram_mmap (fd, memory, block->mr->align, - block->flags &RAM_SHARED ); + block->flags &RAM_SHARED , block->flags &RAM_PMEM );if (area ==MAP_FAILED ) {error_setg_errno (errp, errno,"unable to map backing store for guest RAM" );
include/qemu/mmap-alloc.h +20 -1
@@ -7,7 +7,26 @@ size_t qemu_fd_getpagesize(int fd);size_t qemu_mempath_getpagesize (const char *mem_path); -void *qemu_ram_mmap (int fd,size_t size,size_t align,bool shared); +/** +* qemu_ram_mmap: mmap the specified file or device. +* +* Parameters: +* @fd: the file or the device to mmap +* @size: the number of bytes to be mmaped +* @align: if not zero, specify the alignment of the starting mapping address; +* otherwise, the alignment in use will be determined by QEMU. +* @shared: map has RAM_SHARED flag. +* @is_pmem: map has RAM_PMEM flag. +* +* Return: +* On success, return a pointer to the mapped area. +* On failure, return MAP_FAILED. +*/ +void *qemu_ram_mmap (int fd, +size_t size, +size_t align, +bool shared, +bool is_pmem);void qemu_ram_munmap (void *ptr,size_t size);
util/mmap-alloc.c +5 -1
@@ -75,7 +75,11 @@ size_t qemu_mempath_getpagesize(const char *mem_path)return getpagesize (); } -void *qemu_ram_mmap (int fd,size_t size,size_t align,bool shared) +void *qemu_ram_mmap (int fd, +size_t size, +size_t align, +bool shared, +bool is_pmem) {/* * Note: this always allocates at least one extra page of virtual address
util/oslib-posix.c +1 -1
@@ -203,7 +203,7 @@ void *qemu_memalign(size_t alignment, size_t size)void *qemu_anon_ram_alloc (size_t size,uint64_t *alignment,bool shared) {size_t align =QEMU_VMALLOC_ALIGN ; -void *ptr =qemu_ram_mmap (-1 , size, align, shared); +void *ptr =qemu_ram_mmap (-1 , size, align, shared, false);if (ptr ==MAP_FAILED ) {return NULL ;
Signature · 1 line
2.7.4
PATCH v13 2/5 scripts/update-linux-headers: add linux/mman.h
In reply to Zhang, YiFrom: Zhang Yi <[email protected]>
Add linux/mman.h,asm/mman.h,asm/mman-common.h to linux-headers, So we can use more mmap2 flags.
--- scripts/update-linux-headers.sh | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
scripts/update-linux-headers.sh +3 -3
@@ -95,7 +95,7 @@ for arch in $ARCHLIST; do rm -rf "$output/linux-headers/asm-$arch" mkdir -p "$output/linux-headers/asm-$arch" - for header in kvm.h unistd.h bitsperlong.h; do + for header in kvm.h unistd.h bitsperlong.h mman.h; do cp "$tmpdir/include/asm/$header" "$output/linux-headers/asm-$arch" done
@@ -126,13 +126,13 @@ done rm -rf "$output/linux-headers/linux" mkdir -p "$output/linux-headers/linux" for header in kvm.h vfio.h vfio_ccw.h vhost.h \ - psci.h psp-sev.h userfaultfd.h; do + psci.h psp-sev.h userfaultfd.h mman.h; do cp "$tmpdir/include/linux/$header" "$output/linux-headers/linux" done rm -rf "$output/linux-headers/asm-generic" mkdir -p "$output/linux-headers/asm-generic" -for header in unistd.h bitsperlong.h; do +for header in unistd.h bitsperlong.h mman-common.h mman.h hugetlb_encode.h; do cp "$tmpdir/include/asm-generic/$header" "$output/linux-headers/asm-generic" done
Signature · 1 line
2.7.4
PATCH v13 3/5 linux-headers: add linux/mman.h.
In reply to Zhang, YiFrom: Zhang Yi <[email protected]>
Update it to 4.20-rc1
--- linux-headers/asm-arm/mman.h | 4 ++ linux-headers/asm-arm64/mman.h | 1 + linux-headers/asm-generic/hugetlb_encode.h | 36 ++++++++++ linux-headers/asm-generic/mman-common.h | 77 ++++++++++++++++++++ linux-headers/asm-generic/mman.h | 24 +++++++ linux-headers/asm-mips/mman.h | 108 +++++++++++++++++++++++++++++ linux-headers/asm-powerpc/mman.h | 39 +++++++++++ linux-headers/asm-s390/mman.h | 1 + linux-headers/asm-x86/mman.h | 31 +++++++++ linux-headers/linux/mman.h | 38 ++++++++++ 10 files changed, 359 insertions(+) create mode 100644 linux-headers/asm-arm/mman.h create mode 100644 linux-headers/asm-arm64/mman.h create mode 100644 linux-headers/asm-generic/hugetlb_encode.h create mode 100644 linux-headers/asm-generic/mman-common.h create mode 100644 linux-headers/asm-generic/mman.h create mode 100644 linux-headers/asm-mips/mman.h create mode 100644 linux-headers/asm-powerpc/mman.h create mode 100644 linux-headers/asm-s390/mman.h create mode 100644 linux-headers/asm-x86/mman.h create mode 100644 linux-headers/linux/mman.h
linux-headers/asm-arm/mman.h new file +4 -0
@@ -0,0 +1,4 @@ +#include <asm-generic/mman.h> + +#define arch_mmap_check (addr, len, flags) \ + (((flags) & MAP_FIXED && (addr) < FIRST_USER_ADDRESS) ? -EINVAL : 0)
linux-headers/asm-arm64/mman.h new file +1 -0
@@ -0,0 +1 @@ +#include <asm-generic/mman.h>
linux-headers/asm-generic/hugetlb_encode.h new file +36 -0
@@ -0,0 +1,36 @@ +#ifndef _ASM_GENERIC_HUGETLB_ENCODE_H_ +#define _ASM_GENERIC_HUGETLB_ENCODE_H_ + +/* +* Several system calls take a flag to request "hugetlb" huge pages. +* Without further specification, these system calls will use the +* system's default huge page size. If a system supports multiple +* huge page sizes, the desired huge page size can be specified in +* bits [26:31] of the flag arguments. The value in these 6 bits +* will encode the log2 of the huge page size. +* +* The following definitions are associated with this huge page size +* encoding in flag arguments. System call specific header files +* that use this encoding should include this file. They can then +* provide definitions based on these with their own specific prefix. +* for example: +* #define MAP_HUGE_SHIFT HUGETLB_FLAG_ENCODE_SHIFT +*/ + +#define HUGETLB_FLAG_ENCODE_SHIFT 26 +#define HUGETLB_FLAG_ENCODE_MASK 0x3f + +#define HUGETLB_FLAG_ENCODE_64KB (16 << HUGETLB_FLAG_ENCODE_SHIFT) +#define HUGETLB_FLAG_ENCODE_512KB (19 << HUGETLB_FLAG_ENCODE_SHIFT) +#define HUGETLB_FLAG_ENCODE_1MB (20 << HUGETLB_FLAG_ENCODE_SHIFT) +#define HUGETLB_FLAG_ENCODE_2MB (21 << HUGETLB_FLAG_ENCODE_SHIFT) +#define HUGETLB_FLAG_ENCODE_8MB (23 << HUGETLB_FLAG_ENCODE_SHIFT) +#define HUGETLB_FLAG_ENCODE_16MB (24 << HUGETLB_FLAG_ENCODE_SHIFT) +#define HUGETLB_FLAG_ENCODE_32MB (25 << HUGETLB_FLAG_ENCODE_SHIFT) +#define HUGETLB_FLAG_ENCODE_256MB (28 << HUGETLB_FLAG_ENCODE_SHIFT) +#define HUGETLB_FLAG_ENCODE_512MB (29 << HUGETLB_FLAG_ENCODE_SHIFT) +#define HUGETLB_FLAG_ENCODE_1GB (30 << HUGETLB_FLAG_ENCODE_SHIFT) +#define HUGETLB_FLAG_ENCODE_2GB (31 << HUGETLB_FLAG_ENCODE_SHIFT) +#define HUGETLB_FLAG_ENCODE_16GB (34 << HUGETLB_FLAG_ENCODE_SHIFT) + +#endif /* _ASM_GENERIC_HUGETLB_ENCODE_H_ */
linux-headers/asm-generic/mman-common.h new file +77 -0
77 lines not on this page · raw message
linux-headers/asm-generic/mman.h new file +24 -0
@@ -0,0 +1,24 @@ +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */ +#ifndef __ASM_GENERIC_MMAN_H +#define __ASM_GENERIC_MMAN_H + +#include <asm-generic/mman-common.h> + +#define MAP_GROWSDOWN 0x0100/* stack-like segment */ +#define MAP_DENYWRITE 0x0800/* ETXTBSY */ +#define MAP_EXECUTABLE 0x1000/* mark it as an executable */ +#define MAP_LOCKED 0x2000/* pages are locked */ +#define MAP_NORESERVE 0x4000/* don't check for reservations */ +#define MAP_POPULATE 0x8000/* populate (prefault) pagetables */ +#define MAP_NONBLOCK 0x10000/* do not block on IO */ +#define MAP_STACK 0x20000/* give out an address that is best suited for process/thread stacks */ +#define MAP_HUGETLB 0x40000/* create a huge page mapping */ +#define MAP_SYNC 0x80000/* perform synchronous page faults for the mapping */ + +/* Bits [26:31] are reserved, see mman-common.h for MAP_HUGETLB usage */ + +#define MCL_CURRENT 1/* lock all current mappings */ +#define MCL_FUTURE 2/* lock all future mappings */ +#define MCL_ONFAULT 4/* lock all pages that are faulted in */ + +#endif /* __ASM_GENERIC_MMAN_H */
linux-headers/asm-mips/mman.h new file +108 -0
108 lines not on this page · raw message
linux-headers/asm-powerpc/mman.h new file +39 -0
@@ -0,0 +1,39 @@ +/* SPDX-License-Identifier: GPL-2.0+ WITH Linux-syscall-note */ +/* +* This program is free software; you can redistribute it and/or +* modify it under the terms of the GNU General Public License +* as published by the Free Software Foundation; either version +* 2 of the License, or (at your option) any later version. +*/ +#ifndef _ASM_POWERPC_MMAN_H +#define _ASM_POWERPC_MMAN_H + +#include <asm-generic/mman-common.h> + + +#define PROT_SAO 0x10/* Strong Access Ordering */ + +#define MAP_RENAME MAP_ANONYMOUS/* In SunOS terminology */ +#define MAP_NORESERVE 0x40/* don't reserve swap pages */ +#define MAP_LOCKED 0x80 + +#define MAP_GROWSDOWN 0x0100/* stack-like segment */ +#define MAP_DENYWRITE 0x0800/* ETXTBSY */ +#define MAP_EXECUTABLE 0x1000/* mark it as an executable */ + +#define MCL_CURRENT 0x2000/* lock all currently mapped pages */ +#define MCL_FUTURE 0x4000/* lock all additions to address space */ +#define MCL_ONFAULT 0x8000/* lock all pages that are faulted in */ + +#define MAP_POPULATE 0x8000/* populate (prefault) pagetables */ +#define MAP_NONBLOCK 0x10000/* do not block on IO */ +#define MAP_STACK 0x20000/* give out an address that is best suited for process/thread stacks */ +#define MAP_HUGETLB 0x40000/* create a huge page mapping */ + +/* Override any generic PKEY permission defines */ +#define PKEY_DISABLE_EXECUTE 0x4 +#undef PKEY_ACCESS_MASK +#define PKEY_ACCESS_MASK (PKEY_DISABLE_ACCESS |\ + PKEY_DISABLE_WRITE |\ + PKEY_DISABLE_EXECUTE) +#endif /* _ASM_POWERPC_MMAN_H */
linux-headers/asm-s390/mman.h new file +1 -0
@@ -0,0 +1 @@ +#include <asm-generic/mman.h>
linux-headers/asm-x86/mman.h new file +31 -0
@@ -0,0 +1,31 @@ +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */ +#ifndef _ASM_X86_MMAN_H +#define _ASM_X86_MMAN_H + +#define MAP_32BIT 0x40/* only give out 32bit addresses */ + +#ifdef CONFIG_X86_INTEL_MEMORY_PROTECTION_KEYS +/* +* Take the 4 protection key bits out of the vma->vm_flags +* value and turn them in to the bits that we can put in +* to a pte. +* +* Only override these if Protection Keys are available +* (which is only on 64-bit). +*/ +#define arch_vm_get_page_prot (vm_flags) __pgprot( \ + ((vm_flags) & VM_PKEY_BIT0 ? _PAGE_PKEY_BIT0 : 0) | \ + ((vm_flags) & VM_PKEY_BIT1 ? _PAGE_PKEY_BIT1 : 0) | \ + ((vm_flags) & VM_PKEY_BIT2 ? _PAGE_PKEY_BIT2 : 0) | \ + ((vm_flags) & VM_PKEY_BIT3 ? _PAGE_PKEY_BIT3 : 0)) + +#define arch_calc_vm_prot_bits (prot, key) ( \ + ((key) & 0x1 ? VM_PKEY_BIT0 : 0) | \ + ((key) & 0x2 ? VM_PKEY_BIT1 : 0) | \ + ((key) & 0x4 ? VM_PKEY_BIT2 : 0) | \ + ((key) & 0x8 ? VM_PKEY_BIT3 : 0)) +#endif + +#include <asm-generic/mman.h> + +#endif /* _ASM_X86_MMAN_H */
linux-headers/linux/mman.h new file +38 -0
@@ -0,0 +1,38 @@ +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */ +#ifndef _LINUX_MMAN_H +#define _LINUX_MMAN_H + +#include <asm/mman.h> +#include <asm-generic/hugetlb_encode.h> + +#define MREMAP_MAYMOVE 1 +#define MREMAP_FIXED 2 + +#define OVERCOMMIT_GUESS 0 +#define OVERCOMMIT_ALWAYS 1 +#define OVERCOMMIT_NEVER 2 + +/* +* Huge page size encoding when MAP_HUGETLB is specified, and a huge page +* size other than the default is desired. See hugetlb_encode.h. +* All known huge page size encodings are provided here. It is the +* responsibility of the application to know which sizes are supported on +* the running system. See mmap(2) man page for details. +*/ +#define MAP_HUGE_SHIFT HUGETLB_FLAG_ENCODE_SHIFT +#define MAP_HUGE_MASK HUGETLB_FLAG_ENCODE_MASK + +#define MAP_HUGE_64KB HUGETLB_FLAG_ENCODE_64KB +#define MAP_HUGE_512KB HUGETLB_FLAG_ENCODE_512KB +#define MAP_HUGE_1MB HUGETLB_FLAG_ENCODE_1MB +#define MAP_HUGE_2MB HUGETLB_FLAG_ENCODE_2MB +#define MAP_HUGE_8MB HUGETLB_FLAG_ENCODE_8MB +#define MAP_HUGE_16MB HUGETLB_FLAG_ENCODE_16MB +#define MAP_HUGE_32MB HUGETLB_FLAG_ENCODE_32MB +#define MAP_HUGE_256MB HUGETLB_FLAG_ENCODE_256MB +#define MAP_HUGE_512MB HUGETLB_FLAG_ENCODE_512MB +#define MAP_HUGE_1GB HUGETLB_FLAG_ENCODE_1GB +#define MAP_HUGE_2GB HUGETLB_FLAG_ENCODE_2GB +#define MAP_HUGE_16GB HUGETLB_FLAG_ENCODE_16GB + +#endif /* _LINUX_MMAN_H */
Signature · 1 line
2.7.4
PATCH v13 4/5 util/mmap-alloc: support MAP_SYNC in qemu_ram_mmap()
In reply to Zhang, YiFrom: Zhang Yi <[email protected]>
When a file supporting DAX is used as vNVDIMM backend, mmap it with MAP_SYNC flag in addition which can ensure file system metadata synced in each guest writes to the backend file, without other QEMU actions (e.g., periodic fsync() by QEMU).
Current, We have below different possible use cases:
1. pmem=on is set, shared=on is set, MAP_SYNC supported:
a: backend is a dax supporting file.
- MAP_SYNC will active.
b: backend is not a dax supporting file.
- mmap will trigger a warning. then MAP_SYNC flag will be ignored
2. The rest of cases:
- we will never pass the MAP_SYNC to mmap2
--- util/mmap-alloc.c | 45 ++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 44 insertions(+), 1 deletion(-)
util/mmap-alloc.c +44 -1
@@ -10,6 +10,13 @@* later. See the COPYING file in the top-level directory. */ +#ifdef CONFIG_LINUX +#include <linux/mman.h> +#else /* !CONFIG_LINUX */ +#define MAP_SYNC 0x0 +#define MAP_SHARED_VALIDATE 0x0 +#endif /* CONFIG_LINUX */ +#include "qemu/osdep.h" #include "qemu/mmap-alloc.h" #include "qemu/host-utils.h"
@@ -101,6 +108,8 @@ void *qemu_ram_mmap(int fd,#else void *ptr =mmap (0 , total,PROT_NONE ,MAP_ANONYMOUS |MAP_PRIVATE ,-1 ,0 );#endif +int mmap_flags; +int map_sync_flags =0 ;size_t offset;void *ptr1;
@@ -111,13 +120,47 @@ void *qemu_ram_mmap(int fd,assert (is_power_of_2 (align));/* Always align to host page size */ assert (align >=getpagesize ()); + mmap_flags = shared ?MAP_SHARED :MAP_PRIVATE ; +if (shared && is_pmem) { + map_sync_flags =MAP_SYNC |MAP_SHARED_VALIDATE ; + mmap_flags |= map_sync_flags; + } offset =QEMU_ALIGN_UP ((uintptr_t )ptr, align) - (uintptr_t )ptr; ptr1 =mmap (ptr + offset, size,PROT_READ |PROT_WRITE ,MAP_FIXED | (fd ==-1 ?MAP_ANONYMOUS :0 ) | - (shared ?MAP_SHARED :MAP_PRIVATE ), + mmap_flags, fd,0 ); + + +if (ptr1 ==MAP_FAILED && map_sync_flags) { +if (errno ==ENOTSUP ) { +char *proc_link, *file_name; +int len; + proc_link =g_strdup_printf ("/proc/self/fd/%d" , fd); + file_name =g_malloc0 (PATH_MAX ); + len =readlink (proc_link, file_name,PATH_MAX -1 ); +if (len <0 ) { + len =0 ; + } + file_name[len] ='\0' ; +fprintf (stderr,"Warning: requesting persistence across crashes " +"for backend file %s failed. Proceeding without " +"persistence, data might become corrupted in case of host " +"crash.\n" , file_name); +g_free (proc_link); +g_free (file_name); + } +/* if map failed with MAP_SHARED_VALIDATE | MAP_SYNC, +* we will remove these flags to handle compatibility. +*/ + ptr1 =mmap (ptr + offset, size,PROT_READ |PROT_WRITE , +MAP_FIXED | + (fd ==-1 ?MAP_ANONYMOUS :0 ) | +MAP_SHARED , + fd,0 ); + }if (ptr1 ==MAP_FAILED ) {munmap (ptr, total);return MAP_FAILED ;
Signature · 1 line
2.7.4
Re: [Qemu-devel] [PATCH V13 4/5] util/mmap-alloc: support MAP_SYNC in qemu_ram_mmap()
Hi,
I found out that this series missed QEMU 4.0 and I was going to queue for 4.1, but unfortunately this patch conflicts with:
commit 2044c3e7116eeac0449dcb4a4130cc8f8b9310da
Author: Murilo Opsfelder Araujo <[email protected]> Date: Wed Jan 30 21:36:04 2019 -0200
mmap-alloc: unfold qemu_ram_mmap()
Unfold parts of qemu_ram_mmap() for the sake of understanding, moving declarations to the top, and keeping architecture-specifics in the ifdef-else blocks. No changes in the function behaviour.
Give ptr and ptr1 meaningful names:
ptr -> guardptr : pointer to the PROT_NONE guard region
ptr1 -> ptr : pointer to the mapped memory returned to caller
I'm queueing patches 1-3 into machine-next[1], so only patches 4 and 5 need to be refreshed and resubmitted.
[1] https://github.com/ehabkost/qemu.git machine-next
Quoted 104 lines · Yi
From: Zhang Yi <[email protected]>
When a file supporting DAX is used as vNVDIMM backend, mmap it with
MAP_SYNCflag in addition which can ensure file system metadata synced in each guest writes to the backend file, without other QEMU actions (e.g., periodicfsync()by QEMU).Current, We have below different possible use cases:
1. pmem=on is set, shared=on is set,
MAP_SYNCsupported:
a: backend is a dax supporting file.
-MAP_SYNCwill active.
b: backend is not a dax supporting file.
- mmap will trigger a warning. thenMAP_SYNCflag will be ignored2. The rest of cases:
- we will never pass theMAP_SYNCto mmap2Signed-off-by Haozhong Zhang <[email protected]>Signed-off-by Zhang Yi <[email protected]>--- util/mmap-alloc.c | 45 ++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 44 insertions(+), 1 deletion(-)util/mmap-alloc.c +44
-1@@ -10,6 +10,13 @@* later. See the COPYING file in the top-level directory. */ +#ifdef CONFIG_LINUX +#include <linux/mman.h> +#else /* !CONFIG_LINUX */ +#define MAP_SYNC 0x0 +#define MAP_SHARED_VALIDATE 0x0 +#endif /* CONFIG_LINUX */ +#include "qemu/osdep.h" #include "qemu/mmap-alloc.h" #include "qemu/host-utils.h" @@ -101,6 +108,8 @@ void *qemu_ram_mmap(int fd,#else void *ptr =mmap (0 , total,PROT_NONE ,MAP_ANONYMOUS |MAP_PRIVATE ,-1 ,0 );#endif +int mmap_flags; +int map_sync_flags =0 ;size_t offset;void *ptr1;@@ -111,13 +120,47 @@ void *qemu_ram_mmap(int fd,assert (is_power_of_2 (align));/* Always align to host page size */ assert (align >=getpagesize ()); + mmap_flags = shared ?MAP_SHARED :MAP_PRIVATE ; +if (shared && is_pmem) { + map_sync_flags =MAP_SYNC |MAP_SHARED_VALIDATE ; + mmap_flags |= map_sync_flags; + } offset =QEMU_ALIGN_UP ((uintptr_t )ptr, align) - (uintptr_t )ptr; ptr1 =mmap (ptr + offset, size,PROT_READ |PROT_WRITE ,MAP_FIXED | (fd ==-1 ?MAP_ANONYMOUS :0 ) | - (shared ?MAP_SHARED :MAP_PRIVATE ), + mmap_flags, fd,0 ); + + +if (ptr1 ==MAP_FAILED && map_sync_flags) { +if (errno ==ENOTSUP ) { +char *proc_link, *file_name; +int len; + proc_link =g_strdup_printf ("/proc/self/fd/%d" , fd); + file_name =g_malloc0 (PATH_MAX ); + len =readlink (proc_link, file_name,PATH_MAX -1 ); +if (len <0 ) { + len =0 ; + } + file_name[len] ='\0' ; +fprintf (stderr,"Warning: requesting persistence across crashes " +"for backend file %s failed. Proceeding without " +"persistence, data might become corrupted in case of host " +"crash.\n" , file_name); +g_free (proc_link); +g_free (file_name); + } +/* if map failed with MAP_SHARED_VALIDATE | MAP_SYNC, +* we will remove these flags to handle compatibility. +*/ + ptr1 =mmap (ptr + offset, size,PROT_READ |PROT_WRITE , +MAP_FIXED | + (fd ==-1 ?MAP_ANONYMOUS :0 ) | +MAP_SHARED , + fd,0 ); + }if (ptr1 ==MAP_FAILED ) {munmap (ptr, total);return MAP_FAILED ;Signature · 1 line
2.7.4
Signature · 1 line
Eduardo
Re: [Qemu-devel] [PATCH V13 4/5] util/mmap-alloc: support MAP_SYNC in qemu_ram_mmap()
Quoted 10 lines · Eduardo Habkost
Hi,
I found out that this series missed QEMU 4.0 and I was going to queue for 4.1, but unfortunately this patch conflicts with:
commit
2044c3e7116eeac0449dcb4a4130cc8f8b9310daAuthor: Murilo Opsfelder Araujo <[email protected]> Date: Wed Jan 30 21:36:04 2019 -0200mmap-alloc: unfold
qemu_ram_mmap()
[...]
Quoted 16 lines · Yi
Quoted 15 lines
From: Zhang Yi <[email protected]>
When a file supporting DAX is used as vNVDIMM backend, mmap it with
MAP_SYNCflag in addition which can ensure file system metadata synced in each guest writes to the backend file, without other QEMU actions (e.g., periodicfsync()by QEMU).Current, We have below different possible use cases:
1. pmem=on is set, shared=on is set,
MAP_SYNCsupported:
a: backend is a dax supporting file.
-MAP_SYNCwill active.
b: backend is not a dax supporting file.
- mmap will trigger a warning. thenMAP_SYNCflag will be ignored
Quoted 5 lines
Quoted 5 lines
2. The rest of cases:
- we will never pass theMAP_SYNCto mmap2Signed-off-by Haozhong Zhang <[email protected]>Signed-off-by Zhang Yi <[email protected]>
Untested patch rebase is below. Can Intel help test it and submit v14?
--- util/mmap-alloc.c | 41 +++++++++++++++++++++++++++++++++++++++-- 1 file changed, 39 insertions(+), 2 deletions(-)
util/mmap-alloc.c +39 -2
@@ -10,6 +10,13 @@* later. See the COPYING file in the top-level directory. */ +#ifdef CONFIG_LINUX +#include <linux/mman.h> +#else /* !CONFIG_LINUX */ +#define MAP_SYNC 0x0 +#define MAP_SHARED_VALIDATE 0x0 +#endif /* CONFIG_LINUX */ +#include "qemu/osdep.h" #include "qemu/mmap-alloc.h" #include "qemu/host-utils.h"
@@ -82,6 +89,7 @@ void *qemu_ram_mmap(int fd, bool is_pmem) {int flags; +int map_sync_flags =0 ;int guardfd;size_t offset;size_t pagesize;
@@ -132,10 +140,39 @@ void *qemu_ram_mmap(int fd, flags =MAP_FIXED ; flags |= fd ==-1 ?MAP_ANONYMOUS :0 ; flags |= shared ?MAP_SHARED :MAP_PRIVATE ; +if (shared && is_pmem) { + map_sync_flags =MAP_SYNC |MAP_SHARED_VALIDATE ; + } offset =QEMU_ALIGN_UP ((uintptr_t )guardptr, align) - (uintptr_t )guardptr; - ptr =mmap (guardptr + offset, size,PROT_READ |PROT_WRITE , flags, fd,0 ); - + ptr =mmap (guardptr + offset, size,PROT_READ |PROT_WRITE , + flags | map_sync_flags, fd,0 ); + +if (ptr ==MAP_FAILED && map_sync_flags) { +if (errno ==ENOTSUP ) { +char *proc_link, *file_name; +int len; + proc_link =g_strdup_printf ("/proc/self/fd/%d" , fd); + file_name =g_malloc0 (PATH_MAX ); + len =readlink (proc_link, file_name,PATH_MAX -1 ); +if (len <0 ) { + len =0 ; + } + file_name[len] ='\0' ; +fprintf (stderr,"Warning: requesting persistence across crashes " +"for backend file %s failed. Proceeding without " +"persistence, data might become corrupted in case of host " +"crash.\n" , file_name); +g_free (proc_link); +g_free (file_name); + } +/* +* if map failed with MAP_SHARED_VALIDATE | MAP_SYNC, +* we will remove these flags to handle compatibility. +*/ + ptr =mmap (guardptr + offset, size,PROT_READ |PROT_WRITE , + flags, fd,0 ); + }if (ptr ==MAP_FAILED ) {munmap (guardptr, total);return MAP_FAILED ;
Signature · 1 line
2.18.0.rc1.1.g3f1ff2140
Re: [Qemu-devel] [PATCH V13 4/5] util/mmap-alloc: support MAP_SYNC in qemu_ram_mmap()
Quoted 28 lines · Eduardo Habkost
Quoted 10 lines · Eduardo Habkost
Hi,
I found out that this series missed QEMU 4.0 and I was going to queue for 4.1, but unfortunately this patch conflicts with:
commit
2044c3e7116eeac0449dcb4a4130cc8f8b9310daAuthor: Murilo Opsfelder Araujo <[email protected]> Date: Wed Jan 30 21:36:04 2019 -0200mmap-alloc: unfold
qemu_ram_mmap()[...]
Quoted 16 lines · Yi
Quoted 15 lines
From: Zhang Yi <[email protected]>
When a file supporting DAX is used as vNVDIMM backend, mmap it with
MAP_SYNCflag in addition which can ensure file system metadata synced in each guest writes to the backend file, without other QEMU actions (e.g., periodicfsync()by QEMU).Current, We have below different possible use cases:
1. pmem=on is set, shared=on is set,
MAP_SYNCsupported:
a: backend is a dax supporting file.
-MAP_SYNCwill active.
b: backend is not a dax supporting file.
- mmap will trigger a warning. thenMAP_SYNCflag will be ignored
Quoted 9 lines
Quoted 5 lines
Quoted 5 lines
2. The rest of cases:
- we will never pass theMAP_SYNCto mmap2Signed-off-by Haozhong Zhang <[email protected]>Signed-off-by Zhang Yi <[email protected]>Untested patch rebase is below. Can Intel help test it and submit v14?
Thanks Eduardo, the patch looks good to me and I have tested it.
Signature · 2 lines
Wei Yang Help you, Help me
PATCH v13 5/5 docs: Added MAP_SYNC documentation
In reply to Zhang, YiFrom: Zhang Yi <[email protected]>
--- docs/nvdimm.txt | 25 ++++++++++++++++++++++--- qemu-options.hx | 5 +++++ 2 files changed, 27 insertions(+), 3 deletions(-)
docs/nvdimm.txt +22 -3
@@ -143,9 +143,28 @@ Guest Data Persistence ---------------------- Though QEMU supports multiple types of vNVDIMM backends on Linux, -currently the only one that can guarantee the guest write persistence -is the device DAX on the real NVDIMM device (e.g., /dev/dax0.0), to -which all guest access do not involve any host-side kernel cache. +the only backend that can guarantee the guest write persistence is: + +A. DAX device (e.g., /dev/dax0.0, ) or +B. DAX file(mounted with dax option) + +both are use real NVDIMM device as backend, which supporting direct +access for files(no page cache). + +When using B (A file supporting direct mapping of persistent memory) +as a backend, write persistence is guaranteed if the host kernel has +support for the MAP_SYNC flag in the mmap system call (available +since Linux 4.15 and on certain distro kernels) and additionally +both 'pmem' and 'share' flags are set to 'on' on the backend. + +If these conditions are not satisfied i.e. if either 'pmem' or 'share' +are not set, if the backend file does not support DAX or if MAP_SYNC +is not supported by the host kernel, write persistence is not +guaranteed after a system crash. For compatibility reasons, these +conditions are silently ignored if not satisfied. Currently, no way +is provided to test for them. +For more details, please reference mmap(2) man page: +http://man7.org/linux/man-pages/man2/mmap.2.html. When using other types of backends, it's suggested to set 'unarmed' option of '-device nvdimm' to 'on', which sets the unarmed flag of the
qemu-options.hx +5 -0
@@ -4002,6 +4002,11 @@ using the SNIA NVM programming model (e.g. Intel NVDIMM). If @option{pmem} is set to 'on', QEMU will take necessary operations to guarantee the persistence of its own writes to @option{mem-path} (e.g. in vNVDIMM label emulation and live migration). +Also, we will map the backend-file with MAP_SYNC flag, which ensures the +file metadata is in sync for @option{mem-path} in case of host crash +or a power failure. MAP_SYNC requires support from both the host kernel +(since Linux kernel 4.15) and the filesystem of @option{mem-path} mounted +with DAX option. @item -object memory-backend-ram,id=@var{id},merge=@var{on|off},dump=@var{on|off},share=@var{on|off},prealloc=@var{on|off},size=@var{size},host-nodes=@var{host-nodes},policy=@var{default|preferred|bind|interleave}
Signature · 1 line
2.7.4
Re: [Qemu-devel] [PATCH V13 5/5] docs: Added MAP_SYNC documentation
Quoted 16 lines · Yi
From: Zhang Yi <[email protected]>
Signed-off-by Zhang Yi <[email protected]>--- docs/nvdimm.txt | 25 ++++++++++++++++++++++--- qemu-options.hx | 5 +++++ 2 files changed, 27 insertions(+), 3 deletions(-)docs/nvdimm.txt +0
-0@@ -143,9 +143,28 @@ Guest Data Persistence ---------------------- Though QEMU supports multiple types of vNVDIMM backends on Linux,
docs/nvdimm.txt Guest Data Persistence · line 152
Quoted 10 lines
docs/nvdimm.txt +7
-3-currently the only one that can guarantee the guest write persistence -is the device DAX on the real NVDIMM device (e.g., /dev/dax0.0), to -which all guest access do not involve any host-side kernel cache. +the only backend that can guarantee the guest write persistence is: + +A. DAX device (e.g., /dev/dax0.0, ) or +B. DAX file(mounted with dax option) + +both are use real NVDIMM device as backend, which supporting direct +access for files(no page cache).
So I would just drop above two lines.
How does one know that there's support for direct access? it's just a question of kernel right?
And I don't think it matter wrt "real NVDIMM" - e.g. it could be vNVDIMM in a nested virt scenario.
It will still survive host (i.e. L1 guest) crash.
Besides, the above two lines aren't grammatical.
Quoted 36 lines
docs/nvdimm.txt +15
-0+ +When using B (A file supporting direct mapping of persistent memory) +as a backend, write persistence is guaranteed if the host kernel has +support for the MAP_SYNC flag in the mmap system call (available +since Linux 4.15 and on certain distro kernels) and additionally +both 'pmem' and 'share' flags are set to 'on' on the backend. + +If these conditions are not satisfied i.e. if either 'pmem' or 'share' +are not set, if the backend file does not support DAX or if MAP_SYNC +is not supported by the host kernel, write persistence is not +guaranteed after a system crash. For compatibility reasons, these +conditions are silently ignored if not satisfied. Currently, no way +is provided to test for them. +For more details, please reference mmap(2) man page: +http://man7.org/linux/man-pages/man2/mmap.2.html. When using other types of backends, it's suggested to set 'unarmed' option of '-device nvdimm' to 'on', which sets the unarmed flag of theqemu-options.hx +5
-0@@ -4002,6 +4002,11 @@ using the SNIA NVM programming model (e.g. Intel NVDIMM). If @option{pmem} is set to 'on', QEMU will take necessary operations to guarantee the persistence of its own writes to @option{mem-path} (e.g. in vNVDIMM label emulation and live migration). +Also, we will map the backend-file with MAP_SYNC flag, which ensures the +file metadata is in sync for @option{mem-path} in case of host crash +or a power failure. MAP_SYNC requires support from both the host kernel +(since Linux kernel 4.15) and the filesystem of @option{mem-path} mounted +with DAX option. @item -object memory-backend-ram,id=@var{id},merge=@var{on|off},dump=@var{on|off},share=@var{on|off},prealloc=@var{on|off},size=@var{size},host-nodes=@var{host-nodes},policy=@var{default|preferred|bind|interleave}Signature · 1 line
2.7.4
Re: [Qemu-devel] [PATCH V13 5/5] docs: Added MAP_SYNC documentation
Quoted 30 lines · Michael S. Tsirkin
Quoted 16 lines · Yi
From: Zhang Yi <[email protected]>
Signed-off-by Zhang Yi <[email protected]>--- docs/nvdimm.txt | 25 ++++++++++++++++++++++--- qemu-options.hx | 5 +++++ 2 files changed, 27 insertions(+), 3 deletions(-)docs/nvdimm.txt +0
-0@@ -143,9 +143,28 @@ Guest Data Persistence ---------------------- Though QEMU supports multiple types of vNVDIMM backends on Linux,docs/nvdimm.txt Guest Data Persistence · line 152
Quoted 10 lines
docs/nvdimm.txt +7
-3-currently the only one that can guarantee the guest write persistence -is the device DAX on the real NVDIMM device (e.g., /dev/dax0.0), to -which all guest access do not involve any host-side kernel cache. +the only backend that can guarantee the guest write persistence is: + +A. DAX device (e.g., /dev/dax0.0, ) or +B. DAX file(mounted with dax option) + +both are use real NVDIMM device as backend, which supporting direct +access for files(no page cache).So I would just drop above two lines.
Quoted 8 lines
How does one know that there's support for direct access? it's just a question of kernel right?
And I don't think it matter wrt "real NVDIMM" - e.g. it could be vNVDIMM in a nested virt scenario.
It will still survive host (i.e. L1 guest) crash.Besides, the above two lines aren't grammatical.
Ah.. Reasonable, will drop these two lines.
Thanks for your kindly review. Michael.
Quoted 37 lines
Quoted 37 lines
docs/nvdimm.txt +15
-0+ +When using B (A file supporting direct mapping of persistent memory) +as a backend, write persistence is guaranteed if the host kernel has +support for the MAP_SYNC flag in the mmap system call (available +since Linux 4.15 and on certain distro kernels) and additionally +both 'pmem' and 'share' flags are set to 'on' on the backend. + +If these conditions are not satisfied i.e. if either 'pmem' or 'share' +are not set, if the backend file does not support DAX or if MAP_SYNC +is not supported by the host kernel, write persistence is not +guaranteed after a system crash. For compatibility reasons, these +conditions are silently ignored if not satisfied. Currently, no way +is provided to test for them. +For more details, please reference mmap(2) man page: +http://man7.org/linux/man-pages/man2/mmap.2.html. When using other types of backends, it's suggested to set 'unarmed' option of '-device nvdimm' to 'on', which sets the unarmed flag of theqemu-options.hx +5
-0@@ -4002,6 +4002,11 @@ using the SNIA NVM programming model (e.g. Intel NVDIMM). If @option{pmem} is set to 'on', QEMU will take necessary operations to guarantee the persistence of its own writes to @option{mem-path} (e.g. in vNVDIMM label emulation and live migration). +Also, we will map the backend-file with MAP_SYNC flag, which ensures the +file metadata is in sync for @option{mem-path} in case of host crash +or a power failure. MAP_SYNC requires support from both the host kernel +(since Linux kernel 4.15) and the filesystem of @option{mem-path} mounted +with DAX option. @item -object memory-backend-ram,id=@var{id},merge=@var{on|off},dump=@var{on|off},share=@var{on|off},prealloc=@var{on|off},size=@var{size},host-nodes=@var{host-nodes},policy=@var{default|preferred|bind|interleave}Signature · 1 line
2.7.4
Quoted 11 lines · Yi
Linux 4.15 introduces a new mmap flag
MAP_SYNC, which can be used to guarantee the write persistence to mmap'ed files supporting DAX (e.g., files on ext4/xfs file system mounted with '-o dax').A description of
MAP_SYNCandMAP_SHARED_VALIDATEcan be found athttps://patchwork.kernel.org/patch/10028151/In order to make sure that the file metadata is in sync after a fault while we are writing a shared DAX supporting backend files, this patch-set enables QEMU to use
MAP_SYNCflag for memory-backend-dax-file.
Quoted 6 lines
As the DAX vs DMA truncated issue was solved, we refined the code and send out this feature for the v5 version.
We will pass
MAP_SYNCto mmap(2); ifMAP_SYNCis supported and 'share=on' & 'pmem=on'.
Or QEMU will not pass this flag to mmap(2)
Overall this looks reasonable to me.
Besides the documentation nit:
Quoted 162 lines
Test with below cases:
1. pmem=on is set, shared=on is set,MAP_SYNCsupported:
a: backend is a dax supporting file.
1) start VM1 with options:
-object memory-backend-file,id=nv_be4,share,mem-path=${DAX_FILE_1},size=${DAX_FILE_SIZE_1},align=128M,pmem=on,share=on -device nvdimm,id=nv4,memdev=nv_be4,label-size=2M.2) start VM2 with options:
-object memory-backend-file,id=nv_be4,share,mem-path=${DAX_FILE_2,size=${DAX_FILE_SIZE_2},align=128M,pmem=on,share=on -device nvdimm,id=nv4,memdev=nv_be4,label-size=2M.3) live migrate from VM1 to VM2.
4) Suddly let Host crash or power failure.
5) check
DAX_FILE_1andDAX_FILE_2, no corrupt.b: backend is a regular file.
1) start with options
-object memory-backend-file,id=nv_be4,share,mem-path=${REG_FILE},size=${REG_FILE_SIZE},align=128M,pmem=on,share=on -device nvdimm,id=nv4,memdev=nv_be4,label-size=2M.will warning "failed to validate with mapping flags: Operation not supported"
FILE_1andFILE_2random corrupt.2. Other cases:
FILE_1andFILE_2random corrupt.Changes in V13:
* 4/5 Micheal: move the inlcude tommap_alloc.c.
* 4/5 Micheal: refine the warning message.
* 5/5 Micheal: refine the Documentations.Changes in V12:
* 2/5: Micheal: Updateupdate-linux-headers.sh
* 3/5: Micheal: Use script update addlinux/mman.h
* 4/5: Pankaj,Micheal: 1) fallback to mmap withoutMAP_SYNC&MAP_SHARED_VALIDATEif sync not supported or failed
2) Replace the include with 3/5 addedlinux/mman.h
* 5/5: Micheal: Refine the Documentations.Changes in V11:
* 1/3: Micheal: Change to just add a boolis_pmeminqemu_ram_mmap.
* 2/3: Micheal: Fix the compatibility for old kernel.
* 2/3&3/3: Micheal&Eduardo :Update the behavior below:
Waning at no-dax and continue withoutMAP_SYNC.
Test if fails again for compatibility, then remove theMAP_VALIDATEand silently proceed.Changes in V10:
* 4/4: refine the document.
* 3/4: Reviewed-by: Stefano Garzarella <[email protected]>
* 2/4: refine the commit message, AddedMAP_SHARED_VALIDATE.
* 2/4: Fix the wrong include headerChanges in V9:
* 1/6: Reviewed-by: Eduardo Habkost <[email protected]>
* 2/6: New Added: Micheal: use sparse feature defineRAM_FLAG. since I don't have much knowledge about the sparse feature, @Micheal Could you add some documentation/commit message on this patch? Thank you very much.
* 3/6: from 2/5: Eduardo: updated the commit message.
* 4/6: from 3/5: Micheal: don't ignoreMAP_SYNCfailures silently.
* 5/6: from 4/5: Eduardo: updated the commit message.
* 6/6: from 5/5: Micheal: Drop the sync option, document theMAP_SYNC.Changes in v8:
* Micheal: 3/5, remove the duplicated define in theos_dep.h
* Micheal: 2/5, make type define safety.
* Micheal: 2/5, fixed the incorrect defineMAP_SHAREonqemu_anon_ram_alloc.
* 4/6 removed, we remove the on/off/auto define of sync, as by now,MAP_SYNConly worked with pmem=on.
* @Micheal, I still reuse theRAM_SYNCflag, it is much straightforward to parse all the flags in one parameter.Changes in v7:
* Micheal: [3,4,6]/6 limited the "sync" flag only on a nvdimm backend.(pmem=on)Changes in v6:
* Pankaj: 3/7 are squashed with 2/7
* Pankaj: 7/7 update comments to "consistent filesystem metadata".
* Pankaj, Igor: 1/7 Added Reviewed-by in patch-1/7
* Stefan, 4/7 move the include header from "/linux/mman.h" to "osdep.h"
* Stefan, 5/7 Add missing "munmap"
* Stefan, 2/7 refine the shared/flag.Changes in v5:
* Add patch 1 to fix a memory leak issue.
* Refine the patch 4-6
* Remove the patch 3 as we already change the parameter from "shared" to "flags"Changes in v4:
* Add patch 1-3 to switch some functions to a single 'flags' parameters. (Michael S. Tsirkin)
* v3 patch 1-3 become v4 patch 4-6.
* Patch 4: move definitions ofMAP_SYNCandMAP_SHARED_VALIDATEto a new header file underinclude/standard-headers/linux/. (Michael S. Tsirkin)
* Patch 6: refine the description of the 'sync' option. (Michael S. Tsirkin)Changes in v3:
* Patch 1: addMAP_SHARED_VALIDATEin both sync=on and sync=auto cases, and add back the retry mechanism.MAP_SYNCwill be ignored by Linux kernel 4.15 ifMAP_SHARED_VALIDATEis missed.
* Patch 1: defineMAP_SYNCandMAP_SHARED_VALIDATEas 0 on non-Linux platforms in order to makeqemu_ram_mmap()compile on those platforms.
* Patch 2&3: include more information in error messages of memory-backend in hope to help user to identify the error. (Dr. David Alan Gilbert)
* Patch 3: fix typo in the commit message. (Dr. David Alan Gilbert)Changes in v2:
* Add 'sync' option to control the use ofMAP_SYNC. (Eduardo Habkost)
* Remove the unnecessary set ofMAP_SHARED_VALIDATEin some cases and the retry mechanism inqemu_ram_mmap(). (Michael S. Tsirkin)
* Move OS dependent definitions ofMAP_SYNCandMAP_SHARED_VALIDATEtoosdep.h. (Michael S. Tsirkin)Zhang Yi (5):
util/mmap-alloc: Add a 'is_pmem' parameter toqemu_ram_mmapscripts/update-linux-headers: addlinux/mman.h
linux-headers: addlinux/mman.h.
util/mmap-alloc: supportMAP_SYNCinqemu_ram_mmap()
docs: AddedMAP_SYNCdocumentationZhang Yi (5):
util/mmap-alloc: Add a 'is_pmem' parameter toqemu_ram_mmapscripts/update-linux-headers: addlinux/mman.h
linux-headers: addlinux/mman.h.
util/mmap-alloc: supportMAP_SYNCinqemu_ram_mmap()
docs: AddedMAP_SYNCdocumentationdocs/nvdimm.txt | 25 ++++++- exec.c | 2 +- include/qemu/mmap-alloc.h | 21 +++++- linux-headers/asm-arm/mman.h | 4 ++ linux-headers/asm-arm64/mman.h | 1 + linux-headers/asm-generic/hugetlb_encode.h | 36 ++++++++++ linux-headers/asm-generic/mman-common.h | 77 ++++++++++++++++++++ linux-headers/asm-generic/mman.h | 24 +++++++ linux-headers/asm-mips/mman.h | 108 +++++++++++++++++++++++++++++ linux-headers/asm-powerpc/mman.h | 39 +++++++++++ linux-headers/asm-s390/mman.h | 1 + linux-headers/asm-x86/mman.h | 31 +++++++++ linux-headers/linux/mman.h | 38 ++++++++++ qemu-options.hx | 5 ++ scripts/update-linux-headers.sh | 6 +- util/mmap-alloc.c | 51 +++++++++++++- util/oslib-posix.c | 2 +- 17 files changed, 460 insertions(+), 11 deletions(-) create mode 100644 linux-headers/asm-arm/mman.h create mode 100644 linux-headers/asm-arm64/mman.h create mode 100644 linux-headers/asm-generic/hugetlb_encode.h create mode 100644 linux-headers/asm-generic/mman-common.h create mode 100644 linux-headers/asm-generic/mman.h create mode 100644 linux-headers/asm-mips/mman.h create mode 100644 linux-headers/asm-powerpc/mman.h create mode 100644 linux-headers/asm-s390/mman.h create mode 100644 linux-headers/asm-x86/mman.h create mode 100644 linux-headers/linux/mman.hSignature · 1 line
2.7.4
Quoted 190 lines
Linux 4.15 introduces a new mmap flag
MAP_SYNC, which can be used to guarantee the write persistence to mmap'ed files supporting DAX (e.g., files on ext4/xfs file system mounted with '-o dax').A description of
MAP_SYNCandMAP_SHARED_VALIDATEcan be found athttps://patchwork.kernel.org/patch/10028151/In order to make sure that the file metadata is in sync after a fault while we are writing a shared DAX supporting backend files, this patch-set enables QEMU to use
MAP_SYNCflag for memory-backend-dax-file.As the DAX vs DMA truncated issue was solved, we refined the code and send out this feature for the v5 version.
We will pass
MAP_SYNCto mmap(2); ifMAP_SYNCis supported and 'share=on' & 'pmem=on'.
Or QEMU will not pass this flag to mmap(2)Test with below cases:
1. pmem=on is set, shared=on is set,MAP_SYNCsupported:
a: backend is a dax supporting file.
1) start VM1 with options:
-object memory-backend-file,id=nv_be4,share,mem-path=${DAX_FILE_1},size=${DAX_FILE_SIZE_1},align=128M,pmem=on,share=on -device nvdimm,id=nv4,memdev=nv_be4,label-size=2M.2) start VM2 with options:
-object memory-backend-file,id=nv_be4,share,mem-path=${DAX_FILE_2,size=${DAX_FILE_SIZE_2},align=128M,pmem=on,share=on -device nvdimm,id=nv4,memdev=nv_be4,label-size=2M.3) live migrate from VM1 to VM2.
4) Suddly let Host crash or power failure.
5) check
DAX_FILE_1andDAX_FILE_2, no corrupt.b: backend is a regular file.
1) start with options
-object memory-backend-file,id=nv_be4,share,mem-path=${REG_FILE},size=${REG_FILE_SIZE},align=128M,pmem=on,share=on -device nvdimm,id=nv4,memdev=nv_be4,label-size=2M.will warning "failed to validate with mapping flags: Operation not supported"
FILE_1andFILE_2random corrupt.2. Other cases:
FILE_1andFILE_2random corrupt.Changes in V13:
* 4/5 Micheal: move the inlcude tommap_alloc.c.
* 4/5 Micheal: refine the warning message.
* 5/5 Micheal: refine the Documentations.Changes in V12:
* 2/5: Micheal: Updateupdate-linux-headers.sh
* 3/5: Micheal: Use script update addlinux/mman.h
* 4/5: Pankaj,Micheal: 1) fallback to mmap withoutMAP_SYNC&MAP_SHARED_VALIDATEif sync not supported or failed
2) Replace the include with 3/5 addedlinux/mman.h
* 5/5: Micheal: Refine the Documentations.Changes in V11:
* 1/3: Micheal: Change to just add a boolis_pmeminqemu_ram_mmap.
* 2/3: Micheal: Fix the compatibility for old kernel.
* 2/3&3/3: Micheal&Eduardo :Update the behavior below:
Waning at no-dax and continue withoutMAP_SYNC.
Test if fails again for compatibility, then remove theMAP_VALIDATEand silently proceed.Changes in V10:
* 4/4: refine the document.
* 3/4: Reviewed-by: Stefano Garzarella <[email protected]>
* 2/4: refine the commit message, AddedMAP_SHARED_VALIDATE.
* 2/4: Fix the wrong include headerChanges in V9:
* 1/6: Reviewed-by: Eduardo Habkost <[email protected]>
* 2/6: New Added: Micheal: use sparse feature defineRAM_FLAG. since I don't have much knowledge about the sparse feature, @Micheal Could you
add some documentation/commit message on this patch? Thank you very much.
* 3/6: from 2/5: Eduardo: updated the commit message.
* 4/6: from 3/5: Micheal: don't ignoreMAP_SYNCfailures silently.
* 5/6: from 4/5: Eduardo: updated the commit message.
* 6/6: from 5/5: Micheal: Drop the sync option, document theMAP_SYNC.Changes in v8:
* Micheal: 3/5, remove the duplicated define in theos_dep.h
* Micheal: 2/5, make type define safety.
* Micheal: 2/5, fixed the incorrect defineMAP_SHAREonqemu_anon_ram_alloc.
* 4/6 removed, we remove the on/off/auto define of sync, as by now,MAP_SYNConly worked with pmem=on.
* @Micheal, I still reuse theRAM_SYNCflag, it is much straightforward to parse
all the flags in one parameter.Changes in v7:
* Micheal: [3,4,6]/6 limited the "sync" flag only on a nvdimm backend.(pmem=on)Changes in v6:
* Pankaj: 3/7 are squashed with 2/7
* Pankaj: 7/7 update comments to "consistent filesystem metadata".
* Pankaj, Igor: 1/7 Added Reviewed-by in patch-1/7
* Stefan, 4/7 move the include header from "/linux/mman.h" to "osdep.h"
* Stefan, 5/7 Add missing "munmap"
* Stefan, 2/7 refine the shared/flag.Changes in v5:
* Add patch 1 to fix a memory leak issue.
* Refine the patch 4-6
* Remove the patch 3 as we already change the parameter from "shared" to "flags"Changes in v4:
* Add patch 1-3 to switch some functions to a single 'flags' parameters. (Michael S. Tsirkin)
* v3 patch 1-3 become v4 patch 4-6.
* Patch 4: move definitions ofMAP_SYNCandMAP_SHARED_VALIDATEto a new header file underinclude/standard-headers/linux/. (Michael S. Tsirkin)
* Patch 6: refine the description of the 'sync' option. (Michael S. Tsirkin)Changes in v3:
* Patch 1: addMAP_SHARED_VALIDATEin both sync=on and sync=auto cases, and add back the retry mechanism.MAP_SYNCwill be ignored by Linux kernel 4.15 ifMAP_SHARED_VALIDATEis missed.
* Patch 1: defineMAP_SYNCandMAP_SHARED_VALIDATEas 0 on non-Linux platforms in order to makeqemu_ram_mmap()compile on those platforms.
* Patch 2&3: include more information in error messages of memory-backend in hope to help user to identify the error. (Dr. David Alan Gilbert)
* Patch 3: fix typo in the commit message. (Dr. David Alan Gilbert)Changes in v2:
* Add 'sync' option to control the use ofMAP_SYNC. (Eduardo Habkost)
* Remove the unnecessary set ofMAP_SHARED_VALIDATEin some cases and the retry mechanism inqemu_ram_mmap(). (Michael S. Tsirkin)
* Move OS dependent definitions ofMAP_SYNCandMAP_SHARED_VALIDATEtoosdep.h. (Michael S. Tsirkin)Zhang Yi (5):
util/mmap-alloc: Add a 'is_pmem' parameter toqemu_ram_mmapscripts/update-linux-headers: addlinux/mman.h
linux-headers: addlinux/mman.h.
util/mmap-alloc: supportMAP_SYNCinqemu_ram_mmap()
docs: AddedMAP_SYNCdocumentationZhang Yi (5):
util/mmap-alloc: Add a 'is_pmem' parameter toqemu_ram_mmapscripts/update-linux-headers: addlinux/mman.h
linux-headers: addlinux/mman.h.
util/mmap-alloc: supportMAP_SYNCinqemu_ram_mmap()
docs: AddedMAP_SYNCdocumentationdocs/nvdimm.txt | 25 ++++++- exec.c | 2 +- include/qemu/mmap-alloc.h | 21 +++++- linux-headers/asm-arm/mman.h | 4 ++ linux-headers/asm-arm64/mman.h | 1 + linux-headers/asm-generic/hugetlb_encode.h | 36 ++++++++++ linux-headers/asm-generic/mman-common.h | 77 ++++++++++++++++++++ linux-headers/asm-generic/mman.h | 24 +++++++ linux-headers/asm-mips/mman.h | 108 +++++++++++++++++++++++++++++ linux-headers/asm-powerpc/mman.h | 39 +++++++++++ linux-headers/asm-s390/mman.h | 1 + linux-headers/asm-x86/mman.h | 31 +++++++++ linux-headers/linux/mman.h | 38 ++++++++++ qemu-options.hx | 5 ++ scripts/update-linux-headers.sh | 6 +- util/mmap-alloc.c | 51 +++++++++++++- util/oslib-posix.c | 2 +- 17 files changed, 460 insertions(+), 11 deletions(-) create mode 100644 linux-headers/asm-arm/mman.h create mode 100644 linux-headers/asm-arm64/mman.h create mode 100644 linux-headers/asm-generic/hugetlb_encode.h create mode 100644 linux-headers/asm-generic/mman-common.h create mode 100644 linux-headers/asm-generic/mman.h create mode 100644 linux-headers/asm-mips/mman.h create mode 100644 linux-headers/asm-powerpc/mman.h create mode 100644 linux-headers/asm-s390/mman.h create mode 100644 linux-headers/asm-x86/mman.h create mode 100644 linux-headers/linux/mman.h
Quoted 2 lines
--
2.7.4
This patch series look good to me.
With updated patch 5, feel free to add.