virtio pmem: user document
PATCH 1/1 virtio pmem: user document
This patch documents the steps to use virtio pmem. It also documents other useful information about virtio pmem e.g use-case, comparison with Qemu NVDIMM backend and current limitations.
--- docs/virtio-pmem.txt | 65 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 65 insertions(+) create mode 100644 docs/virtio-pmem.txt
docs/virtio-pmem.txt new file +65 -0
@@ -0,0 +1,65 @@ + +QEMU virtio pmem +=================== + + This document explains the usage of virtio pmem device + which is available since QEMU v4.1.0. + + The virtio pmem is paravirtualized persistent memory device + on regular(non-NVDIMM) storage. + +Usecase +-------- + Allows to bypass the guest page cache and directly use host page cache. + This reduces guest memory footprint as host can make efficient memory + reclaim decisions under memory pressure. + +o How does virtio-pmem compare to the nvdimm emulation supported by QEMU? + + NVDIMM emulation on regular(non-NVDIMM) host storage does not persists + the guest writes as there are no defined semantecs in the device specification. + With virtio pmem device, guest write persistence on non-NVDIMM storage is + supported. + +virtio pmem usage +----------------- + virtio pmem device is created with a memory-backend-file with the below + options: + + -machine pc -m 8G,slots=$N,maxmem=$MAX_SIZE + -object memory-backend-file,id=mem1,share,mem-path=$PATH,size=$SIZE + -device virtio-pmem-pci,memdev=mem1,id=nv1 + + where: + - "object memory-backend-file,id=mem1,share,mem-path=$PATH,size=$VIRTIO_PMEM_SIZE" + creates a backend storage of size $SIZE on a file $PATH. All + accesses to the virtio pmem device go to the file $PATH. + + - "device virtio-pmem-pci,id=nvdimm1,memdev=mem1" creates a virtio pmem + device whose storage is provided by above memory backend device. + + Multiple virtio pmem devices can be created if multiple pairs of "-object" + and "-device" are provided. + +Hotplug +------- +Accomplished by two monitor commands "object_add" and "device_add". + +For example, the following commands add another 4GB virtio pmem device to +the guest: + + (qemu) object_add memory-backend-file,id=mem2,share=on,mem-path=virtio_pmem2.img,size=4G + (qemu) device_add virtio-pmem-pci,id=virtio_pmem2,memdev=mem2 + +Guest Data Persistence +---------------------- +Guest data persistence on non-NVDIMM requires guest userspace application to +perform fsync/msync. This is different than real nvdimm backend where no additional +fsync/msync is required for data persistence. + +Limitations +------------ +- Real nvdimm device backend is not supported. +- virtio pmem hotunplug is not supported. +- ACPI NVDIMM features like regions/namespaces are not supported. +- ndctl command is not supported.
Signature · 1 line
2.21.0
PATCH 1/1 virtio pmem: user document
This patch documents the steps to use virtio pmem. It also documents other useful information about virtio pmem e.g use-case, comparison with Qemu NVDIMM backend and current limitations.
--- docs/virtio-pmem.txt | 65 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 65 insertions(+) create mode 100644 docs/virtio-pmem.txt
docs/virtio-pmem.txt new file +65 -0
@@ -0,0 +1,65 @@ + +QEMU virtio pmem +=================== + + This document explains the usage of virtio pmem device + which is available since QEMU v4.1.0. + + The virtio pmem is paravirtualized persistent memory device + on regular(non-NVDIMM) storage. + +Usecase +-------- + Allows to bypass the guest page cache and directly use host page cache. + This reduces guest memory footprint as host can make efficient memory + reclaim decisions under memory pressure. + +o How does virtio-pmem compare to the nvdimm emulation supported by QEMU? + + NVDIMM emulation on regular(non-NVDIMM) host storage does not persists + the guest writes as there are no defined semantecs in the device specification. + With virtio pmem device, guest write persistence on non-NVDIMM storage is + supported. + +virtio pmem usage +----------------- + virtio pmem device is created with a memory-backend-file with the below + options: + + -machine pc -m 8G,slots=$N,maxmem=$MAX_SIZE + -object memory-backend-file,id=mem1,share,mem-path=$PATH,size=$SIZE + -device virtio-pmem-pci,memdev=mem1,id=nv1 + + where: + - "object memory-backend-file,id=mem1,share,mem-path=$PATH,size=$VIRTIO_PMEM_SIZE" + creates a backend storage of size $SIZE on a file $PATH. All + accesses to the virtio pmem device go to the file $PATH. + + - "device virtio-pmem-pci,id=nvdimm1,memdev=mem1" creates a virtio pmem + device whose storage is provided by above memory backend device. + + Multiple virtio pmem devices can be created if multiple pairs of "-object" + and "-device" are provided. + +Hotplug +------- +Accomplished by two monitor commands "object_add" and "device_add". + +For example, the following commands add another 4GB virtio pmem device to +the guest: + + (qemu) object_add memory-backend-file,id=mem2,share=on,mem-path=virtio_pmem2.img,size=4G + (qemu) device_add virtio-pmem-pci,id=virtio_pmem2,memdev=mem2 + +Guest Data Persistence +---------------------- +Guest data persistence on non-NVDIMM requires guest userspace application to +perform fsync/msync. This is different than real nvdimm backend where no additional +fsync/msync is required for data persistence. + +Limitations +------------ +- Real nvdimm device backend is not supported. +- virtio pmem hotunplug is not supported. +- ACPI NVDIMM features like regions/namespaces are not supported. +- ndctl command is not supported.
Signature · 1 line
2.21.0
docs/virtio-pmem.txt
Quoted 12 lines · On Tue, 30 Jul 2019 12:16:57 +0530 Pankaj Gupta <[email protected]> wrote:
This patch documents the steps to use virtio pmem. It also documents other useful information about virtio pmem e.g use-case, comparison with Qemu NVDIMM backend and current limitations.
Signed-off-by Pankaj Gupta <[email protected]>--- docs/virtio-pmem.txt | 65 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 65 insertions(+) create mode 100644 docs/virtio-pmem.txtdocs/virtio-pmem.txt +0
-0
Maybe make this ReST from the start? Should be trivial enough.
docs/virtio-pmem.txt line 5
Quoted 10 lines
new file mode 100644
index 0000000000..fc61eebb20docs/virtio-pmem.txt new file +5
-0@@ -0,0 +1,65 @@ + +QEMU virtio pmem +=================== + + This document explains the usage of virtio pmem device
"setup and usage" ?
docs/virtio-pmem.txt line 8
Quoted 3 lines
docs/virtio-pmem.txt new file +3
-0+ which is available since QEMU v4.1.0. + + The virtio pmem is paravirtualized persistent memory device
"The virtio pmem device is a paravirtualized..."
docs/virtio-pmem.txt line 14
Quoted 6 lines
docs/virtio-pmem.txt new file +6
-0+ on regular(non-NVDIMM) storage. + +Usecase +-------- + Allows to bypass the guest page cache and directly use host page cache. + This reduces guest memory footprint as host can make efficient memory
s/as host/,as the host/
docs/virtio-pmem.txt line 19
Quoted 5 lines
docs/virtio-pmem.txt new file +5
-0+ reclaim decisions under memory pressure. + +o How does virtio-pmem compare to the nvdimm emulation supported by QEMU? + + NVDIMM emulation on regular(non-NVDIMM) host storage does not persists
s/regular(non-NVDIMM)/regular (i.e. non-NVDIMM)/ ? s/persists/persist/
docs/virtio-pmem.txt line 20
Quoted 1 line
docs/virtio-pmem.txt new file +1
-0+ the guest writes as there are no defined semantecs in the device specification.
s/semantecs/semantics/
docs/virtio-pmem.txt line 22
Quoted 2 lines
docs/virtio-pmem.txt new file +2
-0+ With virtio pmem device, guest write persistence on non-NVDIMM storage is + supported.
"The virtio pmem device provides a way to support guest write persistence on non-NVDIMM storage." ?
docs/virtio-pmem.txt line 27
Quoted 5 lines
docs/virtio-pmem.txt new file +5
-0+ +virtio pmem usage +----------------- + virtio pmem device is created with a memory-backend-file with the below + options:
"A virtio pmem device backed by a memory-backend-file can be created on the QEMU command line as in the following example:" ?
docs/virtio-pmem.txt line 29
Quoted 2 lines
docs/virtio-pmem.txt new file +2
-0+ + -machine pc -m 8G,slots=$N,maxmem=$MAX_SIZE
I'm not sure you should explicitly specify the machine type in this example. I think it is fine to say that something is only supported on a subset of machine types, but it should not make its way into an example on how to configure a device and its backing.
Also, maybe fill in more concrete values here? Or split it into a part specifying the syntax (where I'd use <max_size> instead of $MAX_SIZE etc.), and a more concrete example?
docs/virtio-pmem.txt line 39
Quoted 10 lines
docs/virtio-pmem.txt new file +10
-0+ -object memory-backend-file,id=mem1,share,mem-path=$PATH,size=$SIZE + -device virtio-pmem-pci,memdev=mem1,id=nv1 + + where: + - "object memory-backend-file,id=mem1,share,mem-path=$PATH,size=$VIRTIO_PMEM_SIZE" + creates a backend storage of size $SIZE on a file $PATH. All + accesses to the virtio pmem device go to the file $PATH. + + - "device virtio-pmem-pci,id=nvdimm1,memdev=mem1" creates a virtio pmem + device whose storage is provided by above memory backend device.
"a virtio pmem PCI device" ?
docs/virtio-pmem.txt line 46
Quoted 7 lines
docs/virtio-pmem.txt new file +7
-0+ + Multiple virtio pmem devices can be created if multiple pairs of "-object" + and "-device" are provided. + +Hotplug +------- +Accomplished by two monitor commands "object_add" and "device_add".
Hm... what about the following instead:
"Virtio pmem devices can be hotplugged via the QEMU monitor. First, the memory backing has to be added via 'object_add'; afterwards, the virtio pmem device can be added via 'device_add'."
docs/virtio-pmem.txt line 56
Quoted 10 lines
docs/virtio-pmem.txt new file +10
-0+ +For example, the following commands add another 4GB virtio pmem device to +the guest: + + (qemu) object_add memory-backend-file,id=mem2,share=on,mem-path=virtio_pmem2.img,size=4G + (qemu) device_add virtio-pmem-pci,id=virtio_pmem2,memdev=mem2 + +Guest Data Persistence +---------------------- +Guest data persistence on non-NVDIMM requires guest userspace application to
s/application/applications/ ?
docs/virtio-pmem.txt line 57
Quoted 1 line
docs/virtio-pmem.txt new file +1
-0+perform fsync/msync. This is different than real nvdimm backend where no additional
s/than/from a/ ?
docs/virtio-pmem.txt line 58
Quoted 1 line
docs/virtio-pmem.txt new file +1
-0+fsync/msync is required for data persistence.
Should we be a bit more verbose on what which guest applications are supposed to do? I.e., how do they know they need to do fsync/msync, when should they do it, and what are the consequences if they don't?
Quoted 7 lines
docs/virtio-pmem.txt new file +7
-0+ +Limitations +------------ +- Real nvdimm device backend is not supported. +- virtio pmem hotunplug is not supported. +- ACPI NVDIMM features like regions/namespaces are not supported. +- ndctl command is not supported.
Quoted 16 lines
Quoted 11 lines · On Tue, 30 Jul 2019 12:16:57 +0530 Pankaj Gupta <[email protected]> wrote:
This patch documents the steps to use virtio pmem. It also documents other useful information about virtio pmem e.g use-case, comparison with Qemu NVDIMM backend and current limitations.
Signed-off-by Pankaj Gupta <[email protected]>--- docs/virtio-pmem.txt | 65 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 65 insertions(+) create mode 100644 docs/virtio-pmem.txt
Quoted 3 lines
docs/virtio-pmem.txt
Quoted 1 line
docs/virtio-pmem.txt +0
-0Maybe make this ReST from the start? Should be trivial enough.
o.k
Quoted 13 lines
docs/virtio-pmem.txt line 5
Quoted 11 lines
new file mode 100644
index 0000000000..fc61eebb20docs/virtio-pmem.txt new file +5
-0@@ -0,0 +1,65 @@ + +QEMU virtio pmem +=================== + + This document explains the usage of virtio pmem device"setup and usage" ?
o.k
Quoted 6 lines
docs/virtio-pmem.txt line 8
Quoted 4 lines
docs/virtio-pmem.txt new file +3
-0+ which is available since QEMU v4.1.0. + + The virtio pmem is paravirtualized persistent memory device"The virtio pmem device is a paravirtualized..."
sure.
Quoted 9 lines
docs/virtio-pmem.txt line 14
Quoted 7 lines
docs/virtio-pmem.txt new file +6
-0+ on regular(non-NVDIMM) storage. + +Usecase +-------- + Allows to bypass the guest page cache and directly use host page cache. + This reduces guest memory footprint as host can make efficient memorys/as host/,as the host/
sure
Quoted 9 lines
docs/virtio-pmem.txt line 19
Quoted 6 lines
docs/virtio-pmem.txt new file +5
-0+ reclaim decisions under memory pressure. + +o How does virtio-pmem compare to the nvdimm emulation supported by QEMU? + + NVDIMM emulation on regular(non-NVDIMM) host storage does not persistss/regular(non-NVDIMM)/regular (i.e. non-NVDIMM)/ ? s/persists/persist/
yes, to both.
Quoted 5 lines
Quoted 3 lines
docs/virtio-pmem.txt new file +1
-0+ the guest writes as there are no defined semantecs in the devicespecification.
s/semantecs/semantics/
ah...spell checker :(
Quoted 8 lines
Quoted 4 lines
+ With virtio pmem device, guest write persistence on non-NVDIMM storage is
+ supported."The virtio pmem device provides a way to support guest write persistence on non-NVDIMM storage." ?
Quoted 8 lines
docs/virtio-pmem.txt line 27
Quoted 5 lines
docs/virtio-pmem.txt new file +5
-0+ +virtio pmem usage +----------------- + virtio pmem device is created with a memory-backend-file with the below + options:"A virtio pmem device backed by a memory-backend-file can be created on the QEMU command line as in the following example:" ?
o.k
Quoted 8 lines
docs/virtio-pmem.txt
Quoted 3 lines
docs/virtio-pmem.txt new file +2
-0+ + -machine pc -m 8G,slots=$N,maxmem=$MAX_SIZEI'm not sure you should explicitly specify the machine type in this example. I think it is fine to say that something is only supported on a subset of machine types, but it should not make its way into an example on how to configure a device and its backing.
o.k
Quoted 4 lines
Also, maybe fill in more concrete values here? Or split it into a part specifying the syntax (where I'd use <
max_size> instead of $MAX_SIZE etc.), and a more concrete example?
o.k
Quoted 14 lines
docs/virtio-pmem.txt line 39
Quoted 12 lines
docs/virtio-pmem.txt new file +5
-0+ -object memory-backend-file,id=mem1,share,mem-path=$PATH,size=$SIZE + -device virtio-pmem-pci,memdev=mem1,id=nv1 + + where: + - "objectmemory-backend-file,id=mem1,share,mem-path=$PATH,size=$VIRTIO_PMEM_SIZE"
docs/virtio-pmem.txt new file +5
-0+ creates a backend storage of size $SIZE on a file $PATH. All + accesses to the virtio pmem device go to the file $PATH. + + - "device virtio-pmem-pci,id=nvdimm1,memdev=mem1" creates a virtio pmem + device whose storage is provided by above memory backend device."a virtio pmem PCI device" ?
o.k
Quoted 12 lines
docs/virtio-pmem.txt line 46
Quoted 9 lines
docs/virtio-pmem.txt new file +2
-0+ + Multiple virtio pmem devices can be created if multiple pairs of"-object"
docs/virtio-pmem.txt new file +5
-0+ and "-device" are provided. + +Hotplug +------- +Accomplished by two monitor commands "object_add" and "device_add".Hm... what about the following instead:
Quoted 3 lines
"Virtio pmem devices can be hotplugged via the QEMU monitor. First, the memory backing has to be added via '
object_add'; afterwards, the virtio pmem device can be added via 'device_add'."
o.k
Quoted 16 lines
Quoted 12 lines
docs/virtio-pmem.txt new file +5
-0+ +For example, the following commands add another 4GB virtio pmem device to +the guest: + + (qemu) object_addmemory-backend-file,id=mem2,share=on,mem-path=virtio_pmem2.img,size=4G
+ (qemu)device_addvirtio-pmem-pci,id=virtio_pmem2,memdev=mem2docs/virtio-pmem.txt new file +4
-0+ +Guest Data Persistence +---------------------- +Guest data persistence on non-NVDIMM requires guest userspace applicationQuoted 1 line
to
s/application/applications/ ?
Quoted 4 lines
Quoted 2 lines
+perform fsync/msync. This is different than real nvdimm backend where no additional
s/than/from a/ ?
yes.
Quoted 6 lines
Quoted 2 lines
+fsync/msync is required for data persistence.
Should we be a bit more verbose on what which guest applications are supposed to do? I.e., how do they know they need to do fsync/msync, when should they do it, and what are the consequences if they don't?
o.k.
Quoted 8 lines
docs/virtio-pmem.txt line 65
Quoted 8 lines
docs/virtio-pmem.txt new file +7
-0+ +Limitations +------------ +- Real nvdimm device backend is not supported. +- virtio pmem hotunplug is not supported. +- ACPI NVDIMM features like regions/namespaces are not supported. +- ndctl command is not supported.
Thank you for the review.
Best regards,
Pankaj