virtio pmem driver
Subsystems (12)
v12 cover virtio pmem driver
This patch series is ready to be merged via nvdimm tree as discussed with Dan. We have ack/review on XFS, EXT4 & VIRTIO patches. Device mapper change is also reviewed.
Mike, Can you please provide ack for device mapper change i.e patch4.
This version has changed implementation for patch 4 as suggested by 'Mike'. Keeping all the existing r-o-bs. Jakob CCed also tested the patch series and confirmed the working of v9.
---
This patch series has implementation for "virtio pmem". "virtio pmem" is fake persistent memory(nvdimm) in guest which allows to bypass the guest page cache. This also implements a VIRTIO based asynchronous flush mechanism.
Sharing guest kernel driver in this patchset with the changes suggested in v4. Tested with Qemu side device emulation [5] for virtio-pmem. Documented the impact of possible page cache side channel attacks with suggested countermeasures.
Details of project idea for 'virtio pmem' flushing interface is shared [3] & [4].
Implementation is divided into two parts:
New virtio pmem guest driver and qemu code changes for new virtio pmem paravirtualized device.
1. Guest virtio-pmem kernel driver
---------------------------------
- Reads persistent memory range from paravirt device and registers with 'nvdimm_bus'.
- 'nvdimm/pmem' driver uses this information to allocate persistent memory region and setup filesystem operations to the allocated memory.
- virtio pmem driver implements asynchronous flushing interface to flush from guest to host.
2. Qemu virtio-pmem device
---------------------------------
- Creates virtio pmem device and exposes a memory range to KVM guest.
- At host side this is file backed memory which acts as persistent memory.
- Qemu side flush uses aio thread pool API's and virtio for asynchronous guest multi request handling.
Virtio-pmem security implications and countermeasures:
-----------------------------------------------------
In previous posting of kernel driver, there was discussion [7] on possible implications of page cache side channel attacks with virtio pmem. After thorough analysis of details of known side channel attacks, below are the suggestions:
- Depends entirely on how host backing image file is mapped into guest address space.
- virtio-pmem device emulation, by default shared mapping is used to map host backing file. It is recommended to use separate backing file at host side for every guest. This will prevent any possibility of executing common code from multiple guests and any chance of inferring guest local data based based on execution time.
- If backing file is required to be shared among multiple guests it is recommended to don't support host page cache eviction commands from the guest driver. This will avoid any possibility of inferring guest local data or host data from another guest.
- Proposed device specification [6] for virtio-pmem device with details of possible security implications and suggested countermeasures for device emulation.
Virtio-pmem errors handling:
----------------------------------------
Checked behaviour of virtio-pmem for below types of errors Need suggestions on expected behaviour for handling these errors?
- Hardware Errors: Uncorrectable recoverable Errors:
a] virtio-pmem:
- As per current logic if error page belongs to Qemu process, host MCE handler isolates(hwpoison) that page and send SIGBUS. Qemu SIGBUS handler injects exception to KVM guest.
- KVM guest then isolates the page and send SIGBUS to guest userspace process which has mapped the page.
b] Existing implementation for ACPI pmem driver:
- Handles such errors with MCE notifier and creates a list of bad blocks. Read/direct access DAX operation return EIO if accessed memory page fall in bad block list.
- It also starts backgound scrubbing.
- Similar functionality can be reused in virtio-pmem with MCE notifier but without scrubbing(no ACPI/ARS)? Need inputs to confirm if this behaviour is ok or needs any change?
Changes from PATCH v11: [1]
- Change implmentation for setting of synchronous DAX type for device mapper - Mike
Changes from PATCH v10: [2]
- Rebased on Linux-5.2-rc4
Changes from PATCH v9:
- Kconfig help text add two spaces - Randy
- Fixed libnvdimm 'bio' include warning - Dan
- virtio-pmem, separate request/resp struct and move to uapi file with updated license - DavidH
- Use virtio32* type for req/resp endianess - DavidH
- Added tested-by & ack-by of Jakob
- Rebased to 5.2-rc1
Changes from PATCH v8:
- Set device mapper synchronous if all target devices support - Dan
- Move virtio_pmem.h to nvdimm directory - Dan
- Style, indentation & better error messages in patch 2 - DavidH
- Added MST's ack in patch 2.
Changes from PATCH v7:
- Corrected pending request queue logic (patch 2) - Jakub Staroń
- Used unsigned long flags for passing DAXDEV_F_SYNC (patch 3) - Dan
- Fixed typo => vma 'flag' to 'vm_flag' (patch 4)
- Added rob in patch 6 & patch 2
Changes from PATCH v6:
- Corrected comment format in patch 5 & patch 6. [Dave]
- Changed variable declaration indentation in patch 6 [Darrick]
- Add Reviewed-by tag by 'Jan Kara' in patch 4 & patch 5
Changes from PATCH v5:
Changes suggested in by - [Cornelia, Yuval]
- Remove assignment chaining in virtio driver
- Better error message and remove not required free
- Check nd_region before use
Changes suggested by - [Jan Kara]
- dax_synchronous() for !CONFIG_DAX
- Correct 'daxdev_mapping_supported' comment and non-dax implementation
Changes suggested by - [Dan Williams]
- Pass meaningful flag 'DAXDEV_F_SYNC' to alloc_dax
- Gate nvdimm_flush instead of additional async parameter
- Move block chaining logic to flush callback than common nvdimm_flush
- Use NULL flush callback for generic flush for better readability [Dan, Jan]
- Use virtio device id 27 from 25(already used) - [MST]
Changes from PATCH v4:
- Factor out MAP_SYNC supported functionality to a common helper [Dave, Darrick, Jan]
- Comment, indentation and virtqueue_kick failure handle - Yuval Shaia
Changes from PATCH v3:
- Use generic dax_synchronous() helper to check for DAXDEV_SYNC flag - [Dan, Darrick, Jan]
- Add 'is_nvdimm_async' function
- Document page cache side channel attacks implications & countermeasures - [Dave Chinner, Michael]
Changes from PATCH v2:
- Disable MAP_SYNC for ext4 & XFS filesystems - [Dan]
- Use name 'virtio pmem' in place of 'fake dax'
Changes from PATCH v1:
- 0-day build test for build dependency on libnvdimm
Changes suggested by - [Dan Williams]
- Split the driver into two parts virtio & pmem
- Move queuing of async block request to block layer
- Add "sync" parameter in nvdimm_flush function
- Use indirect call for nvdimm_flush
- Don’t move declarations to common global header e.g nd.h
- nvdimm_flush() return 0 or -EIO if it fails
- Teach nsio_rw_bytes() that the flush can fail
- Rename nvdimm_flush() to generic_nvdimm_flush()
- Use 'nd_region->provider_data' for long dereferencing
- Remove virtio_pmem_freeze/restore functions
- Remove BSD license text with SPDX license text
- Add might_sleep() in virtio_pmem_flush - [Luiz]
- Make spin_lock_irqsave() narrow
Pankaj Gupta (7):
libnvdimm: nd_region flush callback support
virtio-pmem: Add virtio-pmem guest driver
libnvdimm: add nd_region buffered dax_dev flag
dax: check synchronous mapping is supported
dm: dm: Enable synchronous dax
ext4: disable map_sync for virtio pmem
xfs: disable map_sync for virtio pmem
[1] https://lkml.org/lkml/2019/6/10/209
[2] https://lkml.org/lkml/2019/5/21/569
[3] https://www.spinics.net/lists/kvm/msg149761.html
[4] https://www.spinics.net/lists/kvm/msg153095.html
[5] https://marc.info/?l=qemu-devel&m=155860751202202&w=2
[6] https://lists.oasis-open.org/archives/virtio-dev/201903/msg00083.html
[7] https://lkml.org/lkml/2019/1/9/1191
drivers/acpi/nfit/core.c | 4 - drivers/dax/bus.c | 2 drivers/dax/super.c | 19 +++++ drivers/md/dm-table.c | 24 +++++-- drivers/md/dm.c | 5 - drivers/md/dm.h | 5 + drivers/nvdimm/Makefile | 1 drivers/nvdimm/claim.c | 6 + drivers/nvdimm/nd.h | 1 drivers/nvdimm/nd_virtio.c | 124 +++++++++++++++++++++++++++++++++++++++ drivers/nvdimm/pmem.c | 18 +++-- drivers/nvdimm/region_devs.c | 33 +++++++++- drivers/nvdimm/virtio_pmem.c | 122 ++++++++++++++++++++++++++++++++++++++ drivers/nvdimm/virtio_pmem.h | 55 +++++++++++++++++ drivers/virtio/Kconfig | 11 +++ fs/ext4/file.c | 10 +-- fs/xfs/xfs_file.c | 9 +- include/linux/dax.h | 26 +++++++- include/linux/libnvdimm.h | 10 ++- include/uapi/linux/virtio_ids.h | 1 include/uapi/linux/virtio_pmem.h | 35 +++++++++++ 21 files changed, 488 insertions(+), 33 deletions(-)
_______________________________________________
Linux-nvdimm mailing list
[email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm
PATCH v12 1/7 libnvdimm: nd_region flush callback support
This patch adds functionality to perform flush from guest to host over VIRTIO. We are registering a callback based on 'nd_region' type. virtio_pmem driver requires this special flush function. For rest of the region types we are registering existing flush function. Report error returned by host fsync failure to userspace.
--- drivers/acpi/nfit/core.c | 4 ++-- drivers/nvdimm/claim.c | 6 ++++-- drivers/nvdimm/nd.h | 1 + drivers/nvdimm/pmem.c | 13 ++++++++----- drivers/nvdimm/region_devs.c | 26 ++++++++++++++++++++++++-- include/linux/libnvdimm.h | 9 ++++++++- 6 files changed, 47 insertions(+), 12 deletions(-)
drivers/acpi/nfit/core.c +2 -2
@@ -2434,7 +2434,7 @@ static void write_blk_ctl(struct nfit_blk *nfit_blk, unsigned int bw, offset =to_interleave_offset (offset, mmio);writeq (cmd, mmio->addr.base + offset); -nvdimm_flush (nfit_blk->nd_region); +nvdimm_flush (nfit_blk->nd_region,NULL );if (nfit_blk->dimm_flags &NFIT_BLK_DCR_LATCH )readq (mmio->addr.base + offset);
@@ -2483,7 +2483,7 @@ static int acpi_nfit_blk_single_io(struct nfit_blk *nfit_blk, }if (rw) -nvdimm_flush (nfit_blk->nd_region); +nvdimm_flush (nfit_blk->nd_region,NULL ); rc =read_blk_stat (nfit_blk, lane) ? -EIO :0 ;return rc;
drivers/nvdimm/claim.c +4 -2
@@ -263,7 +263,7 @@ static int nsio_rw_bytes(struct nd_namespace_common *ndns,struct nd_namespace_io *nsio =to_nd_namespace_io (&ndns->dev);unsigned int sz_align =ALIGN (size + (offset & (512 -1 )),512 );sector_t sector = offset >>9 ; -int rc =0 ; +int rc =0 , ret =0 ;if (unlikely (!size))return 0 ;
@@ -301,7 +301,9 @@ static int nsio_rw_bytes(struct nd_namespace_common *ndns, }memcpy_flushcache (nsio->addr + offset, buf, size); -nvdimm_flush (to_nd_region (ndns->dev.parent)); + ret =nvdimm_flush (to_nd_region (ndns->dev.parent),NULL ); +if (ret) + rc = ret;return rc; }
drivers/nvdimm/nd.h +1 -0
@@ -159,6 +159,7 @@ struct nd_region {struct badblocks bb;struct nd_interleave_set *nd_set;struct nd_percpu_lane __percpu *lane; +int (*flush)(struct nd_region *nd_region,struct bio *bio);struct nd_mapping mapping[0 ]; };
drivers/nvdimm/pmem.c +8 -5
@@ -192,6 +192,7 @@ static blk_status_t pmem_do_bvec(struct pmem_device *pmem, struct page *page,static blk_qc_t pmem_make_request (struct request_queue *q,struct bio *bio) { +int ret =0 ;blk_status_t rc =0 ;bool do_acct;unsigned long start;
@@ -201,7 +202,7 @@ static blk_qc_t pmem_make_request(struct request_queue *q, struct bio *bio)struct nd_region *nd_region =to_region (pmem);if (bio->bi_opf &REQ_PREFLUSH ) -nvdimm_flush (nd_region); + ret =nvdimm_flush (nd_region, bio); do_acct =nd_iostat_start (bio, &start);bio_for_each_segment (bvec, bio, iter) {
@@ -216,7 +217,10 @@ static blk_qc_t pmem_make_request(struct request_queue *q, struct bio *bio)nd_iostat_end (bio, start);if (bio->bi_opf &REQ_FUA ) -nvdimm_flush (nd_region); + ret =nvdimm_flush (nd_region, bio); + +if (ret) + bio->bi_status =errno_to_blk_status (ret);bio_endio (bio);return BLK_QC_T_NONE ;
@@ -469,7 +473,6 @@ static int pmem_attach_disk(struct device *dev, }dax_write_cache (dax_dev,nvdimm_has_cache (nd_region)); pmem->dax_dev = dax_dev; - gendev =disk_to_dev (disk); gendev->groups = pmem_attribute_groups;
@@ -527,14 +530,14 @@ static int nd_pmem_remove(struct device *dev)sysfs_put (pmem->bb_state); pmem->bb_state =NULL ; } -nvdimm_flush (to_nd_region (dev->parent)); +nvdimm_flush (to_nd_region (dev->parent),NULL );return 0 ; }static void nd_pmem_shutdown (struct device *dev) { -nvdimm_flush (to_nd_region (dev->parent)); +nvdimm_flush (to_nd_region (dev->parent),NULL ); }static void nd_pmem_notify (struct device *dev,enum nvdimm_event event)
drivers/nvdimm/region_devs.c +24 -2
@@ -295,7 +295,9 @@ static ssize_t deep_flush_store(struct device *dev, struct device_attribute *attreturn rc;if (!flush)return -EINVAL ; -nvdimm_flush (nd_region); + rc =nvdimm_flush (nd_region,NULL ); +if (rc) +return rc;return len; }
@@ -1085,6 +1087,11 @@ static struct nd_region *nd_region_create(struct nvdimm_bus *nvdimm_bus, dev->of_node = ndr_desc->of_node; nd_region->ndr_size =resource_size (ndr_desc->res); nd_region->ndr_start = ndr_desc->res->start; +if (ndr_desc->flush) + nd_region->flush = ndr_desc->flush; +else + nd_region->flush =NULL ; +nd_device_register (dev);return nd_region;
@@ -1125,11 +1132,24 @@ struct nd_region *nvdimm_volatile_region_create(struct nvdimm_bus *nvdimm_bus, }EXPORT_SYMBOL_GPL (nvdimm_volatile_region_create); +int nvdimm_flush (struct nd_region *nd_region,struct bio *bio) +{ +int rc =0 ; + +if (!nd_region->flush) + rc =generic_nvdimm_flush (nd_region); +else { +if (nd_region->flush (nd_region, bio)) + rc = -EIO ; + } + +return rc; +}/** * nvdimm_flush - flush any posted write queues between the cpu and pmem media * @nd_region: blk or interleaved pmem region */ -void nvdimm_flush (struct nd_region *nd_region) +int generic_nvdimm_flush (struct nd_region *nd_region) {struct nd_region_data *ndrd =dev_get_drvdata (&nd_region->dev);int i, idx;
@@ -1153,6 +1173,8 @@ void nvdimm_flush(struct nd_region *nd_region)if (ndrd_get_flush_wpq (ndrd, i,0 ))writeq (1 ,ndrd_get_flush_wpq (ndrd, i, idx));wmb (); + +return 0 ; }EXPORT_SYMBOL_GPL (nvdimm_flush);
include/linux/libnvdimm.h +8 -1
@@ -19,6 +19,7 @@#include <linux/types.h> #include <linux/uuid.h> #include <linux/spinlock.h> +#include <linux/bio.h> struct badrange_entry {u64 start;
@@ -65,6 +66,9 @@ enum {*/ ND_REGION_PERSIST_MEMCTRL =2 , +/* Platform provides asynchronous flush mechanism */ +ND_REGION_ASYNC =3 , +/* mark newly adjusted resources as requiring a label update */ DPA_RESOURCE_ADJUSTED = 1 << 0, };
@@ -121,6 +125,7 @@ struct nd_mapping_desc {int position; }; +struct nd_region ;struct nd_region_desc {struct resource *res;struct nd_mapping_desc *mapping;
@@ -133,6 +138,7 @@ struct nd_region_desc {int target_node;unsigned long flags;struct device_node *of_node; +int (*flush)(struct nd_region *nd_region,struct bio *bio); };struct device ;
@@ -260,7 +266,8 @@ unsigned long nd_blk_memremap_flags(struct nd_blk_region *ndbr);unsigned int nd_region_acquire_lane (struct nd_region *nd_region);void nd_region_release_lane (struct nd_region *nd_region,unsigned int lane);u64 nd_fletcher64 (void *addr,size_t len,bool le); -void nvdimm_flush (struct nd_region *nd_region); +int nvdimm_flush (struct nd_region *nd_region,struct bio *bio); +int generic_nvdimm_flush (struct nd_region *nd_region);int nvdimm_has_flush (struct nd_region *nd_region);int nvdimm_has_cache (struct nd_region *nd_region);int nvdimm_in_overwrite (struct nvdimm *nvdimm);
Signature · 1 line
2.20.1
PATCH v12 1/7 libnvdimm: nd_region flush callback support
In reply to Pankaj GuptaThis patch adds functionality to perform flush from guest to host over VIRTIO. We are registering a callback based on 'nd_region' type. virtio_pmem driver requires this special flush function. For rest of the region types we are registering existing flush function. Report error returned by host fsync failure to userspace.
--- drivers/acpi/nfit/core.c | 4 ++-- drivers/nvdimm/claim.c | 6 ++++-- drivers/nvdimm/nd.h | 1 + drivers/nvdimm/pmem.c | 13 ++++++++----- drivers/nvdimm/region_devs.c | 26 ++++++++++++++++++++++++-- include/linux/libnvdimm.h | 9 ++++++++- 6 files changed, 47 insertions(+), 12 deletions(-)
drivers/acpi/nfit/core.c +2 -2
@@ -2434,7 +2434,7 @@ static void write_blk_ctl(struct nfit_blk *nfit_blk, unsigned int bw, offset =to_interleave_offset (offset, mmio);writeq (cmd, mmio->addr.base + offset); -nvdimm_flush (nfit_blk->nd_region); +nvdimm_flush (nfit_blk->nd_region,NULL );if (nfit_blk->dimm_flags &NFIT_BLK_DCR_LATCH )readq (mmio->addr.base + offset);
@@ -2483,7 +2483,7 @@ static int acpi_nfit_blk_single_io(struct nfit_blk *nfit_blk, }if (rw) -nvdimm_flush (nfit_blk->nd_region); +nvdimm_flush (nfit_blk->nd_region,NULL ); rc =read_blk_stat (nfit_blk, lane) ? -EIO :0 ;return rc;
drivers/nvdimm/claim.c +4 -2
@@ -263,7 +263,7 @@ static int nsio_rw_bytes(struct nd_namespace_common *ndns,struct nd_namespace_io *nsio =to_nd_namespace_io (&ndns->dev);unsigned int sz_align =ALIGN (size + (offset & (512 -1 )),512 );sector_t sector = offset >>9 ; -int rc =0 ; +int rc =0 , ret =0 ;if (unlikely (!size))return 0 ;
@@ -301,7 +301,9 @@ static int nsio_rw_bytes(struct nd_namespace_common *ndns, }memcpy_flushcache (nsio->addr + offset, buf, size); -nvdimm_flush (to_nd_region (ndns->dev.parent)); + ret =nvdimm_flush (to_nd_region (ndns->dev.parent),NULL ); +if (ret) + rc = ret;return rc; }
drivers/nvdimm/nd.h +1 -0
@@ -159,6 +159,7 @@ struct nd_region {struct badblocks bb;struct nd_interleave_set *nd_set;struct nd_percpu_lane __percpu *lane; +int (*flush)(struct nd_region *nd_region,struct bio *bio);struct nd_mapping mapping[0 ]; };
drivers/nvdimm/pmem.c +8 -5
@@ -192,6 +192,7 @@ static blk_status_t pmem_do_bvec(struct pmem_device *pmem, struct page *page,static blk_qc_t pmem_make_request (struct request_queue *q,struct bio *bio) { +int ret =0 ;blk_status_t rc =0 ;bool do_acct;unsigned long start;
@@ -201,7 +202,7 @@ static blk_qc_t pmem_make_request(struct request_queue *q, struct bio *bio)struct nd_region *nd_region =to_region (pmem);if (bio->bi_opf &REQ_PREFLUSH ) -nvdimm_flush (nd_region); + ret =nvdimm_flush (nd_region, bio); do_acct =nd_iostat_start (bio, &start);bio_for_each_segment (bvec, bio, iter) {
@@ -216,7 +217,10 @@ static blk_qc_t pmem_make_request(struct request_queue *q, struct bio *bio)nd_iostat_end (bio, start);if (bio->bi_opf &REQ_FUA ) -nvdimm_flush (nd_region); + ret =nvdimm_flush (nd_region, bio); + +if (ret) + bio->bi_status =errno_to_blk_status (ret);bio_endio (bio);return BLK_QC_T_NONE ;
@@ -469,7 +473,6 @@ static int pmem_attach_disk(struct device *dev, }dax_write_cache (dax_dev,nvdimm_has_cache (nd_region)); pmem->dax_dev = dax_dev; - gendev =disk_to_dev (disk); gendev->groups = pmem_attribute_groups;
@@ -527,14 +530,14 @@ static int nd_pmem_remove(struct device *dev)sysfs_put (pmem->bb_state); pmem->bb_state =NULL ; } -nvdimm_flush (to_nd_region (dev->parent)); +nvdimm_flush (to_nd_region (dev->parent),NULL );return 0 ; }static void nd_pmem_shutdown (struct device *dev) { -nvdimm_flush (to_nd_region (dev->parent)); +nvdimm_flush (to_nd_region (dev->parent),NULL ); }static void nd_pmem_notify (struct device *dev,enum nvdimm_event event)
drivers/nvdimm/region_devs.c +24 -2
@@ -295,7 +295,9 @@ static ssize_t deep_flush_store(struct device *dev, struct device_attribute *attreturn rc;if (!flush)return -EINVAL ; -nvdimm_flush (nd_region); + rc =nvdimm_flush (nd_region,NULL ); +if (rc) +return rc;return len; }
@@ -1085,6 +1087,11 @@ static struct nd_region *nd_region_create(struct nvdimm_bus *nvdimm_bus, dev->of_node = ndr_desc->of_node; nd_region->ndr_size =resource_size (ndr_desc->res); nd_region->ndr_start = ndr_desc->res->start; +if (ndr_desc->flush) + nd_region->flush = ndr_desc->flush; +else + nd_region->flush =NULL ; +nd_device_register (dev);return nd_region;
@@ -1125,11 +1132,24 @@ struct nd_region *nvdimm_volatile_region_create(struct nvdimm_bus *nvdimm_bus, }EXPORT_SYMBOL_GPL (nvdimm_volatile_region_create); +int nvdimm_flush (struct nd_region *nd_region,struct bio *bio) +{ +int rc =0 ; + +if (!nd_region->flush) + rc =generic_nvdimm_flush (nd_region); +else { +if (nd_region->flush (nd_region, bio)) + rc = -EIO ; + } + +return rc; +}/** * nvdimm_flush - flush any posted write queues between the cpu and pmem media * @nd_region: blk or interleaved pmem region */ -void nvdimm_flush (struct nd_region *nd_region) +int generic_nvdimm_flush (struct nd_region *nd_region) {struct nd_region_data *ndrd =dev_get_drvdata (&nd_region->dev);int i, idx;
@@ -1153,6 +1173,8 @@ void nvdimm_flush(struct nd_region *nd_region)if (ndrd_get_flush_wpq (ndrd, i,0 ))writeq (1 ,ndrd_get_flush_wpq (ndrd, i, idx));wmb (); + +return 0 ; }EXPORT_SYMBOL_GPL (nvdimm_flush);
include/linux/libnvdimm.h +8 -1
@@ -19,6 +19,7 @@#include <linux/types.h> #include <linux/uuid.h> #include <linux/spinlock.h> +#include <linux/bio.h> struct badrange_entry {u64 start;
@@ -65,6 +66,9 @@ enum {*/ ND_REGION_PERSIST_MEMCTRL =2 , +/* Platform provides asynchronous flush mechanism */ +ND_REGION_ASYNC =3 , +/* mark newly adjusted resources as requiring a label update */ DPA_RESOURCE_ADJUSTED = 1 << 0, };
@@ -121,6 +125,7 @@ struct nd_mapping_desc {int position; }; +struct nd_region ;struct nd_region_desc {struct resource *res;struct nd_mapping_desc *mapping;
@@ -133,6 +138,7 @@ struct nd_region_desc {int target_node;unsigned long flags;struct device_node *of_node; +int (*flush)(struct nd_region *nd_region,struct bio *bio); };struct device ;
@@ -260,7 +266,8 @@ unsigned long nd_blk_memremap_flags(struct nd_blk_region *ndbr);unsigned int nd_region_acquire_lane (struct nd_region *nd_region);void nd_region_release_lane (struct nd_region *nd_region,unsigned int lane);u64 nd_fletcher64 (void *addr,size_t len,bool le); -void nvdimm_flush (struct nd_region *nd_region); +int nvdimm_flush (struct nd_region *nd_region,struct bio *bio); +int generic_nvdimm_flush (struct nd_region *nd_region);int nvdimm_has_flush (struct nd_region *nd_region);int nvdimm_has_cache (struct nd_region *nd_region);int nvdimm_in_overwrite (struct nvdimm *nvdimm);
Signature · 1 line
2.20.1
PATCH v12 2/7 virtio-pmem: Add virtio pmem driver
In reply to Pankaj GuptaThis patch adds virtio-pmem driver for KVM guest.
Guest reads the persistent memory range information from Qemu over VIRTIO and registers it on nvdimm_bus. It also creates a nd_region object with the persistent memory range information so that existing 'nvdimm/pmem' driver can reserve this into system memory map. This way 'virtio-pmem' driver uses existing functionality of pmem driver to register persistent memory compatible for DAX capable filesystems.
This also provides function to perform guest flush over VIRTIO from 'pmem' driver when userspace performs flush on DAX memory range.
--- drivers/nvdimm/Makefile | 1 + drivers/nvdimm/nd_virtio.c | 124 +++++++++++++++++++++++++++++++ drivers/nvdimm/virtio_pmem.c | 122 ++++++++++++++++++++++++++++++ drivers/nvdimm/virtio_pmem.h | 55 ++++++++++++++ drivers/virtio/Kconfig | 11 +++ include/uapi/linux/virtio_ids.h | 1 + include/uapi/linux/virtio_pmem.h | 35 +++++++++ 7 files changed, 349 insertions(+) create mode 100644 drivers/nvdimm/nd_virtio.c create mode 100644 drivers/nvdimm/virtio_pmem.c create mode 100644 drivers/nvdimm/virtio_pmem.h create mode 100644 include/uapi/linux/virtio_pmem.h
drivers/nvdimm/Makefile +1 -0
@@ -5,6 +5,7 @@ obj-$(CONFIG_ND_BTT) += nd_btt.o obj-$(CONFIG_ND_BLK) += nd_blk.o obj-$(CONFIG_X86_PMEM_LEGACY) += nd_e820.o obj-$(CONFIG_OF_PMEM) += of_pmem.o +obj-$(CONFIG_VIRTIO_PMEM) += virtio_pmem.o nd_virtio.o nd_pmem-y := pmem.o
drivers/nvdimm/nd_virtio.c new file +124 -0
124 lines not on this page · raw message
drivers/nvdimm/virtio_pmem.c new file +122 -0
122 lines not on this page · raw message
drivers/nvdimm/virtio_pmem.h new file +55 -0
55 lines not on this page · raw message
drivers/virtio/Kconfig +11 -0
@@ -42,6 +42,17 @@ config VIRTIO_PCI_LEGACY If unsure, say Y. +config VIRTIO_PMEM + tristate "Support for virtio pmem driver" + depends on VIRTIO + depends on LIBNVDIMM + help + This driver provides access to virtio-pmem devices, storage devices + that are mapped into the physical address space - similar to NVDIMMs + - with a virtio-based flushing interface. + + If unsure, say Y. + config VIRTIO_BALLOON tristate "Virtio balloon driver" depends on VIRTIO
include/uapi/linux/virtio_ids.h +1 -0
@@ -43,5 +43,6 @@#define VIRTIO_ID_INPUT 18/* virtio input */ #define VIRTIO_ID_VSOCK 19/* virtio vsock transport */ #define VIRTIO_ID_CRYPTO 20/* virtio crypto */ +#define VIRTIO_ID_PMEM 27/* virtio pmem */ #endif /* _LINUX_VIRTIO_IDS_H */
include/uapi/linux/virtio_pmem.h new file +35 -0
@@ -0,0 +1,35 @@ +/* SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause */ +/* +* Definitions for virtio-pmem devices. +* +* Copyright (C) 2019 Red Hat, Inc. +* +* Author(s): Pankaj Gupta <[email protected]> +*/ + +#ifndef _UAPI_LINUX_VIRTIO_PMEM_H +#define _UAPI_LINUX_VIRTIO_PMEM_H + +#include <linux/types.h> +#include <linux/virtio_types.h> +#include <linux/virtio_ids.h> +#include <linux/virtio_config.h> + +struct virtio_pmem_config { +__le64 start; +__le64 size; +}; + +#define VIRTIO_PMEM_REQ_TYPE_FLUSH 0 + +struct virtio_pmem_resp { +/* Host return status corresponding to flush request */ +__virtio32 ret; +}; + +struct virtio_pmem_req { +/* command type */ +__virtio32 type; +}; + +#endif
Signature · 1 line
2.20.1
Re: [PATCH v12 2/7] virtio-pmem: Add virtio pmem driver
Quoted 20 lines · On Tue, 11 Jun 2019 22:07:57 +0530 Pankaj Gupta <[email protected]> wrote:
This patch adds virtio-pmem driver for KVM guest.
Guest reads the persistent memory range information from Qemu over VIRTIO and registers it on
nvdimm_bus. It also creates and_regionobject with the persistent memory range information so that existing 'nvdimm/pmem' driver can reserve this into system memory map. This way 'virtio-pmem' driver uses existing functionality of pmem driver to register persistent memory compatible for DAX capable filesystems.This also provides function to perform guest flush over VIRTIO from 'pmem' driver when userspace performs flush on DAX memory range.
Signed-off-by Pankaj Gupta <[email protected]>Reviewed-by Yuval Shaia <[email protected]>Acked-by Jakub Staron <[email protected]>Tested-by Jakub Staron <[email protected]>
Quoted 13 lines
--- drivers/nvdimm/Makefile | 1 + drivers/nvdimm/nd_virtio.c | 124 +++++++++++++++++++++++++++++++ drivers/nvdimm/virtio_pmem.c | 122 ++++++++++++++++++++++++++++++ drivers/nvdimm/virtio_pmem.h | 55 ++++++++++++++ drivers/virtio/Kconfig | 11 +++ include/uapi/linux/virtio_ids.h | 1 + include/uapi/linux/virtio_pmem.h | 35 +++++++++ 7 files changed, 349 insertions(+) create mode 100644 drivers/nvdimm/nd_virtio.c create mode 100644 drivers/nvdimm/virtio_pmem.c create mode 100644 drivers/nvdimm/virtio_pmem.h create mode 100644 include/uapi/linux/virtio_pmem.h
Sorry about being late to the party; this one has been sitting in my 'to review' queue for far too long :(
(...)
Quoted 31 lines
drivers/nvdimm/nd_virtio.c new file +25
-0@@ -0,0 +1,124 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* virtio_pmem.c: Virtio pmem Driver +* +* Discovers persistent memory range information +* from host and provides a virtio based flushing +* interface. +*/ +#include "virtio_pmem.h" +#include "nd.h" + +/* The interrupt handler */ +void host_ack (struct virtqueue *vq) +{ +struct virtio_pmem *vpmem = vq->vdev->priv; +struct virtio_pmem_request *req_data, *req_buf; +unsigned long flags; +unsigned int len; + +spin_lock_irqsave (&vpmem->pmem_lock, flags); +while ((req_data =virtqueue_get_buf (vq, &len)) !=NULL ) { + req_data->done = true; +wake_up (&req_data->host_acked); + +if (!list_empty (&vpmem->req_list)) {
drivers/nvdimm/nd_virtio.c line 35
Quoted 10 lines
drivers/nvdimm/nd_virtio.c new file +10
-0+ req_buf =list_first_entry (&vpmem->req_list, + struct virtio_pmem_request, list); + req_buf->wq_buf_avail = true; +wake_up (&req_buf->wq_buf); +list_del (&req_buf->list); + } + } +spin_unlock_irqrestore (&vpmem->pmem_lock, flags); +} +EXPORT_SYMBOL_GPL (host_ack);
Nit: 'host_ack' looks a bit generic for an exported function... would 'virtio_pmem_host_ack' maybe be better?
drivers/nvdimm/nd_virtio.c line 38
Quoted 3 lines
drivers/nvdimm/nd_virtio.c new file +3
-0+ +/* The request submission function */ +int virtio_pmem_flush (struct nd_region *nd_region)
I don't see an EXPORT_SYMBOL_GPL() for this function... should it get one, or should it be made static?
Quoted 24 lines
drivers/nvdimm/nd_virtio.c new file +24
-0+{ +struct virtio_device *vdev = nd_region->provider_data; +struct virtio_pmem *vpmem = vdev->priv; +struct virtio_pmem_request *req_data; +struct scatterlist *sgs[2 ], sg, ret; +unsigned long flags; +int err, err1; + +might_sleep (); + req_data =kmalloc (sizeof (*req_data),GFP_KERNEL ); +if (!req_data) +return -ENOMEM ; + + req_data->done = false; +init_waitqueue_head (&req_data->host_acked); +init_waitqueue_head (&req_data->wq_buf); +INIT_LIST_HEAD (&req_data->list); + req_data->req.type =cpu_to_virtio32 (vdev,VIRTIO_PMEM_REQ_TYPE_FLUSH ); +sg_init_one (&sg, &req_data->req,sizeof (req_data->req)); + sgs[0 ] = &sg; +sg_init_one (&ret, &req_data->resp.ret,sizeof (req_data->resp)); + sgs[1 ] = &ret; + +spin_lock_irqsave (&vpmem->pmem_lock, flags);
drivers/nvdimm/nd_virtio.c line 72
Quoted 10 lines
drivers/nvdimm/nd_virtio.c new file +10
-0+/* +* If virtqueue_add_sgs returns -ENOSPC then req_vq virtual +* queue does not have free descriptor. We add the request +* to req_list and wait for host_ack to wake us up when free +* slots are available. +*/ +while ((err =virtqueue_add_sgs (vpmem->req_vq, sgs,1 ,1 , req_data, +GFP_ATOMIC )) == -ENOSPC ) { + +dev_err (&vdev->dev,"failed to send command to virtio pmem device, no free slots in the virtqueue\n" );
Hm... by the comment above I would have thought that this is not really an error, but rather a temporary condition? Maybe downgrade this to dev_info()?
Quoted 4 lines
drivers/nvdimm/nd_virtio.c new file +4
-0+ req_data->wq_buf_avail = false; +list_add_tail (&req_data->list, &vpmem->req_list); +spin_unlock_irqrestore (&vpmem->pmem_lock, flags); +
drivers/nvdimm/nd_virtio.c line 86
Quoted 10 lines
drivers/nvdimm/nd_virtio.c new file +10
-0+/* A host response results in "host_ack" getting called */ +wait_event (req_data->wq_buf, req_data->wq_buf_avail); +spin_lock_irqsave (&vpmem->pmem_lock, flags); + } + err1 =virtqueue_kick (vpmem->req_vq); +spin_unlock_irqrestore (&vpmem->pmem_lock, flags); +/* +* virtqueue_add_sgs failed with error different than -ENOSPC, we can't +* do anything about that. +*/
Does it make sense to kick if you couldn't add at all?
drivers/nvdimm/nd_virtio.c line 88
Quoted 2 lines
drivers/nvdimm/nd_virtio.c new file +2
-0+if (err || !err1) { +dev_info (&vdev->dev,"failed to send command to virtio pmem device\n" );
If this is dev_info, I think the error above really should be dev_info as well (and maybe also log the error value)?
Quoted 7 lines
drivers/nvdimm/nd_virtio.c new file +7
-0+ err = -EIO ; + }else { +/* A host repsonse results in "host_ack" getting called */ +wait_event (req_data->host_acked, req_data->done); + err =virtio32_to_cpu (vdev, req_data->resp.ret); + } +
drivers/nvdimm/nd_virtio.c line 105
Quoted 10 lines
drivers/nvdimm/nd_virtio.c new file +10
-0+kfree (req_data); +return err; +}; + +/* The asynchronous flush callback function */ +int async_pmem_flush (struct nd_region *nd_region,struct bio *bio) +{ +/* Create child bio for asynchronous flush and chain with +* parent bio. Otherwise directly call nd_region flush. +*/
Nit: The comment should start with an otherwise empty /* line.
Quoted 9 lines
drivers/nvdimm/nd_virtio.c new file +9
-0+if (bio && bio->bi_iter.bi_sector !=-1 ) { +struct bio *child =bio_alloc (GFP_ATOMIC ,0 ); + +if (!child) +return -ENOMEM ; +bio_copy_dev (child, bio); + child->bi_opf =REQ_PREFLUSH ; + child->bi_iter.bi_sector =-1 ; +bio_chain (child, bio);
drivers/nvdimm/nd_virtio.c line 124
Quoted 10 lines
drivers/nvdimm/nd_virtio.c new file +10
-0+submit_bio (child); +return 0 ; + } +if (virtio_pmem_flush (nd_region)) +return -EIO ; + +return 0 ; +}; +EXPORT_SYMBOL_GPL (async_pmem_flush); +MODULE_LICENSE ("GPL" );
(...)
I have only some more minor comments; on the whole, this looks good to me.
Re: [PATCH v12 2/7] virtio-pmem: Add virtio pmem driver
Hi Cornelia,
Quoted 78 lines
Quoted 20 lines · On Tue, 11 Jun 2019 22:07:57 +0530 Pankaj Gupta <[email protected]> wrote:
This patch adds virtio-pmem driver for KVM guest.
Guest reads the persistent memory range information from Qemu over VIRTIO and registers it on
nvdimm_bus. It also creates and_regionobject with the persistent memory range information so that existing 'nvdimm/pmem' driver can reserve this into system memory map. This way 'virtio-pmem' driver uses existing functionality of pmem driver to register persistent memory compatible for DAX capable filesystems.This also provides function to perform guest flush over VIRTIO from 'pmem' driver when userspace performs flush on DAX memory range.
Signed-off-by Pankaj Gupta <[email protected]>Reviewed-by Yuval Shaia <[email protected]>Acked-by Jakub Staron <[email protected]>Tested-by Jakub Staron <[email protected]>Quoted 13 lines
--- drivers/nvdimm/Makefile | 1 + drivers/nvdimm/nd_virtio.c | 124 +++++++++++++++++++++++++++++++ drivers/nvdimm/virtio_pmem.c | 122 ++++++++++++++++++++++++++++++ drivers/nvdimm/virtio_pmem.h | 55 ++++++++++++++ drivers/virtio/Kconfig | 11 +++ include/uapi/linux/virtio_ids.h | 1 + include/uapi/linux/virtio_pmem.h | 35 +++++++++ 7 files changed, 349 insertions(+) create mode 100644 drivers/nvdimm/nd_virtio.c create mode 100644 drivers/nvdimm/virtio_pmem.c create mode 100644 drivers/nvdimm/virtio_pmem.h create mode 100644 include/uapi/linux/virtio_pmem.hSorry about being late to the party; this one has been sitting in my 'to review' queue for far too long :(
(...)
Quoted 36 lines
drivers/nvdimm/nd_virtio.c new file +30
-0@@ -0,0 +1,124 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* virtio_pmem.c: Virtio pmem Driver +* +* Discovers persistent memory range information +* from host and provides a virtio based flushing +* interface. +*/ +#include "virtio_pmem.h" +#include "nd.h" + +/* The interrupt handler */ +void host_ack (struct virtqueue *vq) +{ +struct virtio_pmem *vpmem = vq->vdev->priv; +struct virtio_pmem_request *req_data, *req_buf; +unsigned long flags; +unsigned int len; + +spin_lock_irqsave (&vpmem->pmem_lock, flags); +while ((req_data =virtqueue_get_buf (vq, &len)) !=NULL ) { + req_data->done = true; +wake_up (&req_data->host_acked); + +if (!list_empty (&vpmem->req_list)) { + req_buf =list_first_entry (&vpmem->req_list, + struct virtio_pmem_request, list); + req_buf->wq_buf_avail = true; +wake_up (&req_buf->wq_buf); +list_del (&req_buf->list);
Quoted 8 lines
drivers/nvdimm/nd_virtio.c line 35
Quoted 5 lines
drivers/nvdimm/nd_virtio.c new file +5
-0+ } + } +spin_unlock_irqrestore (&vpmem->pmem_lock, flags); +} +EXPORT_SYMBOL_GPL (host_ack);Nit: '
host_ack' looks a bit generic for an exported function... would 'virtio_pmem_host_ack' maybe be better?
Yes, this looks better. Changed.
Quoted 7 lines
drivers/nvdimm/nd_virtio.c line 38
Quoted 4 lines
drivers/nvdimm/nd_virtio.c new file +3
-0+ +/* The request submission function */ +int virtio_pmem_flush (struct nd_region *nd_region)I don't see an
EXPORT_SYMBOL_GPL()for this function... should it get one, or should it be made static?
Made static. Leftover from last refactor of 'asyc_pmem_flush'.
Quoted 32 lines
Quoted 32 lines
drivers/nvdimm/nd_virtio.c new file +31
-0+{ + struct virtio_device *vdev = nd_region->provider_data; + struct virtio_pmem *vpmem = vdev->priv; + struct virtio_pmem_request *req_data; + struct scatterlist *sgs[2], sg, ret; + unsigned long flags; + int err, err1; + + might_sleep(); + req_data = kmalloc(sizeof(*req_data), GFP_KERNEL); + if (!req_data) + return -ENOMEM; + + req_data->done = false; + init_waitqueue_head(&req_data->host_acked); + init_waitqueue_head(&req_data->wq_buf); + INIT_LIST_HEAD(&req_data->list); + req_data->req.type = cpu_to_virtio32(vdev, VIRTIO_PMEM_REQ_TYPE_FLUSH); + sg_init_one(&sg, &req_data->req, sizeof(req_data->req)); + sgs[0] = &sg; + sg_init_one(&ret, &req_data->resp.ret, sizeof(req_data->resp)); + sgs[1] = &ret; + + spin_lock_irqsave(&vpmem->pmem_lock, flags); + /* + * If virtqueue_add_sgs returns -ENOSPC then req_vq virtual + * queue does not have free descriptor. We add the request + * to req_list and wait for host_ack to wake us up when free + * slots are available. + */ + while ((err = virtqueue_add_sgs(vpmem->req_vq, sgs, 1, 1, req_data,
Quoted 8 lines
Quoted 4 lines
drivers/nvdimm/nd_virtio.c new file +3
-0+ GFP_ATOMIC)) == -ENOSPC) { + + dev_err(&vdev->dev, "failed to send command to virtio pmem device, nofree slots in the virtqueue\n");
Hm... by the comment above I would have thought that this is not really an error, but rather a temporary condition? Maybe downgrade this to
dev_info()?
o.k.
Quoted 17 lines
Quoted 5 lines
drivers/nvdimm/nd_virtio.c new file +4
-0+ req_data->wq_buf_avail = false; +list_add_tail (&req_data->list, &vpmem->req_list); +spin_unlock_irqrestore (&vpmem->pmem_lock, flags); +drivers/nvdimm/nd_virtio.c
Quoted 10 lines
drivers/nvdimm/nd_virtio.c new file +10
-0+/* A host response results in "host_ack" getting called */ +wait_event (req_data->wq_buf, req_data->wq_buf_avail); +spin_lock_irqsave (&vpmem->pmem_lock, flags); + } + err1 =virtqueue_kick (vpmem->req_vq); +spin_unlock_irqrestore (&vpmem->pmem_lock, flags); +/* +* virtqueue_add_sgs failed with error different than -ENOSPC, we can't +* do anything about that. +*/ Does it make sense to kick if you couldn't add at all?
When we could not add because of -ENOSPC we are waiting and when buffer is added then only we do a kick. For any other error which might be a rare occurrence, I think kick is harmless here and keeps the code clean?
Quoted 6 lines
drivers/nvdimm/nd_virtio.c
Quoted 3 lines
drivers/nvdimm/nd_virtio.c new file +2
-0+if (err || !err1) { +dev_info (&vdev->dev,"failed to send command to virtio pmem device\n" );If this is
dev_info, I think the error above really should bedev_infoas well (and maybe also log the error value)?
o.k.
Quoted 12 lines
Quoted 12 lines
drivers/nvdimm/nd_virtio.c new file +11
-0+ err = -EIO ; + }else { +/* A host repsonse results in "host_ack" getting called */ +wait_event (req_data->host_acked, req_data->done); + err =virtio32_to_cpu (vdev, req_data->resp.ret); + } + +kfree (req_data); +return err; +}; +
Quoted 8 lines
drivers/nvdimm/nd_virtio.c
Quoted 6 lines
drivers/nvdimm/nd_virtio.c new file +6
-0+/* The asynchronous flush callback function */ +int async_pmem_flush (struct nd_region *nd_region,struct bio *bio) +{ +/* Create child bio for asynchronous flush and chain with +* parent bio. Otherwise directly call nd_region flush. +*/ Nit: The comment should start with an otherwise empty /* line.
yes.
Quoted 23 lines
Quoted 20 lines
drivers/nvdimm/nd_virtio.c new file +19
-0+if (bio && bio->bi_iter.bi_sector !=-1 ) { +struct bio *child =bio_alloc (GFP_ATOMIC ,0 ); + +if (!child) +return -ENOMEM ; +bio_copy_dev (child, bio); + child->bi_opf =REQ_PREFLUSH ; + child->bi_iter.bi_sector =-1 ; +bio_chain (child, bio); +submit_bio (child); +return 0 ; + } +if (virtio_pmem_flush (nd_region)) +return -EIO ; + +return 0 ; +}; +EXPORT_SYMBOL_GPL (async_pmem_flush); +MODULE_LICENSE ("GPL" );(...)
Quoted 2 lines
I have only some more minor comments; on the whole, this looks good to me.
Sure, Thank you. Attaching below on top changes on current patch2 based on your suggestions. Let me know if these are okay and then will send official v13 to for upstream merging.
Thanks,
Pankaj
===============
drivers/nvdimm/nd_virtio.c +8 -6
@@ -10,7 +10,7 @@#include "nd.h" /* The interrupt handler */ -void host_ack (struct virtqueue *vq) +void virtio_pmem_host_ack (struct virtqueue *vq) {struct virtio_pmem *vpmem = vq->vdev->priv;struct virtio_pmem_request *req_data, *req_buf;
@@ -32,10 +32,10 @@ void host_ack(struct virtqueue *vq) }spin_unlock_irqrestore (&vpmem->pmem_lock, flags); } -EXPORT_SYMBOL_GPL (host_ack); +EXPORT_SYMBOL_GPL (virtio_pmem_host_ack);/* The request submission function */ -int virtio_pmem_flush (struct nd_region *nd_region) +static int virtio_pmem_flush (struct nd_region *nd_region) {struct virtio_device *vdev = nd_region->provider_data;struct virtio_pmem *vpmem = vdev->priv;
@@ -69,7 +69,7 @@ int virtio_pmem_flush(struct nd_region *nd_region)while ((err =virtqueue_add_sgs (vpmem->req_vq, sgs,1 ,1 , req_data,GFP_ATOMIC )) == -ENOSPC ) { -dev_err (&vdev->dev,"failed to send command to virtio pmem device, no free slots in the virtqueue\n" ); +dev_info (&vdev->dev,"failed to send command to virtio pmem device, no free slots in the virtqueue\n" ); req_data->wq_buf_avail = false;list_add_tail (&req_data->list, &vpmem->req_list);spin_unlock_irqrestore (&vpmem->pmem_lock, flags);
@@ -90,7 +90,8 @@ int virtio_pmem_flush(struct nd_region *nd_region) }else {/* A host repsonse results in "host_ack" getting called */ wait_event (req_data->host_acked, req_data->done); - err =virtio32_to_cpu (vdev, req_data->resp.ret); +if ((err =virtio32_to_cpu (vdev, req_data->resp.ret))) + err = -EIO ; }kfree (req_data);
@@ -100,7 +101,8 @@ int virtio_pmem_flush(struct nd_region *nd_region)/* The asynchronous flush callback function */ int async_pmem_flush (struct nd_region *nd_region,struct bio *bio) { -/* Create child bio for asynchronous flush and chain with +/* +* Create child bio for asynchronous flush and chain with * parent bio. Otherwise directly call nd_region flush. */ if (bio && bio->bi_iter.bi_sector !=-1 ) {
drivers/nvdimm/virtio_pmem.c +1 -1
@@ -19,7 +19,7 @@ static int init_vq(struct virtio_pmem *vpmem) {/* single vq */ vpmem->req_vq =virtio_find_single_vq (vpmem->vdev, - host_ack,"flush_queue" ); + virtio_pmem_host_ack,"flush_queue" );if (IS_ERR (vpmem->req_vq))return PTR_ERR (vpmem->req_vq);
drivers/nvdimm/virtio_pmem.h +1 -1
@@ -50,6 +50,6 @@ struct virtio_pmem { uint64_t size; }; -void host_ack (struct virtqueue *vq); +void virtio_pmem_host_ack (struct virtqueue *vq);int async_pmem_flush (struct nd_region *nd_region,struct bio *bio);#endif
Re: [PATCH v12 2/7] virtio-pmem: Add virtio pmem driver
Hi Pankaj,
Quoted 15 lines · On Tue, 11 Jun 2019 23:34:50 -0400 (EDT) Pankaj Gupta <[email protected]>…
Hi Cornelia,
Quoted 4 lines
On Tue, 11 Jun 2019 22:07:57 +0530
Pankaj Gupta <[email protected]> wrote:Quoted 8 lines
Review
Quoted 6 lines
(quoted diff) +6
-0+ err1 = virtqueue_kick(vpmem->req_vq); + spin_unlock_irqrestore(&vpmem->pmem_lock, flags); + /* + * virtqueue_add_sgs failed with error different than -ENOSPC, we can't + * do anything about that. + */Does it make sense to kick if you couldn't add at all?
Quoted 3 lines
When we could not add because of
-ENOSPCwe are waiting and when buffer is added then only we do a kick. For any other error which might be a rare occurrence, I think kick is harmless here and keeps the code clean?
Yes, I agree it does not hurt. Let's keep it as-is.
Quoted 3 lines
Sure, Thank you. Attaching below on top changes on current patch2 based on your suggestions. Let me know if these are okay and then will send official v13 to for upstream merging.
Looks good to me, except for one change.
[Again sorry for the late review, did not want to get the version numbers up :)]
Quoted 42 lines
Thanks,
Pankaj===============
drivers/nvdimm/nd_virtio.c +4
-4@@ -10,7 +10,7 @@#include "nd.h" /* The interrupt handler */ -void host_ack (struct virtqueue *vq) +void virtio_pmem_host_ack (struct virtqueue *vq) {struct virtio_pmem *vpmem = vq->vdev->priv;struct virtio_pmem_request *req_data, *req_buf;@@ -32,10 +32,10 @@ void host_ack(struct virtqueue *vq) }spin_unlock_irqrestore (&vpmem->pmem_lock, flags); } -EXPORT_SYMBOL_GPL (host_ack); +EXPORT_SYMBOL_GPL (virtio_pmem_host_ack);/* The request submission function */ -int virtio_pmem_flush (struct nd_region *nd_region) +static int virtio_pmem_flush (struct nd_region *nd_region) {struct virtio_device *vdev = nd_region->provider_data;struct virtio_pmem *vpmem = vdev->priv;@@ -69,7 +69,7 @@ int virtio_pmem_flush(struct nd_region *nd_region)while ((err =virtqueue_add_sgs (vpmem->req_vq, sgs,1 ,1 , req_data,GFP_ATOMIC )) == -ENOSPC ) { -dev_err (&vdev->dev,"failed to send command to virtio pmem device, no free slots in the virtqueue\n" ); +dev_info (&vdev->dev,"failed to send command to virtio pmem device, no free slots in the virtqueue\n" ); req_data->wq_buf_avail = false;list_add_tail (&req_data->list, &vpmem->req_list);spin_unlock_irqrestore (&vpmem->pmem_lock, flags);
drivers/nvdimm/nd_virtio.c int virtio_pmem_flush(struct nd_region *nd_region) · line 94
Quoted 6 lines
drivers/nvdimm/nd_virtio.c +2
-1@@ -90,7 +90,8 @@ int virtio_pmem_flush(struct nd_region *nd_region) }else {/* A host repsonse results in "host_ack" getting called */ wait_event (req_data->host_acked, req_data->done); - err =virtio32_to_cpu (vdev, req_data->resp.ret); +if ((err =virtio32_to_cpu (vdev, req_data->resp.ret))) + err = -EIO ;
Hm, why are you making this change? I think the previous code was fine.
Quoted 38 lines
drivers/nvdimm/nd_virtio.c +2
-1}kfree (req_data);@@ -100,7 +101,8 @@ int virtio_pmem_flush(struct nd_region *nd_region)/* The asynchronous flush callback function */ int async_pmem_flush (struct nd_region *nd_region,struct bio *bio) { -/* Create child bio for asynchronous flush and chain with +/* +* Create child bio for asynchronous flush and chain with * parent bio. Otherwise directly call nd_region flush. */ if (bio && bio->bi_iter.bi_sector !=-1 ) {drivers/nvdimm/virtio_pmem.c +1
-1@@ -19,7 +19,7 @@ static int init_vq(struct virtio_pmem *vpmem) {/* single vq */ vpmem->req_vq =virtio_find_single_vq (vpmem->vdev, - host_ack,"flush_queue" ); + virtio_pmem_host_ack,"flush_queue" );if (IS_ERR (vpmem->req_vq))return PTR_ERR (vpmem->req_vq);drivers/nvdimm/virtio_pmem.h +1
-1@@ -50,6 +50,6 @@ struct virtio_pmem { uint64_t size; }; -void host_ack (struct virtqueue *vq); +void virtio_pmem_host_ack (struct virtqueue *vq);int async_pmem_flush (struct nd_region *nd_region,struct bio *bio);#endif
Re: [Qemu-devel] [PATCH v12 2/7] virtio-pmem: Add virtio pmem driver
Quoted 23 lines
Hi Pankaj,
Quoted 16 lines · On Tue, 11 Jun 2019 23:34:50 -0400 (EDT) Pankaj Gupta <[email protected]>…
Hi Cornelia,
Quoted 13 lines
Review
Quoted 7 lines · On Tue, 11 Jun 2019 22:07:57 +0530 Pankaj Gupta <[email protected]> wrote:
(quoted diff) +4
-0+ err1 = virtqueue_kick(vpmem->req_vq); + spin_unlock_irqrestore(&vpmem->pmem_lock, flags); + /* + * virtqueue_add_sgs failed with error different than -ENOSPC, wecan't
(quoted diff) +2
-0+ * do anything about that. + */Does it make sense to kick if you couldn't add at all?
Quoted 7 lines
Quoted 5 lines
When we could not add because of
-ENOSPCwe are waiting and when buffer is added
then only we do a kick. For any other error which might be a rare occurrence, I think
kick is harmless here and keeps the code clean?Yes, I agree it does not hurt. Let's keep it as-is.
Sure.
Quoted 7 lines
Quoted 5 lines
Sure, Thank you. Attaching below on top changes on current patch2 based on your suggestions. Let me know if these are okay and then will send official v13 to for upstream merging.
Looks good to me, except for one change.
Sure. Will send v13 shortly.
Quoted 3 lines
[Again sorry for the late review, did not want to get the version numbers up :)]
Thank you :)
Quoted 37 lines
Quoted 37 lines
Thanks,
Pankaj===============
drivers/nvdimm/nd_virtio.c +3
-3@@ -10,7 +10,7 @@#include "nd.h" /* The interrupt handler */ -void host_ack (struct virtqueue *vq) +void virtio_pmem_host_ack (struct virtqueue *vq) {struct virtio_pmem *vpmem = vq->vdev->priv;struct virtio_pmem_request *req_data, *req_buf;@@ -32,10 +32,10 @@ void host_ack(struct virtqueue *vq) }spin_unlock_irqrestore (&vpmem->pmem_lock, flags); } -EXPORT_SYMBOL_GPL (host_ack); +EXPORT_SYMBOL_GPL (virtio_pmem_host_ack);/* The request submission function */ -int virtio_pmem_flush (struct nd_region *nd_region) +static int virtio_pmem_flush (struct nd_region *nd_region) {struct virtio_device *vdev = nd_region->provider_data;struct virtio_pmem *vpmem = vdev->priv;@@ -69,7 +69,7 @@ int virtio_pmem_flush(struct nd_region *nd_region)while ((err =virtqueue_add_sgs (vpmem->req_vq, sgs,1 ,1 , req_data,GFP_ATOMIC )) == -ENOSPC ) {
Quoted 16 lines
Quoted 8 lines
drivers/nvdimm/nd_virtio.c +1
-1- dev_err(&vdev->dev, "failed to send command to virtio pmem device, no free slots in the virtqueue\n"); + dev_info(&vdev->dev, "failed to send command to virtio pmem device, no free slots in the virtqueue\n"); req_data->wq_buf_avail = false;list_add_tail(&req_data->list, &vpmem->req_list); spin_unlock_irqrestore(&vpmem->pmem_lock, flags);drivers/nvdimm/nd_virtio.c int virtio_pmem_flush(struct nd_region *nd_region) · line 94
Quoted 6 lines
drivers/nvdimm/nd_virtio.c +2
-1@@ -90,7 +90,8 @@ int virtio_pmem_flush(struct nd_region *nd_region) }else {/* A host repsonse results in "host_ack" getting called */ wait_event (req_data->host_acked, req_data->done); - err =virtio32_to_cpu (vdev, req_data->resp.ret); +if ((err =virtio32_to_cpu (vdev, req_data->resp.ret))) + err = -EIO ;Hm, why are you making this change? I think the previous code was fine.
Yes, Something came to my mind while making the change but I agree will keep this as it was before.
Quoted 43 lines
Quoted 40 lines
drivers/nvdimm/nd_virtio.c +2
-1}kfree (req_data);@@ -100,7 +101,8 @@ int virtio_pmem_flush(struct nd_region *nd_region)/* The asynchronous flush callback function */ int async_pmem_flush (struct nd_region *nd_region,struct bio *bio) { -/* Create child bio for asynchronous flush and chain with +/* +* Create child bio for asynchronous flush and chain with * parent bio. Otherwise directly call nd_region flush. */ if (bio && bio->bi_iter.bi_sector !=-1 ) {drivers/nvdimm/virtio_pmem.c +1
-1@@ -19,7 +19,7 @@ static int init_vq(struct virtio_pmem *vpmem) {/* single vq */ vpmem->req_vq =virtio_find_single_vq (vpmem->vdev, - host_ack,"flush_queue" ); + virtio_pmem_host_ack,"flush_queue" );if (IS_ERR (vpmem->req_vq))return PTR_ERR (vpmem->req_vq);drivers/nvdimm/virtio_pmem.h +1
-1@@ -50,6 +50,6 @@ struct virtio_pmem { uint64_t size; }; -void host_ack (struct virtqueue *vq); +void virtio_pmem_host_ack (struct virtqueue *vq);int async_pmem_flush (struct nd_region *nd_region,struct bio *bio);#endif
Re: [Qemu-devel] [PATCH v12 2/7] virtio-pmem: Add virtio pmem driver
In reply to Cornelia HuckQuoted 23 lines
Hi Pankaj,
Quoted 16 lines · On Tue, 11 Jun 2019 23:34:50 -0400 (EDT) Pankaj Gupta <[email protected]>…
Hi Cornelia,
Quoted 13 lines
Review
Quoted 7 lines · On Tue, 11 Jun 2019 22:07:57 +0530 Pankaj Gupta <[email protected]> wrote:
(quoted diff) +4
-0+ err1 = virtqueue_kick(vpmem->req_vq); + spin_unlock_irqrestore(&vpmem->pmem_lock, flags); + /* + * virtqueue_add_sgs failed with error different than -ENOSPC, wecan't
(quoted diff) +2
-0+ * do anything about that. + */Does it make sense to kick if you couldn't add at all?
Quoted 7 lines
Quoted 5 lines
When we could not add because of
-ENOSPCwe are waiting and when buffer is added
then only we do a kick. For any other error which might be a rare occurrence, I think
kick is harmless here and keeps the code clean?Yes, I agree it does not hurt. Let's keep it as-is.
Sure.
Quoted 7 lines
Quoted 5 lines
Sure, Thank you. Attaching below on top changes on current patch2 based on your suggestions. Let me know if these are okay and then will send official v13 to for upstream merging.
Looks good to me, except for one change.
Sure. Will send v13 shortly.
Quoted 3 lines
[Again sorry for the late review, did not want to get the version numbers up :)]
Thank you :)
Quoted 37 lines
Quoted 37 lines
Thanks,
Pankaj===============
drivers/nvdimm/nd_virtio.c +3
-3@@ -10,7 +10,7 @@#include "nd.h" /* The interrupt handler */ -void host_ack (struct virtqueue *vq) +void virtio_pmem_host_ack (struct virtqueue *vq) {struct virtio_pmem *vpmem = vq->vdev->priv;struct virtio_pmem_request *req_data, *req_buf;@@ -32,10 +32,10 @@ void host_ack(struct virtqueue *vq) }spin_unlock_irqrestore (&vpmem->pmem_lock, flags); } -EXPORT_SYMBOL_GPL (host_ack); +EXPORT_SYMBOL_GPL (virtio_pmem_host_ack);/* The request submission function */ -int virtio_pmem_flush (struct nd_region *nd_region) +static int virtio_pmem_flush (struct nd_region *nd_region) {struct virtio_device *vdev = nd_region->provider_data;struct virtio_pmem *vpmem = vdev->priv;@@ -69,7 +69,7 @@ int virtio_pmem_flush(struct nd_region *nd_region)while ((err =virtqueue_add_sgs (vpmem->req_vq, sgs,1 ,1 , req_data,GFP_ATOMIC )) == -ENOSPC ) {
Quoted 16 lines
Quoted 8 lines
drivers/nvdimm/nd_virtio.c +1
-1- dev_err(&vdev->dev, "failed to send command to virtio pmem device, no free slots in the virtqueue\n"); + dev_info(&vdev->dev, "failed to send command to virtio pmem device, no free slots in the virtqueue\n"); req_data->wq_buf_avail = false;list_add_tail(&req_data->list, &vpmem->req_list); spin_unlock_irqrestore(&vpmem->pmem_lock, flags);drivers/nvdimm/nd_virtio.c int virtio_pmem_flush(struct nd_region *nd_region) · line 94
Quoted 6 lines
drivers/nvdimm/nd_virtio.c +2
-1@@ -90,7 +90,8 @@ int virtio_pmem_flush(struct nd_region *nd_region) }else {/* A host repsonse results in "host_ack" getting called */ wait_event (req_data->host_acked, req_data->done); - err =virtio32_to_cpu (vdev, req_data->resp.ret); +if ((err =virtio32_to_cpu (vdev, req_data->resp.ret))) + err = -EIO ;Hm, why are you making this change? I think the previous code was fine.
Yes, Something came to my mind while making the change but I agree will keep this as it was before.
Quoted 43 lines
Quoted 40 lines
drivers/nvdimm/nd_virtio.c +2
-1}kfree (req_data);@@ -100,7 +101,8 @@ int virtio_pmem_flush(struct nd_region *nd_region)/* The asynchronous flush callback function */ int async_pmem_flush (struct nd_region *nd_region,struct bio *bio) { -/* Create child bio for asynchronous flush and chain with +/* +* Create child bio for asynchronous flush and chain with * parent bio. Otherwise directly call nd_region flush. */ if (bio && bio->bi_iter.bi_sector !=-1 ) {drivers/nvdimm/virtio_pmem.c +1
-1@@ -19,7 +19,7 @@ static int init_vq(struct virtio_pmem *vpmem) {/* single vq */ vpmem->req_vq =virtio_find_single_vq (vpmem->vdev, - host_ack,"flush_queue" ); + virtio_pmem_host_ack,"flush_queue" );if (IS_ERR (vpmem->req_vq))return PTR_ERR (vpmem->req_vq);drivers/nvdimm/virtio_pmem.h +1
-1@@ -50,6 +50,6 @@ struct virtio_pmem { uint64_t size; }; -void host_ack (struct virtqueue *vq); +void virtio_pmem_host_ack (struct virtqueue *vq);int async_pmem_flush (struct nd_region *nd_region,struct bio *bio);#endif
Re: [PATCH v12 2/7] virtio-pmem: Add virtio pmem driver
In reply to Pankaj GuptaHi Pankaj,
Quoted 15 lines · On Tue, 11 Jun 2019 23:34:50 -0400 (EDT) Pankaj Gupta <[email protected]>…
Hi Cornelia,
Quoted 4 lines
On Tue, 11 Jun 2019 22:07:57 +0530
Pankaj Gupta <[email protected]> wrote:Quoted 8 lines
Review
Quoted 6 lines
(quoted diff) +6
-0+ err1 = virtqueue_kick(vpmem->req_vq); + spin_unlock_irqrestore(&vpmem->pmem_lock, flags); + /* + * virtqueue_add_sgs failed with error different than -ENOSPC, we can't + * do anything about that. + */Does it make sense to kick if you couldn't add at all?
Quoted 3 lines
When we could not add because of
-ENOSPCwe are waiting and when buffer is added then only we do a kick. For any other error which might be a rare occurrence, I think kick is harmless here and keeps the code clean?
Yes, I agree it does not hurt. Let's keep it as-is.
Quoted 3 lines
Sure, Thank you. Attaching below on top changes on current patch2 based on your suggestions. Let me know if these are okay and then will send official v13 to for upstream merging.
Looks good to me, except for one change.
[Again sorry for the late review, did not want to get the version numbers up :)]
Quoted 42 lines
Thanks,
Pankaj===============
drivers/nvdimm/nd_virtio.c +4
-4@@ -10,7 +10,7 @@#include "nd.h" /* The interrupt handler */ -void host_ack (struct virtqueue *vq) +void virtio_pmem_host_ack (struct virtqueue *vq) {struct virtio_pmem *vpmem = vq->vdev->priv;struct virtio_pmem_request *req_data, *req_buf;@@ -32,10 +32,10 @@ void host_ack(struct virtqueue *vq) }spin_unlock_irqrestore (&vpmem->pmem_lock, flags); } -EXPORT_SYMBOL_GPL (host_ack); +EXPORT_SYMBOL_GPL (virtio_pmem_host_ack);/* The request submission function */ -int virtio_pmem_flush (struct nd_region *nd_region) +static int virtio_pmem_flush (struct nd_region *nd_region) {struct virtio_device *vdev = nd_region->provider_data;struct virtio_pmem *vpmem = vdev->priv;@@ -69,7 +69,7 @@ int virtio_pmem_flush(struct nd_region *nd_region)while ((err =virtqueue_add_sgs (vpmem->req_vq, sgs,1 ,1 , req_data,GFP_ATOMIC )) == -ENOSPC ) { -dev_err (&vdev->dev,"failed to send command to virtio pmem device, no free slots in the virtqueue\n" ); +dev_info (&vdev->dev,"failed to send command to virtio pmem device, no free slots in the virtqueue\n" ); req_data->wq_buf_avail = false;list_add_tail (&req_data->list, &vpmem->req_list);spin_unlock_irqrestore (&vpmem->pmem_lock, flags);
drivers/nvdimm/nd_virtio.c int virtio_pmem_flush(struct nd_region *nd_region) · line 94
Quoted 6 lines
drivers/nvdimm/nd_virtio.c +2
-1@@ -90,7 +90,8 @@ int virtio_pmem_flush(struct nd_region *nd_region) }else {/* A host repsonse results in "host_ack" getting called */ wait_event (req_data->host_acked, req_data->done); - err =virtio32_to_cpu (vdev, req_data->resp.ret); +if ((err =virtio32_to_cpu (vdev, req_data->resp.ret))) + err = -EIO ;
Hm, why are you making this change? I think the previous code was fine.
Quoted 38 lines
drivers/nvdimm/nd_virtio.c +2
-1}kfree (req_data);@@ -100,7 +101,8 @@ int virtio_pmem_flush(struct nd_region *nd_region)/* The asynchronous flush callback function */ int async_pmem_flush (struct nd_region *nd_region,struct bio *bio) { -/* Create child bio for asynchronous flush and chain with +/* +* Create child bio for asynchronous flush and chain with * parent bio. Otherwise directly call nd_region flush. */ if (bio && bio->bi_iter.bi_sector !=-1 ) {drivers/nvdimm/virtio_pmem.c +1
-1@@ -19,7 +19,7 @@ static int init_vq(struct virtio_pmem *vpmem) {/* single vq */ vpmem->req_vq =virtio_find_single_vq (vpmem->vdev, - host_ack,"flush_queue" ); + virtio_pmem_host_ack,"flush_queue" );if (IS_ERR (vpmem->req_vq))return PTR_ERR (vpmem->req_vq);drivers/nvdimm/virtio_pmem.h +1
-1@@ -50,6 +50,6 @@ struct virtio_pmem { uint64_t size; }; -void host_ack (struct virtqueue *vq); +void virtio_pmem_host_ack (struct virtqueue *vq);int async_pmem_flush (struct nd_region *nd_region,struct bio *bio);#endif
Re: [PATCH v12 2/7] virtio-pmem: Add virtio pmem driver
In reply to Cornelia HuckHi Cornelia,
Quoted 78 lines
Quoted 20 lines · On Tue, 11 Jun 2019 22:07:57 +0530 Pankaj Gupta <[email protected]> wrote:
This patch adds virtio-pmem driver for KVM guest.
Guest reads the persistent memory range information from Qemu over VIRTIO and registers it on
nvdimm_bus. It also creates and_regionobject with the persistent memory range information so that existing 'nvdimm/pmem' driver can reserve this into system memory map. This way 'virtio-pmem' driver uses existing functionality of pmem driver to register persistent memory compatible for DAX capable filesystems.This also provides function to perform guest flush over VIRTIO from 'pmem' driver when userspace performs flush on DAX memory range.
Signed-off-by Pankaj Gupta <[email protected]>Reviewed-by Yuval Shaia <[email protected]>Acked-by Jakub Staron <[email protected]>Tested-by Jakub Staron <[email protected]>Quoted 13 lines
--- drivers/nvdimm/Makefile | 1 + drivers/nvdimm/nd_virtio.c | 124 +++++++++++++++++++++++++++++++ drivers/nvdimm/virtio_pmem.c | 122 ++++++++++++++++++++++++++++++ drivers/nvdimm/virtio_pmem.h | 55 ++++++++++++++ drivers/virtio/Kconfig | 11 +++ include/uapi/linux/virtio_ids.h | 1 + include/uapi/linux/virtio_pmem.h | 35 +++++++++ 7 files changed, 349 insertions(+) create mode 100644 drivers/nvdimm/nd_virtio.c create mode 100644 drivers/nvdimm/virtio_pmem.c create mode 100644 drivers/nvdimm/virtio_pmem.h create mode 100644 include/uapi/linux/virtio_pmem.hSorry about being late to the party; this one has been sitting in my 'to review' queue for far too long :(
(...)
Quoted 36 lines
drivers/nvdimm/nd_virtio.c new file +30
-0@@ -0,0 +1,124 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* virtio_pmem.c: Virtio pmem Driver +* +* Discovers persistent memory range information +* from host and provides a virtio based flushing +* interface. +*/ +#include "virtio_pmem.h" +#include "nd.h" + +/* The interrupt handler */ +void host_ack (struct virtqueue *vq) +{ +struct virtio_pmem *vpmem = vq->vdev->priv; +struct virtio_pmem_request *req_data, *req_buf; +unsigned long flags; +unsigned int len; + +spin_lock_irqsave (&vpmem->pmem_lock, flags); +while ((req_data =virtqueue_get_buf (vq, &len)) !=NULL ) { + req_data->done = true; +wake_up (&req_data->host_acked); + +if (!list_empty (&vpmem->req_list)) { + req_buf =list_first_entry (&vpmem->req_list, + struct virtio_pmem_request, list); + req_buf->wq_buf_avail = true; +wake_up (&req_buf->wq_buf); +list_del (&req_buf->list);
Quoted 8 lines
drivers/nvdimm/nd_virtio.c line 35
Quoted 5 lines
drivers/nvdimm/nd_virtio.c new file +5
-0+ } + } +spin_unlock_irqrestore (&vpmem->pmem_lock, flags); +} +EXPORT_SYMBOL_GPL (host_ack);Nit: '
host_ack' looks a bit generic for an exported function... would 'virtio_pmem_host_ack' maybe be better?
Yes, this looks better. Changed.
Quoted 7 lines
drivers/nvdimm/nd_virtio.c line 38
Quoted 4 lines
drivers/nvdimm/nd_virtio.c new file +3
-0+ +/* The request submission function */ +int virtio_pmem_flush (struct nd_region *nd_region)I don't see an
EXPORT_SYMBOL_GPL()for this function... should it get one, or should it be made static?
Made static. Leftover from last refactor of 'asyc_pmem_flush'.
Quoted 32 lines
Quoted 32 lines
drivers/nvdimm/nd_virtio.c new file +31
-0+{ + struct virtio_device *vdev = nd_region->provider_data; + struct virtio_pmem *vpmem = vdev->priv; + struct virtio_pmem_request *req_data; + struct scatterlist *sgs[2], sg, ret; + unsigned long flags; + int err, err1; + + might_sleep(); + req_data = kmalloc(sizeof(*req_data), GFP_KERNEL); + if (!req_data) + return -ENOMEM; + + req_data->done = false; + init_waitqueue_head(&req_data->host_acked); + init_waitqueue_head(&req_data->wq_buf); + INIT_LIST_HEAD(&req_data->list); + req_data->req.type = cpu_to_virtio32(vdev, VIRTIO_PMEM_REQ_TYPE_FLUSH); + sg_init_one(&sg, &req_data->req, sizeof(req_data->req)); + sgs[0] = &sg; + sg_init_one(&ret, &req_data->resp.ret, sizeof(req_data->resp)); + sgs[1] = &ret; + + spin_lock_irqsave(&vpmem->pmem_lock, flags); + /* + * If virtqueue_add_sgs returns -ENOSPC then req_vq virtual + * queue does not have free descriptor. We add the request + * to req_list and wait for host_ack to wake us up when free + * slots are available. + */ + while ((err = virtqueue_add_sgs(vpmem->req_vq, sgs, 1, 1, req_data,
Quoted 8 lines
Quoted 4 lines
drivers/nvdimm/nd_virtio.c new file +3
-0+ GFP_ATOMIC)) == -ENOSPC) { + + dev_err(&vdev->dev, "failed to send command to virtio pmem device, nofree slots in the virtqueue\n");
Hm... by the comment above I would have thought that this is not really an error, but rather a temporary condition? Maybe downgrade this to
dev_info()?
o.k.
Quoted 17 lines
Quoted 5 lines
drivers/nvdimm/nd_virtio.c new file +4
-0+ req_data->wq_buf_avail = false; +list_add_tail (&req_data->list, &vpmem->req_list); +spin_unlock_irqrestore (&vpmem->pmem_lock, flags); +drivers/nvdimm/nd_virtio.c
Quoted 10 lines
drivers/nvdimm/nd_virtio.c new file +10
-0+/* A host response results in "host_ack" getting called */ +wait_event (req_data->wq_buf, req_data->wq_buf_avail); +spin_lock_irqsave (&vpmem->pmem_lock, flags); + } + err1 =virtqueue_kick (vpmem->req_vq); +spin_unlock_irqrestore (&vpmem->pmem_lock, flags); +/* +* virtqueue_add_sgs failed with error different than -ENOSPC, we can't +* do anything about that. +*/ Does it make sense to kick if you couldn't add at all?
When we could not add because of -ENOSPC we are waiting and when buffer is added then only we do a kick. For any other error which might be a rare occurrence, I think kick is harmless here and keeps the code clean?
Quoted 6 lines
drivers/nvdimm/nd_virtio.c
Quoted 3 lines
drivers/nvdimm/nd_virtio.c new file +2
-0+if (err || !err1) { +dev_info (&vdev->dev,"failed to send command to virtio pmem device\n" );If this is
dev_info, I think the error above really should bedev_infoas well (and maybe also log the error value)?
o.k.
Quoted 12 lines
Quoted 12 lines
drivers/nvdimm/nd_virtio.c new file +11
-0+ err = -EIO ; + }else { +/* A host repsonse results in "host_ack" getting called */ +wait_event (req_data->host_acked, req_data->done); + err =virtio32_to_cpu (vdev, req_data->resp.ret); + } + +kfree (req_data); +return err; +}; +
Quoted 8 lines
drivers/nvdimm/nd_virtio.c
Quoted 6 lines
drivers/nvdimm/nd_virtio.c new file +6
-0+/* The asynchronous flush callback function */ +int async_pmem_flush (struct nd_region *nd_region,struct bio *bio) +{ +/* Create child bio for asynchronous flush and chain with +* parent bio. Otherwise directly call nd_region flush. +*/ Nit: The comment should start with an otherwise empty /* line.
yes.
Quoted 23 lines
Quoted 20 lines
drivers/nvdimm/nd_virtio.c new file +19
-0+if (bio && bio->bi_iter.bi_sector !=-1 ) { +struct bio *child =bio_alloc (GFP_ATOMIC ,0 ); + +if (!child) +return -ENOMEM ; +bio_copy_dev (child, bio); + child->bi_opf =REQ_PREFLUSH ; + child->bi_iter.bi_sector =-1 ; +bio_chain (child, bio); +submit_bio (child); +return 0 ; + } +if (virtio_pmem_flush (nd_region)) +return -EIO ; + +return 0 ; +}; +EXPORT_SYMBOL_GPL (async_pmem_flush); +MODULE_LICENSE ("GPL" );(...)
Quoted 2 lines
I have only some more minor comments; on the whole, this looks good to me.
Sure, Thank you. Attaching below on top changes on current patch2 based on your suggestions. Let me know if these are okay and then will send official v13 to for upstream merging.
Thanks,
Pankaj
===============
drivers/nvdimm/nd_virtio.c +8 -6
@@ -10,7 +10,7 @@#include "nd.h" /* The interrupt handler */ -void host_ack (struct virtqueue *vq) +void virtio_pmem_host_ack (struct virtqueue *vq) {struct virtio_pmem *vpmem = vq->vdev->priv;struct virtio_pmem_request *req_data, *req_buf;
@@ -32,10 +32,10 @@ void host_ack(struct virtqueue *vq) }spin_unlock_irqrestore (&vpmem->pmem_lock, flags); } -EXPORT_SYMBOL_GPL (host_ack); +EXPORT_SYMBOL_GPL (virtio_pmem_host_ack);/* The request submission function */ -int virtio_pmem_flush (struct nd_region *nd_region) +static int virtio_pmem_flush (struct nd_region *nd_region) {struct virtio_device *vdev = nd_region->provider_data;struct virtio_pmem *vpmem = vdev->priv;
@@ -69,7 +69,7 @@ int virtio_pmem_flush(struct nd_region *nd_region)while ((err =virtqueue_add_sgs (vpmem->req_vq, sgs,1 ,1 , req_data,GFP_ATOMIC )) == -ENOSPC ) { -dev_err (&vdev->dev,"failed to send command to virtio pmem device, no free slots in the virtqueue\n" ); +dev_info (&vdev->dev,"failed to send command to virtio pmem device, no free slots in the virtqueue\n" ); req_data->wq_buf_avail = false;list_add_tail (&req_data->list, &vpmem->req_list);spin_unlock_irqrestore (&vpmem->pmem_lock, flags);
@@ -90,7 +90,8 @@ int virtio_pmem_flush(struct nd_region *nd_region) }else {/* A host repsonse results in "host_ack" getting called */ wait_event (req_data->host_acked, req_data->done); - err =virtio32_to_cpu (vdev, req_data->resp.ret); +if ((err =virtio32_to_cpu (vdev, req_data->resp.ret))) + err = -EIO ; }kfree (req_data);
@@ -100,7 +101,8 @@ int virtio_pmem_flush(struct nd_region *nd_region)/* The asynchronous flush callback function */ int async_pmem_flush (struct nd_region *nd_region,struct bio *bio) { -/* Create child bio for asynchronous flush and chain with +/* +* Create child bio for asynchronous flush and chain with * parent bio. Otherwise directly call nd_region flush. */ if (bio && bio->bi_iter.bi_sector !=-1 ) {
drivers/nvdimm/virtio_pmem.c +1 -1
@@ -19,7 +19,7 @@ static int init_vq(struct virtio_pmem *vpmem) {/* single vq */ vpmem->req_vq =virtio_find_single_vq (vpmem->vdev, - host_ack,"flush_queue" ); + virtio_pmem_host_ack,"flush_queue" );if (IS_ERR (vpmem->req_vq))return PTR_ERR (vpmem->req_vq);
drivers/nvdimm/virtio_pmem.h +1 -1
@@ -50,6 +50,6 @@ struct virtio_pmem { uint64_t size; }; -void host_ack (struct virtqueue *vq); +void virtio_pmem_host_ack (struct virtqueue *vq);int async_pmem_flush (struct nd_region *nd_region,struct bio *bio);#endif
Re: [PATCH v12 2/7] virtio-pmem: Add virtio pmem driver
In reply to Pankaj GuptaQuoted 20 lines · On Tue, 11 Jun 2019 22:07:57 +0530 Pankaj Gupta <[email protected]> wrote:
This patch adds virtio-pmem driver for KVM guest.
Guest reads the persistent memory range information from Qemu over VIRTIO and registers it on
nvdimm_bus. It also creates and_regionobject with the persistent memory range information so that existing 'nvdimm/pmem' driver can reserve this into system memory map. This way 'virtio-pmem' driver uses existing functionality of pmem driver to register persistent memory compatible for DAX capable filesystems.This also provides function to perform guest flush over VIRTIO from 'pmem' driver when userspace performs flush on DAX memory range.
Signed-off-by Pankaj Gupta <[email protected]>Reviewed-by Yuval Shaia <[email protected]>Acked-by Jakub Staron <[email protected]>Tested-by Jakub Staron <[email protected]>
Quoted 13 lines
--- drivers/nvdimm/Makefile | 1 + drivers/nvdimm/nd_virtio.c | 124 +++++++++++++++++++++++++++++++ drivers/nvdimm/virtio_pmem.c | 122 ++++++++++++++++++++++++++++++ drivers/nvdimm/virtio_pmem.h | 55 ++++++++++++++ drivers/virtio/Kconfig | 11 +++ include/uapi/linux/virtio_ids.h | 1 + include/uapi/linux/virtio_pmem.h | 35 +++++++++ 7 files changed, 349 insertions(+) create mode 100644 drivers/nvdimm/nd_virtio.c create mode 100644 drivers/nvdimm/virtio_pmem.c create mode 100644 drivers/nvdimm/virtio_pmem.h create mode 100644 include/uapi/linux/virtio_pmem.h
Sorry about being late to the party; this one has been sitting in my 'to review' queue for far too long :(
(...)
Quoted 31 lines
drivers/nvdimm/nd_virtio.c new file +25
-0@@ -0,0 +1,124 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* virtio_pmem.c: Virtio pmem Driver +* +* Discovers persistent memory range information +* from host and provides a virtio based flushing +* interface. +*/ +#include "virtio_pmem.h" +#include "nd.h" + +/* The interrupt handler */ +void host_ack (struct virtqueue *vq) +{ +struct virtio_pmem *vpmem = vq->vdev->priv; +struct virtio_pmem_request *req_data, *req_buf; +unsigned long flags; +unsigned int len; + +spin_lock_irqsave (&vpmem->pmem_lock, flags); +while ((req_data =virtqueue_get_buf (vq, &len)) !=NULL ) { + req_data->done = true; +wake_up (&req_data->host_acked); + +if (!list_empty (&vpmem->req_list)) {
drivers/nvdimm/nd_virtio.c line 35
Quoted 10 lines
drivers/nvdimm/nd_virtio.c new file +10
-0+ req_buf =list_first_entry (&vpmem->req_list, + struct virtio_pmem_request, list); + req_buf->wq_buf_avail = true; +wake_up (&req_buf->wq_buf); +list_del (&req_buf->list); + } + } +spin_unlock_irqrestore (&vpmem->pmem_lock, flags); +} +EXPORT_SYMBOL_GPL (host_ack);
Nit: 'host_ack' looks a bit generic for an exported function... would 'virtio_pmem_host_ack' maybe be better?
drivers/nvdimm/nd_virtio.c line 38
Quoted 3 lines
drivers/nvdimm/nd_virtio.c new file +3
-0+ +/* The request submission function */ +int virtio_pmem_flush (struct nd_region *nd_region)
I don't see an EXPORT_SYMBOL_GPL() for this function... should it get one, or should it be made static?
Quoted 24 lines
drivers/nvdimm/nd_virtio.c new file +24
-0+{ +struct virtio_device *vdev = nd_region->provider_data; +struct virtio_pmem *vpmem = vdev->priv; +struct virtio_pmem_request *req_data; +struct scatterlist *sgs[2 ], sg, ret; +unsigned long flags; +int err, err1; + +might_sleep (); + req_data =kmalloc (sizeof (*req_data),GFP_KERNEL ); +if (!req_data) +return -ENOMEM ; + + req_data->done = false; +init_waitqueue_head (&req_data->host_acked); +init_waitqueue_head (&req_data->wq_buf); +INIT_LIST_HEAD (&req_data->list); + req_data->req.type =cpu_to_virtio32 (vdev,VIRTIO_PMEM_REQ_TYPE_FLUSH ); +sg_init_one (&sg, &req_data->req,sizeof (req_data->req)); + sgs[0 ] = &sg; +sg_init_one (&ret, &req_data->resp.ret,sizeof (req_data->resp)); + sgs[1 ] = &ret; + +spin_lock_irqsave (&vpmem->pmem_lock, flags);
drivers/nvdimm/nd_virtio.c line 72
Quoted 10 lines
drivers/nvdimm/nd_virtio.c new file +10
-0+/* +* If virtqueue_add_sgs returns -ENOSPC then req_vq virtual +* queue does not have free descriptor. We add the request +* to req_list and wait for host_ack to wake us up when free +* slots are available. +*/ +while ((err =virtqueue_add_sgs (vpmem->req_vq, sgs,1 ,1 , req_data, +GFP_ATOMIC )) == -ENOSPC ) { + +dev_err (&vdev->dev,"failed to send command to virtio pmem device, no free slots in the virtqueue\n" );
Hm... by the comment above I would have thought that this is not really an error, but rather a temporary condition? Maybe downgrade this to dev_info()?
Quoted 4 lines
drivers/nvdimm/nd_virtio.c new file +4
-0+ req_data->wq_buf_avail = false; +list_add_tail (&req_data->list, &vpmem->req_list); +spin_unlock_irqrestore (&vpmem->pmem_lock, flags); +
drivers/nvdimm/nd_virtio.c line 86
Quoted 10 lines
drivers/nvdimm/nd_virtio.c new file +10
-0+/* A host response results in "host_ack" getting called */ +wait_event (req_data->wq_buf, req_data->wq_buf_avail); +spin_lock_irqsave (&vpmem->pmem_lock, flags); + } + err1 =virtqueue_kick (vpmem->req_vq); +spin_unlock_irqrestore (&vpmem->pmem_lock, flags); +/* +* virtqueue_add_sgs failed with error different than -ENOSPC, we can't +* do anything about that. +*/
Does it make sense to kick if you couldn't add at all?
drivers/nvdimm/nd_virtio.c line 88
Quoted 2 lines
drivers/nvdimm/nd_virtio.c new file +2
-0+if (err || !err1) { +dev_info (&vdev->dev,"failed to send command to virtio pmem device\n" );
If this is dev_info, I think the error above really should be dev_info as well (and maybe also log the error value)?
Quoted 7 lines
drivers/nvdimm/nd_virtio.c new file +7
-0+ err = -EIO ; + }else { +/* A host repsonse results in "host_ack" getting called */ +wait_event (req_data->host_acked, req_data->done); + err =virtio32_to_cpu (vdev, req_data->resp.ret); + } +
drivers/nvdimm/nd_virtio.c line 105
Quoted 10 lines
drivers/nvdimm/nd_virtio.c new file +10
-0+kfree (req_data); +return err; +}; + +/* The asynchronous flush callback function */ +int async_pmem_flush (struct nd_region *nd_region,struct bio *bio) +{ +/* Create child bio for asynchronous flush and chain with +* parent bio. Otherwise directly call nd_region flush. +*/
Nit: The comment should start with an otherwise empty /* line.
Quoted 9 lines
drivers/nvdimm/nd_virtio.c new file +9
-0+if (bio && bio->bi_iter.bi_sector !=-1 ) { +struct bio *child =bio_alloc (GFP_ATOMIC ,0 ); + +if (!child) +return -ENOMEM ; +bio_copy_dev (child, bio); + child->bi_opf =REQ_PREFLUSH ; + child->bi_iter.bi_sector =-1 ; +bio_chain (child, bio);
drivers/nvdimm/nd_virtio.c line 124
Quoted 10 lines
drivers/nvdimm/nd_virtio.c new file +10
-0+submit_bio (child); +return 0 ; + } +if (virtio_pmem_flush (nd_region)) +return -EIO ; + +return 0 ; +}; +EXPORT_SYMBOL_GPL (async_pmem_flush); +MODULE_LICENSE ("GPL" );
(...)
I have only some more minor comments; on the whole, this looks good to me.
PATCH v12 2/7 virtio-pmem: Add virtio pmem driver
In reply to Pankaj GuptaThis patch adds virtio-pmem driver for KVM guest.
Guest reads the persistent memory range information from Qemu over VIRTIO and registers it on nvdimm_bus. It also creates a nd_region object with the persistent memory range information so that existing 'nvdimm/pmem' driver can reserve this into system memory map. This way 'virtio-pmem' driver uses existing functionality of pmem driver to register persistent memory compatible for DAX capable filesystems.
This also provides function to perform guest flush over VIRTIO from 'pmem' driver when userspace performs flush on DAX memory range.
--- drivers/nvdimm/Makefile | 1 + drivers/nvdimm/nd_virtio.c | 124 +++++++++++++++++++++++++++++++ drivers/nvdimm/virtio_pmem.c | 122 ++++++++++++++++++++++++++++++ drivers/nvdimm/virtio_pmem.h | 55 ++++++++++++++ drivers/virtio/Kconfig | 11 +++ include/uapi/linux/virtio_ids.h | 1 + include/uapi/linux/virtio_pmem.h | 35 +++++++++ 7 files changed, 349 insertions(+) create mode 100644 drivers/nvdimm/nd_virtio.c create mode 100644 drivers/nvdimm/virtio_pmem.c create mode 100644 drivers/nvdimm/virtio_pmem.h create mode 100644 include/uapi/linux/virtio_pmem.h
drivers/nvdimm/Makefile +1 -0
@@ -5,6 +5,7 @@ obj-$(CONFIG_ND_BTT) += nd_btt.o obj-$(CONFIG_ND_BLK) += nd_blk.o obj-$(CONFIG_X86_PMEM_LEGACY) += nd_e820.o obj-$(CONFIG_OF_PMEM) += of_pmem.o +obj-$(CONFIG_VIRTIO_PMEM) += virtio_pmem.o nd_virtio.o nd_pmem-y := pmem.o
drivers/nvdimm/nd_virtio.c new file +124 -0
124 lines not on this page · raw message
drivers/nvdimm/virtio_pmem.c new file +122 -0
122 lines not on this page · raw message
drivers/nvdimm/virtio_pmem.h new file +55 -0
55 lines not on this page · raw message
drivers/virtio/Kconfig +11 -0
@@ -42,6 +42,17 @@ config VIRTIO_PCI_LEGACY If unsure, say Y. +config VIRTIO_PMEM + tristate "Support for virtio pmem driver" + depends on VIRTIO + depends on LIBNVDIMM + help + This driver provides access to virtio-pmem devices, storage devices + that are mapped into the physical address space - similar to NVDIMMs + - with a virtio-based flushing interface. + + If unsure, say Y. + config VIRTIO_BALLOON tristate "Virtio balloon driver" depends on VIRTIO
include/uapi/linux/virtio_ids.h +1 -0
@@ -43,5 +43,6 @@#define VIRTIO_ID_INPUT 18/* virtio input */ #define VIRTIO_ID_VSOCK 19/* virtio vsock transport */ #define VIRTIO_ID_CRYPTO 20/* virtio crypto */ +#define VIRTIO_ID_PMEM 27/* virtio pmem */ #endif /* _LINUX_VIRTIO_IDS_H */
include/uapi/linux/virtio_pmem.h new file +35 -0
@@ -0,0 +1,35 @@ +/* SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause */ +/* +* Definitions for virtio-pmem devices. +* +* Copyright (C) 2019 Red Hat, Inc. +* +* Author(s): Pankaj Gupta <[email protected]> +*/ + +#ifndef _UAPI_LINUX_VIRTIO_PMEM_H +#define _UAPI_LINUX_VIRTIO_PMEM_H + +#include <linux/types.h> +#include <linux/virtio_types.h> +#include <linux/virtio_ids.h> +#include <linux/virtio_config.h> + +struct virtio_pmem_config { +__le64 start; +__le64 size; +}; + +#define VIRTIO_PMEM_REQ_TYPE_FLUSH 0 + +struct virtio_pmem_resp { +/* Host return status corresponding to flush request */ +__virtio32 ret; +}; + +struct virtio_pmem_req { +/* command type */ +__virtio32 type; +}; + +#endif
Signature · 1 line
2.20.1
PATCH v12 3/7 libnvdimm: add dax_dev sync flag
In reply to Pankaj GuptaThis patch adds 'DAXDEV_SYNC' flag which is set for nd_region doing synchronous flush. This later is used to disable MAP_SYNC functionality for ext4 & xfs filesystem for devices don't support synchronous flush.
--- drivers/dax/bus.c | 2 +- drivers/dax/super.c | 19 ++++++++++++++++++- drivers/md/dm.c | 3 ++- drivers/nvdimm/pmem.c | 5 ++++- drivers/nvdimm/region_devs.c | 7 +++++++ include/linux/dax.h | 9 +++++++-- include/linux/libnvdimm.h | 1 + 7 files changed, 40 insertions(+), 6 deletions(-)
drivers/dax/bus.c +1 -1
@@ -388,7 +388,7 @@ struct dev_dax *__devm_create_dev_dax(struct dax_region *dax_region, int id,* No 'host' or dax_operations since there is no access to this * device outside of mmap of the resulting character device. */ - dax_dev =alloc_dax (dev_dax,NULL ,NULL ); + dax_dev =alloc_dax (dev_dax,NULL ,NULL ,DAXDEV_F_SYNC );if (!dax_dev) goto err;
drivers/dax/super.c +18 -1
@@ -186,6 +186,8 @@ enum dax_device_flags {DAXDEV_ALIVE ,/* gate whether dax_flush() calls the low level flush routine */ DAXDEV_WRITE_CACHE , +/* flag to check if device supports synchronous flush */ + DAXDEV_SYNC, };/**
@@ -354,6 +356,18 @@ bool dax_write_cache_enabled(struct dax_device *dax_dev) }EXPORT_SYMBOL_GPL (dax_write_cache_enabled); +bool dax_synchronous (struct dax_device *dax_dev) +{ +return test_bit (DAXDEV_SYNC , &dax_dev->flags); +} +EXPORT_SYMBOL_GPL (dax_synchronous); + +void set_dax_synchronous (struct dax_device *dax_dev) +{ +set_bit (DAXDEV_SYNC , &dax_dev->flags); +} +EXPORT_SYMBOL_GPL (set_dax_synchronous); +bool dax_alive (struct dax_device *dax_dev) {lockdep_assert_held (&dax_srcu);
@@ -508,7 +522,7 @@ static void dax_add_host(struct dax_device *dax_dev, const char *host) }struct dax_device *alloc_dax (void *private,const char *__host, -const struct dax_operations *ops) +const struct dax_operations *ops,unsigned long flags) {struct dax_device *dax_dev;const char *host;
@@ -531,6 +545,9 @@ struct dax_device *alloc_dax(void *private, const char *__host,dax_add_host (dax_dev, host); dax_dev->ops = ops; dax_dev->private = private; +if (flags &DAXDEV_F_SYNC ) +set_dax_synchronous (dax_dev); +return dax_dev; err_dev:
drivers/md/dm.c +2 -1
@@ -1970,7 +1970,8 @@ static struct mapped_device *alloc_dev(int minor)sprintf (md->disk->disk_name,"dm-%d" , minor);if (IS_ENABLED (CONFIG_DAX_DRIVER )) { - md->dax_dev =alloc_dax (md, md->disk->disk_name, &dm_dax_ops); + md->dax_dev =alloc_dax (md, md->disk->disk_name, + &dm_dax_ops,0 );if (!md->dax_dev) goto bad; }
drivers/nvdimm/pmem.c +4 -1
@@ -365,6 +365,7 @@ static int pmem_attach_disk(struct device *dev,struct gendisk *disk;void *addr;int rc; +unsigned long flags =0UL ; pmem =devm_kzalloc (dev,sizeof (*pmem),GFP_KERNEL );if (!pmem)
@@ -462,7 +463,9 @@ static int pmem_attach_disk(struct device *dev,nvdimm_badblocks_populate (nd_region, &pmem->bb, &bb_res); disk->bb = &pmem->bb; - dax_dev =alloc_dax (pmem, disk->disk_name, &pmem_dax_ops); +if (is_nvdimm_sync (nd_region)) + flags =DAXDEV_F_SYNC ; + dax_dev =alloc_dax (pmem, disk->disk_name, &pmem_dax_ops, flags);if (!dax_dev) {put_disk (disk);return -ENOMEM ;
drivers/nvdimm/region_devs.c +7 -0
@@ -1197,6 +1197,13 @@ int nvdimm_has_cache(struct nd_region *nd_region) }EXPORT_SYMBOL_GPL (nvdimm_has_cache); +bool is_nvdimm_sync (struct nd_region *nd_region) +{ +return is_nd_pmem (&nd_region->dev) && + !test_bit (ND_REGION_ASYNC , &nd_region->flags); +} +EXPORT_SYMBOL_GPL (is_nvdimm_sync); +struct conflict_context {struct nd_region *nd_region;resource_size_t start, size;
include/linux/dax.h +7 -2
@@ -7,6 +7,9 @@#include <linux/radix-tree.h> #include <asm/pgtable.h> +/* Flag for synchronous flush */ +#define DAXDEV_F_SYNC (1UL << 0) +typedef unsigned long dax_entry_t ;struct iomap_ops ;
@@ -32,18 +35,20 @@ extern struct attribute_group dax_attribute_group;#if IS_ENABLED (CONFIG_DAX )struct dax_device *dax_get_by_host (const char *host);struct dax_device *alloc_dax (void *private,const char *host, -const struct dax_operations *ops); +const struct dax_operations *ops,unsigned long flags);void put_dax (struct dax_device *dax_dev);void kill_dax (struct dax_device *dax_dev);void dax_write_cache (struct dax_device *dax_dev,bool wc);bool dax_write_cache_enabled (struct dax_device *dax_dev); +bool dax_synchronous (struct dax_device *dax_dev); +void set_dax_synchronous (struct dax_device *dax_dev);#else static inline struct dax_device *dax_get_by_host (const char *host) {return NULL ; }static inline struct dax_device *alloc_dax (void *private,const char *host, -const struct dax_operations *ops) +const struct dax_operations *ops,unsigned long flags) {/* * Callers should check IS_ENABLED(CONFIG_DAX) to know if this
include/linux/libnvdimm.h +1 -0
@@ -264,6 +264,7 @@ void nvdimm_flush(struct nd_region *nd_region);int nvdimm_has_flush (struct nd_region *nd_region);int nvdimm_has_cache (struct nd_region *nd_region);int nvdimm_in_overwrite (struct nvdimm *nvdimm); +bool is_nvdimm_sync (struct nd_region *nd_region);static inline int nvdimm_ctl (struct nvdimm *nvdimm,unsigned int cmd,void *buf,unsigned int buf_len,int *cmd_rc)
Signature · 1 line
2.20.1
PATCH v12 3/7 libnvdimm: add dax_dev sync flag
In reply to Pankaj GuptaThis patch adds 'DAXDEV_SYNC' flag which is set for nd_region doing synchronous flush. This later is used to disable MAP_SYNC functionality for ext4 & xfs filesystem for devices don't support synchronous flush.
--- drivers/dax/bus.c | 2 +- drivers/dax/super.c | 19 ++++++++++++++++++- drivers/md/dm.c | 3 ++- drivers/nvdimm/pmem.c | 5 ++++- drivers/nvdimm/region_devs.c | 7 +++++++ include/linux/dax.h | 9 +++++++-- include/linux/libnvdimm.h | 1 + 7 files changed, 40 insertions(+), 6 deletions(-)
drivers/dax/bus.c +1 -1
@@ -388,7 +388,7 @@ struct dev_dax *__devm_create_dev_dax(struct dax_region *dax_region, int id,* No 'host' or dax_operations since there is no access to this * device outside of mmap of the resulting character device. */ - dax_dev =alloc_dax (dev_dax,NULL ,NULL ); + dax_dev =alloc_dax (dev_dax,NULL ,NULL ,DAXDEV_F_SYNC );if (!dax_dev) goto err;
drivers/dax/super.c +18 -1
@@ -186,6 +186,8 @@ enum dax_device_flags {DAXDEV_ALIVE ,/* gate whether dax_flush() calls the low level flush routine */ DAXDEV_WRITE_CACHE , +/* flag to check if device supports synchronous flush */ + DAXDEV_SYNC, };/**
@@ -354,6 +356,18 @@ bool dax_write_cache_enabled(struct dax_device *dax_dev) }EXPORT_SYMBOL_GPL (dax_write_cache_enabled); +bool dax_synchronous (struct dax_device *dax_dev) +{ +return test_bit (DAXDEV_SYNC , &dax_dev->flags); +} +EXPORT_SYMBOL_GPL (dax_synchronous); + +void set_dax_synchronous (struct dax_device *dax_dev) +{ +set_bit (DAXDEV_SYNC , &dax_dev->flags); +} +EXPORT_SYMBOL_GPL (set_dax_synchronous); +bool dax_alive (struct dax_device *dax_dev) {lockdep_assert_held (&dax_srcu);
@@ -508,7 +522,7 @@ static void dax_add_host(struct dax_device *dax_dev, const char *host) }struct dax_device *alloc_dax (void *private,const char *__host, -const struct dax_operations *ops) +const struct dax_operations *ops,unsigned long flags) {struct dax_device *dax_dev;const char *host;
@@ -531,6 +545,9 @@ struct dax_device *alloc_dax(void *private, const char *__host,dax_add_host (dax_dev, host); dax_dev->ops = ops; dax_dev->private = private; +if (flags &DAXDEV_F_SYNC ) +set_dax_synchronous (dax_dev); +return dax_dev; err_dev:
drivers/md/dm.c +2 -1
@@ -1970,7 +1970,8 @@ static struct mapped_device *alloc_dev(int minor)sprintf (md->disk->disk_name,"dm-%d" , minor);if (IS_ENABLED (CONFIG_DAX_DRIVER )) { - md->dax_dev =alloc_dax (md, md->disk->disk_name, &dm_dax_ops); + md->dax_dev =alloc_dax (md, md->disk->disk_name, + &dm_dax_ops,0 );if (!md->dax_dev) goto bad; }
drivers/nvdimm/pmem.c +4 -1
@@ -365,6 +365,7 @@ static int pmem_attach_disk(struct device *dev,struct gendisk *disk;void *addr;int rc; +unsigned long flags =0UL ; pmem =devm_kzalloc (dev,sizeof (*pmem),GFP_KERNEL );if (!pmem)
@@ -462,7 +463,9 @@ static int pmem_attach_disk(struct device *dev,nvdimm_badblocks_populate (nd_region, &pmem->bb, &bb_res); disk->bb = &pmem->bb; - dax_dev =alloc_dax (pmem, disk->disk_name, &pmem_dax_ops); +if (is_nvdimm_sync (nd_region)) + flags =DAXDEV_F_SYNC ; + dax_dev =alloc_dax (pmem, disk->disk_name, &pmem_dax_ops, flags);if (!dax_dev) {put_disk (disk);return -ENOMEM ;
drivers/nvdimm/region_devs.c +7 -0
@@ -1197,6 +1197,13 @@ int nvdimm_has_cache(struct nd_region *nd_region) }EXPORT_SYMBOL_GPL (nvdimm_has_cache); +bool is_nvdimm_sync (struct nd_region *nd_region) +{ +return is_nd_pmem (&nd_region->dev) && + !test_bit (ND_REGION_ASYNC , &nd_region->flags); +} +EXPORT_SYMBOL_GPL (is_nvdimm_sync); +struct conflict_context {struct nd_region *nd_region;resource_size_t start, size;
include/linux/dax.h +7 -2
@@ -7,6 +7,9 @@#include <linux/radix-tree.h> #include <asm/pgtable.h> +/* Flag for synchronous flush */ +#define DAXDEV_F_SYNC (1UL << 0) +typedef unsigned long dax_entry_t ;struct iomap_ops ;
@@ -32,18 +35,20 @@ extern struct attribute_group dax_attribute_group;#if IS_ENABLED (CONFIG_DAX )struct dax_device *dax_get_by_host (const char *host);struct dax_device *alloc_dax (void *private,const char *host, -const struct dax_operations *ops); +const struct dax_operations *ops,unsigned long flags);void put_dax (struct dax_device *dax_dev);void kill_dax (struct dax_device *dax_dev);void dax_write_cache (struct dax_device *dax_dev,bool wc);bool dax_write_cache_enabled (struct dax_device *dax_dev); +bool dax_synchronous (struct dax_device *dax_dev); +void set_dax_synchronous (struct dax_device *dax_dev);#else static inline struct dax_device *dax_get_by_host (const char *host) {return NULL ; }static inline struct dax_device *alloc_dax (void *private,const char *host, -const struct dax_operations *ops) +const struct dax_operations *ops,unsigned long flags) {/* * Callers should check IS_ENABLED(CONFIG_DAX) to know if this
include/linux/libnvdimm.h +1 -0
@@ -264,6 +264,7 @@ void nvdimm_flush(struct nd_region *nd_region);int nvdimm_has_flush (struct nd_region *nd_region);int nvdimm_has_cache (struct nd_region *nd_region);int nvdimm_in_overwrite (struct nvdimm *nvdimm); +bool is_nvdimm_sync (struct nd_region *nd_region);static inline int nvdimm_ctl (struct nvdimm *nvdimm,unsigned int cmd,void *buf,unsigned int buf_len,int *cmd_rc)
Signature · 1 line
2.20.1
PATCH v12 4/7 dm: enable synchronous dax
In reply to Pankaj GuptaThis patch sets dax device 'DAXDEV_SYNC' flag if all the target devices of device mapper support synchrononous DAX. If device mapper consists of both synchronous and asynchronous dax devices, we don't set 'DAXDEV_SYNC' flag.
'dm_table_supports_dax' is refactored to pass 'iterate_devices_fn' as argument so that the callers can pass the appropriate functions.
--- drivers/md/dm-table.c | 24 ++++++++++++++++++------ drivers/md/dm.c | 2 +- drivers/md/dm.h | 5 ++++- 3 files changed, 23 insertions(+), 8 deletions(-)
drivers/md/dm-table.c +18 -6
@@ -881,7 +881,7 @@ void dm_table_set_type(struct dm_table *t, enum dm_queue_mode type)EXPORT_SYMBOL_GPL (dm_table_set_type);/* validate the dax capability of the target device span */ -static int device_supports_dax (struct dm_target *ti,struct dm_dev *dev, +int device_supports_dax (struct dm_target *ti,struct dm_dev *dev,sector_t start,sector_t len,void *data) {int blocksize = *(int *) data;
@@ -890,7 +890,15 @@ static int device_supports_dax(struct dm_target *ti, struct dm_dev *dev, start, len); } -bool dm_table_supports_dax (struct dm_table *t,int blocksize) +/* Check devices support synchronous DAX */ +static int device_synchronous (struct dm_target *ti,struct dm_dev *dev, +sector_t start,sector_t len,void *data) +{ +return dax_synchronous (dev->dax_dev); +} + +bool dm_table_supports_dax (struct dm_table *t, +iterate_devices_callout_fn iterate_fn,int *blocksize) {struct dm_target *ti;unsigned i;
@@ -903,8 +911,7 @@ bool dm_table_supports_dax(struct dm_table *t, int blocksize)return false;if (!ti->type->iterate_devices || - !ti->type->iterate_devices (ti, device_supports_dax, - &blocksize)) + !ti->type->iterate_devices (ti, iterate_fn, blocksize))return false; }
@@ -940,6 +947,7 @@ static int dm_table_determine_type(struct dm_table *t)struct dm_target *tgt;struct list_head *devices =dm_table_get_devices (t);enum dm_queue_mode live_md_type =dm_get_md_type (t->md); +int page_size =PAGE_SIZE ;if (t->type !=DM_TYPE_NONE ) {/* target already set the table's type */
@@ -984,7 +992,7 @@ static int dm_table_determine_type(struct dm_table *t) verify_bio_based:/* We must use this table as bio-based */ t->type =DM_TYPE_BIO_BASED ; -if (dm_table_supports_dax (t,PAGE_SIZE ) || +if (dm_table_supports_dax (t, device_supports_dax, &page_size) || (list_empty (devices) && live_md_type ==DM_TYPE_DAX_BIO_BASED )) { t->type =DM_TYPE_DAX_BIO_BASED ; }else {
@@ -1883,6 +1891,7 @@ void dm_table_set_restrictions(struct dm_table *t, struct request_queue *q, struct queue_limits *limits) {bool wc = false, fua = false; +int page_size =PAGE_SIZE ;/* * Copy table's limits to the DM device's request_queue
@@ -1910,8 +1919,11 @@ void dm_table_set_restrictions(struct dm_table *t, struct request_queue *q, }blk_queue_write_cache (q, wc, fua); -if (dm_table_supports_dax (t,PAGE_SIZE )) +if (dm_table_supports_dax (t, device_supports_dax, &page_size)) {blk_queue_flag_set (QUEUE_FLAG_DAX , q); +if (dm_table_supports_dax (t, device_synchronous,NULL )) +set_dax_synchronous (t->md->dax_dev); + }else blk_queue_flag_clear (QUEUE_FLAG_DAX , q);
drivers/md/dm.c +1 -1
@@ -1119,7 +1119,7 @@ static bool dm_dax_supported(struct dax_device *dax_dev, struct block_device *bdif (!map)return false; - ret =dm_table_supports_dax (map, blocksize); + ret =dm_table_supports_dax (map, device_supports_dax, &blocksize);dm_put_live_table (md, srcu_idx);
drivers/md/dm.h +4 -1
@@ -72,7 +72,10 @@ bool dm_table_bio_based(struct dm_table *t);bool dm_table_request_based (struct dm_table *t);void dm_table_free_md_mempools (struct dm_table *t);struct dm_md_mempools *dm_table_get_md_mempools (struct dm_table *t); -bool dm_table_supports_dax (struct dm_table *t,int blocksize); +bool dm_table_supports_dax (struct dm_table *t,iterate_devices_callout_fn fn, +int *blocksize); +int device_supports_dax (struct dm_target *ti,struct dm_dev *dev, +sector_t start,sector_t len,void *data);void dm_lock_md_type (struct mapped_device *md);void dm_unlock_md_type (struct mapped_device *md);
Signature · 1 line
2.20.1
Re: [PATCH v12 4/7] dm: enable synchronous dax
Quoted 10 lines · Pankaj Gupta
This patch sets dax device '
DAXDEV_SYNC' flag if all the target devices of device mapper support synchrononous DAX. If device mapper consists of both synchronous and asynchronous dax devices, we don't set 'DAXDEV_SYNC' flag.'
dm_table_supports_dax' is refactored to pass 'iterate_devices_fn' as argument so that the callers can pass the appropriate functions.Suggested-by Mike Snitzer <[email protected]>Signed-off-by Pankaj Gupta <[email protected]>
Thanks, and for the benefit of others, passing function pointers like this is perfectly fine IMHO because this code is _not_ in the fast path. These methods are only for device creation.
Re: [Qemu-devel] [PATCH v12 4/7] dm: enable synchronous dax
Quoted 12 lines
Quoted 8 lines · Pankaj Gupta
This patch sets dax device '
DAXDEV_SYNC' flag if all the target devices of device mapper support synchrononous DAX. If device mapper consists of both synchronous and asynchronous dax devices, we don't set 'DAXDEV_SYNC' flag.'
dm_table_supports_dax' is refactored to pass 'iterate_devices_fn' as argument so that the callers can pass the appropriate functions.
Quoted 8 lines
Quoted 2 lines
Suggested-by Mike Snitzer <[email protected]>Signed-off-by Pankaj Gupta <[email protected]>Thanks, and for the benefit of others, passing function pointers like this is perfectly fine IMHO because this code is _not_ in the fast path. These methods are only for device creation.
Reviewed-by Mike Snitzer <[email protected]>
Thank you, Mike
Best regards,
Pankaj
Quoted 2 lines
Re: [Qemu-devel] [PATCH v12 4/7] dm: enable synchronous dax
In reply to Mike SnitzerQuoted 12 lines
Quoted 8 lines · Pankaj Gupta
This patch sets dax device '
DAXDEV_SYNC' flag if all the target devices of device mapper support synchrononous DAX. If device mapper consists of both synchronous and asynchronous dax devices, we don't set 'DAXDEV_SYNC' flag.'
dm_table_supports_dax' is refactored to pass 'iterate_devices_fn' as argument so that the callers can pass the appropriate functions.
Quoted 8 lines
Quoted 2 lines
Suggested-by Mike Snitzer <[email protected]>Signed-off-by Pankaj Gupta <[email protected]>Thanks, and for the benefit of others, passing function pointers like this is perfectly fine IMHO because this code is _not_ in the fast path. These methods are only for device creation.
Reviewed-by Mike Snitzer <[email protected]>
Thank you, Mike
Best regards,
Pankaj
Quoted 2 lines
Re: [PATCH v12 4/7] dm: enable synchronous dax
In reply to Pankaj GuptaQuoted 10 lines · Pankaj Gupta
This patch sets dax device '
DAXDEV_SYNC' flag if all the target devices of device mapper support synchrononous DAX. If device mapper consists of both synchronous and asynchronous dax devices, we don't set 'DAXDEV_SYNC' flag.'
dm_table_supports_dax' is refactored to pass 'iterate_devices_fn' as argument so that the callers can pass the appropriate functions.Suggested-by Mike Snitzer <[email protected]>Signed-off-by Pankaj Gupta <[email protected]>
Thanks, and for the benefit of others, passing function pointers like this is perfectly fine IMHO because this code is _not_ in the fast path. These methods are only for device creation.
PATCH v12 4/7 dm: enable synchronous dax
In reply to Pankaj GuptaThis patch sets dax device 'DAXDEV_SYNC' flag if all the target devices of device mapper support synchrononous DAX. If device mapper consists of both synchronous and asynchronous dax devices, we don't set 'DAXDEV_SYNC' flag.
'dm_table_supports_dax' is refactored to pass 'iterate_devices_fn' as argument so that the callers can pass the appropriate functions.
--- drivers/md/dm-table.c | 24 ++++++++++++++++++------ drivers/md/dm.c | 2 +- drivers/md/dm.h | 5 ++++- 3 files changed, 23 insertions(+), 8 deletions(-)
drivers/md/dm-table.c +18 -6
@@ -881,7 +881,7 @@ void dm_table_set_type(struct dm_table *t, enum dm_queue_mode type)EXPORT_SYMBOL_GPL (dm_table_set_type);/* validate the dax capability of the target device span */ -static int device_supports_dax (struct dm_target *ti,struct dm_dev *dev, +int device_supports_dax (struct dm_target *ti,struct dm_dev *dev,sector_t start,sector_t len,void *data) {int blocksize = *(int *) data;
@@ -890,7 +890,15 @@ static int device_supports_dax(struct dm_target *ti, struct dm_dev *dev, start, len); } -bool dm_table_supports_dax (struct dm_table *t,int blocksize) +/* Check devices support synchronous DAX */ +static int device_synchronous (struct dm_target *ti,struct dm_dev *dev, +sector_t start,sector_t len,void *data) +{ +return dax_synchronous (dev->dax_dev); +} + +bool dm_table_supports_dax (struct dm_table *t, +iterate_devices_callout_fn iterate_fn,int *blocksize) {struct dm_target *ti;unsigned i;
@@ -903,8 +911,7 @@ bool dm_table_supports_dax(struct dm_table *t, int blocksize)return false;if (!ti->type->iterate_devices || - !ti->type->iterate_devices (ti, device_supports_dax, - &blocksize)) + !ti->type->iterate_devices (ti, iterate_fn, blocksize))return false; }
@@ -940,6 +947,7 @@ static int dm_table_determine_type(struct dm_table *t)struct dm_target *tgt;struct list_head *devices =dm_table_get_devices (t);enum dm_queue_mode live_md_type =dm_get_md_type (t->md); +int page_size =PAGE_SIZE ;if (t->type !=DM_TYPE_NONE ) {/* target already set the table's type */
@@ -984,7 +992,7 @@ static int dm_table_determine_type(struct dm_table *t) verify_bio_based:/* We must use this table as bio-based */ t->type =DM_TYPE_BIO_BASED ; -if (dm_table_supports_dax (t,PAGE_SIZE ) || +if (dm_table_supports_dax (t, device_supports_dax, &page_size) || (list_empty (devices) && live_md_type ==DM_TYPE_DAX_BIO_BASED )) { t->type =DM_TYPE_DAX_BIO_BASED ; }else {
@@ -1883,6 +1891,7 @@ void dm_table_set_restrictions(struct dm_table *t, struct request_queue *q, struct queue_limits *limits) {bool wc = false, fua = false; +int page_size =PAGE_SIZE ;/* * Copy table's limits to the DM device's request_queue
@@ -1910,8 +1919,11 @@ void dm_table_set_restrictions(struct dm_table *t, struct request_queue *q, }blk_queue_write_cache (q, wc, fua); -if (dm_table_supports_dax (t,PAGE_SIZE )) +if (dm_table_supports_dax (t, device_supports_dax, &page_size)) {blk_queue_flag_set (QUEUE_FLAG_DAX , q); +if (dm_table_supports_dax (t, device_synchronous,NULL )) +set_dax_synchronous (t->md->dax_dev); + }else blk_queue_flag_clear (QUEUE_FLAG_DAX , q);
drivers/md/dm.c +1 -1
@@ -1119,7 +1119,7 @@ static bool dm_dax_supported(struct dax_device *dax_dev, struct block_device *bdif (!map)return false; - ret =dm_table_supports_dax (map, blocksize); + ret =dm_table_supports_dax (map, device_supports_dax, &blocksize);dm_put_live_table (md, srcu_idx);
drivers/md/dm.h +4 -1
@@ -72,7 +72,10 @@ bool dm_table_bio_based(struct dm_table *t);bool dm_table_request_based (struct dm_table *t);void dm_table_free_md_mempools (struct dm_table *t);struct dm_md_mempools *dm_table_get_md_mempools (struct dm_table *t); -bool dm_table_supports_dax (struct dm_table *t,int blocksize); +bool dm_table_supports_dax (struct dm_table *t,iterate_devices_callout_fn fn, +int *blocksize); +int device_supports_dax (struct dm_target *ti,struct dm_dev *dev, +sector_t start,sector_t len,void *data);void dm_lock_md_type (struct mapped_device *md);void dm_unlock_md_type (struct mapped_device *md);
Signature · 1 line
2.20.1
PATCH v12 5/7 dax: check synchronous mapping is supported
In reply to Pankaj GuptaThis patch introduces 'daxdev_mapping_supported' helper which checks if 'MAP_SYNC' is supported with filesystem mapping. It also checks if corresponding dax_device is synchronous. Virtio pmem device is asynchronous and does not not support VM_SYNC.
--- include/linux/dax.h | 17 +++++++++++++++++ 1 file changed, 17 insertions(+)
include/linux/dax.h +17 -0
@@ -42,6 +42,18 @@ void dax_write_cache(struct dax_device *dax_dev, bool wc);bool dax_write_cache_enabled (struct dax_device *dax_dev);bool dax_synchronous (struct dax_device *dax_dev);void set_dax_synchronous (struct dax_device *dax_dev); +/* +* Check if given mapping is supported by the file / underlying device. +*/ +static inline bool daxdev_mapping_supported (struct vm_area_struct *vma, +struct dax_device *dax_dev) +{ +if (!(vma->vm_flags &VM_SYNC )) +return true; +if (!IS_DAX (file_inode (vma->vm_file))) +return false; +return dax_synchronous (dax_dev); +}#else static inline struct dax_device *dax_get_by_host (const char *host) {
@@ -69,6 +81,11 @@ static inline bool dax_write_cache_enabled(struct dax_device *dax_dev) {return false; } +static inline bool daxdev_mapping_supported (struct vm_area_struct *vma, +struct dax_device *dax_dev) +{ +return !(vma->vm_flags &VM_SYNC ); +}#endif struct writeback_control ;
Signature · 1 line
2.20.1
PATCH v12 6/7 ext4: disable map_sync for async flush
In reply to Pankaj GuptaDont support 'MAP_SYNC' with non-DAX files and DAX files with asynchronous dax_device. Virtio pmem provides asynchronous host page cache flush mechanism. We don't support 'MAP_SYNC' with virtio pmem and ext4.
--- fs/ext4/file.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-)
fs/ext4/file.c +6 -4
@@ -360,15 +360,17 @@ static const struct vm_operations_struct ext4_file_vm_ops = {static int ext4_file_mmap (struct file *file,struct vm_area_struct *vma) {struct inode *inode = file->f_mapping->host; +struct ext4_sb_info *sbi =EXT4_SB (inode->i_sb); +struct dax_device *dax_dev = sbi->s_daxdev; -if (unlikely (ext4_forced_shutdown (EXT4_SB (inode->i_sb)))) +if (unlikely (ext4_forced_shutdown (sbi)))return -EIO ;/* -* We don't support synchronous mappings for non-DAX files. At least -* until someone comes with a sensible use case. +* We don't support synchronous mappings for non-DAX files and +* for DAX files if underneath dax_device is not synchronous. */ -if (!IS_DAX (file_inode (file)) && (vma->vm_flags &VM_SYNC )) +if (!daxdev_mapping_supported (vma, dax_dev))return -EOPNOTSUPP ;file_accessed (file);
Signature · 1 line
2.20.1
PATCH v12 6/7 ext4: disable map_sync for async flush
In reply to Pankaj GuptaDont support 'MAP_SYNC' with non-DAX files and DAX files with asynchronous dax_device. Virtio pmem provides asynchronous host page cache flush mechanism. We don't support 'MAP_SYNC' with virtio pmem and ext4.
--- fs/ext4/file.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-)
fs/ext4/file.c +6 -4
@@ -360,15 +360,17 @@ static const struct vm_operations_struct ext4_file_vm_ops = {static int ext4_file_mmap (struct file *file,struct vm_area_struct *vma) {struct inode *inode = file->f_mapping->host; +struct ext4_sb_info *sbi =EXT4_SB (inode->i_sb); +struct dax_device *dax_dev = sbi->s_daxdev; -if (unlikely (ext4_forced_shutdown (EXT4_SB (inode->i_sb)))) +if (unlikely (ext4_forced_shutdown (sbi)))return -EIO ;/* -* We don't support synchronous mappings for non-DAX files. At least -* until someone comes with a sensible use case. +* We don't support synchronous mappings for non-DAX files and +* for DAX files if underneath dax_device is not synchronous. */ -if (!IS_DAX (file_inode (file)) && (vma->vm_flags &VM_SYNC )) +if (!daxdev_mapping_supported (vma, dax_dev))return -EOPNOTSUPP ;file_accessed (file);
Signature · 1 line
2.20.1
PATCH v12 7/7 xfs: disable map_sync for async flush
In reply to Pankaj GuptaDont support 'MAP_SYNC' with non-DAX files and DAX files with asynchronous dax_device. Virtio pmem provides asynchronous host page cache flush mechanism. We don't support 'MAP_SYNC' with virtio pmem and xfs.
--- fs/xfs/xfs_file.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-)
fs/xfs/xfs_file.c +6 -3
@@ -1203,11 +1203,14 @@ xfs_file_mmap(struct file *filp, struct vm_area_struct *vma) { + struct dax_device *dax_dev; + + dax_dev =xfs_find_daxdev_for_inode (file_inode (filp));/* -* We don't support synchronous mappings for non-DAX files. At least -* until someone comes with a sensible use case. +* We don't support synchronous mappings for non-DAX files and +* for DAX files if underneath dax_device is not synchronous. */ -if (!IS_DAX (file_inode (filp)) && (vma->vm_flags &VM_SYNC )) +if (!daxdev_mapping_supported (vma, dax_dev))return -EOPNOTSUPP ;file_accessed (filp);
Signature · 1 line
2.20.1
PATCH v12 7/7 xfs: disable map_sync for async flush
In reply to Pankaj GuptaDont support 'MAP_SYNC' with non-DAX files and DAX files with asynchronous dax_device. Virtio pmem provides asynchronous host page cache flush mechanism. We don't support 'MAP_SYNC' with virtio pmem and xfs.
--- fs/xfs/xfs_file.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-)
fs/xfs/xfs_file.c +6 -3
@@ -1203,11 +1203,14 @@ xfs_file_mmap(struct file *filp, struct vm_area_struct *vma) { + struct dax_device *dax_dev; + + dax_dev =xfs_find_daxdev_for_inode (file_inode (filp));/* -* We don't support synchronous mappings for non-DAX files. At least -* until someone comes with a sensible use case. +* We don't support synchronous mappings for non-DAX files and +* for DAX files if underneath dax_device is not synchronous. */ -if (!IS_DAX (file_inode (filp)) && (vma->vm_flags &VM_SYNC )) +if (!daxdev_mapping_supported (vma, dax_dev))return -EOPNOTSUPP ;file_accessed (filp);
Signature · 1 line
2.20.1