Skip to contentSkip to conversation

block/nvme: Align block pages queue to host page size

PATCH Superseded Philippe Mathieu-Daudé 5 participants 19 messages May 4, 2020 – May 7, 2020 Block layer

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

cover block/nvme: Align block pages queue to host page size

The most important patch of this series is 'Align block pages queue to host page size', it start fixing an issue on PowerPC hosts. It is not sufficient, but the rest is VFIO specific, so I prefer to send it separately.

The other patches are documentation I was writing down while looking at the code.

The last patch is about the emulated device and writeback. Marked as RFC as I don't understand writeback/dirty_log_mask yet.

Philippe Mathieu-Daudé (6):
qemu/osdep: Document qemu_memalign() and friends qemu/bitmap: Document bitmap_new() returned pointer sysemu/block-backend: Document blk_read()/blk_pwrite() block/block: Document BlockSizes fields block/nvme: Align block pages queue to host page size hw/block/nvme: Make device target agnostic

 include/block/block.h          |  8 ++++++--
 include/qemu/bitmap.h          |  2 ++
 include/qemu/osdep.h           |  3 +++
 include/sysemu/block-backend.h | 26 ++++++++++++++++++++++++++
 block/nvme.c                   |  2 +-
 hw/block/nvme.c                |  4 +---
 hw/block/Makefile.objs         |  2 +-
 7 files changed, 40 insertions(+), 7 deletions(-)
Signature · 1 line
2.21.3
#1 lore raw

PATCH 1/6 qemu/osdep: Document qemu_memalign() and friends

Document allocator functions that require a specific de-allocator call.

---
 include/block/block.h          | 4 ++++
 include/qemu/osdep.h           | 3 +++
 include/sysemu/block-backend.h | 2 ++
 3 files changed, 9 insertions(+)
include/block/block.h +4 -0
index 8b62429aa4..c57fdecf9a 100644
@@ -526,9 +526,13 @@ void bdrv_img_create(const char *filename, const char *fmt,
 size_t bdrv_min_mem_align(BlockDriverState *bs);
 /* Returns optimal alignment in bytes for bounce buffer */
 size_t bdrv_opt_mem_align(BlockDriverState *bs);
+/* callers must free the returned pointer with qemu_vfree() */
 void *qemu_blockalign(BlockDriverState *bs, size_t size);
+/* callers must free the returned pointer with qemu_vfree() */
 void *qemu_blockalign0(BlockDriverState *bs, size_t size);
+/* callers must free the returned pointer with qemu_vfree() */
 void *qemu_try_blockalign(BlockDriverState *bs, size_t size);
+/* callers must free the returned pointer with qemu_vfree() */
 void *qemu_try_blockalign0(BlockDriverState *bs, size_t size);
 bool bdrv_qiov_is_aligned(BlockDriverState *bs, QEMUIOVector *qiov);
 
include/qemu/osdep.h +3 -0
index 20f5c5f197..778c459c22 100644
@@ -294,8 +294,11 @@ extern int daemon(int, int);
 #endif
 
 int qemu_daemon(int nochdir, int noclose);
+/* callers must free the returned pointer with qemu_vfree() */
 void *qemu_try_memalign(size_t alignment, size_t size);
+/* callers must free the returned pointer with qemu_vfree() */
 void *qemu_memalign(size_t alignment, size_t size);
+/* callers must free the returned pointer with qemu_anon_ram_free() */
 void *qemu_anon_ram_alloc(size_t size, uint64_t *align, bool shared);
 void qemu_vfree(void *ptr);
 void qemu_anon_ram_free(void *ptr, size_t size);
include/sysemu/block-backend.h +2 -0
index 34de7faa81..f2dcf63ae3 100644
@@ -203,7 +203,9 @@ uint32_t blk_get_request_alignment(BlockBackend *blk);
 uint32_t blk_get_max_transfer(BlockBackend *blk);
 int blk_get_max_iov(BlockBackend *blk);
 void blk_set_guest_block_size(BlockBackend *blk, int align);
+/* callers must free the returned pointer with qemu_vfree() */
 void *blk_try_blockalign(BlockBackend *blk, size_t size);
+/* callers must free the returned pointer with qemu_vfree() */
 void *blk_blockalign(BlockBackend *blk, size_t size);
 bool blk_op_is_blocked(BlockBackend *blk, BlockOpType op, Error **errp);
 void blk_op_unblock(BlockBackend *blk, BlockOpType op, Error *reason);
Signature · 1 line
2.21.3
#3 lore raw

PATCH 2/6 qemu/bitmap: Document bitmap_new() returned pointer

In reply to Philippe Mathieu-Daudé
---
 include/qemu/bitmap.h | 2 ++
 1 file changed, 2 insertions(+)
include/qemu/bitmap.h +2 -0
index 82a1d2f41f..0b390ff576 100644
@@ -90,12 +90,14 @@ int slow_bitmap_intersects(const unsigned long *bitmap1,
                            const unsigned long *bitmap2, long bits);
 long slow_bitmap_count_one(const unsigned long *bitmap, long nbits);
 
+/* callers must free the returned pointer with g_free() */
 static inline unsigned long *bitmap_try_new(long nbits)
 {
     long len = BITS_TO_LONGS(nbits) * sizeof(unsigned long);
     return g_try_malloc0(len);
 }
 
+/* callers must free the returned pointer with g_free() */
 static inline unsigned long *bitmap_new(long nbits)
 {
     unsigned long *ptr = bitmap_try_new(nbits);
Signature · 1 line
2.21.3
#5 lore raw

PATCH 3/6 sysemu/block-backend: Document blk_read()/blk_pwrite()

In reply to Philippe Mathieu-Daudé

The blk_read()/blk_pwrite() return value is not obvious, document it.

---
 include/sysemu/block-backend.h | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)
include/sysemu/block-backend.h +24 -0
index f2dcf63ae3..823b8e94a7 100644
@@ -153,7 +153,31 @@ BlockAIOCB *blk_aio_pwrite_zeroes(BlockBackend *blk, int64_t offset,
                                   int bytes, BdrvRequestFlags flags,
                                   BlockCompletionFunc *cb, void *opaque);
 int blk_make_zero(BlockBackend *blk, BdrvRequestFlags flags);
+
+/**
+ * blk_pread:
+ *
+ * @blk - the block backend where the buffer content is going to be read from
+ * @offset: position in bytes to read at
+ * @buf: the data buffer
+ * @bytes: number of bytes to read
+ *
+ * Returns: the number of bytes read on success, or a negative errno otherwise.
+ */
 int blk_pread(BlockBackend *blk, int64_t offset, void *buf, int bytes);
+
+/**
+ * blk_pwrite:
+ *
+ * @blk - the block backend where the buffer content is going to be written to
+ * @offset: position in bytes to write at
+ * @buf: the data buffer
+ * @bytes: number of bytes of @buf to write
+ * @flags: request flags
+ *
+ * Returns: the number of bytes consumed on success,
+ *          or a negative errno otherwise.
+ */
 int blk_pwrite(BlockBackend *blk, int64_t offset, const void *buf, int bytes,
                BdrvRequestFlags flags);
 int64_t blk_getlength(BlockBackend *blk);
Signature · 1 line
2.21.3
#6 lore raw

Re: [PATCH 3/6] sysemu/block-backend: Document blk_read()/blk_pwrite()

include/sysemu/block-backend.h BlockAIOCB *blk_aio_pwrite_zeroes(BlockBackend *blk, int64_t offset, · line 160

Quoted 5 lines · Philippe Mathieu-Daudé
include/sysemu/block-backend.h +5 -0
+
+/**
+ * blk_pread:
+ *
+ * @blk - the block backend where the buffer content is going to be read from

s/ - /: /

include/sysemu/block-backend.h BlockAIOCB *blk_aio_pwrite_zeroes(BlockBackend *blk, int64_t offset, · line 172

Quoted 5 lines
include/sysemu/block-backend.h +5 -0
+
+/**
+ * blk_pwrite:
+ *
+ * @blk - the block backend where the buffer content is going to be written to

s/ - /: /

#7 lore raw

PATCH 4/6 block/block: Document BlockSizes fields

In reply to Philippe Mathieu-Daudé

As it is not obvious for a block neophyte what means the 'log' value, document it.

---
 include/block/block.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
include/block/block.h +2 -2
index c57fdecf9a..94517c92b6 100644
@@ -86,8 +86,8 @@ typedef enum {
 } BdrvRequestFlags;
 
 typedef struct BlockSizes {
-    uint32_t phys;
-    uint32_t log;
+    uint32_t phys;  /* physical block size */
+    uint32_t log;   /* logical block size */
 } BlockSizes;
 
 typedef struct HDGeometry {
Signature · 1 line
2.21.3
#9 lore raw

PATCH 5/6 block/nvme: Align block pages queue to host page size

In reply to Philippe Mathieu-Daudé

In nvme_create_queue_pair() we create a page list using qemu_blockalign(), then map it with qemu_vfio_dma_map():

q->prp_list_pages = qemu_blockalign0(bs, s->page_size * NVME_QUEUE_SIZE); r = qemu_vfio_dma_map(s->vfio, q->prp_list_pages,

                        s->page_size * NVME_QUEUE_SIZE, ...);

With:

  s->page_size = MAX(4096, 1 << (12 + ((cap >> 48) & 0xF)));

The qemu_vfio_dma_map() documentation says "The caller need to make sure the area is aligned to page size". While we use multiple s->page_size as alignment, it might be not sufficient on some hosts. Use the qemu_real_host_page_size value to be sure the host alignment is respected.

---
Cc: Cédric Le Goater <[email protected]>
Cc: David Gibson <[email protected]>
Cc: Laurent Vivier <[email protected]>

---
 block/nvme.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
block/nvme.c +1 -1
index 7b7c0cc5d6..bde0d28b39 100644
@@ -627,7 +627,7 @@ static int nvme_init(BlockDriverState *bs, const char *device, int namespace,
 
     s->page_size = MAX(4096, 1 << (12 + ((cap >> 48) & 0xF)));
     s->doorbell_scale = (4 << (((cap >> 32) & 0xF))) / sizeof(uint32_t);
-    bs->bl.opt_mem_alignment = s->page_size;
+    bs->bl.opt_mem_alignment = MAX(qemu_real_host_page_size, s->page_size);
     timeout_ms = MIN(500 * ((cap >> 24) & 0xFF), 30000);
 
     /* Reset device to get a clean state. */
Signature · 1 line
2.21.3
Reviewed-by on v1 5/6
#10 lore raw
Quoted 11 lines · Philippe Mathieu-Daudé

In nvme_create_queue_pair() we create a page list using qemu_blockalign(), then map it with qemu_vfio_dma_map():

q->prp_list_pages = qemu_blockalign0(bs, s->page_size * NVME_QUEUE_SIZE); r = qemu_vfio_dma_map(s->vfio, q->prp_list_pages,

                        s->page_size * NVME_QUEUE_SIZE, ...);

With:

  s->page_size = MAX(4096, 1 << (12 + ((cap >> 48) & 0xF)));
Quoted 7 lines

The qemu_vfio_dma_map() documentation says "The caller need to make sure the area is aligned to page size". While we use multiple s->page_size as alignment, it might be not sufficient on some hosts. Use the qemu_real_host_page_size value to be sure the host alignment is respected.

Quoted 21 lines

---
Cc: Cédric Le Goater <[email protected]>
Cc: David Gibson <[email protected]>
Cc: Laurent Vivier <[email protected]>

---
 block/nvme.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
block/nvme.c +1 -1
index 7b7c0cc5d6..bde0d28b39 100644
@@ -627,7 +627,7 @@ static int nvme_init(BlockDriverState *bs, const char *device, int namespace,
 
     s->page_size = MAX(4096, 1 << (12 + ((cap >> 48) & 0xF)));
     s->doorbell_scale = (4 << (((cap >> 32) & 0xF))) / sizeof(uint32_t);
-    bs->bl.opt_mem_alignment = s->page_size;
+    bs->bl.opt_mem_alignment = MAX(qemu_real_host_page_size, s->page_size);
     timeout_ms = MIN(500 * ((cap >> 24) & 0xFF), 30000);
 
     /* Reset device to get a clean state. */
Signature · 4 lines
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson
#11 lore raw
In reply to Philippe Mathieu-Daudé
Quoted 31 lines · Philippe Mathieu-Daudé

In nvme_create_queue_pair() we create a page list using qemu_blockalign(), then map it with qemu_vfio_dma_map():

q->prp_list_pages = qemu_blockalign0(bs, s->page_size * NVME_QUEUE_SIZE); r = qemu_vfio_dma_map(s->vfio, q->prp_list_pages,

                        s->page_size * NVME_QUEUE_SIZE, ...);

With:

  s->page_size = MAX(4096, 1 << (12 + ((cap >> 48) & 0xF)));

The qemu_vfio_dma_map() documentation says "The caller need to make sure the area is aligned to page size". While we use multiple s->page_size as alignment, it might be not sufficient on some hosts. Use the qemu_real_host_page_size value to be sure the host alignment is respected.

---
Cc: Cédric Le Goater <[email protected]>
Cc: David Gibson <[email protected]>
Cc: Laurent Vivier <[email protected]>

---
 block/nvme.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
block/nvme.c +0 -0
index 7b7c0cc5d6..bde0d28b39 100644

block/nvme.c static int nvme_init(BlockDriverState *bs, const char *device, int namespace, · line 630

Quoted 5 lines
block/nvme.c +1 -1
@@ -627,7 +627,7 @@ static int nvme_init(BlockDriverState *bs, const char *device, int namespace,
 
     s->page_size = MAX(4096, 1 << (12 + ((cap >> 48) & 0xF)));
     s->doorbell_scale = (4 << (((cap >> 32) & 0xF))) / sizeof(uint32_t);
-    bs->bl.opt_mem_alignment = s->page_size;
+    bs->bl.opt_mem_alignment = MAX(qemu_real_host_page_size, s->page_size);

Why don't you replace directly the "4096" in s->page_size by qemu_real_host_page_size?

I think this was the purpose of MAX(4096, ...) to align to the host page size.

Thanks,
Laurnt

#12 lore raw
Quoted 41 lines · Laurent Vivier
Quoted 31 lines · Philippe Mathieu-Daudé

In nvme_create_queue_pair() we create a page list using qemu_blockalign(), then map it with qemu_vfio_dma_map():

q->prp_list_pages = qemu_blockalign0(bs, s->page_size * NVME_QUEUE_SIZE); r = qemu_vfio_dma_map(s->vfio, q->prp_list_pages,

                        s->page_size * NVME_QUEUE_SIZE, ...);

With:

  s->page_size = MAX(4096, 1 << (12 + ((cap >> 48) & 0xF)));

The qemu_vfio_dma_map() documentation says "The caller need to make sure the area is aligned to page size". While we use multiple s->page_size as alignment, it might be not sufficient on some hosts. Use the qemu_real_host_page_size value to be sure the host alignment is respected.

---
Cc: Cédric Le Goater <[email protected]>
Cc: David Gibson <[email protected]>
Cc: Laurent Vivier <[email protected]>

---
 block/nvme.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
block/nvme.c +0 -0
index 7b7c0cc5d6..bde0d28b39 100644

block/nvme.c static int nvme_init(BlockDriverState *bs, const char *device, int namespace, · line 630

Quoted 5 lines
block/nvme.c +1 -1
@@ -627,7 +627,7 @@ static int nvme_init(BlockDriverState *bs, const char *device, int namespace,
 
     s->page_size = MAX(4096, 1 << (12 + ((cap >> 48) & 0xF)));
     s->doorbell_scale = (4 << (((cap >> 32) & 0xF))) / sizeof(uint32_t);
-    bs->bl.opt_mem_alignment = s->page_size;
+    bs->bl.opt_mem_alignment = MAX(qemu_real_host_page_size, s->page_size);

Why don't you replace directly the "4096" in s->page_size by qemu_real_host_page_size?

Quoted 2 lines

I think this was the purpose of MAX(4096, ...) to align to the host page size.

oh, I see, page_size is used for a lot of things than can be broken, and this is what is done in bdrv_opt_mem_align() for instance, 4096 is the sector size not the host page size.

BTW, you need the same change as in nvme_init() in nvme_refresh_limits(), I think.

Thanks,
Laurent

#13 lore raw
Quoted 45 lines · Laurent Vivier
Quoted 41 lines · Laurent Vivier
Quoted 31 lines · Philippe Mathieu-Daudé

In nvme_create_queue_pair() we create a page list using qemu_blockalign(), then map it with qemu_vfio_dma_map():

   q->prp_list_pages = qemu_blockalign0(bs, s->page_size * NVME_QUEUE_SIZE);
   r = qemu_vfio_dma_map(s->vfio, q->prp_list_pages,
                         s->page_size * NVME_QUEUE_SIZE, ...);

With:

   s->page_size = MAX(4096, 1 << (12 + ((cap >> 48) & 0xF)));

The qemu_vfio_dma_map() documentation says "The caller need to make sure the area is aligned to page size". While we use multiple s->page_size as alignment, it might be not sufficient on some hosts. Use the qemu_real_host_page_size value to be sure the host alignment is respected.

---
Cc: Cédric Le Goater <[email protected]>
Cc: David Gibson <[email protected]>
Cc: Laurent Vivier <[email protected]>

---
  block/nvme.c | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
block/nvme.c +0 -0
index 7b7c0cc5d6..bde0d28b39 100644

block/nvme.c static int nvme_init(BlockDriverState *bs, const char *device, int namespace, · line 630

Quoted 5 lines
block/nvme.c +1 -1
@@ -627,7 +627,7 @@ static int nvme_init(BlockDriverState *bs, const char *device, int namespace,
 
      s->page_size = MAX(4096, 1 << (12 + ((cap >> 48) & 0xF)));
      s->doorbell_scale = (4 << (((cap >> 32) & 0xF))) / sizeof(uint32_t);
-    bs->bl.opt_mem_alignment = s->page_size;
+    bs->bl.opt_mem_alignment = MAX(qemu_real_host_page_size, s->page_size);

Why don't you replace directly the "4096" in s->page_size by qemu_real_host_page_size?

Quoted 2 lines

I think this was the purpose of MAX(4096, ...) to align to the host page size.

Quoted 6 lines

oh, I see, page_size is used for a lot of things than can be broken, and this is what is done in bdrv_opt_mem_align() for instance, 4096 is the sector size not the host page size.

BTW, you need the same change as in nvme_init() in nvme_refresh_limits(), I think.

nvme_init() seems OK, but you are right, I missed nvme_refresh_limits()... Thanks!

Quoted 5 lines

Thanks,
Laurent

#14 lore raw
In reply to Philippe Mathieu-Daudé
Quoted 32 lines · Philippe Mathieu-Daudé

In nvme_create_queue_pair() we create a page list using qemu_blockalign(), then map it with qemu_vfio_dma_map():

   q->prp_list_pages = qemu_blockalign0(bs, s->page_size * NVME_QUEUE_SIZE);
   r = qemu_vfio_dma_map(s->vfio, q->prp_list_pages,
                         s->page_size * NVME_QUEUE_SIZE, ...);

With:

   s->page_size = MAX(4096, 1 << (12 + ((cap >> 48) & 0xF)));

The qemu_vfio_dma_map() documentation says "The caller need to make sure the area is aligned to page size". While we use multiple s->page_size as alignment, it might be not sufficient on some hosts. Use the qemu_real_host_page_size value to be sure the host alignment is respected.

---
Cc: Cédric Le Goater <[email protected]>
Cc: David Gibson <[email protected]>
Cc: Laurent Vivier <[email protected]>

---
  block/nvme.c | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
block/nvme.c +0 -0
index 7b7c0cc5d6..bde0d28b39 100644

block/nvme.c static int nvme_init(BlockDriverState *bs, const char *device, int namespace, · line 633

Quoted 8 lines
block/nvme.c +1 -1
@@ -627,7 +627,7 @@ static int nvme_init(BlockDriverState *bs, const char *device, int namespace,
 
      s->page_size = MAX(4096, 1 << (12 + ((cap >> 48) & 0xF)));
      s->doorbell_scale = (4 << (((cap >> 32) & 0xF))) / sizeof(uint32_t);
-    bs->bl.opt_mem_alignment = s->page_size;
+    bs->bl.opt_mem_alignment = MAX(qemu_real_host_page_size, s->page_size);
      timeout_ms = MIN(500 * ((cap >> 24) & 0xFF), 30000);
 
      /* Reset device to get a clean state. */

Please disregard this patch as it is incomplete.

#15 lore raw

RFC PATCH 6/6 hw/block/nvme: Make device target agnostic

In reply to Philippe Mathieu-Daudé

The NVMe device should not use target specific API. Use memory_region_do_writeback() (which was introduced in commit 61c490e25e0, after the NVMe emulated device was added) to replace qemu_ram_writeback().

---
RFC because I have no clue how dirty_log_mask works.

Cc Paolo Bonzini <[email protected]>
Cc Beata Michalska <[email protected]>
---
 hw/block/nvme.c        | 4 +---
 hw/block/Makefile.objs | 2 +-
 2 files changed, 2 insertions(+), 4 deletions(-)
hw/block/nvme.c +1 -3
index 9b453423cf..9b0ac0ea2a 100644
@@ -46,7 +46,6 @@
 #include "qapi/visitor.h"
 #include "sysemu/hostmem.h"
 #include "sysemu/block-backend.h"
-#include "exec/ram_addr.h"
 
 #include "qemu/log.h"
 #include "qemu/module.h"
@@ -1207,8 +1206,7 @@ static uint64_t nvme_mmio_read(void *opaque, hwaddr addr, unsigned size)
          */
         if (addr == 0xE08 &&
             (NVME_PMRCAP_PMRWBM(n->bar.pmrcap) & 0x02)) {
-            qemu_ram_writeback(n->pmrdev->mr.ram_block,
-                               0, n->pmrdev->size);
+            memory_region_do_writeback(&n->pmrdev->mr, 0, n->pmrdev->size);
         }
         memcpy(&val, ptr + addr, size);
     } else {
hw/block/Makefile.objs +1 -1
index 47960b5f0d..8855c22656 100644
@@ -13,6 +13,6 @@ common-obj-$(CONFIG_SH4) += tc58128.o
 
 obj-$(CONFIG_VIRTIO_BLK) += virtio-blk.o
 obj-$(CONFIG_VHOST_USER_BLK) += vhost-user-blk.o
-obj-$(CONFIG_NVME_PCI) += nvme.o
+common-obj-$(CONFIG_NVME_PCI) += nvme.o
 
 obj-y += dataplane/
Signature · 1 line
2.21.3
#16 lore raw

Re: [RFC PATCH 6/6] hw/block/nvme: Make device target agnostic

+Keith new email

Quoted 51 lines · Philippe Mathieu-Daudé

The NVMe device should not use target specific API. Use memory_region_do_writeback() (which was introduced in commit 61c490e25e0, after the NVMe emulated device was added) to replace qemu_ram_writeback().

---
RFC because I have no clue how dirty_log_mask works.

Cc Paolo Bonzini <[email protected]>
Cc Beata Michalska <[email protected]>
---
  hw/block/nvme.c        | 4 +---
  hw/block/Makefile.objs | 2 +-
  2 files changed, 2 insertions(+), 4 deletions(-)
hw/block/nvme.c +1 -3
index 9b453423cf..9b0ac0ea2a 100644
@@ -46,7 +46,6 @@
  #include "qapi/visitor.h"
  #include "sysemu/hostmem.h"
  #include "sysemu/block-backend.h"
-#include "exec/ram_addr.h"
 
  #include "qemu/log.h"
  #include "qemu/module.h"
@@ -1207,8 +1206,7 @@ static uint64_t nvme_mmio_read(void *opaque, hwaddr addr, unsigned size)
           */
          if (addr == 0xE08 &&
              (NVME_PMRCAP_PMRWBM(n->bar.pmrcap) & 0x02)) {
-            qemu_ram_writeback(n->pmrdev->mr.ram_block,
-                               0, n->pmrdev->size);
+            memory_region_do_writeback(&n->pmrdev->mr, 0, n->pmrdev->size);
          }
          memcpy(&val, ptr + addr, size);
      } else {
hw/block/Makefile.objs +1 -1
index 47960b5f0d..8855c22656 100644
@@ -13,6 +13,6 @@ common-obj-$(CONFIG_SH4) += tc58128.o
 
  obj-$(CONFIG_VIRTIO_BLK) += virtio-blk.o
  obj-$(CONFIG_VHOST_USER_BLK) += vhost-user-blk.o
-obj-$(CONFIG_NVME_PCI) += nvme.o
+common-obj-$(CONFIG_NVME_PCI) += nvme.o
 
  obj-y += dataplane/
#17 lore raw

Re: [RFC PATCH 6/6] hw/block/nvme: Make device target agnostic

Quoted 29 lines · Philippe Mathieu-Daudé

+Keith new email

Quoted 26 lines · Philippe Mathieu-Daudé

The NVMe device should not use target specific API. Use memory_region_do_writeback() (which was introduced in commit 61c490e25e0, after the NVMe emulated device was added) to replace qemu_ram_writeback().

---
RFC because I have no clue how dirty_log_mask works.

Cc Paolo Bonzini <[email protected]>
Cc Beata Michalska <[email protected]>
---
  hw/block/nvme.c        | 4 +---
  hw/block/Makefile.objs | 2 +-
  2 files changed, 2 insertions(+), 4 deletions(-)
hw/block/nvme.c +0 -1
index 9b453423cf..9b0ac0ea2a 100644
@@ -46,7 +46,6 @@
  #include "qapi/visitor.h"
  #include "sysemu/hostmem.h"
  #include "sysemu/block-backend.h"
-#include "exec/ram_addr.h"
  #include "qemu/log.h"
Quoted 8 lines

hw/block/nvme.c static uint64_t nvme_mmio_read(void *opaque, hwaddr addr, unsigned size) · line 1209

Quoted 8 lines
hw/block/nvme.c +1 -2
  #include "qemu/module.h"
@@ -1207,8 +1206,7 @@ static uint64_t nvme_mmio_read(void *opaque, hwaddr addr, unsigned size)
           */
          if (addr == 0xE08 &&
              (NVME_PMRCAP_PMRWBM(n->bar.pmrcap) & 0x02)) {
-            qemu_ram_writeback(n->pmrdev->mr.ram_block,
-                               0, n->pmrdev->size);
+            memory_region_do_writeback(&n->pmrdev->mr, 0, n->pmrdev->size);

qemu_ram_write() is being called because we need to msync or persist pmem here.

The memory_region_do_writeback() API is not equivalent, its purpose is for dirty write logging (which we don't care about here because the writes themselves will already have been logged when the guest performed them).

I think qemu_ram_writeback() should just be made common so that this code isn't target-specific. Maybe it should be renamed to qemu_ram_msync() to avoid confusion with dirty write APIs.

Stefan

#18 lore raw

Re: [RFC PATCH 6/6] hw/block/nvme: Make device target agnostic

Quoted 7 lines · Stefan Hajnoczi
Quoted 4 lines

hw/block/nvme.c static uint64_t nvme_mmio_read(void *opaque, hwaddr addr, unsigned size) · line 1209

Quoted 4 lines
              (NVME_PMRCAP_PMRWBM(n->bar.pmrcap) & 0x02)) {
hw/block/nvme.c +1 -2
-            qemu_ram_writeback(n->pmrdev->mr.ram_block,
-                               0, n->pmrdev->size);
+            memory_region_do_writeback(&n->pmrdev->mr, 0, n->pmrdev->size);

qemu_ram_write() is being called because we need to msync or persist pmem here.

Quoted 8 lines

The memory_region_do_writeback() API is not equivalent, its purpose is for dirty write logging (which we don't care about here because the writes themselves will already have been logged when the guest performed them).

I think qemu_ram_writeback() should just be made common so that this code isn't target-specific. Maybe it should be renamed to qemu_ram_msync() to avoid confusion with dirty write APIs.

Yes, we can add qemu_ram_msync and memory_region_msync.

Paolo