Skip to contentSkip to conversation

hw/pvrdma: PVRDMA device implementation

PATCH New Marcel Apfelbaum 1 participant 11 messages Feb 14, 2018

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

v11 cover hw/pvrdma: PVRDMA device implementation

V10 -> V11:
- Addressed Michael S. Tsirkin comments:
- Split the standard-headers patch in two, one dealing with the update-linux-headers script while the other adds the imported headers.
- Add comments to the update-linux-headers script explaining the sed transformations.
- Added Zhu Yanjun's R-B tags (rdma patches review -- Thanks!)
- Added Gal Hammer's R-B tag (update-linux-headers patch review (Thanks!)
- Rebased on latest master

V9 -> V10:
- Addressed Peter Maydell's comments:
- Modified license to "version 2 or any later version"
- Added license comment on top of code files
- Move the kernel headers to "standard-headers" and modified the update-linux-headers script to import them and update the types for QEMU. (this patch has no R-B tag, maybe someone can have a look, I am not sure who can review it)
- Got an R-B from Eduardo on memory-ram-backend patch (thanks!)
- Split the pvrdma implementation patch into 6 patches, preserving Dotan Barak R-B since no semantic changes were made, only a mechanical split.
- Rebased on latest master

V8 -> V9:
- Addressed Dotan Barak's (offline) comments:
- use g_malloc instead of malloc
- re-arrange structs for better padding
- some cosmetic changes
- do not try to fetch CQE when CQ is going down
- init state of QP changed to RESET
- modify poll_cq
- add fix to qkey handling so now qkey=0 is also supported
- add validation to gid_index
- fix memory leak with ah_key ref
- Addressed Eduardo Habkost comments:
- Add the mem-backed-ram "share" option to qemu-options.hx.
- Rebased on latest master

V7 -> V8:
- Addressed Michael S. Tsirkin comments:
- fail to init the pvrdma device if target page size is different from the host size, or if the guest RAM is not backed by memory and shared.
- Update documentation to include a note on huge memory regions registration and remove not needed info.
- Removed "pci/shpc: Move function to generic header file" since it appears in latest maintainer pull request
- Rebased on master

V6 -> V7:
- Addressed Philippe Mathieu-Daudé comments
- modified pow2roundup32 signature
- added his RB tag (thanks)
- Addressed Corenlia Huck comments:
- Compiled the pvrdma for all archs and not only x86/arm (thanks)
- Fixed typo in documentation
- Rebased on latest master

V5 -> V6:
- Found a ppc machine and solved ppc compilation issues
- Tried to fix the s390x issue (still looking of a machine)

V4 -> V5:
- Fixed (at least tried to) compilation issues

V3 -> V4:
- Fixed documentation (added more impl details)
- Fixed compilation errors discovered by patchew.
- Addressed Michael S. Tsirkin comments:
- Removed unnecessary typedefs and replace them with macros in VMware header files, together with explanations.
- Moved more code from vmw specific to rdma generic code.
- Added page size limitations to the documentation.

V2 -> V3:
- Addressed Michael S. Tsirkin and Philippe Mathieu-Daudé comments:
- Moved the device to hw/rdma
- Addressed Michael S. Tsirkin comments:
- Split the code into generic (hw/rdma) and VMWare specific (hw/rdma/vmw)
- Added more details to documentation - VMware guest-host protocol.
- Remove mad processing
- limited the memory the Guest can pin.
- Addressed Philippe Mathieu-Daudé comment:
- s/roundup_pow_of_two/pow2roundup32 and move it to qemu/host-utils.h
- Added Shamit Rabinovici's review to documentation
- Rebased to latest master

RFC -> V2:
- Full implementation of the pvrdma device
- Backend is an ibdevice interface, no need for the KDBR module

General description
===================
PVRDMA is the QEMU implementation of VMware's paravirtualized RDMA device. It works with its Linux Kernel driver AS IS, no need for any special guest modifications.

While it complies with the VMware device, it can also communicate with bare metal RDMA-enabled machines and does not require an RDMA HCA in the host, it can work with Soft-RoCE (rxe).

It does not require the whole guest RAM to be pinned allowing memory over-commit and, even if not implemented yet, migration support will be possible with some HW assistance.

Design
======
- Follows the behavior of VMware's pvrdma device, however is not tightly coupled with it and most of the code can be reused if we decide to continue to a Virtio based RDMA device.

- It exposes 3 BARs:
BAR 0 - MSIX, utilize 3 vectors for command ring, async events and
completions
BAR 1 - Configuration of registers
BAR 2 - UAR, used to pass HW commands from driver.

- The device performs internal management of the RDMA resources (PDs, CQs, QPs, ...), meaning the objects are not directly coupled to a physical RDMA device resources.

The pvrdma backend is an ibdevice interface that can be exposed either by a Soft-RoCE(rxe) device on machines with no RDMA device, or an HCA SRIOV function(VF/PF).
Note that ibdevice interfaces can't be shared between pvrdma devices, each one requiring a separate instance (rxe or SRIOV VF).

Tests and performance
=====================
Tested with SoftRoCE backend (rxe)/Mellanox ConnectX3, and Mellanox ConnectX4 HCAs with:
- VMs in the same host
- VMs in different hosts
- VMs to bare metal.

The best performance achieved with ConnectX HCAs and buffer size bigger than 1MB which was the line rate ~ 50Gb/s.
The conclusion is that using the PVRDMA device there are no actual performance penalties compared to bare metal for big enough buffers (which is quite common when using RDMA), while allowing memory overcommit.

Marcel Apfelbaum (5):
mem: add share parameter to memory-backend-ram
docs: add pvrdma device documentation. scripts/update-linux-headers: import pvrdma headers include/standard-headers: add pvrdma related headers
MAINTAINERS: add entry for hw/rdma

Yuval Shaia (5):
hw/rdma: Add wrappers and macros
hw/rdma: Definitions for rdma device and rdma resource manager hw/rdma: Implementation of generic rdma device layers hw/rdma: PVRDMA commands and data-path ops
hw/rdma: Implementation of PVRDMA device

 MAINTAINERS                                        |   8 +
 Makefile.objs                                      |   2 +
 backends/hostmem-file.c                            |  25 +-
 backends/hostmem-ram.c                             |   4 +-
 backends/hostmem.c                                 |  21 +
 configure                                          |   9 +-
 docs/pvrdma.txt                                    | 255 +++++++
 exec.c                                             |  26 +-
 hw/Makefile.objs                                   |   1 +
 hw/rdma/Makefile.objs                              |   5 +
 hw/rdma/rdma_backend.c                             | 818 +++++++++++++++++++++
 hw/rdma/rdma_backend.h                             |  98 +++
 hw/rdma/rdma_backend_defs.h                        |  62 ++
 hw/rdma/rdma_rm.c                                  | 544 ++++++++++++++
 hw/rdma/rdma_rm.h                                  |  69 ++
 hw/rdma/rdma_rm_defs.h                             | 104 +++
 hw/rdma/rdma_utils.c                               |  51 ++
 hw/rdma/rdma_utils.h                               |  43 ++
 hw/rdma/trace-events                               |   5 +
 hw/rdma/vmw/pvrdma.h                               | 122 +++
 hw/rdma/vmw/pvrdma_cmd.c                           | 673 +++++++++++++++++
 hw/rdma/vmw/pvrdma_dev_ring.c                      | 155 ++++
 hw/rdma/vmw/pvrdma_dev_ring.h                      |  42 ++
 hw/rdma/vmw/pvrdma_main.c                          | 670 +++++++++++++++++
 hw/rdma/vmw/pvrdma_qp_ops.c                        | 222 ++++++
 hw/rdma/vmw/pvrdma_qp_ops.h                        |  27 +
 hw/rdma/vmw/trace-events                           |   5 +
 include/exec/memory.h                              |  23 +
 include/exec/ram_addr.h                            |   3 +-
 include/hw/pci/pci_ids.h                           |   3 +
 include/qemu/osdep.h                               |   2 +-
 .../infiniband/hw/vmw_pvrdma/pvrdma_dev_api.h      | 667 +++++++++++++++++
 .../drivers/infiniband/hw/vmw_pvrdma/pvrdma_ring.h | 114 +++
 .../infiniband/hw/vmw_pvrdma/pvrdma_verbs.h        | 383 ++++++++++
 include/standard-headers/rdma/vmw_pvrdma-abi.h     | 293 ++++++++
 include/sysemu/hostmem.h                           |   2 +-
 include/sysemu/kvm.h                               |   2 +-
 memory.c                                           |  16 +-
 qemu-options.hx                                    |  10 +-
 scripts/update-linux-headers.sh                    |  30 +
 target/s390x/kvm.c                                 |   4 +-
 util/oslib-posix.c                                 |   4 +-
 util/oslib-win32.c                                 |   2 +-
 43 files changed, 5570 insertions(+), 54 deletions(-)
 create mode 100644 docs/pvrdma.txt
 create mode 100644 hw/rdma/Makefile.objs
 create mode 100644 hw/rdma/rdma_backend.c
 create mode 100644 hw/rdma/rdma_backend.h
 create mode 100644 hw/rdma/rdma_backend_defs.h
 create mode 100644 hw/rdma/rdma_rm.c
 create mode 100644 hw/rdma/rdma_rm.h
 create mode 100644 hw/rdma/rdma_rm_defs.h
 create mode 100644 hw/rdma/rdma_utils.c
 create mode 100644 hw/rdma/rdma_utils.h
 create mode 100644 hw/rdma/trace-events
 create mode 100644 hw/rdma/vmw/pvrdma.h
 create mode 100644 hw/rdma/vmw/pvrdma_cmd.c
 create mode 100644 hw/rdma/vmw/pvrdma_dev_ring.c
 create mode 100644 hw/rdma/vmw/pvrdma_dev_ring.h
 create mode 100644 hw/rdma/vmw/pvrdma_main.c
 create mode 100644 hw/rdma/vmw/pvrdma_qp_ops.c
 create mode 100644 hw/rdma/vmw/pvrdma_qp_ops.h
 create mode 100644 hw/rdma/vmw/trace-events
 create mode 100644 include/standard-headers/drivers/infiniband/hw/vmw_pvrdma/pvrdma_dev_api.h
 create mode 100644 include/standard-headers/drivers/infiniband/hw/vmw_pvrdma/pvrdma_ring.h
 create mode 100644 include/standard-headers/drivers/infiniband/hw/vmw_pvrdma/pvrdma_verbs.h
 create mode 100644 include/standard-headers/rdma/vmw_pvrdma-abi.h
Signature · 1 line
2.13.5
#1 lore raw

PATCH v11 1/10 mem: add share parameter to memory-backend-ram

Currently only file backed memory backend can be created with a "share" flag in order to allow sharing guest RAM with other processes in the host.

Add the "share" flag also to RAM Memory Backend
in order to allow remapping parts of the guest RAM to different host virtual addresses. This is needed by the RDMA devices in order to remap non-contiguous QEMU virtual addresses to a contiguous virtual address range.

Moved the "share" flag to the Host Memory base class, modified phys_mem_alloc to include the new parameter and a new interface memory_region_init_ram_shared_nomigrate.

There are no functional changes if the new flag is not used.

---
 backends/hostmem-file.c  | 25 +------------------------
 backends/hostmem-ram.c   |  4 ++--
 backends/hostmem.c       | 21 +++++++++++++++++++++
 exec.c                   | 26 +++++++++++++++-----------
 include/exec/memory.h    | 23 +++++++++++++++++++++++
 include/exec/ram_addr.h  |  3 ++-
 include/qemu/osdep.h     |  2 +-
 include/sysemu/hostmem.h |  2 +-
 include/sysemu/kvm.h     |  2 +-
 memory.c                 | 16 +++++++++++++---
 qemu-options.hx          | 10 +++++++++-
 target/s390x/kvm.c       |  4 ++--
 util/oslib-posix.c       |  4 ++--
 util/oslib-win32.c       |  2 +-
 14 files changed, 94 insertions(+), 50 deletions(-)
backends/hostmem-file.c +1 -24
index e319ec1ad8..134b08d63a 100644
@@ -31,7 +31,6 @@ typedef struct HostMemoryBackendFile HostMemoryBackendFile;
 struct HostMemoryBackendFile {
     HostMemoryBackend parent_obj;
 
-    bool share;
     bool discard_data;
     char *mem_path;
     uint64_t align;
@@ -59,7 +58,7 @@ file_backend_memory_alloc(HostMemoryBackend *backend, Error **errp)
         path = object_get_canonical_path(OBJECT(backend));
         memory_region_init_ram_from_file(&backend->mr, OBJECT(backend),
                                  path,
-                                 backend->size, fb->align, fb->share,
+                                 backend->size, fb->align, backend->share,
                                  fb->mem_path, errp);
         g_free(path);
     }
@@ -86,25 +85,6 @@ static void set_mem_path(Object *o, const char *str, Error **errp)
     fb->mem_path = g_strdup(str);
 }
 
-static bool file_memory_backend_get_share(Object *o, Error **errp)
-{
-    HostMemoryBackendFile *fb = MEMORY_BACKEND_FILE(o);
-
-    return fb->share;
-}
-
-static void file_memory_backend_set_share(Object *o, bool value, Error **errp)
-{
-    HostMemoryBackend *backend = MEMORY_BACKEND(o);
-    HostMemoryBackendFile *fb = MEMORY_BACKEND_FILE(o);
-
-    if (host_memory_backend_mr_inited(backend)) {
-        error_setg(errp, "cannot change property value");
-        return;
-    }
-    fb->share = value;
-}
-
 static bool file_memory_backend_get_discard_data(Object *o, Error **errp)
 {
     return MEMORY_BACKEND_FILE(o)->discard_data;
@@ -171,9 +151,6 @@ file_backend_class_init(ObjectClass *oc, void *data)
     bc->alloc = file_backend_memory_alloc;
     oc->unparent = file_backend_unparent;
 
-    object_class_property_add_bool(oc, "share",
-        file_memory_backend_get_share, file_memory_backend_set_share,
-        &error_abort);
     object_class_property_add_bool(oc, "discard-data",
         file_memory_backend_get_discard_data, file_memory_backend_set_discard_data,
         &error_abort);
backends/hostmem-ram.c +2 -2
index 38977be73e..7ddd08d370 100644
@@ -28,8 +28,8 @@ ram_backend_memory_alloc(HostMemoryBackend *backend, Error **errp)
     }
 
     path = object_get_canonical_path_component(OBJECT(backend));
-    memory_region_init_ram_nomigrate(&backend->mr, OBJECT(backend), path,
-                           backend->size, errp);
+    memory_region_init_ram_shared_nomigrate(&backend->mr, OBJECT(backend), path,
+                           backend->size, backend->share, errp);
     g_free(path);
 }
 
backends/hostmem.c +21 -0
index 81d14554a7..8aa0412032 100644
@@ -368,6 +368,24 @@ static void set_id(Object *o, const char *str, Error **errp)
     backend->id = g_strdup(str);
 }
 
+static bool host_memory_backend_get_share(Object *o, Error **errp)
+{
+    HostMemoryBackend *backend = MEMORY_BACKEND(o);
+
+    return backend->share;
+}
+
+static void host_memory_backend_set_share(Object *o, bool value, Error **errp)
+{
+    HostMemoryBackend *backend = MEMORY_BACKEND(o);
+
+    if (host_memory_backend_mr_inited(backend)) {
+        error_setg(errp, "cannot change property value");
+        return;
+    }
+    backend->share = value;
+}
+
 static void
 host_memory_backend_class_init(ObjectClass *oc, void *data)
 {
@@ -398,6 +416,9 @@ host_memory_backend_class_init(ObjectClass *oc, void *data)
         host_memory_backend_get_policy,
         host_memory_backend_set_policy, &error_abort);
     object_class_property_add_str(oc, "id", get_id, set_id, &error_abort);
+    object_class_property_add_bool(oc, "share",
+        host_memory_backend_get_share, host_memory_backend_set_share,
+        &error_abort);
 }
 
 static void host_memory_backend_finalize(Object *o)
exec.c +15 -11
index e8d7b335b6..4d8addb263 100644
@@ -1285,7 +1285,7 @@ static int subpage_register (subpage_t *mmio, uint32_t start, uint32_t end,
                              uint16_t section);
 static subpage_t *subpage_init(FlatView *fv, hwaddr base);
 
-static void *(*phys_mem_alloc)(size_t size, uint64_t *align) =
+static void *(*phys_mem_alloc)(size_t size, uint64_t *align, bool shared) =
                                qemu_anon_ram_alloc;
 
 /*
@@ -1293,7 +1293,7 @@ static void *(*phys_mem_alloc)(size_t size, uint64_t *align) =
  * Accelerators with unusual needs may need this.  Hopefully, we can
  * get rid of it eventually.
  */
-void phys_mem_set_alloc(void *(*alloc)(size_t, uint64_t *align))
+void phys_mem_set_alloc(void *(*alloc)(size_t, uint64_t *align, bool shared))
 {
     phys_mem_alloc = alloc;
 }
@@ -1921,7 +1921,7 @@ static void dirty_memory_extend(ram_addr_t old_ram_size,
     }
 }
 
-static void ram_block_add(RAMBlock *new_block, Error **errp)
+static void ram_block_add(RAMBlock *new_block, Error **errp, bool shared)
 {
     RAMBlock *block;
     RAMBlock *last_block = NULL;
@@ -1944,7 +1944,7 @@ static void ram_block_add(RAMBlock *new_block, Error **errp)
             }
         } else {
             new_block->host = phys_mem_alloc(new_block->max_length,
-                                             &new_block->mr->align);
+                                             &new_block->mr->align, shared);
             if (!new_block->host) {
                 error_setg_errno(errp, errno,
                                  "cannot set up guest memory '%s'",
@@ -2049,7 +2049,7 @@ RAMBlock *qemu_ram_alloc_from_fd(ram_addr_t size, MemoryRegion *mr,
         return NULL;
     }
 
-    ram_block_add(new_block, &local_err);
+    ram_block_add(new_block, &local_err, share);
     if (local_err) {
         g_free(new_block);
         error_propagate(errp, local_err);
@@ -2091,7 +2091,7 @@ RAMBlock *qemu_ram_alloc_internal(ram_addr_t size, ram_addr_t max_size,
                                   void (*resized)(const char*,
                                                   uint64_t length,
                                                   void *host),
-                                  void *host, bool resizeable,
+                                  void *host, bool resizeable, bool share,
                                   MemoryRegion *mr, Error **errp)
 {
     RAMBlock *new_block;
@@ -2114,7 +2114,7 @@ RAMBlock *qemu_ram_alloc_internal(ram_addr_t size, ram_addr_t max_size,
     if (resizeable) {
         new_block->flags |= RAM_RESIZEABLE;
     }
-    ram_block_add(new_block, &local_err);
+    ram_block_add(new_block, &local_err, share);
     if (local_err) {
         g_free(new_block);
         error_propagate(errp, local_err);
@@ -2126,12 +2126,15 @@ RAMBlock *qemu_ram_alloc_internal(ram_addr_t size, ram_addr_t max_size,
 RAMBlock *qemu_ram_alloc_from_ptr(ram_addr_t size, void *host,
                                    MemoryRegion *mr, Error **errp)
 {
-    return qemu_ram_alloc_internal(size, size, NULL, host, false, mr, errp);
+    return qemu_ram_alloc_internal(size, size, NULL, host, false,
+                                   false, mr, errp);
 }
 
-RAMBlock *qemu_ram_alloc(ram_addr_t size, MemoryRegion *mr, Error **errp)
+RAMBlock *qemu_ram_alloc(ram_addr_t size, bool share,
+                         MemoryRegion *mr, Error **errp)
 {
-    return qemu_ram_alloc_internal(size, size, NULL, NULL, false, mr, errp);
+    return qemu_ram_alloc_internal(size, size, NULL, NULL, false,
+                                   share, mr, errp);
 }
 
 RAMBlock *qemu_ram_alloc_resizeable(ram_addr_t size, ram_addr_t maxsz,
@@ -2140,7 +2143,8 @@ RAMBlock *qemu_ram_alloc_resizeable(ram_addr_t size, ram_addr_t maxsz,
                                                      void *host),
                                      MemoryRegion *mr, Error **errp)
 {
-    return qemu_ram_alloc_internal(size, maxsz, resized, NULL, true, mr, errp);
+    return qemu_ram_alloc_internal(size, maxsz, resized, NULL, true,
+                                   false, mr, errp);
 }
 
 static void reclaim_ramblock(RAMBlock *block)
include/exec/memory.h +23 -0
index fff9b1d871..15e81113ba 100644
@@ -436,6 +436,29 @@ void memory_region_init_ram_nomigrate(MemoryRegion *mr,
                                       Error **errp);
 
 /**
+ * memory_region_init_ram_shared_nomigrate:  Initialize RAM memory region.
+ *                                           Accesses into the region will
+ *                                           modify memory directly.
+ *
+ * @mr: the #MemoryRegion to be initialized.
+ * @owner: the object that tracks the region's reference count
+ * @name: Region name, becomes part of RAMBlock name used in migration stream
+ *        must be unique within any device
+ * @size: size of the region.
+ * @share: allow remapping RAM to different addresses
+ * @errp: pointer to Error*, to store an error if it happens.
+ *
+ * Note that this function is similar to memory_region_init_ram_nomigrate.
+ * The only difference is part of the RAM region can be remapped.
+ */
+void memory_region_init_ram_shared_nomigrate(MemoryRegion *mr,
+                                             struct Object *owner,
+                                             const char *name,
+                                             uint64_t size,
+                                             bool share,
+                                             Error **errp);
+
+/**
  * memory_region_init_resizeable_ram:  Initialize memory region with resizeable
  *                                     RAM.  Accesses into the region will
  *                                     modify memory directly.  Only an initial
include/exec/ram_addr.h +2 -1
index 7633ef6342..cf2446a176 100644
@@ -80,7 +80,8 @@ RAMBlock *qemu_ram_alloc_from_fd(ram_addr_t size, MemoryRegion *mr,
                                  Error **errp);
 RAMBlock *qemu_ram_alloc_from_ptr(ram_addr_t size, void *host,
                                   MemoryRegion *mr, Error **errp);
-RAMBlock *qemu_ram_alloc(ram_addr_t size, MemoryRegion *mr, Error **errp);
+RAMBlock *qemu_ram_alloc(ram_addr_t size, bool share, MemoryRegion *mr,
+                         Error **errp);
 RAMBlock *qemu_ram_alloc_resizeable(ram_addr_t size, ram_addr_t max_size,
                                     void (*resized)(const char*,
                                                     uint64_t length,
include/qemu/osdep.h +1 -1
index adb3758275..41658060a7 100644
@@ -255,7 +255,7 @@ extern int daemon(int, int);
 int qemu_daemon(int nochdir, int noclose);
 void *qemu_try_memalign(size_t alignment, size_t size);
 void *qemu_memalign(size_t alignment, size_t size);
-void *qemu_anon_ram_alloc(size_t size, uint64_t *align);
+void *qemu_anon_ram_alloc(size_t size, uint64_t *align, bool shared);
 void qemu_vfree(void *ptr);
 void qemu_anon_ram_free(void *ptr, size_t size);
 
include/sysemu/hostmem.h +1 -1
index 621a3f9d42..d5ab0b99c6 100644
@@ -54,7 +54,7 @@ struct HostMemoryBackend {
     char *id;
     uint64_t size;
     bool merge, dump;
-    bool prealloc, force_prealloc, is_mapped;
+    bool prealloc, force_prealloc, is_mapped, share;
     DECLARE_BITMAP(host_nodes, MAX_NODES + 1);
     HostMemPolicy policy;
 
include/sysemu/kvm.h +1 -1
index bbf12a1723..85002ac49a 100644
@@ -248,7 +248,7 @@ int kvm_on_sigbus(int code, void *addr);
 
 /* interface with exec.c */
 
-void phys_mem_set_alloc(void *(*alloc)(size_t, uint64_t *align));
+void phys_mem_set_alloc(void *(*alloc)(size_t, uint64_t *align, bool shared));
 
 /* internal API */
 
memory.c +13 -3
index c7f6588452..6515131ac2 100644
@@ -1539,11 +1539,21 @@ void memory_region_init_ram_nomigrate(MemoryRegion *mr,
                                       uint64_t size,
                                       Error **errp)
 {
+    memory_region_init_ram_shared_nomigrate(mr, owner, name, size, false, errp);
+}
+
+void memory_region_init_ram_shared_nomigrate(MemoryRegion *mr,
+                                             Object *owner,
+                                             const char *name,
+                                             uint64_t size,
+                                             bool share,
+                                             Error **errp)
+{
     memory_region_init(mr, owner, name, size);
     mr->ram = true;
     mr->terminates = true;
     mr->destructor = memory_region_destructor_ram;
-    mr->ram_block = qemu_ram_alloc(size, mr, errp);
+    mr->ram_block = qemu_ram_alloc(size, share, mr, errp);
     mr->dirty_log_mask = tcg_enabled() ? (1 << DIRTY_MEMORY_CODE) : 0;
 }
 
@@ -1654,7 +1664,7 @@ void memory_region_init_rom_nomigrate(MemoryRegion *mr,
     mr->readonly = true;
     mr->terminates = true;
     mr->destructor = memory_region_destructor_ram;
-    mr->ram_block = qemu_ram_alloc(size, mr, errp);
+    mr->ram_block = qemu_ram_alloc(size, false, mr, errp);
     mr->dirty_log_mask = tcg_enabled() ? (1 << DIRTY_MEMORY_CODE) : 0;
 }
 
@@ -1673,7 +1683,7 @@ void memory_region_init_rom_device_nomigrate(MemoryRegion *mr,
     mr->terminates = true;
     mr->rom_device = true;
     mr->destructor = memory_region_destructor_ram;
-    mr->ram_block = qemu_ram_alloc(size, mr, errp);
+    mr->ram_block = qemu_ram_alloc(size, false,  mr, errp);
 }
 
 void memory_region_init_iommu(void *_iommu_mr,
qemu-options.hx +9 -1
index 5050a49a5e..8ccd5dcaa6 100644
@@ -3975,6 +3975,14 @@ The @option{share} boolean option determines whether the memory
 region is marked as private to QEMU, or shared. The latter allows
 a co-operating external process to access the QEMU memory region.
 
+The @option{share} is also required for pvrdma devices due to
+limitations in the RDMA API provided by Linux.
+
+Setting share=on might affect the ability to configure NUMA
+bindings for the memory backend under some circumstances, see
+Documentation/vm/numa_memory_policy.txt on the Linux kernel
+source tree for additional details.
+
 Setting the @option{discard-data} boolean option to @var{on}
 indicates that file contents can be destroyed when QEMU exits,
 to avoid unnecessarily flushing data to the backing file.  Note
@@ -4017,7 +4025,7 @@ requires an alignment different than the default one used by QEMU, eg
 the device DAX /dev/dax0.0 requires 2M alignment rather than 4K. In
 such cases, users can specify the required alignment via this option.
 
-@item -object memory-backend-ram,id=@var{id},merge=@var{on|off},dump=@var{on|off},prealloc=@var{on|off},size=@var{size},host-nodes=@var{host-nodes},policy=@var{default|preferred|bind|interleave}
+@item -object memory-backend-ram,id=@var{id},merge=@var{on|off},dump=@var{on|off},share=@var{on|off},prealloc=@var{on|off},size=@var{size},host-nodes=@var{host-nodes},policy=@var{default|preferred|bind|interleave}
 
 Creates a memory backend object, which can be used to back the guest RAM.
 Memory backend objects offer more control than the @option{-m} option that is
target/s390x/kvm.c +2 -2
index 0301e9d519..e13c8907df 100644
@@ -144,7 +144,7 @@ static int cap_gs;
 
 static int active_cmma;
 
-static void *legacy_s390_alloc(size_t size, uint64_t *align);
+static void *legacy_s390_alloc(size_t size, uint64_t *align, bool shared);
 
 static int kvm_s390_query_mem_limit(uint64_t *memory_limit)
 {
@@ -752,7 +752,7 @@ int kvm_s390_mem_op(S390CPU *cpu, vaddr addr, uint8_t ar, void *hostbuf,
  * to grow. We also have to use MAP parameters that avoid
  * read-only mapping of guest pages.
  */
-static void *legacy_s390_alloc(size_t size, uint64_t *align)
+static void *legacy_s390_alloc(size_t size, uint64_t *align, bool shared)
 {
     void *mem;
 
util/oslib-posix.c +2 -2
index 4655bc1f89..13b6f8d776 100644
@@ -127,10 +127,10 @@ void *qemu_memalign(size_t alignment, size_t size)
 }
 
 /* alloc shared memory pages */
-void *qemu_anon_ram_alloc(size_t size, uint64_t *alignment)
+void *qemu_anon_ram_alloc(size_t size, uint64_t *alignment, bool shared)
 {
     size_t align = QEMU_VMALLOC_ALIGN;
-    void *ptr = qemu_ram_mmap(-1, size, align, false);
+    void *ptr = qemu_ram_mmap(-1, size, align, shared);
 
     if (ptr == MAP_FAILED) {
         return NULL;
util/oslib-win32.c +1 -1
index 69a6286d50..bb5ad28bd3 100644
@@ -67,7 +67,7 @@ void *qemu_memalign(size_t alignment, size_t size)
     return qemu_oom_check(qemu_try_memalign(alignment, size));
 }
 
-void *qemu_anon_ram_alloc(size_t size, uint64_t *align)
+void *qemu_anon_ram_alloc(size_t size, uint64_t *align, bool shared)
 {
     void *ptr;
 
Signature · 1 line
2.13.5
#2 lore raw

PATCH v11 2/10 docs: add pvrdma device documentation.

In reply to Marcel Apfelbaum
---
 docs/pvrdma.txt | 255 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 255 insertions(+)
 create mode 100644 docs/pvrdma.txt
docs/pvrdma.txt new file +255 -0
new file mode 100644
index 0000000000..5599318159
@@ -0,0 +1,255 @@
+Paravirtualized RDMA Device (PVRDMA)
+====================================
+
+
+1. Description
+===============
+PVRDMA is the QEMU implementation of VMware's paravirtualized RDMA device.
+It works with its Linux Kernel driver AS IS, no need for any special guest
+modifications.
+
+While it complies with the VMware device, it can also communicate with bare
+metal RDMA-enabled machines and does not require an RDMA HCA in the host, it
+can work with Soft-RoCE (rxe).
+
+It does not require the whole guest RAM to be pinned allowing memory
+over-commit and, even if not implemented yet, migration support will be
+possible with some HW assistance.
+
+A project presentation accompany this document:
+- http://events.linuxfoundation.org/sites/events/files/slides/lpc-2017-pvrdma-marcel-apfelbaum-yuval-shaia.pdf
+
+
+
+2. Setup
+========
+
+
+2.1 Guest setup
+===============
+Fedora 27+ kernels work out of the box, older distributions
+require updating the kernel to 4.14 to include the pvrdma driver.
+
+However the libpvrdma library needed by User Level Software is still
+not available as part of the distributions, so the rdma-core library
+needs to be compiled and optionally installed.
+
+Please follow the instructions at:
+  https://github.com/linux-rdma/rdma-core.git
+
+
+2.2 Host Setup
+==============
+The pvrdma backend is an ibdevice interface that can be exposed
+either by a Soft-RoCE(rxe) device on machines with no RDMA device,
+or an HCA SRIOV function(VF/PF).
+Note that ibdevice interfaces can't be shared between pvrdma devices,
+each one requiring a separate instance (rxe or SRIOV VF).
+
+
+2.2.1 Soft-RoCE backend(rxe)
+===========================
+A stable version of rxe is required, Fedora 27+ or a Linux
+Kernel 4.14+ is preferred.
+
+The rdma_rxe module is part of the Linux Kernel but not loaded by default.
+Install the User Level library (librxe) following the instructions from:
+https://github.com/SoftRoCE/rxe-dev/wiki/rxe-dev:-Home
+
+Associate an ETH interface with rxe by running:
+   rxe_cfg add eth0
+An rxe0 ibdevice interface will be created and can be used as pvrdma backend.
+
+
+2.2.2 RDMA device Virtual Function backend
+==========================================
+Nothing special is required, the pvrdma device can work not only with
+Ethernet Links, but also Infinibands Links.
+All is needed is an ibdevice with an active port, for Mellanox cards
+will be something like mlx5_6 which can be the backend.
+
+
+2.2.3 QEMU setup
+================
+Configure QEMU with --enable-rdma flag, installing
+the required RDMA libraries.
+
+
+
+3. Usage
+========
+Currently the device is working only with memory backed RAM
+and it must be mark as "shared":
+   -m 1G \
+   -object memory-backend-ram,id=mb1,size=1G,share \
+   -numa node,memdev=mb1 \
+
+The pvrdma device is composed of two functions:
+ - Function 0 is a vmxnet Ethernet Device which is redundant in Guest
+   but is required to pass the ibdevice GID using its MAC.
+   Examples:
+     For an rxe backend using eth0 interface it will use its mac:
+       -device vmxnet3,addr=<slot>.0,multifunction=on,mac=<eth0 MAC>
+     For an SRIOV VF, we take the Ethernet Interface exposed by it:
+       -device vmxnet3,multifunction=on,mac=<RoCE eth MAC>
+ - Function 1 is the actual device:
+       -device pvrdma,addr=<slot>.1,backend-dev=<ibdevice>,backend-gid-idx=<gid>,backend-port=<port>
+   where the ibdevice can be rxe or RDMA VF (e.g. mlx5_4)
+ Note: Pay special attention that the GID at backend-gid-idx matches vmxnet's MAC.
+ The rules of conversion are part of the RoCE spec, but since manual conversion
+ is not required, spotting problems is not hard:
+    Example: GID: fe80:0000:0000:0000:7efe:90ff:fecb:743a
+             MAC: 7c:fe:90:cb:74:3a
+    Note the difference between the first byte of the MAC and the GID.
+
+
+
+4. Implementation details
+=========================
+
+
+4.1 Overview
+============
+The device acts like a proxy between the Guest Driver and the host
+ibdevice interface.
+On configuration path:
+ - For every hardware resource request (PD/QP/CQ/...) the pvrdma will request
+   a resource from the backend interface, maintaining a 1-1 mapping
+   between the guest and host.
+On data path:
+ - Every post_send/receive received from the guest will be converted into
+   a post_send/receive for the backend. The buffers data will not be touched
+   or copied resulting in near bare-metal performance for large enough buffers.
+ - Completions from the backend interface will result in completions for
+   the pvrdma device.
+
+
+4.2 PCI BARs
+============
+PCI Bars:
+	BAR 0 - MSI-X
+        MSI-X vectors:
+		(0) Command - used when execution of a command is completed.
+		(1) Async - not in use.
+		(2) Completion - used when a completion event is placed in
+		  device's CQ ring.
+	BAR 1 - Registers
+        --------------------------------------------------------
+        | VERSION |  DSR | CTL | REQ | ERR |  ICR | IMR  | MAC |
+        --------------------------------------------------------
+		DSR - Address of driver/device shared memory used
+              for the command channel, used for passing:
+			    - General info such as driver version
+			    - Address of 'command' and 'response'
+			    - Address of async ring
+			    - Address of device's CQ ring
+			    - Device capabilities
+		CTL - Device control operations (activate, reset etc)
+		IMG - Set interrupt mask
+		REQ - Command execution register
+		ERR - Operation status
+
+	BAR 2 - UAR
+        ---------------------------------------------------------
+        | QP_NUM  | SEND/RECV Flag ||  CQ_NUM |   ARM/POLL Flag |
+        ---------------------------------------------------------
+		- Offset 0 used for QP operations (send and recv)
+		- Offset 4 used for CQ operations (arm and poll)
+
+
+4.3 Major flows
+===============
+
+4.3.1 Create CQ
+===============
+    - Guest driver
+        - Allocates pages for CQ ring
+        - Creates page directory (pdir) to hold CQ ring's pages
+        - Initializes CQ ring
+        - Initializes 'Create CQ' command object (cqe, pdir etc)
+        - Copies the command to 'command' address
+        - Writes 0 into REQ register
+    - Device
+        - Reads the request object from the 'command' address
+        - Allocates CQ object and initialize CQ ring based on pdir
+        - Creates the backend CQ
+        - Writes operation status to ERR register
+        - Posts command-interrupt to guest
+    - Guest driver
+        - Reads the HW response code from ERR register
+
+4.3.2 Create QP
+===============
+    - Guest driver
+        - Allocates pages for send and receive rings
+        - Creates page directory(pdir) to hold the ring's pages
+        - Initializes 'Create QP' command object (max_send_wr,
+          send_cq_handle, recv_cq_handle, pdir etc)
+        - Copies the object to 'command' address
+        - Write 0 into REQ register
+    - Device
+        - Reads the request object from 'command' address
+        - Allocates the QP object and initialize
+            - Send and recv rings based on pdir
+            - Send and recv ring state
+        - Creates the backend QP
+        - Writes the operation status to ERR register
+        - Posts command-interrupt to guest
+    - Guest driver
+        - Reads the HW response code from ERR register
+
+4.3.3 Post receive
+==================
+    - Guest driver
+        - Initializes a wqe and place it on recv ring
+        - Write to qpn|qp_recv_bit (31) to QP offset in UAR
+    - Device
+        - Extracts qpn from UAR
+        - Walks through the ring and does the following for each wqe
+            - Prepares the backend CQE context to be used when
+              receiving completion from backend (wr_id, op_code, emu_cq_num)
+            - For each sge prepares backend sge
+            - Calls backend's post_recv
+
+4.3.4 Process backend events
+============================
+    - Done by a dedicated thread used to process backend events;
+      at initialization is attached to the device and creates
+      the communication channel.
+    - Thread main loop:
+        - Polls for completions
+        - Extracts QEMU _cq_num, wr_id and op_code from context
+        - Writes CQE to CQ ring
+        - Writes CQ number to device CQ
+        - Sends completion-interrupt to guest
+        - Deallocates context
+        - Acks the event to backend
+
+
+
+5. Limitations
+==============
+- The device obviously is limited by the Guest Linux Driver features implementation
+  of the VMware device API.
+- Memory registration mechanism requires mremap for every page in the buffer in order
+  to map it to a contiguous virtual address range. Since this is not the data path
+  it should not matter much. If the default max mr size is increased, be aware that
+  memory registration can take up to 0.5 seconds for 1GB of memory.
+- The device requires target page size to be the same as the host page size,
+  otherwise it will fail to init.
+- QEMU cannot map guest RAM from a file descriptor if a pvrdma device is attached,
+  so it can't work with huge pages. The limitation will be addressed in the future,
+  however QEMU allocates Guest RAM with MADV_HUGEPAGE so if there are enough huge
+  pages available, QEMU will use them. QEMU will fail to init if the requirements
+  are not met.
+
+
+
+6. Performance
+==============
+By design the pvrdma device exits on each post-send/receive, so for small buffers
+the performance is affected; however for medium buffers it will became close to
+bare metal and from 1MB buffers and  up it reaches bare metal performance.
+(tested with 2 VMs, the pvrdma devices connected to 2 VFs of the same device)
+
+All the above assumes no memory registration is done on data path.
Signature · 1 line
2.13.5
#3 lore raw

PATCH v11 3/10 scripts/update-linux-headers: import pvrdma headers

In reply to Marcel Apfelbaum

Modify the script to import the headers used by the pvrdma device. Part of them are interfaces between the guest driver and the device, import them under include/standart-headers/drivers/infiniband/... .

Remove the unused functions from pvrdma_verbs.h avoiding the unnecessary import of several infiniband/networking/other headers.

---
 scripts/update-linux-headers.sh | 30 ++++++++++++++++++++++++++++++
 1 file changed, 30 insertions(+)
scripts/update-linux-headers.sh +30 -0
index 135a10d96a..be065704df 100755
@@ -38,6 +38,7 @@ cp_portable() {
                                      -e 'linux/if_ether' \
                                      -e 'input-event-codes' \
                                      -e 'sys/' \
+                                     -e 'pvrdma_verbs' \
                                      > /dev/null
     then
         echo "Unexpected #include in input file $f".
@@ -46,6 +47,7 @@ cp_portable() {
 
     header=$(basename "$f");
     sed -e 's/__u\([0-9][0-9]*\)/uint\1_t/g' \
+        -e 's/u\([0-9][0-9]*\)/uint\1_t/g' \
         -e 's/__s\([0-9][0-9]*\)/int\1_t/g' \
         -e 's/__le\([0-9][0-9]*\)/uint\1_t/g' \
         -e 's/__be\([0-9][0-9]*\)/uint\1_t/g' \
@@ -56,6 +58,7 @@ cp_portable() {
         -e 's/__inline__/inline/' \
         -e '/sys\/ioctl.h/d' \
         -e 's/SW_MAX/SW_MAX_/' \
+        -e 's/atomic_t/int/' \
         "$f" > "$to/$header";
 }
 
@@ -147,6 +150,33 @@ for i in "$tmpdir"/include/linux/*virtio*.h "$tmpdir/include/linux/input.h" \
     cp_portable "$i" "$output/include/standard-headers/linux"
 done
 
+rm -rf "$output/include/standard-headers/drivers/infiniband/hw/vmw_pvrdma"
+mkdir -p "$output/include/standard-headers/drivers/infiniband/hw/vmw_pvrdma"
+
+# Remove the unused functions from pvrdma_verbs.h avoiding the unnecessary
+# import of several infiniband/networking/other headers
+tmp_pvrdma_verbs="$tmpdir/pvrdma_verbs.h"
+# Parse the entire file instead of single lines to match
+# function declarations expanding over multiple lines
+# and strip the declarations starting with pvrdma prefix.
+sed  -e '1h;2,$H;$!d;g'  -e 's/[^};]*pvrdma[^(| ]*([^)]*);//g' \
+    "$linux/drivers/infiniband/hw/vmw_pvrdma/pvrdma_verbs.h" > \
+    "$tmp_pvrdma_verbs";
+
+for i in "$linux/drivers/infiniband/hw/vmw_pvrdma/pvrdma_ring.h" \
+         "$linux/drivers/infiniband/hw/vmw_pvrdma/pvrdma_dev_api.h" \
+         "$tmp_pvrdma_verbs"; do \
+    cp_portable "$i" \
+         "$output/include/standard-headers/drivers/infiniband/hw/vmw_pvrdma/"
+done
+
+rm -rf "$output/include/standard-headers/rdma/"
+mkdir -p "$output/include/standard-headers/rdma/"
+for i in "$tmpdir/include/rdma/vmw_pvrdma-abi.h"; do
+    cp_portable "$i" \
+         "$output/include/standard-headers/rdma/"
+done
+
 cat <<EOF >$output/include/standard-headers/linux/types.h
 /* For QEMU all types are already defined via osdep.h, so this
  * header does not need to do anything.
Signature · 1 line
2.13.5
#4 lore raw

PATCH v11 4/10 include/standard-headers: add pvrdma related headers

In reply to Marcel Apfelbaum

Import the headers used by the pvrdma device.
Part of them are interfaces between the guest driver and the device, imported under include/standart-headers/drivers/infiniband/... .

---
 .../infiniband/hw/vmw_pvrdma/pvrdma_dev_api.h      | 667 +++++++++++++++++++++
 .../drivers/infiniband/hw/vmw_pvrdma/pvrdma_ring.h | 114 ++++
 .../infiniband/hw/vmw_pvrdma/pvrdma_verbs.h        | 383 ++++++++++++
 include/standard-headers/rdma/vmw_pvrdma-abi.h     | 293 +++++++++
 4 files changed, 1457 insertions(+)
 create mode 100644 include/standard-headers/drivers/infiniband/hw/vmw_pvrdma/pvrdma_dev_api.h
 create mode 100644 include/standard-headers/drivers/infiniband/hw/vmw_pvrdma/pvrdma_ring.h
 create mode 100644 include/standard-headers/drivers/infiniband/hw/vmw_pvrdma/pvrdma_verbs.h
 create mode 100644 include/standard-headers/rdma/vmw_pvrdma-abi.h
include/standard-headers/drivers/infiniband/hw/vmw_pvrdma/pvrdma_dev_api.h new file +667 -0
new file mode 100644
index 0000000000..422eb3f4c1
@@ -0,0 +1,667 @@
+/*
+ * Copyright (c) 2012-2016 VMware, Inc.  All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of EITHER the GNU General Public License
+ * version 2 as published by the Free Software Foundation or the BSD
+ * 2-Clause License. This program is distributed in the hope that it
+ * will be useful, but WITHOUT ANY WARRANTY; WITHOUT EVEN THE IMPLIED
+ * WARRANTY OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
+ * See the GNU General Public License version 2 for more details at
+ * http://www.gnu.org/licenses/old-licenses/gpl-2.0.en.html.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program available in the file COPYING in the main
+ * directory of this source tree.
+ *
+ * The BSD 2-Clause License
+ *
+ *     Redistribution and use in source and binary forms, with or
+ *     without modification, are permitted provided that the following
+ *     conditions are met:
+ *
+ *      - Redistributions of source code must retain the above
+ *        copyright notice, this list of conditions and the following
+ *        disclaimer.
+ *
+ *      - Redistributions in binary form must reproduce the above
+ *        copyright notice, this list of conditions and the following
+ *        disclaimer in the documentation and/or other materials
+ *        provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
+ * FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
+ * COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT,
+ * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+ * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+ * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
+ * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED
+ * OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef __PVRDMA_DEV_API_H__
+#define __PVRDMA_DEV_API_H__
+
+#include "standard-headers/linux/types.h"
+
+#include "pvrdma_verbs.h"
+
+/*
+ * PVRDMA version macros. Some new features require updates to PVRDMA_VERSION.
+ * These macros allow us to check for different features if necessary.
+ */
+
+#define PVRDMA_ROCEV1_VERSION		17
+#define PVRDMA_ROCEV2_VERSION		18
+#define PVRDMA_VERSION			PVRDMA_ROCEV2_VERSION
+
+#define PVRDMA_BOARD_ID			1
+#define PVRDMA_REV_ID			1
+
+/*
+ * Masks and accessors for page directory, which is a two-level lookup:
+ * page directory -> page table -> page. Only one directory for now, but we
+ * could expand that easily. 9 bits for tables, 9 bits for pages, gives one
+ * gigabyte for memory regions and so forth.
+ */
+
+#define PVRDMA_PDIR_SHIFT		18
+#define PVRDMA_PTABLE_SHIFT		9
+#define PVRDMA_PAGE_DIR_DIR(x)		(((x) >> PVRDMA_PDIR_SHIFT) & 0x1)
+#define PVRDMA_PAGE_DIR_TABLE(x)	(((x) >> PVRDMA_PTABLE_SHIFT) & 0x1ff)
+#define PVRDMA_PAGE_DIR_PAGE(x)		((x) & 0x1ff)
+#define PVRDMA_PAGE_DIR_MAX_PAGES	(1 * 512 * 512)
+#define PVRDMA_MAX_FAST_REG_PAGES	128
+
+/*
+ * Max MSI-X vectors.
+ */
+
+#define PVRDMA_MAX_INTERRUPTS	3
+
+/* Register offsets within PCI resource on BAR1. */
+#define PVRDMA_REG_VERSION	0x00	/* R: Version of device. */
+#define PVRDMA_REG_DSRLOW	0x04	/* W: Device shared region low PA. */
+#define PVRDMA_REG_DSRHIGH	0x08	/* W: Device shared region high PA. */
+#define PVRDMA_REG_CTL		0x0c	/* W: PVRDMA_DEVICE_CTL */
+#define PVRDMA_REG_REQUEST	0x10	/* W: Indicate device request. */
+#define PVRDMA_REG_ERR		0x14	/* R: Device error. */
+#define PVRDMA_REG_ICR		0x18	/* R: Interrupt cause. */
+#define PVRDMA_REG_IMR		0x1c	/* R/W: Interrupt mask. */
+#define PVRDMA_REG_MACL		0x20	/* R/W: MAC address low. */
+#define PVRDMA_REG_MACH		0x24	/* R/W: MAC address high. */
+
+/* Object flags. */
+#define PVRDMA_CQ_FLAG_ARMED_SOL	BIT(0)	/* Armed for solicited-only. */
+#define PVRDMA_CQ_FLAG_ARMED		BIT(1)	/* Armed. */
+#define PVRDMA_MR_FLAG_DMA		BIT(0)	/* DMA region. */
+#define PVRDMA_MR_FLAG_FRMR		BIT(1)	/* Fast reg memory region. */
+
+/*
+ * Atomic operation capability (masked versions are extended atomic
+ * operations.
+ */
+
+#define PVRDMA_ATOMIC_OP_COMP_SWAP	BIT(0)	/* Compare and swap. */
+#define PVRDMA_ATOMIC_OP_FETCH_ADD	BIT(1)	/* Fetch and add. */
+#define PVRDMA_ATOMIC_OP_MASK_COMP_SWAP	BIT(2)	/* Masked compare and swap. */
+#define PVRDMA_ATOMIC_OP_MASK_FETCH_ADD	BIT(3)	/* Masked fetch and add. */
+
+/*
+ * Base Memory Management Extension flags to support Fast Reg Memory Regions
+ * and Fast Reg Work Requests. Each flag represents a verb operation and we
+ * must support all of them to qualify for the BMME device cap.
+ */
+
+#define PVRDMA_BMME_FLAG_LOCAL_INV	BIT(0)	/* Local Invalidate. */
+#define PVRDMA_BMME_FLAG_REMOTE_INV	BIT(1)	/* Remote Invalidate. */
+#define PVRDMA_BMME_FLAG_FAST_REG_WR	BIT(2)	/* Fast Reg Work Request. */
+
+/*
+ * GID types. The interpretation of the gid_types bit field in the device
+ * capabilities will depend on the device mode. For now, the device only
+ * supports RoCE as mode, so only the different GID types for RoCE are
+ * defined.
+ */
+
+#define PVRDMA_GID_TYPE_FLAG_ROCE_V1	BIT(0)
+#define PVRDMA_GID_TYPE_FLAG_ROCE_V2	BIT(1)
+
+/*
+ * Version checks. This checks whether each version supports specific
+ * capabilities from the device.
+ */
+
+#define PVRDMA_IS_VERSION17(_dev)					\
+	(_dev->dsr_version == PVRDMA_ROCEV1_VERSION &&			\
+	 _dev->dsr->caps.gid_types == PVRDMA_GID_TYPE_FLAG_ROCE_V1)
+
+#define PVRDMA_IS_VERSION18(_dev)					\
+	(_dev->dsr_version >= PVRDMA_ROCEV2_VERSION &&			\
+	 (_dev->dsr->caps.gid_types == PVRDMA_GID_TYPE_FLAG_ROCE_V1 ||  \
+	  _dev->dsr->caps.gid_types == PVRDMA_GID_TYPE_FLAG_ROCE_V2))	\
+
+#define PVRDMA_SUPPORTED(_dev)						\
+	((_dev->dsr->caps.mode == PVRDMA_DEVICE_MODE_ROCE) &&		\
+	 (PVRDMA_IS_VERSION17(_dev) || PVRDMA_IS_VERSION18(_dev)))
+
+/*
+ * Get capability values based on device version.
+ */
+
+#define PVRDMA_GET_CAP(_dev, _old_val, _val) \
+	((PVRDMA_IS_VERSION18(_dev)) ? _val : _old_val)
+
+enum pvrdma_pci_resource {
+	PVRDMA_PCI_RESOURCE_MSIX,	/* BAR0: MSI-X, MMIO. */
+	PVRDMA_PCI_RESOURCE_REG,	/* BAR1: Registers, MMIO. */
+	PVRDMA_PCI_RESOURCE_UAR,	/* BAR2: UAR pages, MMIO, 64-bit. */
+	PVRDMA_PCI_RESOURCE_LAST,	/* Last. */
+};
+
+enum pvrdma_device_ctl {
+	PVRDMA_DEVICE_CTL_ACTIVATE,	/* Activate device. */
+	PVRDMA_DEVICE_CTL_UNQUIESCE,	/* Unquiesce device. */
+	PVRDMA_DEVICE_CTL_RESET,	/* Reset device. */
+};
+
+enum pvrdma_intr_vector {
+	PVRDMA_INTR_VECTOR_RESPONSE,	/* Command response. */
+	PVRDMA_INTR_VECTOR_ASYNC,	/* Async events. */
+	PVRDMA_INTR_VECTOR_CQ,		/* CQ notification. */
+	/* Additional CQ notification vectors. */
+};
+
+enum pvrdma_intr_cause {
+	PVRDMA_INTR_CAUSE_RESPONSE	= (1 << PVRDMA_INTR_VECTOR_RESPONSE),
+	PVRDMA_INTR_CAUSE_ASYNC		= (1 << PVRDMA_INTR_VECTOR_ASYNC),
+	PVRDMA_INTR_CAUSE_CQ		= (1 << PVRDMA_INTR_VECTOR_CQ),
+};
+
+enum pvrdma_gos_bits {
+	PVRDMA_GOS_BITS_UNK,		/* Unknown. */
+	PVRDMA_GOS_BITS_32,		/* 32-bit. */
+	PVRDMA_GOS_BITS_64,		/* 64-bit. */
+};
+
+enum pvrdma_gos_type {
+	PVRDMA_GOS_TYPE_UNK,		/* Unknown. */
+	PVRDMA_GOS_TYPE_LINUX,		/* Linux. */
+};
+
+enum pvrdma_device_mode {
+	PVRDMA_DEVICE_MODE_ROCE,	/* RoCE. */
+	PVRDMA_DEVICE_MODE_IWARP,	/* iWarp. */
+	PVRDMA_DEVICE_MODE_IB,		/* InfiniBand. */
+};
+
+struct pvrdma_gos_info {
+	uint32_t gos_bits:2;			/* W: PVRDMA_GOS_BITS_ */
+	uint32_t gos_type:4;			/* W: PVRDMA_GOS_TYPE_ */
+	uint32_t gos_ver:16;			/* W: Guest OS version. */
+	uint32_t gos_misc:10;		/* W: Other. */
+	uint32_t pad;			/* Pad to 8-byte alignment. */
+};
+
+struct pvrdma_device_caps {
+	uint64_t fw_ver;				/* R: Query device. */
+	uint64_t node_guid;
+	uint64_t sys_image_guid;
+	uint64_t max_mr_size;
+	uint64_t page_size_cap;
+	uint64_t atomic_arg_sizes;			/* EX verbs. */
+	uint32_t ex_comp_mask;			/* EX verbs. */
+	uint32_t device_cap_flags2;			/* EX verbs. */
+	uint32_t max_fa_bit_boundary;		/* EX verbs. */
+	uint32_t log_max_atomic_inline_arg;		/* EX verbs. */
+	uint32_t vendor_id;
+	uint32_t vendor_part_id;
+	uint32_t hw_ver;
+	uint32_t max_qp;
+	uint32_t max_qp_wr;
+	uint32_t device_cap_flags;
+	uint32_t max_sge;
+	uint32_t max_sge_rd;
+	uint32_t max_cq;
+	uint32_t max_cqe;
+	uint32_t max_mr;
+	uint32_t max_pd;
+	uint32_t max_qp_rd_atom;
+	uint32_t max_ee_rd_atom;
+	uint32_t max_res_rd_atom;
+	uint32_t max_qp_init_rd_atom;
+	uint32_t max_ee_init_rd_atom;
+	uint32_t max_ee;
+	uint32_t max_rdd;
+	uint32_t max_mw;
+	uint32_t max_raw_ipv6_qp;
+	uint32_t max_raw_ethy_qp;
+	uint32_t max_mcast_grp;
+	uint32_t max_mcast_qp_attach;
+	uint32_t max_total_mcast_qp_attach;
+	uint32_t max_ah;
+	uint32_t max_fmr;
+	uint32_t max_map_per_fmr;
+	uint32_t max_srq;
+	uint32_t max_srq_wr;
+	uint32_t max_srq_sge;
+	uint32_t max_uar;
+	uint32_t gid_tbl_len;
+	uint16_t max_pkeys;
+	uint8_t  local_ca_ack_delay;
+	uint8_t  phys_port_cnt;
+	uint8_t  mode;				/* PVRDMA_DEVICE_MODE_ */
+	uint8_t  atomic_ops;				/* PVRDMA_ATOMIC_OP_* bits */
+	uint8_t  bmme_flags;				/* FRWR Mem Mgmt Extensions */
+	uint8_t  gid_types;				/* PVRDMA_GID_TYPE_FLAG_ */
+	uint32_t max_fast_reg_page_list_len;
+};
+
+struct pvrdma_ring_page_info {
+	uint32_t num_pages;				/* Num pages incl. header. */
+	uint32_t reserved;				/* Reserved. */
+	uint64_t pdir_dma;				/* Page directory PA. */
+};
+
+#pragma pack(push, 1)
+
+struct pvrdma_device_shared_region {
+	uint32_t driver_version;			/* W: Driver version. */
+	uint32_t pad;				/* Pad to 8-byte align. */
+	struct pvrdma_gos_info gos_info;	/* W: Guest OS information. */
+	uint64_t cmd_slot_dma;			/* W: Command slot address. */
+	uint64_t resp_slot_dma;			/* W: Response slot address. */
+	struct pvrdma_ring_page_info async_ring_pages;
+						/* W: Async ring page info. */
+	struct pvrdma_ring_page_info cq_ring_pages;
+						/* W: CQ ring page info. */
+	uint32_t uar_pfn;				/* W: UAR pageframe. */
+	uint32_t pad2;				/* Pad to 8-byte align. */
+	struct pvrdma_device_caps caps;		/* R: Device capabilities. */
+};
+
+#pragma pack(pop)
+
+/* Event types. Currently a 1:1 mapping with enum ib_event. */
+enum pvrdma_eqe_type {
+	PVRDMA_EVENT_CQ_ERR,
+	PVRDMA_EVENT_QP_FATAL,
+	PVRDMA_EVENT_QP_REQ_ERR,
+	PVRDMA_EVENT_QP_ACCESS_ERR,
+	PVRDMA_EVENT_COMM_EST,
+	PVRDMA_EVENT_SQ_DRAINED,
+	PVRDMA_EVENT_PATH_MIG,
+	PVRDMA_EVENT_PATH_MIG_ERR,
+	PVRDMA_EVENT_DEVICE_FATAL,
+	PVRDMA_EVENT_PORT_ACTIVE,
+	PVRDMA_EVENT_PORT_ERR,
+	PVRDMA_EVENT_LID_CHANGE,
+	PVRDMA_EVENT_PKEY_CHANGE,
+	PVRDMA_EVENT_SM_CHANGE,
+	PVRDMA_EVENT_SRQ_ERR,
+	PVRDMA_EVENT_SRQ_LIMIT_REACHED,
+	PVRDMA_EVENT_QP_LAST_WQE_REACHED,
+	PVRDMA_EVENT_CLIENT_REREGISTER,
+	PVRDMA_EVENT_GID_CHANGE,
+};
+
+/* Event queue element. */
+struct pvrdma_eqe {
+	uint32_t type;	/* Event type. */
+	uint32_t info;	/* Handle, other. */
+};
+
+/* CQ notification queue element. */
+struct pvrdma_cqne {
+	uint32_t info;	/* Handle */
+};
+
+enum {
+	PVRDMA_CMD_FIRST,
+	PVRDMA_CMD_QUERY_PORT = PVRDMA_CMD_FIRST,
+	PVRDMA_CMD_QUERY_PKEY,
+	PVRDMA_CMD_CREATE_PD,
+	PVRDMA_CMD_DESTROY_PD,
+	PVRDMA_CMD_CREATE_MR,
+	PVRDMA_CMD_DESTROY_MR,
+	PVRDMA_CMD_CREATE_CQ,
+	PVRDMA_CMD_RESIZE_CQ,
+	PVRDMA_CMD_DESTROY_CQ,
+	PVRDMA_CMD_CREATE_QP,
+	PVRDMA_CMD_MODIFY_QP,
+	PVRDMA_CMD_QUERY_QP,
+	PVRDMA_CMD_DESTROY_QP,
+	PVRDMA_CMD_CREATE_UC,
+	PVRDMA_CMD_DESTROY_UC,
+	PVRDMA_CMD_CREATE_BIND,
+	PVRDMA_CMD_DESTROY_BIND,
+	PVRDMA_CMD_CREATE_SRQ,
+	PVRDMA_CMD_MODIFY_SRQ,
+	PVRDMA_CMD_QUERY_SRQ,
+	PVRDMA_CMD_DESTROY_SRQ,
+	PVRDMA_CMD_MAX,
+};
+
+enum {
+	PVRDMA_CMD_FIRST_RESP = (1 << 31),
+	PVRDMA_CMD_QUERY_PORT_RESP = PVRDMA_CMD_FIRST_RESP,
+	PVRDMA_CMD_QUERY_PKEY_RESP,
+	PVRDMA_CMD_CREATE_PD_RESP,
+	PVRDMA_CMD_DESTROY_PD_RESP_NOOP,
+	PVRDMA_CMD_CREATE_MR_RESP,
+	PVRDMA_CMD_DESTROY_MR_RESP_NOOP,
+	PVRDMA_CMD_CREATE_CQ_RESP,
+	PVRDMA_CMD_RESIZE_CQ_RESP,
+	PVRDMA_CMD_DESTROY_CQ_RESP_NOOP,
+	PVRDMA_CMD_CREATE_QP_RESP,
+	PVRDMA_CMD_MODIFY_QP_RESP,
+	PVRDMA_CMD_QUERY_QP_RESP,
+	PVRDMA_CMD_DESTROY_QP_RESP,
+	PVRDMA_CMD_CREATE_UC_RESP,
+	PVRDMA_CMD_DESTROY_UC_RESP_NOOP,
+	PVRDMA_CMD_CREATE_BIND_RESP_NOOP,
+	PVRDMA_CMD_DESTROY_BIND_RESP_NOOP,
+	PVRDMA_CMD_CREATE_SRQ_RESP,
+	PVRDMA_CMD_MODIFY_SRQ_RESP,
+	PVRDMA_CMD_QUERY_SRQ_RESP,
+	PVRDMA_CMD_DESTROY_SRQ_RESP,
+	PVRDMA_CMD_MAX_RESP,
+};
+
+struct pvrdma_cmd_hdr {
+	uint64_t response;		/* Key for response lookup. */
+	uint32_t cmd;		/* PVRDMA_CMD_ */
+	uint32_t reserved;		/* Reserved. */
+};
+
+struct pvrdma_cmd_resp_hdr {
+	uint64_t response;		/* From cmd hdr. */
+	uint32_t ack;		/* PVRDMA_CMD_XXX_RESP */
+	uint8_t err;			/* Error. */
+	uint8_t reserved[3];		/* Reserved. */
+};
+
+struct pvrdma_cmd_query_port {
+	struct pvrdma_cmd_hdr hdr;
+	uint8_t port_num;
+	uint8_t reserved[7];
+};
+
+struct pvrdma_cmd_query_port_resp {
+	struct pvrdma_cmd_resp_hdr hdr;
+	struct pvrdma_port_attr attrs;
+};
+
+struct pvrdma_cmd_query_pkey {
+	struct pvrdma_cmd_hdr hdr;
+	uint8_t port_num;
+	uint8_t index;
+	uint8_t reserved[6];
+};
+
+struct pvrdma_cmd_query_pkey_resp {
+	struct pvrdma_cmd_resp_hdr hdr;
+	uint16_t pkey;
+	uint8_t reserved[6];
+};
+
+struct pvrdma_cmd_create_uc {
+	struct pvrdma_cmd_hdr hdr;
+	uint32_t pfn; /* UAR page frame number */
+	uint8_t reserved[4];
+};
+
+struct pvrdma_cmd_create_uc_resp {
+	struct pvrdma_cmd_resp_hdr hdr;
+	uint32_t ctx_handle;
+	uint8_t reserved[4];
+};
+
+struct pvrdma_cmd_destroy_uc {
+	struct pvrdma_cmd_hdr hdr;
+	uint32_t ctx_handle;
+	uint8_t reserved[4];
+};
+
+struct pvrdma_cmd_create_pd {
+	struct pvrdma_cmd_hdr hdr;
+	uint32_t ctx_handle;
+	uint8_t reserved[4];
+};
+
+struct pvrdma_cmd_create_pd_resp {
+	struct pvrdma_cmd_resp_hdr hdr;
+	uint32_t pd_handle;
+	uint8_t reserved[4];
+};
+
+struct pvrdma_cmd_destroy_pd {
+	struct pvrdma_cmd_hdr hdr;
+	uint32_t pd_handle;
+	uint8_t reserved[4];
+};
+
+struct pvrdma_cmd_create_mr {
+	struct pvrdma_cmd_hdr hdr;
+	uint64_t start;
+	uint64_t length;
+	uint64_t pdir_dma;
+	uint32_t pd_handle;
+	uint32_t access_flags;
+	uint32_t flags;
+	uint32_t nchunks;
+};
+
+struct pvrdma_cmd_create_mr_resp {
+	struct pvrdma_cmd_resp_hdr hdr;
+	uint32_t mr_handle;
+	uint32_t lkey;
+	uint32_t rkey;
+	uint8_t reserved[4];
+};
+
+struct pvrdma_cmd_destroy_mr {
+	struct pvrdma_cmd_hdr hdr;
+	uint32_t mr_handle;
+	uint8_t reserved[4];
+};
+
+struct pvrdma_cmd_create_cq {
+	struct pvrdma_cmd_hdr hdr;
+	uint64_t pdir_dma;
+	uint32_t ctx_handle;
+	uint32_t cqe;
+	uint32_t nchunks;
+	uint8_t reserved[4];
+};
+
+struct pvrdma_cmd_create_cq_resp {
+	struct pvrdma_cmd_resp_hdr hdr;
+	uint32_t cq_handle;
+	uint32_t cqe;
+};
+
+struct pvrdma_cmd_resize_cq {
+	struct pvrdma_cmd_hdr hdr;
+	uint32_t cq_handle;
+	uint32_t cqe;
+};
+
+struct pvrdma_cmd_resize_cq_resp {
+	struct pvrdma_cmd_resp_hdr hdr;
+	uint32_t cqe;
+	uint8_t reserved[4];
+};
+
+struct pvrdma_cmd_destroy_cq {
+	struct pvrdma_cmd_hdr hdr;
+	uint32_t cq_handle;
+	uint8_t reserved[4];
+};
+
+struct pvrdma_cmd_create_srq {
+	struct pvrdma_cmd_hdr hdr;
+	uint64_t pdir_dma;
+	uint32_t pd_handle;
+	uint32_t nchunks;
+	struct pvrdma_srq_attr attrs;
+	uint8_t srq_type;
+	uint8_t reserved[7];
+};
+
+struct pvrdma_cmd_create_srq_resp {
+	struct pvrdma_cmd_resp_hdr hdr;
+	uint32_t srqn;
+	uint8_t reserved[4];
+};
+
+struct pvrdma_cmd_modify_srq {
+	struct pvrdma_cmd_hdr hdr;
+	uint32_t srq_handle;
+	uint32_t attr_mask;
+	struct pvrdma_srq_attr attrs;
+};
+
+struct pvrdma_cmd_query_srq {
+	struct pvrdma_cmd_hdr hdr;
+	uint32_t srq_handle;
+	uint8_t reserved[4];
+};
+
+struct pvrdma_cmd_query_srq_resp {
+	struct pvrdma_cmd_resp_hdr hdr;
+	struct pvrdma_srq_attr attrs;
+};
+
+struct pvrdma_cmd_destroy_srq {
+	struct pvrdma_cmd_hdr hdr;
+	uint32_t srq_handle;
+	uint8_t reserved[4];
+};
+
+struct pvrdma_cmd_create_qp {
+	struct pvrdma_cmd_hdr hdr;
+	uint64_t pdir_dma;
+	uint32_t pd_handle;
+	uint32_t send_cq_handle;
+	uint32_t recv_cq_handle;
+	uint32_t srq_handle;
+	uint32_t max_send_wr;
+	uint32_t max_recv_wr;
+	uint32_t max_send_sge;
+	uint32_t max_recv_sge;
+	uint32_t max_inline_data;
+	uint32_t lkey;
+	uint32_t access_flags;
+	uint16_t total_chunks;
+	uint16_t send_chunks;
+	uint16_t max_atomic_arg;
+	uint8_t sq_sig_all;
+	uint8_t qp_type;
+	uint8_t is_srq;
+	uint8_t reserved[3];
+};
+
+struct pvrdma_cmd_create_qp_resp {
+	struct pvrdma_cmd_resp_hdr hdr;
+	uint32_t qpn;
+	uint32_t max_send_wr;
+	uint32_t max_recv_wr;
+	uint32_t max_send_sge;
+	uint32_t max_recv_sge;
+	uint32_t max_inline_data;
+};
+
+struct pvrdma_cmd_modify_qp {
+	struct pvrdma_cmd_hdr hdr;
+	uint32_t qp_handle;
+	uint32_t attr_mask;
+	struct pvrdma_qp_attr attrs;
+};
+
+struct pvrdma_cmd_query_qp {
+	struct pvrdma_cmd_hdr hdr;
+	uint32_t qp_handle;
+	uint32_t attr_mask;
+};
+
+struct pvrdma_cmd_query_qp_resp {
+	struct pvrdma_cmd_resp_hdr hdr;
+	struct pvrdma_qp_attr attrs;
+};
+
+struct pvrdma_cmd_destroy_qp {
+	struct pvrdma_cmd_hdr hdr;
+	uint32_t qp_handle;
+	uint8_t reserved[4];
+};
+
+struct pvrdma_cmd_destroy_qp_resp {
+	struct pvrdma_cmd_resp_hdr hdr;
+	uint32_t events_reported;
+	uint8_t reserved[4];
+};
+
+struct pvrdma_cmd_create_bind {
+	struct pvrdma_cmd_hdr hdr;
+	uint32_t mtu;
+	uint32_t vlan;
+	uint32_t index;
+	uint8_t new_gid[16];
+	uint8_t gid_type;
+	uint8_t reserved[3];
+};
+
+struct pvrdma_cmd_destroy_bind {
+	struct pvrdma_cmd_hdr hdr;
+	uint32_t index;
+	uint8_t dest_gid[16];
+	uint8_t reserved[4];
+};
+
+union pvrdma_cmd_req {
+	struct pvrdma_cmd_hdr hdr;
+	struct pvrdma_cmd_query_port query_port;
+	struct pvrdma_cmd_query_pkey query_pkey;
+	struct pvrdma_cmd_create_uc create_uc;
+	struct pvrdma_cmd_destroy_uc destroy_uc;
+	struct pvrdma_cmd_create_pd create_pd;
+	struct pvrdma_cmd_destroy_pd destroy_pd;
+	struct pvrdma_cmd_create_mr create_mr;
+	struct pvrdma_cmd_destroy_mr destroy_mr;
+	struct pvrdma_cmd_create_cq create_cq;
+	struct pvrdma_cmd_resize_cq resize_cq;
+	struct pvrdma_cmd_destroy_cq destroy_cq;
+	struct pvrdma_cmd_create_qp create_qp;
+	struct pvrdma_cmd_modify_qp modify_qp;
+	struct pvrdma_cmd_query_qp query_qp;
+	struct pvrdma_cmd_destroy_qp destroy_qp;
+	struct pvrdma_cmd_create_bind create_bind;
+	struct pvrdma_cmd_destroy_bind destroy_bind;
+	struct pvrdma_cmd_create_srq create_srq;
+	struct pvrdma_cmd_modify_srq modify_srq;
+	struct pvrdma_cmd_query_srq query_srq;
+	struct pvrdma_cmd_destroy_srq destroy_srq;
+};
+
+union pvrdma_cmd_resp {
+	struct pvrdma_cmd_resp_hdr hdr;
+	struct pvrdma_cmd_query_port_resp query_port_resp;
+	struct pvrdma_cmd_query_pkey_resp query_pkey_resp;
+	struct pvrdma_cmd_create_uc_resp create_uc_resp;
+	struct pvrdma_cmd_create_pd_resp create_pd_resp;
+	struct pvrdma_cmd_create_mr_resp create_mr_resp;
+	struct pvrdma_cmd_create_cq_resp create_cq_resp;
+	struct pvrdma_cmd_resize_cq_resp resize_cq_resp;
+	struct pvrdma_cmd_create_qp_resp create_qp_resp;
+	struct pvrdma_cmd_query_qp_resp query_qp_resp;
+	struct pvrdma_cmd_destroy_qp_resp destroy_qp_resp;
+	struct pvrdma_cmd_create_srq_resp create_srq_resp;
+	struct pvrdma_cmd_query_srq_resp query_srq_resp;
+};
+
+#endif /* __PVRDMA_DEV_API_H__ */
include/standard-headers/drivers/infiniband/hw/vmw_pvrdma/pvrdma_ring.h new file +114 -0

114 lines not on this page · raw message

include/standard-headers/drivers/infiniband/hw/vmw_pvrdma/pvrdma_verbs.h new file +383 -0

383 lines not on this page · raw message

include/standard-headers/rdma/vmw_pvrdma-abi.h new file +293 -0

293 lines not on this page · raw message

Signature · 1 line
2.13.5
#5 lore raw

PATCH v11 5/10 hw/rdma: Add wrappers and macros

In reply to Marcel Apfelbaum

From: Yuval Shaia <[email protected]>

As all mapping for this device are from driver to device, declare wrappers on top of pci_dma_*map functions.

In addition, declare macros to be used for debug messages.

---
 hw/Makefile.objs      |  1 +
 hw/rdma/Makefile.objs |  3 +++
 hw/rdma/rdma_utils.c  | 51 +++++++++++++++++++++++++++++++++++++++++++++++++++
 hw/rdma/rdma_utils.h  | 43 +++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 98 insertions(+)
 create mode 100644 hw/rdma/Makefile.objs
 create mode 100644 hw/rdma/rdma_utils.c
 create mode 100644 hw/rdma/rdma_utils.h
hw/Makefile.objs +1 -0
index cf4cb2010b..6a0ffe0afd 100644
@@ -18,6 +18,7 @@ devices-dirs-$(CONFIG_IPMI) += ipmi/
 devices-dirs-$(CONFIG_SOFTMMU) += isa/
 devices-dirs-$(CONFIG_SOFTMMU) += misc/
 devices-dirs-$(CONFIG_SOFTMMU) += net/
+devices-dirs-$(CONFIG_SOFTMMU) += rdma/
 devices-dirs-$(CONFIG_SOFTMMU) += nvram/
 devices-dirs-$(CONFIG_SOFTMMU) += pci/
 devices-dirs-$(CONFIG_PCI) += pci-bridge/ pci-host/
hw/rdma/Makefile.objs new file +3 -0
new file mode 100644
index 0000000000..cdffe4a9a3
@@ -0,0 +1,3 @@
+ifeq ($(CONFIG_RDMA),y)
+obj-$(CONFIG_PCI) += rdma_utils.o
+endif
hw/rdma/rdma_utils.c new file +51 -0

51 lines not on this page · raw message

hw/rdma/rdma_utils.h new file +43 -0

43 lines not on this page · raw message

Signature · 1 line
2.13.5
#6 lore raw

PATCH v11 6/10 hw/rdma: Definitions for rdma device and rdma resource manager

In reply to Marcel Apfelbaum

From: Yuval Shaia <[email protected]>

Definition of various structures and constants used in backend and resource manager modules.

---
 hw/rdma/rdma_backend_defs.h |  62 ++++++++++++++++++++++++++
 hw/rdma/rdma_rm_defs.h      | 104 ++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 166 insertions(+)
 create mode 100644 hw/rdma/rdma_backend_defs.h
 create mode 100644 hw/rdma/rdma_rm_defs.h
hw/rdma/rdma_backend_defs.h new file +62 -0
new file mode 100644
index 0000000000..837e32419c
@@ -0,0 +1,62 @@
+/*
+ *  RDMA device: Definitions of Backend Device structures
+ *
+ * Copyright (C) 2018 Oracle
+ * Copyright (C) 2018 Red Hat Inc
+ *
+ * Authors:
+ *     Yuval Shaia <[email protected]>
+ *     Marcel Apfelbaum <[email protected]>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#ifndef RDMA_BACKEND_DEFS_H
+#define RDMA_BACKEND_DEFS_H
+
+#include <infiniband/verbs.h>
+#include <qemu/thread.h>
+
+typedef struct RdmaDeviceResources RdmaDeviceResources;
+
+typedef struct RdmaBackendThread {
+    QemuThread thread;
+    QemuMutex mutex;
+    bool run;
+} RdmaBackendThread;
+
+typedef struct RdmaBackendDev {
+    struct ibv_device_attr dev_attr;
+    RdmaBackendThread comp_thread;
+    union ibv_gid gid;
+    PCIDevice *dev;
+    RdmaDeviceResources *rdma_dev_res;
+    struct ibv_device *ib_dev;
+    struct ibv_context *context;
+    struct ibv_comp_channel *channel;
+    uint8_t port_num;
+    uint8_t backend_gid_idx;
+} RdmaBackendDev;
+
+typedef struct RdmaBackendPD {
+    struct ibv_pd *ibpd;
+} RdmaBackendPD;
+
+typedef struct RdmaBackendMR {
+    struct ibv_pd *ibpd;
+    struct ibv_mr *ibmr;
+} RdmaBackendMR;
+
+typedef struct RdmaBackendCQ {
+    RdmaBackendDev *backend_dev;
+    struct ibv_cq *ibcq;
+} RdmaBackendCQ;
+
+typedef struct RdmaBackendQP {
+    struct ibv_pd *ibpd;
+    struct ibv_qp *ibqp;
+} RdmaBackendQP;
+
+#endif
hw/rdma/rdma_rm_defs.h new file +104 -0

104 lines not on this page · raw message

Signature · 1 line
2.13.5
#7 lore raw

PATCH v11 7/10 hw/rdma: Implementation of generic rdma device layers

In reply to Marcel Apfelbaum

From: Yuval Shaia <[email protected]>

This layer is composed of two sub-modules, backend and resource manager. Backend sub-module is responsible for all the interaction with IB layers such as ibverbs and umad (external libraries).
Resource manager is a collection of functions and structures to manage RDMA resources such as QPs, CQs and MRs.

---
 Makefile.objs          |   1 +
 configure              |   9 +-
 hw/rdma/Makefile.objs  |   2 +-
 hw/rdma/rdma_backend.c | 818 +++++++++++++++++++++++++++++++++++++++++++++++++
 hw/rdma/rdma_backend.h |  98 ++++++
 hw/rdma/rdma_rm.c      | 544 ++++++++++++++++++++++++++++++++
 hw/rdma/rdma_rm.h      |  69 +++++
 hw/rdma/trace-events   |   5 +
 8 files changed, 1541 insertions(+), 5 deletions(-)
 create mode 100644 hw/rdma/rdma_backend.c
 create mode 100644 hw/rdma/rdma_backend.h
 create mode 100644 hw/rdma/rdma_rm.c
 create mode 100644 hw/rdma/rdma_rm.h
 create mode 100644 hw/rdma/trace-events
Makefile.objs +1 -0
index 2efba6d768..f3a3d28304 100644
@@ -130,6 +130,7 @@ trace-events-subdirs += hw/block/dataplane
 trace-events-subdirs += hw/char
 trace-events-subdirs += hw/intc
 trace-events-subdirs += hw/net
+trace-events-subdirs += hw/rdma
 trace-events-subdirs += hw/virtio
 trace-events-subdirs += hw/audio
 trace-events-subdirs += hw/misc
configure +5 -4
index 913e14839d..ed45a3c4dd 100755
@@ -1572,7 +1572,7 @@ disabled with --disable-FEATURE, default is enabled if available:
   hax             HAX acceleration support
   hvf             Hypervisor.framework acceleration support
   whpx            Windows Hypervisor Platform acceleration support
-  rdma            RDMA-based migration support
+  rdma            Enable RDMA-based migration and PVRDMA support
   vde             support for vde network
   netmap          support for netmap network
   linux-aio       Linux AIO support
@@ -2923,15 +2923,16 @@ if test "$rdma" != "no" ; then
 #include <rdma/rdma_cma.h>
 int main(void) { return 0; }
 EOF
-  rdma_libs="-lrdmacm -libverbs"
+  rdma_libs="-lrdmacm -libverbs -libumad"
   if compile_prog "" "$rdma_libs" ; then
     rdma="yes"
+    libs_softmmu="$libs_softmmu $rdma_libs"
   else
     if test "$rdma" = "yes" ; then
         error_exit \
-            " OpenFabrics librdmacm/libibverbs not present." \
+            " OpenFabrics librdmacm/libibverbs/libibumad not present." \
             " Your options:" \
-            "  (1) Fast: Install infiniband packages from your distro." \
+            "  (1) Fast: Install infiniband packages (devel) from your distro." \
             "  (2) Cleanest: Install libraries from www.openfabrics.org" \
             "  (3) Also: Install softiwarp if you don't have RDMA hardware"
     fi
hw/rdma/Makefile.objs +1 -1
index cdffe4a9a3..6a59bf0d5b 100644
@@ -1,3 +1,3 @@
 ifeq ($(CONFIG_RDMA),y)
-obj-$(CONFIG_PCI) += rdma_utils.o
+obj-$(CONFIG_PCI) += rdma_utils.o rdma_backend.o rdma_rm.o
 endif
hw/rdma/rdma_backend.c new file +818 -0

818 lines not on this page · raw message

hw/rdma/rdma_backend.h new file +98 -0

98 lines not on this page · raw message

hw/rdma/rdma_rm.c new file +544 -0

544 lines not on this page · raw message

hw/rdma/rdma_rm.h new file +69 -0

69 lines not on this page · raw message

hw/rdma/trace-events new file +5 -0
new file mode 100644
index 0000000000..c4c202e647
@@ -0,0 +1,5 @@
+# See docs/tracing.txt for syntax documentation.
+
+#hw/rdma/rdma_backend.c
+create_ah_cache_hit(uint64_t subnet, uint64_t net_id) "subnet = 0x%"PRIx64" net_id = 0x%"PRIx64
+create_ah_cache_miss(uint64_t subnet, uint64_t net_id) "subnet = 0x%"PRIx64" net_id = 0x%"PRIx64
Signature · 1 line
2.13.5
#8 lore raw

PATCH v11 8/10 hw/rdma: PVRDMA commands and data-path ops

In reply to Marcel Apfelbaum

From: Yuval Shaia <[email protected]>

First PVRDMA sub-module - implementation of the PVRDMA device.
- PVRDMA commands such as create CQ and create MR.
- Data path QP operations - post_send and post_recv.
- Completion handler.

---
 hw/rdma/Makefile.objs         |   2 +
 hw/rdma/vmw/pvrdma.h          | 122 ++++++++
 hw/rdma/vmw/pvrdma_cmd.c      | 673 ++++++++++++++++++++++++++++++++++++++++++
 hw/rdma/vmw/pvrdma_dev_ring.c | 155 ++++++++++
 hw/rdma/vmw/pvrdma_dev_ring.h |  42 +++
 hw/rdma/vmw/pvrdma_qp_ops.c   | 222 ++++++++++++++
 hw/rdma/vmw/pvrdma_qp_ops.h   |  27 ++
 7 files changed, 1243 insertions(+)
 create mode 100644 hw/rdma/vmw/pvrdma.h
 create mode 100644 hw/rdma/vmw/pvrdma_cmd.c
 create mode 100644 hw/rdma/vmw/pvrdma_dev_ring.c
 create mode 100644 hw/rdma/vmw/pvrdma_dev_ring.h
 create mode 100644 hw/rdma/vmw/pvrdma_qp_ops.c
 create mode 100644 hw/rdma/vmw/pvrdma_qp_ops.h
hw/rdma/Makefile.objs +2 -0
index 6a59bf0d5b..44a85f687d 100644
@@ -1,3 +1,5 @@
 ifeq ($(CONFIG_RDMA),y)
 obj-$(CONFIG_PCI) += rdma_utils.o rdma_backend.o rdma_rm.o
+obj-$(CONFIG_PCI) += vmw/pvrdma_dev_ring.o vmw/pvrdma_cmd.o \
+                     vmw/pvrdma_qp_ops.o
 endif
hw/rdma/vmw/pvrdma.h new file +122 -0

122 lines not on this page · raw message

hw/rdma/vmw/pvrdma_cmd.c new file +673 -0

673 lines not on this page · raw message

hw/rdma/vmw/pvrdma_dev_ring.c new file +155 -0

155 lines not on this page · raw message

hw/rdma/vmw/pvrdma_dev_ring.h new file +42 -0

42 lines not on this page · raw message

hw/rdma/vmw/pvrdma_qp_ops.c new file +222 -0

222 lines not on this page · raw message

hw/rdma/vmw/pvrdma_qp_ops.h new file +27 -0
new file mode 100644
index 0000000000..ac46bf7fdf
@@ -0,0 +1,27 @@
+/*
+ * QEMU VMWARE paravirtual RDMA QP Operations
+ *
+ * Copyright (C) 2018 Oracle
+ * Copyright (C) 2018 Red Hat Inc
+ *
+ * Authors:
+ *     Yuval Shaia <[email protected]>
+ *     Marcel Apfelbaum <[email protected]>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#ifndef PVRDMA_QP_H
+#define PVRDMA_QP_H
+
+#include "pvrdma.h"
+
+int pvrdma_qp_ops_init(void);
+void pvrdma_qp_ops_fini(void);
+int pvrdma_qp_send(PVRDMADev *dev, uint32_t qp_handle);
+int pvrdma_qp_recv(PVRDMADev *dev, uint32_t qp_handle);
+void pvrdma_cq_poll(RdmaDeviceResources *dev_res, uint32_t cq_handle);
+
+#endif
Signature · 1 line
2.13.5
#9 lore raw

PATCH v11 9/10 hw/rdma: Implementation of PVRDMA device

In reply to Marcel Apfelbaum

From: Yuval Shaia <[email protected]>

PVRDMA is the QEMU implementation of VMware's paravirtualized RDMA device. It works with its Linux Kernel driver AS IS, no need for any special guest modifications.

While it complies with the VMware device, it can also communicate with bare metal RDMA-enabled machines and does not require an RDMA HCA in the host, it can work with Soft-RoCE (rxe).

It does not require the whole guest RAM to be pinned allowing memory over-commit and, even if not implemented yet, migration support will be possible with some HW assistance.

Implementation is divided into 2 components, rdma general and pvRDMA specific functions and structures.

The second PVRDMA sub-module - interaction with PCI layer.
- Device configuration and setup (MSIX, BARs etc).
- Setup of DSR (Device Shared Resources)
- Setup of device ring.
- Device management.

---
 Makefile.objs             |   1 +
 hw/rdma/Makefile.objs     |   2 +-
 hw/rdma/vmw/pvrdma_main.c | 670 ++++++++++++++++++++++++++++++++++++++++++++++
 hw/rdma/vmw/trace-events  |   5 +
 include/hw/pci/pci_ids.h  |   3 +
 5 files changed, 680 insertions(+), 1 deletion(-)
 create mode 100644 hw/rdma/vmw/pvrdma_main.c
 create mode 100644 hw/rdma/vmw/trace-events
Makefile.objs +1 -0
index f3a3d28304..0b3c630719 100644
@@ -131,6 +131,7 @@ trace-events-subdirs += hw/char
 trace-events-subdirs += hw/intc
 trace-events-subdirs += hw/net
 trace-events-subdirs += hw/rdma
+trace-events-subdirs += hw/rdma/vmw
 trace-events-subdirs += hw/virtio
 trace-events-subdirs += hw/audio
 trace-events-subdirs += hw/misc
hw/rdma/Makefile.objs +1 -1
index 44a85f687d..3504c39d21 100644
@@ -1,5 +1,5 @@
 ifeq ($(CONFIG_RDMA),y)
 obj-$(CONFIG_PCI) += rdma_utils.o rdma_backend.o rdma_rm.o
 obj-$(CONFIG_PCI) += vmw/pvrdma_dev_ring.o vmw/pvrdma_cmd.o \
-                     vmw/pvrdma_qp_ops.o
+                     vmw/pvrdma_qp_ops.o vmw/pvrdma_main.o
 endif
hw/rdma/vmw/pvrdma_main.c new file +670 -0

670 lines not on this page · raw message

hw/rdma/vmw/trace-events new file +5 -0
new file mode 100644
index 0000000000..b3f9e2b19f
@@ -0,0 +1,5 @@
+# See docs/tracing.txt for syntax documentation.
+
+# hw/rdma/vmw/pvrdma_main.c
+pvrdma_regs_read(uint64_t addr, uint64_t val) "regs[0x%"PRIx64"] = 0x%"PRIx64
+pvrdma_regs_write(uint64_t addr, uint64_t val) "regs[0x%"PRIx64"] = 0x%"PRIx64
include/hw/pci/pci_ids.h +3 -0
index 35df1874a9..1dbf53627c 100644
@@ -266,4 +266,7 @@
 #define PCI_VENDOR_ID_TEWS               0x1498
 #define PCI_DEVICE_ID_TEWS_TPCI200       0x30C8
 
+#define PCI_VENDOR_ID_VMWARE             0x15ad
+#define PCI_DEVICE_ID_VMWARE_PVRDMA      0x0820
+
 #endif
Signature · 1 line
2.13.5
#10 lore raw

PATCH v11 10/10 MAINTAINERS: add entry for hw/rdma

In reply to Marcel Apfelbaum
---
 MAINTAINERS | 8 ++++++++
 1 file changed, 8 insertions(+)
MAINTAINERS +8 -0
index 57358a08e2..6e7adad1df 100644
@@ -2034,6 +2034,14 @@ F: block/replication.c
 F: tests/test-replication.c
 F: docs/block-replication.txt
 
+PVRDMA
+M: Yuval Shaia <[email protected]>
+M: Marcel Apfelbaum <[email protected]>
+S: Maintained
+F: hw/rdma/*
+F: hw/rdma/vmw/*
+F: docs/pvrdma.txt
+
 Build and test automation
 -------------------------
 Build and test automation
Signature · 1 line
2.13.5