Skip to contentSkip to conversation

Extend ioreq-server to support page write protection

PATCH In review Wei Ye 4 participants 19 messages Sep 3, 2014 – Sep 16, 2014

lore
Share & tools
lore

Wei Ye

author
#0 lore raw

v3 cover Extend ioreq-server to support page write protection

Changes from v1:
- Changes the new p2m type name from p2m_ram_wp to p2m_mmio_write_dm. This means that we treat the pages as a special mmio range instead of ram.
- Move macros to c file since only this file is using them.
- Address various comments from Jan.

Changes from v2:
- Remove excute attribute of the new p2m type p2m_mmio_write_dm
- Use existing rangeset for keeping the write protection page range instead of introducing hash table.
- Some code style fix.

ioreq-server is proposed to forward PIO and MMIO request to multiple device models according to the io range. XenGT (Intel Graphics Virtualization technology, please refer to https://01.org/xen/blogs/srclarkx/2013/graphics-virtualization-
xengt) driver reside in Dom0 as a virtual graphics device model also need to trap and emulate the guest's write operation to some specific memory pages, like the memory pages used by guest graphics driver as PPGTT(per-process graphics translation table). We add an new p2m type "p2m_mmio_write_dm" to trap the page write operation. Page of this new p2m type is read only and for write, the request will go to device model via ioreq-server.

Wei Ye (2):
x86: add p2m_mmio_write_dm
ioreq-server: write protected range and forwarding

 tools/libxc/xc_domain.c          |   30 ++++++++++++++++++++++
 tools/libxc/xenctrl.h            |   18 ++++++++++++++
 xen/arch/x86/hvm/hvm.c           |   51 ++++++++++++++++++++++++++++++++++++--
 xen/arch/x86/mm/p2m-ept.c        |    1 +
 xen/arch/x86/mm/p2m-pt.c         |    1 +
 xen/include/asm-x86/hvm/domain.h |    2 +-
 xen/include/asm-x86/p2m.h        |    1 +
 xen/include/public/hvm/hvm_op.h  |    1 +
 8 files changed, 102 insertions(+), 3 deletions(-)
Signature · 1 line
1.7.9.5
author
#1 lore raw

PATCH v3 1/2 x86: add p2m_mmio_write_dm

Add an new p2m type p2m_mmio_write_dm. Page of this type is read only, and write will go to the device model for emulation just like a mmio.

---
 xen/arch/x86/hvm/hvm.c    |    3 ++-
 xen/arch/x86/mm/p2m-ept.c |    1 +
 xen/arch/x86/mm/p2m-pt.c  |    1 +
 xen/include/asm-x86/p2m.h |    1 +
 4 files changed, 5 insertions(+), 1 deletion(-)
xen/arch/x86/hvm/hvm.c +2 -1
index 17ff011..adc0f93 100644
@@ -2739,7 +2739,8 @@ int hvm_hap_nested_page_fault(paddr_t gpa,
      * to the mmio handler.
      */
     if ( (p2mt == p2m_mmio_dm) || 
-         (access_w && (p2mt == p2m_ram_ro)) )
+         (access_w && ((p2mt == p2m_ram_ro) ||
+         (p2mt == p2m_mmio_write_dm))) )
     {
         put_gfn(p2m->domain, gfn);
 
xen/arch/x86/mm/p2m-ept.c +1 -0
index 15c6e83..e21a92d 100644
@@ -136,6 +136,7 @@ static void ept_p2m_type_to_flags(ept_entry_t *entry, p2m_type_t type, p2m_acces
             entry->x = 0;
             break;
         case p2m_grant_map_ro:
+        case p2m_mmio_write_dm:
             entry->r = 1;
             entry->w = entry->x = 0;
             break;
xen/arch/x86/mm/p2m-pt.c +1 -0
index 085ab6f..6a18606 100644
@@ -94,6 +94,7 @@ static unsigned long p2m_type_to_flags(p2m_type_t t, mfn_t mfn)
     default:
         return flags | _PAGE_NX_BIT;
     case p2m_grant_map_ro:
+    case p2m_mmio_write_dm:
         return flags | P2M_BASE_FLAGS | _PAGE_NX_BIT;
     case p2m_ram_ro:
     case p2m_ram_logdirty:
xen/include/asm-x86/p2m.h +1 -0
index 0ddbadb..523c7a9 100644
@@ -71,6 +71,7 @@ typedef enum {
     p2m_ram_shared = 12,          /* Shared or sharable memory */
     p2m_ram_broken = 13,          /* Broken page, access cause domain crash */
     p2m_map_foreign  = 14,        /* ram pages from foreign domain */
+    p2m_mmio_write_dm = 15,       /* Read-only; write go to the device model */
 } p2m_type_t;
 
 /*
Signature · 1 line
1.7.9.5
#2 lore raw

Re: [PATCH v3 1/2] x86: add p2m_mmio_write_dm

Quoted 10 lines

-----Original Message-----
From: Wei Ye [mailto:[email protected]]
Sent: 03 September 2014 22:53
To: [email protected]
Cc: [email protected]; Ian Campbell; Paul Durrant; Ian Jackson; Stefano Stabellini; [email protected]; Kevin Tian; [email protected]; [email protected]; [email protected]; Keir (Xen.org); Tim (Xen.org); Wei Ye
Subject: [PATCH v3 1/2] x86: add p2m_mmio_write_dm

Quoted 4 lines

Add an new p2m type p2m_mmio_write_dm. Page of this type is read only, and write will go to the device model for emulation just like a mmio.

I realise this is a bit beyond the scope of this what you're doing but it occurs to me that the idea of p2m_mmio_read_dm might be equally useful when emulating MMIO that has read side effects but no write side effects. May save a few VMEXITs here and there.

Quoted 13 lines
---
 xen/arch/x86/hvm/hvm.c    |    3 ++-
 xen/arch/x86/mm/p2m-ept.c |    1 +
 xen/arch/x86/mm/p2m-pt.c  |    1 +
 xen/include/asm-x86/p2m.h |    1 +
 4 files changed, 5 insertions(+), 1 deletion(-)
xen/arch/x86/hvm/hvm.c +0 -0
index 17ff011..adc0f93 100644

xen/arch/x86/hvm/hvm.c int hvm_hap_nested_page_fault(paddr_t gpa, · line 2743

Quoted 6 lines
xen/arch/x86/hvm/hvm.c +2 -1
@@ -2739,7 +2739,8 @@ int hvm_hap_nested_page_fault(paddr_t gpa,
      * to the mmio handler.
      */
     if ( (p2mt == p2m_mmio_dm) ||
-         (access_w && (p2mt == p2m_ram_ro)) )
+         (access_w && ((p2mt == p2m_ram_ro) ||
+         (p2mt == p2m_mmio_write_dm))) )

Personally I would have put the two p2m_mmio checks next to each other, but it doesn't really matter.

Paul

Quoted 44 lines
xen/arch/x86/hvm/hvm.c +0 -0
     {
         put_gfn(p2m->domain, gfn);
 
xen/arch/x86/mm/p2m-ept.c +1 -0
index 15c6e83..e21a92d 100644
@@ -136,6 +136,7 @@ static void ept_p2m_type_to_flags(ept_entry_t
 *entry, p2m_type_t type, p2m_acces
             entry->x = 0;
             break;
         case p2m_grant_map_ro:
+        case p2m_mmio_write_dm:
             entry->r = 1;
             entry->w = entry->x = 0;
             break;
xen/arch/x86/mm/p2m-pt.c +1 -0
index 085ab6f..6a18606 100644
@@ -94,6 +94,7 @@ static unsigned long p2m_type_to_flags(p2m_type_t t,
 mfn_t mfn)
     default:
         return flags | _PAGE_NX_BIT;
     case p2m_grant_map_ro:
+    case p2m_mmio_write_dm:
         return flags | P2M_BASE_FLAGS | _PAGE_NX_BIT;
     case p2m_ram_ro:

case p2m_ram_logdirty:

xen/include/asm-x86/p2m.h +1 -0
index 0ddbadb..523c7a9 100644
@@ -71,6 +71,7 @@ typedef enum {
     p2m_ram_shared = 12,          /* Shared or sharable memory */
     p2m_ram_broken = 13,          /* Broken page, access cause domain crash */
     p2m_map_foreign  = 14,        /* ram pages from foreign domain */
+    p2m_mmio_write_dm = 15,       /* Read-only; write go to the device

model */
} p2m_type_t;

/*
--
1.7.9.5

#3 lore raw

Re: [PATCH v3 1/2] x86: add p2m_mmio_write_dm

Quoted 14 lines

xen/arch/x86/hvm/hvm.c int hvm_hap_nested_page_fault(paddr_t gpa, · line 2743

Quoted 11 lines
Quoted 1 line · On 03.09.14 at 11:54, <[email protected]> wrote:

From: Wei Ye [mailto:[email protected]]

xen/arch/x86/hvm/hvm.c +2 -1
@@ -2739,7 +2739,8 @@ int hvm_hap_nested_page_fault(paddr_t gpa,
      * to the mmio handler.
      */
     if ( (p2mt == p2m_mmio_dm) ||
-         (access_w && (p2mt == p2m_ram_ro)) )
+         (access_w && ((p2mt == p2m_ram_ro) ||
+         (p2mt == p2m_mmio_write_dm))) )

Personally I would have put the two p2m_mmio checks next to each other, but it doesn't really matter.

At the very least the indentation would need adjustment if the ordering is to be kept as it is now.

Jan

author
#4 lore raw

PATCH v3 2/2 ioreq-server: write protected range and forwarding

In reply to Wei Ye

Extend the ioreq-server to write protect pages and forward the write access to the corresponding device model.

---
 tools/libxc/xc_domain.c          |   33 +++++++++++++++++++++++++++
 tools/libxc/xenctrl.h            |   18 +++++++++++++++
 xen/arch/x86/hvm/hvm.c           |   46 ++++++++++++++++++++++++++++++++++++++
 xen/include/asm-x86/hvm/domain.h |    2 +-
 xen/include/public/hvm/hvm_op.h  |    1 +
 5 files changed, 99 insertions(+), 1 deletion(-)
tools/libxc/xc_domain.c +33 -0
index 37ed141..34e6309 100644
@@ -1485,6 +1485,39 @@ int xc_hvm_unmap_pcidev_from_ioreq_server(xc_interface *xch, domid_t domid,
     return rc;
 }
 
+int xc_hvm_map_wp_range_to_ioreq_server(xc_interface *xch, domid_t domid,
+                                        ioservid_t id, uint16_t set,
+                                        uint64_t start, uint64_t end)
+{
+    DECLARE_HYPERCALL;
+    DECLARE_HYPERCALL_BUFFER(xen_hvm_io_range_t, arg);
+    int rc = -1;
+
+    if ( arg == NULL )
+    {
+        errno = -EFAULT;
+        return -1;
+    }
+
+    hypercall.op = __HYPERVISOR_hvm_op;
+    if ( set )
+        hypercall.arg[0] = HVMOP_map_io_range_to_ioreq_server;
+    else 
+        hypercall.arg[0] = HVMOP_unmap_io_range_from_ioreq_server;
+    hypercall.arg[1] = HYPERCALL_BUFFER_AS_ARG(arg);
+
+    arg->domid = domid;
+    arg->id = id;
+    arg->type = HVMOP_IO_RANGE_WP;
+    arg->start = start;
+    arg->end = end;
+
+    rc = do_xen_hypercall(xch, &hypercall);
+
+    xc_hypercall_buffer_free(xch, arg);
+    return rc;
+}
+
 int xc_hvm_destroy_ioreq_server(xc_interface *xch,
                                 domid_t domid,
                                 ioservid_t id)
tools/libxc/xenctrl.h +18 -0
index b55d857..b261602 100644
@@ -1943,6 +1943,24 @@ int xc_hvm_unmap_pcidev_from_ioreq_server(xc_interface *xch,
                                           uint8_t function);
 
 /**
+ * This function registers/deregisters a set of pages to be write protected.
+ *
+ * @parm xch a handle to an open hypervisor interface.
+ * @parm domid the domain id to be serviced
+ * @parm id the IOREQ Server id.
+ * @parm set whether registers or deregisters: 1 for register, 0 for deregister
+ * @parm start start of range
+ * @parm end end of range (inclusive).
+ * @return 0 on success, -1 on failure.
+ */
+int xc_hvm_map_wp_range_to_ioreq_server(xc_interface *xch,
+                                        domid_t domid,
+                                        ioservid_t id,
+                                        uint16_t set,
+                                        uint64_t start,
+                                        uint64_t end);
+
+/**
  * This function destroys an IOREQ Server.
  *
  * @parm xch a handle to an open hypervisor interface.
xen/arch/x86/hvm/hvm.c +46 -0

88 lines not on this page · raw message

xen/include/asm-x86/hvm/domain.h +1 -1
index 291a2e0..b194f9a 100644
@@ -48,7 +48,7 @@ struct hvm_ioreq_vcpu {
     evtchn_port_t    ioreq_evtchn;
 };
 
-#define NR_IO_RANGE_TYPES (HVMOP_IO_RANGE_PCI + 1)
+#define NR_IO_RANGE_TYPES (HVMOP_IO_RANGE_WP + 1)
 #define MAX_NR_IO_RANGES  256
 
 struct hvm_ioreq_server {
xen/include/public/hvm/hvm_op.h +1 -0
index eeb0a60..aedb97f 100644
@@ -323,6 +323,7 @@ struct xen_hvm_io_range {
 # define HVMOP_IO_RANGE_PORT   0 /* I/O port range */
 # define HVMOP_IO_RANGE_MEMORY 1 /* MMIO range */
 # define HVMOP_IO_RANGE_PCI    2 /* PCI segment/bus/dev/func range */
+# define HVMOP_IO_RANGE_WP     3 /* Write protetion range */
     uint64_aligned_t start, end; /* IN - inclusive start and end of range */
 };
 typedef struct xen_hvm_io_range xen_hvm_io_range_t;
Signature · 1 line
1.7.9.5
#5 lore raw

Re: [PATCH v3 2/2] ioreq-server: write protected range and forwarding

Quoted 10 lines

-----Original Message-----
From: Wei Ye [mailto:[email protected]]
Sent: 03 September 2014 22:53
To: [email protected]
Cc: [email protected]; Ian Campbell; Paul Durrant; Ian Jackson; Stefano Stabellini; [email protected]; Kevin Tian; [email protected]; [email protected]; [email protected]; Keir (Xen.org); Tim (Xen.org); Wei Ye
Subject: [PATCH v3 2/2] ioreq-server: write protected range and forwarding

Quoted 3 lines

Extend the ioreq-server to write protect pages and forward the write access to the corresponding device model.

Using rangesets did make the patch somewhat smaller then :-)

Quoted 22 lines
---
 tools/libxc/xc_domain.c          |   33 +++++++++++++++++++++++++++
 tools/libxc/xenctrl.h            |   18 +++++++++++++++
 xen/arch/x86/hvm/hvm.c           |   46
++++++++++++++++++++++++++++++++++++++
 xen/include/asm-x86/hvm/domain.h |    2 +-
 xen/include/public/hvm/hvm_op.h  |    1 +
 5 files changed, 99 insertions(+), 1 deletion(-)
tools/libxc/xc_domain.c +0 -0
index 37ed141..34e6309 100644
@@ -1485,6 +1485,39 @@ int

xc_hvm_unmap_pcidev_from_ioreq_server(xc_interface *xch, domid_t domid,

     return rc;

}

+int xc_hvm_map_wp_range_to_ioreq_server(xc_interface *xch, domid_t domid,

tools/libxc/xc_domain.c int xc_hvm_unmap_pcidev_from_ioreq_server(xc_interface *xch, domid_t domid, · line 1491

Quoted 3 lines
tools/libxc/xc_domain.c +3 -0
+                                        ioservid_t id, uint16_t set,
+                                        uint64_t start, uint64_t end)
+{

Given what I said about the notion of MMIO ranges that need read emulation but no write emulation, and the ranges you're concerned about which require write emulation but no read emulation, I wonder whether this could collapse into the existing xc_hvm_map_io_range_to_ioreq_server() by adding an extra parameter to say whether, for a given range, you want read emulation, write emulation or both.

Quoted 68 lines
tools/libxc/xc_domain.c +29 -0
+    DECLARE_HYPERCALL;
+    DECLARE_HYPERCALL_BUFFER(xen_hvm_io_range_t, arg);
+    int rc = -1;
+
+    if ( arg == NULL )
+    {
+        errno = -EFAULT;
+        return -1;
+    }
+
+    hypercall.op = __HYPERVISOR_hvm_op;
+    if ( set )
+        hypercall.arg[0] = HVMOP_map_io_range_to_ioreq_server;
+    else
+        hypercall.arg[0] = HVMOP_unmap_io_range_from_ioreq_server;
+    hypercall.arg[1] = HYPERCALL_BUFFER_AS_ARG(arg);
+
+    arg->domid = domid;
+    arg->id = id;
+    arg->type = HVMOP_IO_RANGE_WP;
+    arg->start = start;
+    arg->end = end;
+
+    rc = do_xen_hypercall(xch, &hypercall);
+
+    xc_hypercall_buffer_free(xch, arg);
+    return rc;
+}
+
 int xc_hvm_destroy_ioreq_server(xc_interface *xch,
                                 domid_t domid,
                                 ioservid_t id)
tools/libxc/xenctrl.h +6 -0
index b55d857..b261602 100644
@@ -1943,6 +1943,24 @@ int
 xc_hvm_unmap_pcidev_from_ioreq_server(xc_interface *xch,
                                           uint8_t function);
 
 /**
+ * This function registers/deregisters a set of pages to be write protected.
+ *
+ * @parm xch a handle to an open hypervisor interface.
+ * @parm domid the domain id to be serviced
+ * @parm id the IOREQ Server id.
+ * @parm set whether registers or deregisters: 1 for register, 0 for

deregister
+ * @parm start start of range
+ * @parm end end of range (inclusive).
+ * @return 0 on success, -1 on failure.
+ */

tools/libxc/xenctrl.h +8 -0
+int xc_hvm_map_wp_range_to_ioreq_server(xc_interface *xch,
+                                        domid_t domid,
+                                        ioservid_t id,
+                                        uint16_t set,
+                                        uint64_t start,
+                                        uint64_t end);
+
+/**
  * This function destroys an IOREQ Server.
  *
  * @parm xch a handle to an open hypervisor interface.
xen/arch/x86/hvm/hvm.c +0 -0
index adc0f93..f8c4db8 100644

xen/arch/x86/hvm/hvm.c static int hvm_ioreq_server_alloc_rangesets(struct · line 857

Quoted 5 lines
xen/arch/x86/hvm/hvm.c +1 -0
@@ -853,6 +853,7 @@ static int hvm_ioreq_server_alloc_rangesets(struct
 hvm_ioreq_server *s,
                       (i == HVMOP_IO_RANGE_PORT) ? "port" :
                       (i == HVMOP_IO_RANGE_MEMORY) ? "memory" :
                       (i == HVMOP_IO_RANGE_PCI) ? "pci" :
+                      (i == HVMOP_IO_RANGE_WP) ? "write protection" :

Continuing along my train of thought from above, perhaps the rangesets should now be

HVMOP_IO_RANGE_PORT_READ
HMVOP_IO_RANGE_PORT_WRITE
HVMOP_IO_RANGE_MEMORY_READ
HVMOP_IO_RANGE_MEMORY_WRITE
HVMOP_IO_RANGE_PCI

Quoted 54 lines
xen/arch/x86/hvm/hvm.c +0 -0
                       "");
         if ( rc )
             goto fail;
xen/arch/x86/hvm/hvm.c +31 -0
@@ -1138,6 +1139,37 @@ static int hvm_get_ioreq_server_info(struct
 domain *d, ioservid_t id,
     return rc;
 }
 
+static int hvm_change_p2m_type_ioreq_server(struct domain *d, uint16_t
 set,
+                                            uint64_t start, uint64_t end)
+{
+    int rc = -EINVAL;
+    uint64_t gpfn_s, gpfn_e, gpfn;
+    p2m_type_t ot, nt;
+
+    if ( set )
+    {
+        ot = p2m_ram_rw;
+        nt = p2m_mmio_write_dm;
+    }
+    else
+    {
+        ot = p2m_mmio_write_dm;
+        nt = p2m_ram_rw;
+    }
+
+    gpfn_s = start >> PAGE_SHIFT;
+    gpfn_e = end >> PAGE_SHIFT;
+
+    for ( gpfn = gpfn_s; gpfn <= gpfn_e; gpfn++)
+    {
+        rc = p2m_change_type_one(d, gpfn, ot, nt);
+        if ( !rc )
+            break;
+    }
+
+    return rc;
+}
+
 static int hvm_map_io_range_to_ioreq_server(struct domain *d, ioservid_t

id,

                                             uint32_t type, uint64_t start, uint64_t end)

{

xen/arch/x86/hvm/hvm.c +1 -0
@@ -1163,6 +1195,7 @@ static int
 hvm_map_io_range_to_ioreq_server(struct domain *d, ioservid_t id,
             case HVMOP_IO_RANGE_PORT:
             case HVMOP_IO_RANGE_MEMORY:
             case HVMOP_IO_RANGE_PCI:
+            case HVMOP_IO_RANGE_WP:
                 r = s->range[type];
                 break;

xen/arch/x86/hvm/hvm.c static int · line 1220

Quoted 8 lines
xen/arch/x86/hvm/hvm.c +4 -0
@@ -1180,6 +1213,10 @@ static int
 hvm_map_io_range_to_ioreq_server(struct domain *d, ioservid_t id,
                 break;
 
             rc = rangeset_add_range(r, start, end);
+
+            if ( type == HVMOP_IO_RANGE_WP )
+                rc = hvm_change_p2m_type_ioreq_server(d, 1, start, end);
+

Here, for memory ranges, you'd then change the p2m types to p2m_mmio_write_dm, p2m_mmio_read_dm or p2m_mmio_dm depending on whether the a range was being added to the MEMORY_WRITE set, the MEMORY_READ set or both. Obviously you'd need to be careful about ranges with distinct semantics that fall into the same page - but that could be dealt with. E.g. if you had two ranges in the same page, one needing full emulation and one needing write-only emulation then you could set the p2m type to p2m_mmio_dm and handle the read emulation for the second range directly in hvm_send_assist_req_to_ioreq_server().

Paul

Quoted 65 lines
xen/arch/x86/hvm/hvm.c +0 -0
             break;
         }
     }
xen/arch/x86/hvm/hvm.c +5 -0
@@ -1214,6 +1251,7 @@ static int
 hvm_unmap_io_range_from_ioreq_server(struct domain *d, ioservid_t id,
             case HVMOP_IO_RANGE_PORT:
             case HVMOP_IO_RANGE_MEMORY:
             case HVMOP_IO_RANGE_PCI:
+            case HVMOP_IO_RANGE_WP:
                 r = s->range[type];
                 break;
@@ -1231,6 +1269,10 @@ static int
 hvm_unmap_io_range_from_ioreq_server(struct domain *d, ioservid_t id,
                 break;
 
             rc = rangeset_remove_range(r, start, end);
+
+            if ( type == HVMOP_IO_RANGE_WP )
+                rc = hvm_change_p2m_type_ioreq_server(d, 0, start, end);
+
             break;
         }
     }
xen/arch/x86/hvm/hvm.c +4 -0
@@ -2361,6 +2403,10 @@ static struct hvm_ioreq_server
 *hvm_select_ioreq_server(struct domain *d,
             if ( rangeset_contains_range(r, addr, end) )
                 return s;
 
+            r = s->range[HVMOP_IO_RANGE_WP];
+            if ( rangeset_contains_range(r, addr, end) )
+                return s;
+
             break;
         case IOREQ_TYPE_PCI_CONFIG:

if ( rangeset_contains_singleton(r, addr >> 32) )

xen/include/asm- renamed from xen/include/asm-x86/hvm/domain.h +0 -0

x86/hvm/domain.h
index 291a2e0..b194f9a 100644

xen/include/asm-x86/hvm/domain.h +1 -1
@@ -48,7 +48,7 @@ struct hvm_ioreq_vcpu {
     evtchn_port_t    ioreq_evtchn;
 };
 
-#define NR_IO_RANGE_TYPES (HVMOP_IO_RANGE_PCI + 1)
+#define NR_IO_RANGE_TYPES (HVMOP_IO_RANGE_WP + 1)
 #define MAX_NR_IO_RANGES  256
 
 struct hvm_ioreq_server {
xen/include/public/hvm/hvm_op.h +1 -1
index eeb0a60..aedb97f 100644
@@ -323,6 +323,7 @@ struct xen_hvm_io_range {
 # define HVMOP_IO_RANGE_PORT   0 /* I/O port range */
 # define HVMOP_IO_RANGE_MEMORY 1 /* MMIO range */
 # define HVMOP_IO_RANGE_PCI    2 /* PCI segment/bus/dev/func range */
+# define HVMOP_IO_RANGE_WP     3 /* Write protetion range */
     uint64_aligned_t start, end; /* IN - inclusive start and end of range */
 };
 typedef struct xen_hvm_io_range xen_hvm_io_range_t;
--

1.7.9.5

#6 lore raw

Re: [PATCH v3 2/2] ioreq-server: write protected range and forwarding

Quoted 45 lines

From: Paul Durrant [mailto:[email protected]]
Sent: Wednesday, September 03, 2014 3:12 AM

Quoted 10 lines

-----Original Message-----
From: Wei Ye [mailto:[email protected]]
Sent: 03 September 2014 22:53
To: [email protected]
Cc: [email protected]; Ian Campbell; Paul Durrant; Ian Jackson; Stefano Stabellini; [email protected]; Kevin Tian; [email protected]; [email protected]; [email protected]; Keir (Xen.org); Tim (Xen.org); Wei Ye
Subject: [PATCH v3 2/2] ioreq-server: write protected range and forwarding

Quoted 3 lines

Extend the ioreq-server to write protect pages and forward the write access to the corresponding device model.

Using rangesets did make the patch somewhat smaller then :-)

Quoted 3 lines
---
 tools/libxc/xc_domain.c          |   33

+++++++++++++++++++++++++++

Quoted 19 lines
 tools/libxc/xenctrl.h            |   18 +++++++++++++++
 xen/arch/x86/hvm/hvm.c           |   46
++++++++++++++++++++++++++++++++++++++
 xen/include/asm-x86/hvm/domain.h |    2 +-
 xen/include/public/hvm/hvm_op.h  |    1 +
 5 files changed, 99 insertions(+), 1 deletion(-)
tools/libxc/xc_domain.c +0 -0
index 37ed141..34e6309 100644
@@ -1485,6 +1485,39 @@ int

xc_hvm_unmap_pcidev_from_ioreq_server(xc_interface *xch, domid_t domid,

     return rc;

}

+int xc_hvm_map_wp_range_to_ioreq_server(xc_interface *xch, domid_t domid,

tools/libxc/xc_domain.c

Quoted 2 lines
tools/libxc/xc_domain.c +2 -0
+                                        ioservid_t id, uint16_t set,
+                                        uint64_t start, uint64_t

end)

Quoted 8 lines
Quoted 1 line

+{

Given what I said about the notion of MMIO ranges that need read emulation but no write emulation, and the ranges you're concerned about which require write emulation but no read emulation, I wonder whether this could collapse into the existing xc_hvm_map_io_range_to_ioreq_server() by adding an extra parameter to say whether, for a given range, you want read emulation, write emulation or both.

Agree. Add a parameter to indicate r/w permission seems cleaner.

Quoted 77 lines
Quoted 69 lines
tools/libxc/xc_domain.c +29 -0
+    DECLARE_HYPERCALL;
+    DECLARE_HYPERCALL_BUFFER(xen_hvm_io_range_t, arg);
+    int rc = -1;
+
+    if ( arg == NULL )
+    {
+        errno = -EFAULT;
+        return -1;
+    }
+
+    hypercall.op = __HYPERVISOR_hvm_op;
+    if ( set )
+        hypercall.arg[0] = HVMOP_map_io_range_to_ioreq_server;
+    else
+        hypercall.arg[0] = HVMOP_unmap_io_range_from_ioreq_server;
+    hypercall.arg[1] = HYPERCALL_BUFFER_AS_ARG(arg);
+
+    arg->domid = domid;
+    arg->id = id;
+    arg->type = HVMOP_IO_RANGE_WP;
+    arg->start = start;
+    arg->end = end;
+
+    rc = do_xen_hypercall(xch, &hypercall);
+
+    xc_hypercall_buffer_free(xch, arg);
+    return rc;
+}
+
 int xc_hvm_destroy_ioreq_server(xc_interface *xch,
                                 domid_t domid,
                                 ioservid_t id)
tools/libxc/xenctrl.h +6 -0
index b55d857..b261602 100644
@@ -1943,6 +1943,24 @@ int
 xc_hvm_unmap_pcidev_from_ioreq_server(xc_interface *xch,
                                           uint8_t function);
 
 /**
+ * This function registers/deregisters a set of pages to be write protected.
+ *
+ * @parm xch a handle to an open hypervisor interface.
+ * @parm domid the domain id to be serviced
+ * @parm id the IOREQ Server id.
+ * @parm set whether registers or deregisters: 1 for register, 0 for

deregister
+ * @parm start start of range
+ * @parm end end of range (inclusive).
+ * @return 0 on success, -1 on failure.
+ */

tools/libxc/xenctrl.h +8 -0
+int xc_hvm_map_wp_range_to_ioreq_server(xc_interface *xch,
+                                        domid_t domid,
+                                        ioservid_t id,
+                                        uint16_t set,
+                                        uint64_t start,
+                                        uint64_t end);
+
+/**
  * This function destroys an IOREQ Server.
  *
  * @parm xch a handle to an open hypervisor interface.
xen/arch/x86/hvm/hvm.c +0 -0
index adc0f93..f8c4db8 100644

xen/arch/x86/hvm/hvm.c static int hvm_ioreq_server_alloc_rangesets(struct · line 855

Quoted 3 lines
xen/arch/x86/hvm/hvm.c +0 -0
@@ -853,6 +853,7 @@ static int hvm_ioreq_server_alloc_rangesets(struct
 hvm_ioreq_server *s,
                       (i == HVMOP_IO_RANGE_PORT) ? "port" :
                       (i == HVMOP_IO_RANGE_MEMORY) ?

"memory" :

xen/arch/x86/hvm/hvm.c static int hvm_ioreq_server_alloc_rangesets(struct · line 857

Quoted 2 lines
xen/arch/x86/hvm/hvm.c +1 -0
                       (i == HVMOP_IO_RANGE_PCI) ? "pci" :
+                      (i == HVMOP_IO_RANGE_WP) ? "write

protection" :

Quoted 8 lines

Continuing along my train of thought from above, perhaps the rangesets should now be

HVMOP_IO_RANGE_PORT_READ
HMVOP_IO_RANGE_PORT_WRITE
HVMOP_IO_RANGE_MEMORY_READ
HVMOP_IO_RANGE_MEMORY_WRITE
HVMOP_IO_RANGE_PCI

this looks cleaner. Once we start considering the permission, WP becomes only an attribute which can be attached to either port or memory resource. So better not define a new type here.

for now READ can simply fail until we see a real usage.

One side-effect though, is for default r/w emulated case we would double rangesets here.

Quoted 68 lines

xen/arch/x86/hvm/hvm.c static int hvm_get_ioreq_server_info(struct · line 1145

Quoted 12 lines
xen/arch/x86/hvm/hvm.c +0 -0
                       "");
         if ( rc )
             goto fail;
xen/arch/x86/hvm/hvm.c +2 -0
@@ -1138,6 +1139,37 @@ static int hvm_get_ioreq_server_info(struct
 domain *d, ioservid_t id,
     return rc;
 }
 
+static int hvm_change_p2m_type_ioreq_server(struct domain *d, uint16_t
 set,
+                                            uint64_t start,

uint64_t end)

Quoted 20 lines
xen/arch/x86/hvm/hvm.c +20 -0
+{
+    int rc = -EINVAL;
+    uint64_t gpfn_s, gpfn_e, gpfn;
+    p2m_type_t ot, nt;
+
+    if ( set )
+    {
+        ot = p2m_ram_rw;
+        nt = p2m_mmio_write_dm;
+    }
+    else
+    {
+        ot = p2m_mmio_write_dm;
+        nt = p2m_ram_rw;
+    }
+
+    gpfn_s = start >> PAGE_SHIFT;
+    gpfn_e = end >> PAGE_SHIFT;
+
+    for ( gpfn = gpfn_s; gpfn <= gpfn_e; gpfn++)

xen/arch/x86/hvm/hvm.c static int hvm_get_ioreq_server_info(struct domain *d, ioservid_t id,

Quoted 10 lines
xen/arch/x86/hvm/hvm.c +9 -0
+    {
+        rc = p2m_change_type_one(d, gpfn, ot, nt);
+        if ( !rc )
+            break;
+    }
+
+    return rc;
+}
+
 static int hvm_map_io_range_to_ioreq_server(struct domain *d,

ioservid_t

Quoted 2 lines

id,

                                             uint32_t type,

uint64_t start, uint64_t end)

Quoted 11 lines

{

xen/arch/x86/hvm/hvm.c +1 -0
@@ -1163,6 +1195,7 @@ static int
 hvm_map_io_range_to_ioreq_server(struct domain *d, ioservid_t id,
             case HVMOP_IO_RANGE_PORT:
             case HVMOP_IO_RANGE_MEMORY:
             case HVMOP_IO_RANGE_PCI:
+            case HVMOP_IO_RANGE_WP:
                 r = s->range[type];
                 break;

xen/arch/x86/hvm/hvm.c static int · line 1219

Quoted 7 lines
xen/arch/x86/hvm/hvm.c +3 -0
@@ -1180,6 +1213,10 @@ static int
 hvm_map_io_range_to_ioreq_server(struct domain *d, ioservid_t id,
                 break;
 
             rc = rangeset_add_range(r, start, end);
+
+            if ( type == HVMOP_IO_RANGE_WP )
+                rc = hvm_change_p2m_type_ioreq_server(d, 1, start,

end);

xen/arch/x86/hvm/hvm.c static int · line 1220

Quoted 1 line
xen/arch/x86/hvm/hvm.c +1 -0
+
Quoted 9 lines

Here, for memory ranges, you'd then change the p2m types to p2m_mmio_write_dm, p2m_mmio_read_dm or p2m_mmio_dm depending
on whether the a range was being added to the MEMORY_WRITE set, the MEMORY_READ set or both. Obviously you'd need to be careful about ranges with distinct semantics that fall into the same page - but that could be dealt with. E.g. if you had two ranges in the same page, one needing full emulation and one needing write-only emulation then you could set the p2m type to p2m_mmio_dm and handle the read emulation for the second range directly in hvm_send_assist_req_to_ioreq_server().

I prefer to not introducing such complexity now. Just focus on existing usages, i.e. either rw-emulation or w-emulation. We can reserve the type in hypercall path, but p2m_mmio_read_dm can wait until a real usage is required.

Quoted 71 lines

Paul

Quoted 13 lines
xen/arch/x86/hvm/hvm.c +0 -0
             break;
         }
     }
xen/arch/x86/hvm/hvm.c +1 -0
@@ -1214,6 +1251,7 @@ static int
 hvm_unmap_io_range_from_ioreq_server(struct domain *d, ioservid_t id,
             case HVMOP_IO_RANGE_PORT:
             case HVMOP_IO_RANGE_MEMORY:
             case HVMOP_IO_RANGE_PCI:
+            case HVMOP_IO_RANGE_WP:
                 r = s->range[type];
                 break;

xen/arch/x86/hvm/hvm.c static int · line 1275

Quoted 7 lines
xen/arch/x86/hvm/hvm.c +3 -0
@@ -1231,6 +1269,10 @@ static int
 hvm_unmap_io_range_from_ioreq_server(struct domain *d, ioservid_t id,
                 break;
 
             rc = rangeset_remove_range(r, start, end);
+
+            if ( type == HVMOP_IO_RANGE_WP )
+                rc = hvm_change_p2m_type_ioreq_server(d, 0, start,

end);

Quoted 36 lines
xen/arch/x86/hvm/hvm.c +1 -0
+
             break;
         }
     }
xen/arch/x86/hvm/hvm.c +4 -0
@@ -2361,6 +2403,10 @@ static struct hvm_ioreq_server
 *hvm_select_ioreq_server(struct domain *d,
             if ( rangeset_contains_range(r, addr, end) )
                 return s;
 
+            r = s->range[HVMOP_IO_RANGE_WP];
+            if ( rangeset_contains_range(r, addr, end) )
+                return s;
+
             break;
         case IOREQ_TYPE_PCI_CONFIG:

if ( rangeset_contains_singleton(r, addr >> 32) )

xen/include/asm- renamed from xen/include/asm-x86/hvm/domain.h +0 -0

x86/hvm/domain.h
index 291a2e0..b194f9a 100644

xen/include/asm-x86/hvm/domain.h +1 -1
@@ -48,7 +48,7 @@ struct hvm_ioreq_vcpu {
     evtchn_port_t    ioreq_evtchn;
 };
 
-#define NR_IO_RANGE_TYPES (HVMOP_IO_RANGE_PCI + 1)
+#define NR_IO_RANGE_TYPES (HVMOP_IO_RANGE_WP + 1)
 #define MAX_NR_IO_RANGES  256
 
 struct hvm_ioreq_server {
xen/include/public/hvm/hvm_op.h +0 -0
index eeb0a60..aedb97f 100644

xen/include/public/hvm/hvm_op.h struct xen_hvm_io_range { · line 325

Quoted 3 lines
xen/include/public/hvm/hvm_op.h +0 -0
@@ -323,6 +323,7 @@ struct xen_hvm_io_range {
 # define HVMOP_IO_RANGE_PORT   0 /* I/O port range */
 # define HVMOP_IO_RANGE_MEMORY 1 /* MMIO range */
 # define HVMOP_IO_RANGE_PCI    2 /* PCI segment/bus/dev/func

range */

Quoted 6 lines
xen/include/public/hvm/hvm_op.h +1 -1
+# define HVMOP_IO_RANGE_WP     3 /* Write protetion range */
     uint64_aligned_t start, end; /* IN - inclusive start and end of range */
 };
 typedef struct xen_hvm_io_range xen_hvm_io_range_t;
--

1.7.9.5

author
#7 lore raw

Re: [PATCH v3 2/2] ioreq-server: write protected range and forwarding

Quoted 160 lines

-----Original Message-----
From: Tian, Kevin
Sent: Friday, September 5, 2014 7:11 AM
To: Paul Durrant; Ye, Wei; [email protected]
Cc: [email protected]; Ian Campbell; Ian Jackson; Stefano Stabellini; Dugger, Donald D; Zhang, Yang Z; Lv, Zhiyuan; [email protected]; Keir (Xen.org); Tim (Xen.org)
Subject: RE: [PATCH v3 2/2] ioreq-server: write protected range and forwarding

Quoted 41 lines

From: Paul Durrant [mailto:[email protected]]
Sent: Wednesday, September 03, 2014 3:12 AM

Quoted 11 lines

-----Original Message-----
From: Wei Ye [mailto:[email protected]]
Sent: 03 September 2014 22:53
To: [email protected]
Cc: [email protected]; Ian Campbell; Paul Durrant; Ian Jackson; Stefano Stabellini; [email protected]; Kevin Tian; [email protected]; [email protected]; [email protected]; Keir (Xen.org); Tim (Xen.org); Wei Ye
Subject: [PATCH v3 2/2] ioreq-server: write protected range and forwarding

Quoted 3 lines

Extend the ioreq-server to write protect pages and forward the write access to the corresponding device model.

Using rangesets did make the patch somewhat smaller then :-)

Quoted 3 lines
---
 tools/libxc/xc_domain.c          |   33

+++++++++++++++++++++++++++

Quoted 17 lines
 tools/libxc/xenctrl.h            |   18 +++++++++++++++
 xen/arch/x86/hvm/hvm.c           |   46
++++++++++++++++++++++++++++++++++++++
 xen/include/asm-x86/hvm/domain.h |    2 +-
 xen/include/public/hvm/hvm_op.h  |    1 +
 5 files changed, 99 insertions(+), 1 deletion(-)
tools/libxc/xc_domain.c index renamed from tools/libxc/xc_domain.c +0 -0

37ed141..34e6309 100644

tools/libxc/xc_domain.c +0 -0
@@ -1485,6 +1485,39 @@ int

xc_hvm_unmap_pcidev_from_ioreq_server(xc_interface *xch, domid_t domid,

     return rc;

}

Quoted 1 line
Quoted 1 line

+int xc_hvm_map_wp_range_to_ioreq_server(xc_interface *xch,

domid_t

Quoted 11 lines

tools/libxc/xc_domain.c

Quoted 3 lines

domid,

tools/libxc/xc_domain.c +2 -0
+                                        ioservid_t id, uint16_t set,
+                                        uint64_t start, uint64_t

end)

Quoted 1 line

+{

Given what I said about the notion of MMIO ranges that need read emulation but no write emulation, and the ranges you're concerned about which require write emulation but no read emulation, I wonder whether this could collapse into the existing xc_hvm_map_io_range_to_ioreq_server() by adding an extra parameter

to say whether, for a given range, you want read emulation, write

Quoted 1 line

emulation or both.

Agree. Add a parameter to indicate r/w permission seems cleaner.

Quoted 41 lines
Quoted 41 lines
tools/libxc/xc_domain.c +29 -0
+    DECLARE_HYPERCALL;
+    DECLARE_HYPERCALL_BUFFER(xen_hvm_io_range_t, arg);
+    int rc = -1;
+
+    if ( arg == NULL )
+    {
+        errno = -EFAULT;
+        return -1;
+    }
+
+    hypercall.op = __HYPERVISOR_hvm_op;
+    if ( set )
+        hypercall.arg[0] = HVMOP_map_io_range_to_ioreq_server;
+    else
+        hypercall.arg[0] = HVMOP_unmap_io_range_from_ioreq_server;
+    hypercall.arg[1] = HYPERCALL_BUFFER_AS_ARG(arg);
+
+    arg->domid = domid;
+    arg->id = id;
+    arg->type = HVMOP_IO_RANGE_WP;
+    arg->start = start;
+    arg->end = end;
+
+    rc = do_xen_hypercall(xch, &hypercall);
+
+    xc_hypercall_buffer_free(xch, arg);
+    return rc;
+}
+
 int xc_hvm_destroy_ioreq_server(xc_interface *xch,
                                 domid_t domid,
                                 ioservid_t id) diff --git

a/tools/libxc/xenctrl.h b/tools/libxc/xenctrl.h index b55d857..b261602 100644

tools/libxc/xenctrl.h +0 -0
@@ -1943,6 +1943,24 @@ int
 xc_hvm_unmap_pcidev_from_ioreq_server(xc_interface *xch,
                                           uint8_t function);
 
Quoted 2 lines

tools/libxc/xenctrl.h int · line 1947

Quoted 2 lines
tools/libxc/xenctrl.h +1 -0
 /**
+ * This function registers/deregisters a set of pages to be write

protected.

Quoted 36 lines
Quoted 29 lines
tools/libxc/xenctrl.h +6 -0
+ *
+ * @parm xch a handle to an open hypervisor interface.
+ * @parm domid the domain id to be serviced
+ * @parm id the IOREQ Server id.
+ * @parm set whether registers or deregisters: 1 for register, 0
+ for

deregister
+ * @parm start start of range
+ * @parm end end of range (inclusive).
+ * @return 0 on success, -1 on failure.
+ */

tools/libxc/xenctrl.h +8 -0
+int xc_hvm_map_wp_range_to_ioreq_server(xc_interface *xch,
+                                        domid_t domid,
+                                        ioservid_t id,
+                                        uint16_t set,
+                                        uint64_t start,
+                                        uint64_t end);
+
+/**
  * This function destroys an IOREQ Server.
  *
  * @parm xch a handle to an open hypervisor interface.
xen/arch/x86/hvm/hvm.c index renamed from xen/arch/x86/hvm/hvm.c +0 -0

adc0f93..f8c4db8 100644

xen/arch/x86/hvm/hvm.c +0 -0
@@ -853,6 +853,7 @@ static int

hvm_ioreq_server_alloc_rangesets(struct
hvm_ioreq_server *s,

Quoted 2 lines
                       (i == HVMOP_IO_RANGE_PORT) ? "port" :
                       (i == HVMOP_IO_RANGE_MEMORY) ?

"memory" :

Quoted 2 lines
                       (i == HVMOP_IO_RANGE_PCI) ? "pci" :

+ (i == HVMOP_IO_RANGE_WP) ? "write

protection" :

Quoted 8 lines

Continuing along my train of thought from above, perhaps the rangesets should now be

HVMOP_IO_RANGE_PORT_READ
HMVOP_IO_RANGE_PORT_WRITE
HVMOP_IO_RANGE_MEMORY_READ
HVMOP_IO_RANGE_MEMORY_WRITE
HVMOP_IO_RANGE_PCI

Quoted 4 lines

this looks cleaner. Once we start considering the permission, WP becomes only an attribute which can be attached to either port or memory resource. So better not define a new type here.

Shall we also need a type of HVMOP_IO_RANGE_MEMORY_BOTH? Otherwise, if want to set both r/w emulation, user needs to call the interface xc_hvm_map_io_range_to_ioreq_server() twice. One is with the parameter HVMOP_IO_RANGE_MEMORY_READ and the other with parameter HVMOP_IO_RANGE_MEMORY_WRITE.

Wei.

Quoted 160 lines

for now READ can simply fail until we see a real usage.

One side-effect though, is for default r/w emulated case we would double rangesets here.

Quoted 67 lines

xen/arch/x86/hvm/hvm.c static int hvm_get_ioreq_server_info(struct · line 1146

Quoted 13 lines
                       "");
         if ( rc )
             goto fail;
xen/arch/x86/hvm/hvm.c +3 -0
@@ -1138,6 +1139,37 @@ static int hvm_get_ioreq_server_info(struct
 domain *d, ioservid_t id,
     return rc;
 }
 
+static int hvm_change_p2m_type_ioreq_server(struct domain *d,
+uint16_t
 set,
+                                            uint64_t start,

uint64_t end)

Quoted 20 lines
xen/arch/x86/hvm/hvm.c +20 -0
+{
+    int rc = -EINVAL;
+    uint64_t gpfn_s, gpfn_e, gpfn;
+    p2m_type_t ot, nt;
+
+    if ( set )
+    {
+        ot = p2m_ram_rw;
+        nt = p2m_mmio_write_dm;
+    }
+    else
+    {
+        ot = p2m_mmio_write_dm;
+        nt = p2m_ram_rw;
+    }
+
+    gpfn_s = start >> PAGE_SHIFT;
+    gpfn_e = end >> PAGE_SHIFT;
+
+    for ( gpfn = gpfn_s; gpfn <= gpfn_e; gpfn++)

xen/arch/x86/hvm/hvm.c static int hvm_get_ioreq_server_info(struct domain *d, ioservid_t id,

Quoted 10 lines
xen/arch/x86/hvm/hvm.c +9 -0
+    {
+        rc = p2m_change_type_one(d, gpfn, ot, nt);
+        if ( !rc )
+            break;
+    }
+
+    return rc;
+}
+
 static int hvm_map_io_range_to_ioreq_server(struct domain *d,

ioservid_t

Quoted 2 lines

id,

                                             uint32_t type,

uint64_t start, uint64_t end)

Quoted 11 lines

{

xen/arch/x86/hvm/hvm.c +1 -0
@@ -1163,6 +1195,7 @@ static int
 hvm_map_io_range_to_ioreq_server(struct domain *d, ioservid_t id,
             case HVMOP_IO_RANGE_PORT:
             case HVMOP_IO_RANGE_MEMORY:
             case HVMOP_IO_RANGE_PCI:
+            case HVMOP_IO_RANGE_WP:
                 r = s->range[type];
                 break;

xen/arch/x86/hvm/hvm.c static int · line 1219

Quoted 7 lines
xen/arch/x86/hvm/hvm.c +3 -0
@@ -1180,6 +1213,10 @@ static int
 hvm_map_io_range_to_ioreq_server(struct domain *d, ioservid_t id,
                 break;
 
             rc = rangeset_add_range(r, start, end);
+
+            if ( type == HVMOP_IO_RANGE_WP )
+                rc = hvm_change_p2m_type_ioreq_server(d, 1, start,

end);

Quoted 4 lines

xen/arch/x86/hvm/hvm.c static int · line 1220

Quoted 1 line
xen/arch/x86/hvm/hvm.c +1 -0
+

Here, for memory ranges, you'd then change the p2m types to p2m_mmio_write_dm, p2m_mmio_read_dm or p2m_mmio_dm

depending on

Quoted 7 lines

whether the a range was being added to the MEMORY_WRITE set, the MEMORY_READ set or both. Obviously you'd need to be careful about ranges with distinct semantics that fall into the same page - but that could be dealt with. E.g. if you had two ranges in the same page, one needing full emulation and one needing write-only emulation then you could set the p2m type to p2m_mmio_dm and handle the read emulation for the second range directly in hvm_send_assist_req_to_ioreq_server().

I prefer to not introducing such complexity now. Just focus on existing usages, i.e. either rw-emulation or w-emulation. We can reserve the type in hypercall path, but p2m_mmio_read_dm can wait until a real usage is required.

Quoted 8 lines

Paul

Quoted 5 lines
xen/arch/x86/hvm/hvm.c +0 -0
             break;
         }
     }
xen/arch/x86/hvm/hvm.c +0 -0
@@ -1214,6 +1251,7 @@ static int

hvm_unmap_io_range_from_ioreq_server(struct domain *d, ioservid_t

id,

Quoted 9 lines
Quoted 9 lines

case HVMOP_IO_RANGE_PORT:
case HVMOP_IO_RANGE_MEMORY:
case HVMOP_IO_RANGE_PCI:

xen/arch/x86/hvm/hvm.c +1 -0
+            case HVMOP_IO_RANGE_WP:
                 r = s->range[type];
                 break;
@@ -1231,6 +1269,10 @@ static int

hvm_unmap_io_range_from_ioreq_server(struct domain *d, ioservid_t

id,

Quoted 51 lines

xen/arch/x86/hvm/hvm.c static int hvm_map_io_range_to_ioreq_server(struct domain *d, ioservid_t id,

Quoted 6 lines
                 break;

             rc = rangeset_remove_range(r, start, end);
xen/arch/x86/hvm/hvm.c +3 -0
+
+            if ( type == HVMOP_IO_RANGE_WP )
+                rc = hvm_change_p2m_type_ioreq_server(d, 0, start,

end);

Quoted 34 lines
xen/arch/x86/hvm/hvm.c +5 -0
+
             break;
         }
     }
@@ -2361,6 +2403,10 @@ static struct hvm_ioreq_server
 *hvm_select_ioreq_server(struct domain *d,
             if ( rangeset_contains_range(r, addr, end) )
                 return s;
 
+            r = s->range[HVMOP_IO_RANGE_WP];
+            if ( rangeset_contains_range(r, addr, end) )
+                return s;
+
             break;
         case IOREQ_TYPE_PCI_CONFIG:

if ( rangeset_contains_singleton(r, addr >> 32) ) diff
--git a/xen/include/asm-x86/hvm/domain.h b/xen/include/asm- x86/hvm/domain.h index 291a2e0..b194f9a 100644

xen/include/asm-x86/hvm/domain.h +1 -1
@@ -48,7 +48,7 @@ struct hvm_ioreq_vcpu {
     evtchn_port_t    ioreq_evtchn;
 };
 
-#define NR_IO_RANGE_TYPES (HVMOP_IO_RANGE_PCI + 1)
+#define NR_IO_RANGE_TYPES (HVMOP_IO_RANGE_WP + 1)
 #define MAX_NR_IO_RANGES  256
 
 struct hvm_ioreq_server {
(quoted diff) +0 -0

b/xen/include/public/hvm/hvm_op.h index eeb0a60..aedb97f 100644

xen/include/public/hvm/hvm_op.h struct xen_hvm_io_range { · line 325

Quoted 3 lines
xen/include/public/hvm/hvm_op.h +0 -0
@@ -323,6 +323,7 @@ struct xen_hvm_io_range {
 # define HVMOP_IO_RANGE_PORT   0 /* I/O port range */
 # define HVMOP_IO_RANGE_MEMORY 1 /* MMIO range */
 # define HVMOP_IO_RANGE_PCI    2 /* PCI segment/bus/dev/func

range */

Quoted 5 lines
xen/include/public/hvm/hvm_op.h +1 -1
+# define HVMOP_IO_RANGE_WP     3 /* Write protetion range */
     uint64_aligned_t start, end; /* IN - inclusive start and end of
 range */  };  typedef struct xen_hvm_io_range xen_hvm_io_range_t;
--

1.7.9.5

#8 lore raw

Re: [PATCH v3 2/2] ioreq-server: write protected range and forwarding

Quoted 178 lines

-----Original Message-----
From: Ye, Wei [mailto:[email protected]]
Sent: 05 September 2014 01:44
To: Kevin Tian; Paul Durrant; [email protected]
Cc: [email protected]; Ian Campbell; Ian Jackson; Stefano Stabellini; Dugger, Donald D; Zhang, Yang Z; Lv, Zhiyuan; [email protected]; Keir (Xen.org); Tim (Xen.org)
Subject: RE: [PATCH v3 2/2] ioreq-server: write protected range and forwarding

Quoted 5 lines

-----Original Message-----
From: Tian, Kevin
Sent: Friday, September 5, 2014 7:11 AM
To: Paul Durrant; Ye, Wei; [email protected]
Cc: [email protected]; Ian Campbell; Ian Jackson; Stefano Stabellini;

Dugger,

Quoted 1 line

Donald D; Zhang, Yang Z; Lv, Zhiyuan; [email protected]; Keir

(Xen.org);

Quoted 35 lines

Tim (Xen.org)
Subject: RE: [PATCH v3 2/2] ioreq-server: write protected range and forwarding

Quoted 31 lines

From: Paul Durrant [mailto:[email protected]]
Sent: Wednesday, September 03, 2014 3:12 AM

Quoted 11 lines

-----Original Message-----
From: Wei Ye [mailto:[email protected]]
Sent: 03 September 2014 22:53
To: [email protected]
Cc: [email protected]; Ian Campbell; Paul Durrant; Ian Jackson; Stefano Stabellini; [email protected]; Kevin Tian; [email protected]; [email protected]; [email protected]; Keir (Xen.org); Tim (Xen.org); Wei Ye
Subject: [PATCH v3 2/2] ioreq-server: write protected range and forwarding

Quoted 3 lines

Extend the ioreq-server to write protect pages and forward the write access to the corresponding device model.

Using rangesets did make the patch somewhat smaller then :-)

Quoted 3 lines
---
 tools/libxc/xc_domain.c          |   33

+++++++++++++++++++++++++++

Quoted 7 lines
 tools/libxc/xenctrl.h            |   18 +++++++++++++++
 xen/arch/x86/hvm/hvm.c           |   46
++++++++++++++++++++++++++++++++++++++
 xen/include/asm-x86/hvm/domain.h |    2 +-
 xen/include/public/hvm/hvm_op.h  |    1 +
 5 files changed, 99 insertions(+), 1 deletion(-)
Quoted 6 lines
Quoted 6 lines

tools/libxc/xc_domain.c int · line 1485

Quoted 6 lines
tools/libxc/xc_domain.c index renamed from tools/libxc/xc_domain.c +0 -0

37ed141..34e6309 100644

tools/libxc/xc_domain.c +0 -0
@@ -1485,6 +1485,39 @@ int
 xc_hvm_unmap_pcidev_from_ioreq_server(xc_interface *xch,

domid_t

Quoted 111 lines
Quoted 5 lines

tools/libxc/xc_domain.c int xc_hvm_unmap_pcidev_from_ioreq_server(xc_interface *xch, domid_t domid,

Quoted 5 lines
tools/libxc/xc_domain.c +1 -0
 domid,
     return rc;
 }
 
+int xc_hvm_map_wp_range_to_ioreq_server(xc_interface *xch,

domid_t

Quoted 11 lines

tools/libxc/xc_domain.c int · line 1493

Quoted 3 lines
tools/libxc/xc_domain.c +2 -0
 domid,
+                                        ioservid_t id, uint16_t set,
+                                        uint64_t start, uint64_t

end)

tools/libxc/xc_domain.c int · line 1494

Quoted 1 line
tools/libxc/xc_domain.c +1 -0
+{

Given what I said about the notion of MMIO ranges that need read emulation but no write emulation, and the ranges you're concerned about which require write emulation but no read emulation, I wonder whether this could collapse into the existing xc_hvm_map_io_range_to_ioreq_server() by adding an extra parameter

to say whether, for a given range, you want read emulation, write

Quoted 1 line

emulation or both.

Agree. Add a parameter to indicate r/w permission seems cleaner.

Quoted 41 lines
Quoted 41 lines
tools/libxc/xc_domain.c +29 -0
+    DECLARE_HYPERCALL;
+    DECLARE_HYPERCALL_BUFFER(xen_hvm_io_range_t, arg);
+    int rc = -1;
+
+    if ( arg == NULL )
+    {
+        errno = -EFAULT;
+        return -1;
+    }
+
+    hypercall.op = __HYPERVISOR_hvm_op;
+    if ( set )
+        hypercall.arg[0] = HVMOP_map_io_range_to_ioreq_server;
+    else
+        hypercall.arg[0] = HVMOP_unmap_io_range_from_ioreq_server;
+    hypercall.arg[1] = HYPERCALL_BUFFER_AS_ARG(arg);
+
+    arg->domid = domid;
+    arg->id = id;
+    arg->type = HVMOP_IO_RANGE_WP;
+    arg->start = start;
+    arg->end = end;
+
+    rc = do_xen_hypercall(xch, &hypercall);
+
+    xc_hypercall_buffer_free(xch, arg);
+    return rc;
+}
+

int xc_hvm_destroy_ioreq_server(xc_interface *xch,

                                 domid_t domid,
                                 ioservid_t id) diff --git

a/tools/libxc/xenctrl.h b/tools/libxc/xenctrl.h index b55d857..b261602 100644

tools/libxc/xenctrl.h +0 -0
@@ -1943,6 +1943,24 @@ int
 xc_hvm_unmap_pcidev_from_ioreq_server(xc_interface *xch,
                                           uint8_t function);
 
Quoted 2 lines

tools/libxc/xenctrl.h int · line 1947

Quoted 2 lines
tools/libxc/xenctrl.h +1 -0
 /**
+ * This function registers/deregisters a set of pages to be write

protected.

Quoted 36 lines
Quoted 29 lines
tools/libxc/xenctrl.h +6 -0
+ *
+ * @parm xch a handle to an open hypervisor interface.
+ * @parm domid the domain id to be serviced
+ * @parm id the IOREQ Server id.
+ * @parm set whether registers or deregisters: 1 for register, 0
+ for

deregister
+ * @parm start start of range
+ * @parm end end of range (inclusive).
+ * @return 0 on success, -1 on failure.
+ */

tools/libxc/xenctrl.h +8 -0
+int xc_hvm_map_wp_range_to_ioreq_server(xc_interface *xch,
+                                        domid_t domid,
+                                        ioservid_t id,
+                                        uint16_t set,
+                                        uint64_t start,
+                                        uint64_t end);
+
+/**
  * This function destroys an IOREQ Server.
  *
  * @parm xch a handle to an open hypervisor interface.
xen/arch/x86/hvm/hvm.c index renamed from xen/arch/x86/hvm/hvm.c +0 -0

adc0f93..f8c4db8 100644

xen/arch/x86/hvm/hvm.c +0 -0
@@ -853,6 +853,7 @@ static int

hvm_ioreq_server_alloc_rangesets(struct
hvm_ioreq_server *s,

Quoted 2 lines
                       (i == HVMOP_IO_RANGE_PORT) ? "port" :
                       (i == HVMOP_IO_RANGE_MEMORY) ?

"memory" :

Quoted 2 lines
                       (i == HVMOP_IO_RANGE_PCI) ? "pci" :

+ (i == HVMOP_IO_RANGE_WP) ? "write

protection" :

Quoted 8 lines

Continuing along my train of thought from above, perhaps the rangesets should now be

HVMOP_IO_RANGE_PORT_READ
HMVOP_IO_RANGE_PORT_WRITE
HVMOP_IO_RANGE_MEMORY_READ
HVMOP_IO_RANGE_MEMORY_WRITE
HVMOP_IO_RANGE_PCI

Quoted 4 lines

this looks cleaner. Once we start considering the permission, WP becomes only an attribute which can be attached to either port or memory resource. So better not define a new type here.

Quoted 8 lines

Shall we also need a type of HVMOP_IO_RANGE_MEMORY_BOTH? Otherwise, if want to set
both r/w emulation, user needs to call the interface xc_hvm_map_io_range_to_ioreq_server() twice. One
is with the parameter HVMOP_IO_RANGE_MEMORY_READ and the other with parameter
HVMOP_IO_RANGE_MEMORY_WRITE.

That's true, so messing with the types directly is probably not the correct answer. Instead, break direct association between type and rangeset index and then hvmop_map_io_range_to_ioreq_server() can call hvm_map_io_range_to_ioreq_server() multiple times if necessary. You could declare the array ins struct hvm_ioreq_server as:

struct rangeset *range[NR_IO_RANGE_TYPES * 2];

and then calculate index as something like:

if (flags & READ)
index = type;

if (flags & WRITE)
index = type + NR_IO_RANGE_TYPES;

Yes, it does mean double the number of rangesets if you want both read and write, but that's a very small amount of extra memory. If necessary it could be avoided by having a third set of rangesets for READ|WRITE but it would mean potentially having to do two lookups in hvm_select_ioreq_server().

Paul

Quoted 166 lines

Wei.

Quoted 72 lines

for now READ can simply fail until we see a real usage.

One side-effect though, is for default r/w emulated case we would double rangesets here.

Quoted 67 lines

xen/arch/x86/hvm/hvm.c static int hvm_get_ioreq_server_info(struct · line 1146

Quoted 13 lines
                       "");
         if ( rc )
             goto fail;
xen/arch/x86/hvm/hvm.c +3 -0
@@ -1138,6 +1139,37 @@ static int hvm_get_ioreq_server_info(struct
 domain *d, ioservid_t id,
     return rc;
 }
 
+static int hvm_change_p2m_type_ioreq_server(struct domain *d,
+uint16_t
 set,
+                                            uint64_t start,

uint64_t end)

Quoted 20 lines
xen/arch/x86/hvm/hvm.c +20 -0
+{
+    int rc = -EINVAL;
+    uint64_t gpfn_s, gpfn_e, gpfn;
+    p2m_type_t ot, nt;
+
+    if ( set )
+    {
+        ot = p2m_ram_rw;
+        nt = p2m_mmio_write_dm;
+    }
+    else
+    {
+        ot = p2m_mmio_write_dm;
+        nt = p2m_ram_rw;
+    }
+
+    gpfn_s = start >> PAGE_SHIFT;
+    gpfn_e = end >> PAGE_SHIFT;
+
+    for ( gpfn = gpfn_s; gpfn <= gpfn_e; gpfn++)

xen/arch/x86/hvm/hvm.c static int hvm_get_ioreq_server_info(struct domain *d, ioservid_t id,

Quoted 10 lines
xen/arch/x86/hvm/hvm.c +9 -0
+    {
+        rc = p2m_change_type_one(d, gpfn, ot, nt);
+        if ( !rc )
+            break;
+    }
+
+    return rc;
+}
+
 static int hvm_map_io_range_to_ioreq_server(struct domain *d,

ioservid_t

Quoted 2 lines

id,

                                             uint32_t type,

uint64_t start, uint64_t end)

Quoted 11 lines

{

xen/arch/x86/hvm/hvm.c +1 -0
@@ -1163,6 +1195,7 @@ static int
 hvm_map_io_range_to_ioreq_server(struct domain *d, ioservid_t id,
             case HVMOP_IO_RANGE_PORT:
             case HVMOP_IO_RANGE_MEMORY:
             case HVMOP_IO_RANGE_PCI:
+            case HVMOP_IO_RANGE_WP:
                 r = s->range[type];
                 break;

xen/arch/x86/hvm/hvm.c static int · line 1219

Quoted 7 lines
xen/arch/x86/hvm/hvm.c +3 -0
@@ -1180,6 +1213,10 @@ static int
 hvm_map_io_range_to_ioreq_server(struct domain *d, ioservid_t id,
                 break;
 
             rc = rangeset_add_range(r, start, end);
+
+            if ( type == HVMOP_IO_RANGE_WP )
+                rc = hvm_change_p2m_type_ioreq_server(d, 1, start,

end);

Quoted 12 lines
Quoted 4 lines

xen/arch/x86/hvm/hvm.c static int · line 1220

Quoted 1 line
xen/arch/x86/hvm/hvm.c +1 -0
+

Here, for memory ranges, you'd then change the p2m types to p2m_mmio_write_dm, p2m_mmio_read_dm or p2m_mmio_dm

depending on

Quoted 7 lines

whether the a range was being added to the MEMORY_WRITE set, the MEMORY_READ set or both. Obviously you'd need to be careful about ranges with distinct semantics that fall into the same page - but that could be dealt with. E.g. if you had two ranges in the same page, one needing full emulation and one needing write-only emulation then you could set the p2m type to p2m_mmio_dm and handle the read emulation for the second range directly in

hvm_send_assist_req_to_ioreq_server().

Quoted 2 lines

I prefer to not introducing such complexity now. Just focus on existing

usages,

Quoted 5 lines

i.e. either rw-emulation or w-emulation. We can reserve the type in hypercall path, but p2m_mmio_read_dm can wait until a real usage is required.

Quoted 1 line
Quoted 7 lines
Quoted 7 lines

Paul

xen/arch/x86/hvm/hvm.c static int · line 1251

Quoted 5 lines
xen/arch/x86/hvm/hvm.c +0 -0
             break;
         }
     }
xen/arch/x86/hvm/hvm.c +0 -0
@@ -1214,6 +1251,7 @@ static int
 hvm_unmap_io_range_from_ioreq_server(struct domain *d,

ioservid_t

Quoted 10 lines

id,

Quoted 9 lines

xen/arch/x86/hvm/hvm.c static int · line 1269

Quoted 9 lines
xen/arch/x86/hvm/hvm.c +1 -0
             case HVMOP_IO_RANGE_PORT:
             case HVMOP_IO_RANGE_MEMORY:
             case HVMOP_IO_RANGE_PCI:
+            case HVMOP_IO_RANGE_WP:
                 r = s->range[type];
                 break;
@@ -1231,6 +1269,10 @@ static int
 hvm_unmap_io_range_from_ioreq_server(struct domain *d,

ioservid_t

Quoted 52 lines

id,

Quoted 51 lines

xen/arch/x86/hvm/hvm.c static int hvm_unmap_io_range_from_ioreq_server(struct domain *d, ioservid_t id,

Quoted 6 lines
xen/arch/x86/hvm/hvm.c +3 -0
                 break;
 
             rc = rangeset_remove_range(r, start, end);
+
+            if ( type == HVMOP_IO_RANGE_WP )
+                rc = hvm_change_p2m_type_ioreq_server(d, 0, start,

end);

Quoted 34 lines
xen/arch/x86/hvm/hvm.c +1 -0
+
             break;
         }
     }
xen/arch/x86/hvm/hvm.c +4 -0
@@ -2361,6 +2403,10 @@ static struct hvm_ioreq_server
 *hvm_select_ioreq_server(struct domain *d,
             if ( rangeset_contains_range(r, addr, end) )
                 return s;
 
+            r = s->range[HVMOP_IO_RANGE_WP];
+            if ( rangeset_contains_range(r, addr, end) )
+                return s;
+
             break;
         case IOREQ_TYPE_PCI_CONFIG:

if ( rangeset_contains_singleton(r, addr >> 32) ) diff
--git a/xen/include/asm-x86/hvm/domain.h b/xen/include/asm- x86/hvm/domain.h index 291a2e0..b194f9a 100644

xen/include/asm-x86/hvm/domain.h +1 -1
@@ -48,7 +48,7 @@ struct hvm_ioreq_vcpu {
     evtchn_port_t    ioreq_evtchn;
 };
 
-#define NR_IO_RANGE_TYPES (HVMOP_IO_RANGE_PCI + 1)
+#define NR_IO_RANGE_TYPES (HVMOP_IO_RANGE_WP + 1)
 #define MAX_NR_IO_RANGES  256
 
 struct hvm_ioreq_server {
(quoted diff) +0 -0

b/xen/include/public/hvm/hvm_op.h index eeb0a60..aedb97f 100644

xen/include/public/hvm/hvm_op.h struct xen_hvm_io_range { · line 325

Quoted 3 lines
xen/include/public/hvm/hvm_op.h +0 -0
@@ -323,6 +323,7 @@ struct xen_hvm_io_range {
 # define HVMOP_IO_RANGE_PORT   0 /* I/O port range */
 # define HVMOP_IO_RANGE_MEMORY 1 /* MMIO range */
 # define HVMOP_IO_RANGE_PCI    2 /* PCI segment/bus/dev/func

range */

Quoted 5 lines
xen/include/public/hvm/hvm_op.h +1 -1
+# define HVMOP_IO_RANGE_WP     3 /* Write protetion range */
     uint64_aligned_t start, end; /* IN - inclusive start and end of
 range */  };  typedef struct xen_hvm_io_range xen_hvm_io_range_t;
--

1.7.9.5

author
#9 lore raw

Re: [PATCH v3 2/2] ioreq-server: write protected range and forwarding

Quoted 223 lines

-----Original Message-----
From: Paul Durrant [mailto:[email protected]]
Sent: Friday, September 5, 2014 5:02 PM
To: Ye, Wei; Tian, Kevin; [email protected]
Cc: [email protected]; Ian Campbell; Ian Jackson; Stefano Stabellini; Dugger, Donald D; Zhang, Yang Z; Lv, Zhiyuan; [email protected]; Keir (Xen.org); Tim (Xen.org)
Subject: RE: [PATCH v3 2/2] ioreq-server: write protected range and forwarding

Quoted 56 lines

-----Original Message-----
From: Ye, Wei [mailto:[email protected]]
Sent: 05 September 2014 01:44
To: Kevin Tian; Paul Durrant; [email protected]
Cc: [email protected]; Ian Campbell; Ian Jackson; Stefano Stabellini; Dugger, Donald D; Zhang, Yang Z; Lv, Zhiyuan; [email protected]; Keir (Xen.org); Tim (Xen.org)
Subject: RE: [PATCH v3 2/2] ioreq-server: write protected range and forwarding

Quoted 6 lines

-----Original Message-----
From: Tian, Kevin
Sent: Friday, September 5, 2014 7:11 AM
To: Paul Durrant; Ye, Wei; [email protected]
Cc: [email protected]; Ian Campbell; Ian Jackson; Stefano Stabellini;

Dugger,

Quoted 1 line

Donald D; Zhang, Yang Z; Lv, Zhiyuan; [email protected]; Keir

(Xen.org);

Quoted 35 lines

Tim (Xen.org)
Subject: RE: [PATCH v3 2/2] ioreq-server: write protected range and forwarding

Quoted 31 lines

From: Paul Durrant [mailto:[email protected]]
Sent: Wednesday, September 03, 2014 3:12 AM

Quoted 11 lines

-----Original Message-----
From: Wei Ye [mailto:[email protected]]
Sent: 03 September 2014 22:53
To: [email protected]
Cc: [email protected]; Ian Campbell; Paul Durrant; Ian Jackson; Stefano Stabellini; [email protected]; Kevin Tian; [email protected]; [email protected]; [email protected]; Keir (Xen.org); Tim (Xen.org); Wei Ye
Subject: [PATCH v3 2/2] ioreq-server: write protected range and forwarding

Quoted 3 lines

Extend the ioreq-server to write protect pages and forward the write access to the corresponding device model.

Using rangesets did make the patch somewhat smaller then :-)

Quoted 3 lines
---
 tools/libxc/xc_domain.c          |   33

+++++++++++++++++++++++++++

Quoted 7 lines
 tools/libxc/xenctrl.h            |   18 +++++++++++++++
 xen/arch/x86/hvm/hvm.c           |   46
++++++++++++++++++++++++++++++++++++++
 xen/include/asm-x86/hvm/domain.h |    2 +-
 xen/include/public/hvm/hvm_op.h  |    1 +
 5 files changed, 99 insertions(+), 1 deletion(-)
Quoted 26 lines
Quoted 7 lines
Quoted 7 lines

tools/libxc/xc_domain.c int · line 1485

Quoted 7 lines
tools/libxc/xc_domain.c +0 -0

index
37ed141..34e6309 100644

tools/libxc/xc_domain.c +0 -0
@@ -1485,6 +1485,39 @@ int
 xc_hvm_unmap_pcidev_from_ioreq_server(xc_interface *xch,

domid_t

Quoted 18 lines
Quoted 5 lines

tools/libxc/xc_domain.c int xc_hvm_unmap_pcidev_from_ioreq_server(xc_interface *xch, domid_t domid,

Quoted 5 lines
tools/libxc/xc_domain.c +1 -0
 domid,
     return rc;
 }
 
+int xc_hvm_map_wp_range_to_ioreq_server(xc_interface *xch,

domid_t

Quoted 12 lines

tools/libxc/xc_domain.c int · line 1494

Quoted 4 lines
tools/libxc/xc_domain.c +3 -0
 domid,
+                                        ioservid_t id, uint16_t set,
+                                        uint64_t start,
+ uint64_t

end)

tools/libxc/xc_domain.c int · line 1495

Quoted 1 line
tools/libxc/xc_domain.c +1 -0
+{

Given what I said about the notion of MMIO ranges that need read emulation but no write emulation, and the ranges you're concerned about which require write emulation but no read emulation, I wonder whether this could collapse into the existing xc_hvm_map_io_range_to_ioreq_server() by adding an extra

parameter

Quoted 6 lines
Quoted 6 lines

to say whether, for a given range, you want read emulation, write

Quoted 1 line

emulation or both.

Agree. Add a parameter to indicate r/w permission seems cleaner.

Quoted 1 line
Quoted 15 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
tools/libxc/xc_domain.c +7 -0
+    DECLARE_HYPERCALL;
+    DECLARE_HYPERCALL_BUFFER(xen_hvm_io_range_t, arg);
+    int rc = -1;
+
+    if ( arg == NULL )
+    {
+        errno = -EFAULT;
Quoted 8 lines
Quoted 8 lines

tools/libxc/xc_domain.c int xc_hvm_unmap_pcidev_from_ioreq_server(xc_interface *xch, domid_t domid,

Quoted 8 lines
tools/libxc/xc_domain.c +8 -0
+        return -1;
+    }
+
+    hypercall.op = __HYPERVISOR_hvm_op;
+    if ( set )
+        hypercall.arg[0] = HVMOP_map_io_range_to_ioreq_server;
+    else
+        hypercall.arg[0] =

HVMOP_unmap_io_range_from_ioreq_server;

Quoted 69 lines
Quoted 69 lines
Quoted 25 lines
Quoted 25 lines
tools/libxc/xc_domain.c +14 -0
+    hypercall.arg[1] = HYPERCALL_BUFFER_AS_ARG(arg);
+
+    arg->domid = domid;
+    arg->id = id;
+    arg->type = HVMOP_IO_RANGE_WP;
+    arg->start = start;
+    arg->end = end;
+
+    rc = do_xen_hypercall(xch, &hypercall);
+
+    xc_hypercall_buffer_free(xch, arg);
+    return rc;
+}
+
 int xc_hvm_destroy_ioreq_server(xc_interface *xch,
                                 domid_t domid,
                                 ioservid_t id) diff --git

a/tools/libxc/xenctrl.h b/tools/libxc/xenctrl.h index b55d857..b261602 100644

tools/libxc/xenctrl.h +0 -0
@@ -1943,6 +1943,24 @@ int
 xc_hvm_unmap_pcidev_from_ioreq_server(xc_interface *xch,
                                           uint8_t function);
 
Quoted 3 lines

tools/libxc/xenctrl.h int · line 1948

Quoted 3 lines
tools/libxc/xenctrl.h +2 -0
 /**
+ * This function registers/deregisters a set of pages to be
+ write

protected.

Quoted 40 lines
Quoted 30 lines
tools/libxc/xenctrl.h +6 -0
+ *
+ * @parm xch a handle to an open hypervisor interface.
+ * @parm domid the domain id to be serviced
+ * @parm id the IOREQ Server id.
+ * @parm set whether registers or deregisters: 1 for register,
+ 0 for

deregister
+ * @parm start start of range
+ * @parm end end of range (inclusive).
+ * @return 0 on success, -1 on failure.
+ */

tools/libxc/xenctrl.h +8 -0
+int xc_hvm_map_wp_range_to_ioreq_server(xc_interface *xch,
+                                        domid_t domid,
+                                        ioservid_t id,
+                                        uint16_t set,
+                                        uint64_t start,
+                                        uint64_t end);
+
+/**
  * This function destroys an IOREQ Server.
  *
  * @parm xch a handle to an open hypervisor interface.
xen/arch/x86/hvm/hvm.c +0 -0

index
adc0f93..f8c4db8 100644

xen/arch/x86/hvm/hvm.c +0 -0
@@ -853,6 +853,7 @@ static int

hvm_ioreq_server_alloc_rangesets(struct
hvm_ioreq_server *s,

Quoted 2 lines
                       (i == HVMOP_IO_RANGE_PORT) ? "port" :
                       (i == HVMOP_IO_RANGE_MEMORY) ?

"memory" :

Quoted 2 lines
                       (i == HVMOP_IO_RANGE_PCI) ? "pci" :

+ (i == HVMOP_IO_RANGE_WP) ? "write

protection" :

Continuing along my train of thought from above, perhaps the rangesets should now be

Quoted 8 lines
Quoted 8 lines
Quoted 5 lines

HVMOP_IO_RANGE_PORT_READ
HMVOP_IO_RANGE_PORT_WRITE
HVMOP_IO_RANGE_MEMORY_READ
HVMOP_IO_RANGE_MEMORY_WRITE
HVMOP_IO_RANGE_PCI

this looks cleaner. Once we start considering the permission, WP becomes only an attribute which can be attached to either port or

memory resource.

Quoted 7 lines
Quoted 2 lines

So better not define a new type here.

Shall we also need a type of HVMOP_IO_RANGE_MEMORY_BOTH? Otherwise, if want to set
both r/w emulation, user needs to call the interface xc_hvm_map_io_range_to_ioreq_server() twice. One is with the

parameter

Quoted 3 lines

HVMOP_IO_RANGE_MEMORY_READ and the other with parameter HVMOP_IO_RANGE_MEMORY_WRITE.

That's true, so messing with the types directly is probably not the correct answer. Instead, break direct association between type and rangeset index and then hvmop_map_io_range_to_ioreq_server() can call hvm_map_io_range_to_ioreq_server() multiple times if necessary. You could declare the array ins struct hvm_ioreq_server as:

struct rangeset *range[NR_IO_RANGE_TYPES * 2];

and then calculate index as something like:

if (flags & READ)
index = type;

if (flags & WRITE)
index = type + NR_IO_RANGE_TYPES;

Quoted 7 lines

Yes, it does mean double the number of rangesets if you want both read and write, but that's a very small amount of extra memory. If necessary it could be avoided by having a third set of rangesets for READ|WRITE but it would mean potentially having to do two lookups in hvm_select_ioreq_server().

Paul

I'm a little bit confused that like write protection for a true mmio range. If the true mmio range is readable but write is porected, then where is the data can be read from
If the read is not forwarded to device model. Current write protection for normal pages treat write as mmio is a special case. For read, data comes from the normal page, but
wirte go to device model for emulation. And, we also can write directly to the
normal page but forward read to device model. So I think the write protection or
read protection is meaningfull only for normal page protection, not for port or mmio resources. So, double the rangeset that split READ|WRITE sets for existing resource is not very reasonable, am I right?

Wei

#10 lore raw

Re: [PATCH v3 2/2] ioreq-server: write protected range and forwarding

Quoted 79 lines

From: Ye, Wei
Sent: Tuesday, September 09, 2014 11:10 PM

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

adc0f93..f8c4db8 100644

xen/arch/x86/hvm/hvm.c +0 -0
@@ -853,6 +853,7 @@ static int

hvm_ioreq_server_alloc_rangesets(struct
hvm_ioreq_server *s,

                       (i == HVMOP_IO_RANGE_PORT) ?

"port" :

Quoted 46 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 1 line
                       (i == HVMOP_IO_RANGE_MEMORY) ?

"memory" :

Quoted 2 lines
                       (i == HVMOP_IO_RANGE_PCI) ? "pci" :

+ (i == HVMOP_IO_RANGE_WP) ? "write

protection" :

Continuing along my train of thought from above, perhaps the rangesets should now be

Quoted 8 lines
Quoted 8 lines
Quoted 5 lines

HVMOP_IO_RANGE_PORT_READ
HMVOP_IO_RANGE_PORT_WRITE
HVMOP_IO_RANGE_MEMORY_READ
HVMOP_IO_RANGE_MEMORY_WRITE
HVMOP_IO_RANGE_PCI

this looks cleaner. Once we start considering the permission, WP becomes only an attribute which can be attached to either port or

memory resource.

Quoted 7 lines
Quoted 2 lines

So better not define a new type here.

Shall we also need a type of HVMOP_IO_RANGE_MEMORY_BOTH? Otherwise, if want to set
both r/w emulation, user needs to call the interface xc_hvm_map_io_range_to_ioreq_server() twice. One is with the

parameter

Quoted 3 lines

HVMOP_IO_RANGE_MEMORY_READ and the other with parameter HVMOP_IO_RANGE_MEMORY_WRITE.

That's true, so messing with the types directly is probably not the correct answer. Instead, break direct association between type and rangeset index and then hvmop_map_io_range_to_ioreq_server() can call hvm_map_io_range_to_ioreq_server() multiple times if necessary. You could declare the array ins struct hvm_ioreq_server as:

struct rangeset *range[NR_IO_RANGE_TYPES * 2];

and then calculate index as something like:

if (flags & READ)
index = type;

if (flags & WRITE)
index = type + NR_IO_RANGE_TYPES;

Quoted 7 lines

Yes, it does mean double the number of rangesets if you want both read and write, but that's a very small amount of extra memory. If necessary it could be avoided by having a third set of rangesets for READ|WRITE but it would mean potentially having to do two lookups in hvm_select_ioreq_server().

Paul

I'm a little bit confused that like write protection for a true mmio range. If the true
mmio range is readable but write is porected, then where is the data can be read from
If the read is not forwarded to device model. Current write protection for normal pages
treat write as mmio is a special case. For read, data comes from the normal page, but
wirte go to device model for emulation. And, we also can write directly to the normal page but forward read to device model. So I think the write protection or
read protection is meaningfull only for normal page protection, not for port or mmio resources.
So, double the rangeset that split READ|WRITE sets for existing resource is not very reasonable, am I right?

Quoted 1 line

Wei

Ideally there could be such usage of a special device assignment style. For example, have all the writes from the VM forwarded to an agent in Dom0, so writes can be logged and replayed for introspection or high availability purpose, while having most reads from device directly for performance reason if no side-effect.

However it's only an ideal case, so likely the change we made now is incomplete and then anyway more changes are required to make it complete when the ideal case becomes real. Regarding to that, maybe it's better to just introduce a new type, specifically for this write-protected memory page usage:

Quoted 1 line
Quoted 1 line

HVMOP_IO_RANGE_WP_MEMORY

It has effect only on normal memory (add a check on that), w/o impact on existing interfaces.

Paul, how about your thoughts?

Thanks
Kevin

#11 lore raw

Re: [PATCH v3 2/2] ioreq-server: write protected range and forwarding

Quoted 115 lines

-----Original Message-----
From: Tian, Kevin [mailto:[email protected]]
Sent: 10 September 2014 14:02
To: Ye, Wei; Paul Durrant; [email protected]
Cc: [email protected]; Ian Campbell; Ian Jackson; Stefano Stabellini; Dugger, Donald D; Zhang, Yang Z; Lv, Zhiyuan; [email protected]; Keir (Xen.org); Tim (Xen.org)
Subject: RE: [PATCH v3 2/2] ioreq-server: write protected range and forwarding

Quoted 42 lines

From: Ye, Wei
Sent: Tuesday, September 09, 2014 11:10 PM

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

adc0f93..f8c4db8 100644

xen/arch/x86/hvm/hvm.c +0 -0
@@ -853,6 +853,7 @@ static int

hvm_ioreq_server_alloc_rangesets(struct
hvm_ioreq_server *s,

                       (i == HVMOP_IO_RANGE_PORT) ?

"port" :

Quoted 32 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 1 line
                       (i == HVMOP_IO_RANGE_MEMORY) ?

"memory" :

Quoted 2 lines
                       (i == HVMOP_IO_RANGE_PCI) ? "pci" :

+ (i == HVMOP_IO_RANGE_WP) ? "write

protection" :

Continuing along my train of thought from above, perhaps the rangesets should now be

Quoted 8 lines
Quoted 8 lines
Quoted 5 lines

HVMOP_IO_RANGE_PORT_READ
HMVOP_IO_RANGE_PORT_WRITE
HVMOP_IO_RANGE_MEMORY_READ
HVMOP_IO_RANGE_MEMORY_WRITE
HVMOP_IO_RANGE_PCI

this looks cleaner. Once we start considering the permission, WP becomes only an attribute which can be attached to either port or

memory resource.

Quoted 7 lines
Quoted 2 lines

So better not define a new type here.

Shall we also need a type of HVMOP_IO_RANGE_MEMORY_BOTH? Otherwise, if want to set
both r/w emulation, user needs to call the interface xc_hvm_map_io_range_to_ioreq_server() twice. One is with the

parameter

Quoted 3 lines

HVMOP_IO_RANGE_MEMORY_READ and the other with parameter HVMOP_IO_RANGE_MEMORY_WRITE.

That's true, so messing with the types directly is probably not the correct answer. Instead, break direct association between type and rangeset

index

Quoted 8 lines
Quoted 8 lines

and then hvmop_map_io_range_to_ioreq_server() can call hvm_map_io_range_to_ioreq_server() multiple times if necessary. You could declare the array ins struct hvm_ioreq_server as:

struct rangeset *range[NR_IO_RANGE_TYPES * 2];

and then calculate index as something like:

Quoted 7 lines
Quoted 7 lines

if (flags & READ)
index = type;

if (flags & WRITE)
index = type + NR_IO_RANGE_TYPES;

Yes, it does mean double the number of rangesets if you want both read

and

Quoted 1 line
Quoted 1 line

write, but that's a very small amount of extra memory. If necessary it

could

Quoted 1 line
Quoted 1 line

be avoided by having a third set of rangesets for READ|WRITE but it

would

Quoted 5 lines
Quoted 4 lines

mean potentially having to do two lookups in hvm_select_ioreq_server().

Paul

I'm a little bit confused that like write protection for a true mmio range. If

the

Quoted 2 lines

true
mmio range is readable but write is porected, then where is the data can

be

Quoted 6 lines

read from
If the read is not forwarded to device model. Current write protection for normal pages
treat write as mmio is a special case. For read, data comes from the normal page, but
wirte go to device model for emulation. And, we also can write directly to

the

Quoted 1 line

normal page but forward read to device model. So I think the write

protection

Quoted 2 lines

or
read protection is meaningfull only for normal page protection, not for port

or

Quoted 2 lines

mmio resources.
So, double the rangeset that split READ|WRITE sets for existing resource is

not

Quoted 3 lines

very reasonable, am I right?

Wei

Ideally there could be such usage of a special device assignment style. For example, have all the writes from the VM forwarded to an agent in Dom0, so writes can be logged and replayed for introspection or high availability purpose,
while having most reads from device directly for performance reason if no side-effect.

However it's only an ideal case, so likely the change we made now is incomplete
and then anyway more changes are required to make it complete when the ideal case becomes real. Regarding to that, maybe it's better to just introduce
a new type, specifically for this write-protected memory page usage:

Quoted 7 lines
Quoted 1 line
Quoted 1 line

HVMOP_IO_RANGE_WP_MEMORY

It has effect only on normal memory (add a check on that), w/o impact on existing interfaces.

Paul, how about your thoughts?

I really don't see it as a special case. You can either emulate read, write or both. Clearly if you're not emulating both then there has to be a page in the p2m so that the non-emulated case can be handled. Current MMIO is simply the emulate-both case for a range which has no memory backing it. If we get the new additional code right then it would be possible for an emulator registering an MMIO range to add pages into the guest for than range, and then only emulate read or write accesses if it doesn't need to handle both.

Paul

Quoted 2 lines

Thanks
Kevin

#12 lore raw

Re: [PATCH v3 2/2] ioreq-server: write protected range and forwarding

Quoted 132 lines

From: Paul Durrant [mailto:[email protected]]
Sent: Thursday, September 11, 2014 7:39 AM

Quoted 21 lines

-----Original Message-----
From: Tian, Kevin [mailto:[email protected]]
Sent: 10 September 2014 14:02
To: Ye, Wei; Paul Durrant; [email protected]
Cc: [email protected]; Ian Campbell; Ian Jackson; Stefano Stabellini; Dugger, Donald D; Zhang, Yang Z; Lv, Zhiyuan; [email protected]; Keir (Xen.org); Tim (Xen.org)
Subject: RE: [PATCH v3 2/2] ioreq-server: write protected range and forwarding

Quoted 11 lines

From: Ye, Wei
Sent: Tuesday, September 09, 2014 11:10 PM

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

adc0f93..f8c4db8 100644

xen/arch/x86/hvm/hvm.c +0 -0
@@ -853,6 +853,7 @@ static int

hvm_ioreq_server_alloc_rangesets(struct
hvm_ioreq_server *s,

                       (i == HVMOP_IO_RANGE_PORT) ?

"port" :

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
                       (i ==

HVMOP_IO_RANGE_MEMORY) ?

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

"memory" :

Quoted 1 line
                       (i == HVMOP_IO_RANGE_PCI) ?

"pci" :

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

+ (i == HVMOP_IO_RANGE_WP) ?

"write

Quoted 51 lines
Quoted 21 lines
Quoted 21 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

protection" :

Continuing along my train of thought from above, perhaps the rangesets should now be

Quoted 8 lines
Quoted 8 lines
Quoted 5 lines

HVMOP_IO_RANGE_PORT_READ
HMVOP_IO_RANGE_PORT_WRITE
HVMOP_IO_RANGE_MEMORY_READ
HVMOP_IO_RANGE_MEMORY_WRITE
HVMOP_IO_RANGE_PCI

this looks cleaner. Once we start considering the permission, WP becomes only an attribute which can be attached to either port or

memory resource.

Quoted 7 lines
Quoted 2 lines

So better not define a new type here.

Shall we also need a type of HVMOP_IO_RANGE_MEMORY_BOTH? Otherwise, if want to set
both r/w emulation, user needs to call the interface xc_hvm_map_io_range_to_ioreq_server() twice. One is with the

Quoted 7 lines
Quoted 7 lines

parameter

Quoted 3 lines

HVMOP_IO_RANGE_MEMORY_READ and the other with parameter HVMOP_IO_RANGE_MEMORY_WRITE.

That's true, so messing with the types directly is probably not the correct answer. Instead, break direct association between type and rangeset

index

Quoted 8 lines
Quoted 8 lines

and then hvmop_map_io_range_to_ioreq_server() can call hvm_map_io_range_to_ioreq_server() multiple times if necessary. You could declare the array ins struct hvm_ioreq_server as:

struct rangeset *range[NR_IO_RANGE_TYPES * 2];

and then calculate index as something like:

Quoted 7 lines
Quoted 7 lines

if (flags & READ)
index = type;

if (flags & WRITE)
index = type + NR_IO_RANGE_TYPES;

Yes, it does mean double the number of rangesets if you want both read

and

Quoted 1 line
Quoted 1 line

write, but that's a very small amount of extra memory. If necessary it

could

Quoted 1 line
Quoted 1 line

be avoided by having a third set of rangesets for READ|WRITE but it

would

Quoted 2 lines
Quoted 2 lines

mean potentially having to do two lookups in hvm_select_ioreq_server().

Quoted 11 lines
Quoted 3 lines
Quoted 2 lines

Paul

I'm a little bit confused that like write protection for a true mmio range. If

the

Quoted 2 lines

true
mmio range is readable but write is porected, then where is the data can

be

Quoted 4 lines

read from
If the read is not forwarded to device model. Current write protection for normal pages
treat write as mmio is a special case. For read, data comes from the

normal

Quoted 7 lines
Quoted 2 lines

page, but
wirte go to device model for emulation. And, we also can write directly to

the

Quoted 1 line

normal page but forward read to device model. So I think the write

protection

Quoted 2 lines

or
read protection is meaningfull only for normal page protection, not for

port

Quoted 10 lines

or

Quoted 2 lines

mmio resources.
So, double the rangeset that split READ|WRITE sets for existing resource is

not

Quoted 3 lines

very reasonable, am I right?

Wei

Ideally there could be such usage of a special device assignment style. For example, have all the writes from the VM forwarded to an agent in Dom0,

so

Quoted 12 lines

writes can be logged and replayed for introspection or high availability purpose,
while having most reads from device directly for performance reason if no side-effect.

However it's only an ideal case, so likely the change we made now is incomplete
and then anyway more changes are required to make it complete when the ideal case becomes real. Regarding to that, maybe it's better to just introduce
a new type, specifically for this write-protected memory page usage:

Quoted 7 lines
Quoted 1 line
Quoted 1 line

HVMOP_IO_RANGE_WP_MEMORY

It has effect only on normal memory (add a check on that), w/o impact on existing interfaces.

Paul, how about your thoughts?

Quoted 8 lines

I really don't see it as a special case. You can either emulate read, write or both. Clearly if you're not emulating both then there has to be a page in the p2m so that the non-emulated case can be handled. Current MMIO is simply the emulate-both case for a range which has no memory backing it. If we get the new additional code right then it would be possible for an emulator registering an MMIO range to add pages into the guest for than range, and then only emulate read or write accesses if it doesn't need to handle both.

so let's still pursue this interface change like you suggested. However because Wei left Intel, Yu will take this work and please bear some delay for him to ramp up.

Thanks
Kevin

#13 lore raw

Re: [PATCH v3 2/2] ioreq-server: write protected range and forwarding

In reply to Ye, Wei
Quoted 241 lines

-----Original Message-----
From: Ye, Wei [mailto:[email protected]]
Sent: 09 September 2014 23:10
To: Paul Durrant; Kevin Tian; [email protected]
Cc: [email protected]; Ian Campbell; Ian Jackson; Stefano Stabellini; Dugger, Donald D; Zhang, Yang Z; Lv, Zhiyuan; [email protected]; Keir (Xen.org); Tim (Xen.org)
Subject: RE: [PATCH v3 2/2] ioreq-server: write protected range and forwarding

Quoted 5 lines

-----Original Message-----
From: Paul Durrant [mailto:[email protected]]
Sent: Friday, September 5, 2014 5:02 PM
To: Ye, Wei; Tian, Kevin; [email protected]
Cc: [email protected]; Ian Campbell; Ian Jackson; Stefano Stabellini;

Dugger,

Quoted 1 line

Donald D; Zhang, Yang Z; Lv, Zhiyuan; [email protected]; Keir

(Xen.org);

Quoted 209 lines

Tim (Xen.org)
Subject: RE: [PATCH v3 2/2] ioreq-server: write protected range and forwarding

Quoted 56 lines

-----Original Message-----
From: Ye, Wei [mailto:[email protected]]
Sent: 05 September 2014 01:44
To: Kevin Tian; Paul Durrant; [email protected]
Cc: [email protected]; Ian Campbell; Ian Jackson; Stefano Stabellini; Dugger, Donald D; Zhang, Yang Z; Lv, Zhiyuan; [email protected]; Keir (Xen.org); Tim (Xen.org)
Subject: RE: [PATCH v3 2/2] ioreq-server: write protected range and forwarding

Quoted 6 lines

-----Original Message-----
From: Tian, Kevin
Sent: Friday, September 5, 2014 7:11 AM
To: Paul Durrant; Ye, Wei; [email protected]
Cc: [email protected]; Ian Campbell; Ian Jackson; Stefano Stabellini;

Dugger,

Quoted 1 line

Donald D; Zhang, Yang Z; Lv, Zhiyuan; [email protected]; Keir

(Xen.org);

Quoted 35 lines

Tim (Xen.org)
Subject: RE: [PATCH v3 2/2] ioreq-server: write protected range and forwarding

Quoted 31 lines

From: Paul Durrant [mailto:[email protected]]
Sent: Wednesday, September 03, 2014 3:12 AM

Quoted 11 lines

-----Original Message-----
From: Wei Ye [mailto:[email protected]]
Sent: 03 September 2014 22:53
To: [email protected]
Cc: [email protected]; Ian Campbell; Paul Durrant; Ian Jackson; Stefano Stabellini; [email protected]; Kevin Tian; [email protected]; [email protected]; [email protected]; Keir (Xen.org); Tim (Xen.org); Wei Ye
Subject: [PATCH v3 2/2] ioreq-server: write protected range and forwarding

Quoted 3 lines

Extend the ioreq-server to write protect pages and forward the write access to the corresponding device model.

Using rangesets did make the patch somewhat smaller then :-)

Quoted 3 lines
---
 tools/libxc/xc_domain.c          |   33

+++++++++++++++++++++++++++

Quoted 7 lines
 tools/libxc/xenctrl.h            |   18 +++++++++++++++
 xen/arch/x86/hvm/hvm.c           |   46
++++++++++++++++++++++++++++++++++++++
 xen/include/asm-x86/hvm/domain.h |    2 +-
 xen/include/public/hvm/hvm_op.h  |    1 +
 5 files changed, 99 insertions(+), 1 deletion(-)
Quoted 26 lines
Quoted 7 lines
Quoted 7 lines

tools/libxc/xc_domain.c int · line 1485

Quoted 7 lines
tools/libxc/xc_domain.c +0 -0

index
37ed141..34e6309 100644

tools/libxc/xc_domain.c +0 -0
@@ -1485,6 +1485,39 @@ int
 xc_hvm_unmap_pcidev_from_ioreq_server(xc_interface *xch,

domid_t

Quoted 18 lines
Quoted 5 lines

tools/libxc/xc_domain.c int xc_hvm_unmap_pcidev_from_ioreq_server(xc_interface *xch, domid_t domid,

Quoted 5 lines
tools/libxc/xc_domain.c +1 -0
 domid,
     return rc;
 }
 
+int xc_hvm_map_wp_range_to_ioreq_server(xc_interface *xch,

domid_t

Quoted 12 lines

tools/libxc/xc_domain.c int · line 1494

Quoted 4 lines
tools/libxc/xc_domain.c +3 -0
 domid,
+                                        ioservid_t id, uint16_t set,
+                                        uint64_t start,
+ uint64_t

end)

tools/libxc/xc_domain.c int · line 1495

Quoted 1 line
tools/libxc/xc_domain.c +1 -0
+{

Given what I said about the notion of MMIO ranges that need read emulation but no write emulation, and the ranges you're concerned about which require write emulation but no read emulation, I wonder whether this could collapse into the existing xc_hvm_map_io_range_to_ioreq_server() by adding an extra

parameter

Quoted 6 lines
Quoted 6 lines

to say whether, for a given range, you want read emulation, write

Quoted 1 line

emulation or both.

Agree. Add a parameter to indicate r/w permission seems cleaner.

Quoted 1 line
Quoted 15 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
tools/libxc/xc_domain.c +7 -0
+    DECLARE_HYPERCALL;
+    DECLARE_HYPERCALL_BUFFER(xen_hvm_io_range_t, arg);
+    int rc = -1;
+
+    if ( arg == NULL )
+    {
+        errno = -EFAULT;
Quoted 8 lines
Quoted 8 lines

tools/libxc/xc_domain.c int xc_hvm_unmap_pcidev_from_ioreq_server(xc_interface *xch, domid_t domid,

Quoted 8 lines
tools/libxc/xc_domain.c +8 -0
+        return -1;
+    }
+
+    hypercall.op = __HYPERVISOR_hvm_op;
+    if ( set )
+        hypercall.arg[0] = HVMOP_map_io_range_to_ioreq_server;
+    else
+        hypercall.arg[0] =

HVMOP_unmap_io_range_from_ioreq_server;

Quoted 69 lines
Quoted 69 lines
Quoted 25 lines
Quoted 25 lines
tools/libxc/xc_domain.c +14 -0
+    hypercall.arg[1] = HYPERCALL_BUFFER_AS_ARG(arg);
+
+    arg->domid = domid;
+    arg->id = id;
+    arg->type = HVMOP_IO_RANGE_WP;
+    arg->start = start;
+    arg->end = end;
+
+    rc = do_xen_hypercall(xch, &hypercall);
+
+    xc_hypercall_buffer_free(xch, arg);
+    return rc;
+}
+
 int xc_hvm_destroy_ioreq_server(xc_interface *xch,
                                 domid_t domid,
                                 ioservid_t id) diff --git

a/tools/libxc/xenctrl.h b/tools/libxc/xenctrl.h index b55d857..b261602 100644

tools/libxc/xenctrl.h +0 -0
@@ -1943,6 +1943,24 @@ int
 xc_hvm_unmap_pcidev_from_ioreq_server(xc_interface *xch,
                                           uint8_t function);
 
Quoted 3 lines

tools/libxc/xenctrl.h int · line 1948

Quoted 3 lines
tools/libxc/xenctrl.h +2 -0
 /**
+ * This function registers/deregisters a set of pages to be
+ write

protected.

Quoted 40 lines
Quoted 30 lines
tools/libxc/xenctrl.h +6 -0
+ *
+ * @parm xch a handle to an open hypervisor interface.
+ * @parm domid the domain id to be serviced
+ * @parm id the IOREQ Server id.
+ * @parm set whether registers or deregisters: 1 for register,
+ 0 for

deregister
+ * @parm start start of range
+ * @parm end end of range (inclusive).
+ * @return 0 on success, -1 on failure.
+ */

tools/libxc/xenctrl.h +8 -0
+int xc_hvm_map_wp_range_to_ioreq_server(xc_interface *xch,
+                                        domid_t domid,
+                                        ioservid_t id,
+                                        uint16_t set,
+                                        uint64_t start,
+                                        uint64_t end);
+
+/**
  * This function destroys an IOREQ Server.
  *
  * @parm xch a handle to an open hypervisor interface.
xen/arch/x86/hvm/hvm.c +0 -0

index
adc0f93..f8c4db8 100644

xen/arch/x86/hvm/hvm.c +0 -0
@@ -853,6 +853,7 @@ static int

hvm_ioreq_server_alloc_rangesets(struct
hvm_ioreq_server *s,

Quoted 2 lines
                       (i == HVMOP_IO_RANGE_PORT) ? "port" :
                       (i == HVMOP_IO_RANGE_MEMORY) ?

"memory" :

Quoted 2 lines
                       (i == HVMOP_IO_RANGE_PCI) ? "pci" :

+ (i == HVMOP_IO_RANGE_WP) ? "write

protection" :

Continuing along my train of thought from above, perhaps the rangesets should now be

Quoted 8 lines
Quoted 8 lines
Quoted 5 lines

HVMOP_IO_RANGE_PORT_READ
HMVOP_IO_RANGE_PORT_WRITE
HVMOP_IO_RANGE_MEMORY_READ
HVMOP_IO_RANGE_MEMORY_WRITE
HVMOP_IO_RANGE_PCI

this looks cleaner. Once we start considering the permission, WP becomes only an attribute which can be attached to either port or

memory resource.

Quoted 7 lines
Quoted 2 lines

So better not define a new type here.

Shall we also need a type of HVMOP_IO_RANGE_MEMORY_BOTH? Otherwise, if want to set
both r/w emulation, user needs to call the interface xc_hvm_map_io_range_to_ioreq_server() twice. One is with the

parameter

Quoted 3 lines

HVMOP_IO_RANGE_MEMORY_READ and the other with parameter HVMOP_IO_RANGE_MEMORY_WRITE.

That's true, so messing with the types directly is probably not the correct answer. Instead, break direct association between type and rangeset index and then hvmop_map_io_range_to_ioreq_server() can call hvm_map_io_range_to_ioreq_server() multiple times if necessary. You could declare the array ins struct hvm_ioreq_server as:

struct rangeset *range[NR_IO_RANGE_TYPES * 2];

and then calculate index as something like:

Quoted 7 lines

if (flags & READ)
index = type;

if (flags & WRITE)
index = type + NR_IO_RANGE_TYPES;

Yes, it does mean double the number of rangesets if you want both read

and

Quoted 4 lines

write, but that's a very small amount of extra memory. If necessary it could be avoided by having a third set of rangesets for READ|WRITE but it would mean potentially having to do two lookups in hvm_select_ioreq_server().

Quoted 7 lines
Quoted 2 lines

Paul

I'm a little bit confused that like write protection for a true mmio range. If the true
mmio range is readable but write is porected, then where is the data can be read from
If the read is not forwarded to device model.

Clearly the emulator would need to populate that page in the guest p2m if it's not emulating both read and write. But that's what you're already doing in your use-case isn't it?

Quoted 5 lines

Current write protection for
normal pages
treat write as mmio is a special case. For read, data comes from the normal page, but
wirte go to device model for emulation.

Yes, but why is that a special case? All you're essentially doing is populating pages in an MMIO area and then setting protection so you get the intercepts you want. There's already precedent for populating the p2m of MMIO range in the stdvga device model in QEMU (where the VRAM is exposed as a BAR).

Quoted 5 lines

And, we also can write directly to the
normal page but forward read to device model. So I think the write protection or
read protection is meaningfull only for normal page protection, not for port or mmio resources.

But essentially the only difference between MMIO and RAM is whether there are any entries in the p2m.

Paul

Quoted 4 lines

So, double the rangeset that split READ|WRITE sets for existing resource is not very reasonable, am I right?

Wei

#14 lore raw

Re: [PATCH v3 2/2] ioreq-server: write protected range and forwarding

In reply to Wei Ye
Quoted 11 lines
Quoted 1 line
Quoted 1 line · On 03.09.14 at 23:53, <[email protected]> wrote:
xen/arch/x86/hvm/hvm.c +10 -0
+static int hvm_change_p2m_type_ioreq_server(struct domain *d, uint16_t set,
+                                            uint64_t start, uint64_t end)
+{
+    int rc = -EINVAL;
+    uint64_t gpfn_s, gpfn_e, gpfn;
+    p2m_type_t ot, nt;
+
+    if ( set )
+    {
+        ot = p2m_ram_rw;

xen/arch/x86/hvm/hvm.c static int hvm_get_ioreq_server_info(struct domain *d, ioservid_t id, · line 1161

Quoted 10 lines
xen/arch/x86/hvm/hvm.c +10 -0
+        nt = p2m_mmio_write_dm;
+    }
+    else
+    {
+        ot = p2m_mmio_write_dm;
+        nt = p2m_ram_rw;
+    }
+
+    gpfn_s = start >> PAGE_SHIFT;
+    gpfn_e = end >> PAGE_SHIFT;

Considering that the first really ought to be PFN_DOWN() - is the latter really correct? I'd rather expect that to be PFN_UP()...

xen/include/asm-x86/hvm/domain.h struct hvm_ioreq_vcpu { · line 52

Quoted 9 lines
xen/include/asm-x86/hvm/domain.h +1 -1
@@ -48,7 +48,7 @@ struct hvm_ioreq_vcpu {
     evtchn_port_t    ioreq_evtchn;
 };
 
-#define NR_IO_RANGE_TYPES (HVMOP_IO_RANGE_PCI + 1)
+#define NR_IO_RANGE_TYPES (HVMOP_IO_RANGE_WP + 1)
 #define MAX_NR_IO_RANGES  256

So in the end you didn't even find it necessary to bump the limit on the number of ranges per domain? Iirc that was your major objection against using existing infrastructure.

Jan

author
#15 lore raw

Re: [PATCH v3 2/2] ioreq-server: write protected range and forwarding

Quoted 21 lines

-----Original Message-----
From: [email protected] [mailto:xen-devel- [email protected]] On Behalf Of Jan Beulich
Sent: Wednesday, September 3, 2014 9:17 PM
To: Ye, Wei
Cc: Tian, Kevin; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Dugger, Donald D; [email protected]; [email protected]; Lv, Zhiyuan; Zhang, Yang Z
Subject: Re: [Xen-devel] [PATCH v3 2/2] ioreq-server: write protected range and forwarding

Quoted 7 lines
Quoted 1 line
Quoted 1 line · On 03.09.14 at 23:53, <[email protected]> wrote:
xen/include/asm-x86/hvm/domain.h +0 -0
@@ -48,7 +48,7 @@ struct hvm_ioreq_vcpu {
     evtchn_port_t    ioreq_evtchn;
 };
 
Quoted 8 lines

xen/include/asm-x86/hvm/domain.h struct hvm_ioreq_vcpu { · line 52

Quoted 3 lines
xen/include/asm-x86/hvm/domain.h +1 -1
-#define NR_IO_RANGE_TYPES (HVMOP_IO_RANGE_PCI + 1)
+#define NR_IO_RANGE_TYPES (HVMOP_IO_RANGE_WP + 1)
 #define MAX_NR_IO_RANGES  256

So in the end you didn't even find it necessary to bump the limit on the number of ranges per domain? Iirc that was your major objection against using existing infrastructure.

I make an experiment to collect the total amount page of ppgtt that will be protected in one time on the HWS platform. The result show that 512 pages needs to be protected. Though it's larger than the limitation of rangeset, fortunately, most pages are always continuous. So the many pages will be merged into one range. So the total range count is much less than the limitation. And thanks for your remaindering to use the existing rangeset.

Regards
Wei

Quoted 7 lines

Jan

_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xen.org/xen-devel

author
#16 lore raw

Re: [PATCH v3 2/2] ioreq-server: write protected range and forwarding

In reply to Jan Beulich
Quoted 31 lines

-----Original Message-----
From: [email protected] [mailto:xen-devel- [email protected]] On Behalf Of Jan Beulich
Sent: Wednesday, September 3, 2014 9:17 PM
To: Ye, Wei
Cc: Tian, Kevin; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Dugger, Donald D; [email protected]; [email protected]; Lv, Zhiyuan; Zhang, Yang Z
Subject: Re: [Xen-devel] [PATCH v3 2/2] ioreq-server: write protected range and forwarding

Quoted 2 lines
Quoted 1 line
Quoted 1 line · On 03.09.14 at 23:53, <[email protected]> wrote:

+static int hvm_change_p2m_type_ioreq_server(struct domain *d,

uint16_t set,

Quoted 14 lines
xen/arch/x86/hvm/hvm.c +14 -0
+                                            uint64_t start, uint64_t
+end) {
+    int rc = -EINVAL;
+    uint64_t gpfn_s, gpfn_e, gpfn;
+    p2m_type_t ot, nt;
+
+    if ( set )
+    {
+        ot = p2m_ram_rw;
+        nt = p2m_mmio_write_dm;
+    }
+    else
+    {
+        ot = p2m_mmio_write_dm;
Quoted 9 lines

xen/arch/x86/hvm/hvm.c static int hvm_get_ioreq_server_info(struct domain *d, ioservid_t id, · line 1161

Quoted 5 lines
xen/arch/x86/hvm/hvm.c +5 -0
+        nt = p2m_ram_rw;
+    }
+
+    gpfn_s = start >> PAGE_SHIFT;
+    gpfn_e = end >> PAGE_SHIFT;

Considering that the first really ought to be PFN_DOWN() - is the latter really correct? I'd rather expect that to be PFN_UP()...

I think the latter one is also should be PFN_DOWN. Considering a range is [0x0000, 0x1000], then only the first pfn 0 should be changed. Note that the follow loop:
for (gpfn = gpfn_s; gpfn <= gpfn_e; gpfn++)
The gpfn_e is included. If PFN_UP is to calculate the gpfn_e, there's wrong p2m type change to pfn 1.

Wei

Quoted 5 lines

_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xen.org/xen-devel

#17 lore raw

Re: [PATCH v3 2/2] ioreq-server: write protected range and forwarding

Quoted 42 lines
Quoted 31 lines
Quoted 1 line · On 10.09.14 at 07:32, <[email protected]> wrote:

-----Original Message-----
From: [email protected] [mailto:xen-devel- [email protected]] On Behalf Of Jan Beulich
Sent: Wednesday, September 3, 2014 9:17 PM
To: Ye, Wei
Cc: Tian, Kevin; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Dugger, Donald D; [email protected]; [email protected]; Lv, Zhiyuan; Zhang, Yang Z
Subject: Re: [Xen-devel] [PATCH v3 2/2] ioreq-server: write protected range and forwarding

Quoted 2 lines
Quoted 1 line
Quoted 1 line · On 03.09.14 at 23:53, <[email protected]> wrote:

+static int hvm_change_p2m_type_ioreq_server(struct domain *d,

uint16_t set,

Quoted 13 lines
xen/arch/x86/hvm/hvm.c +13 -0
+                                            uint64_t start, uint64_t
+end) {
+    int rc = -EINVAL;
+    uint64_t gpfn_s, gpfn_e, gpfn;
+    p2m_type_t ot, nt;
+
+    if ( set )
+    {
+        ot = p2m_ram_rw;
+        nt = p2m_mmio_write_dm;
+    }
+    else
+    {
Quoted 8 lines

xen/arch/x86/hvm/hvm.c static int hvm_get_ioreq_server_info(struct domain *d, ioservid_t id, · line 1161

Quoted 6 lines
xen/arch/x86/hvm/hvm.c +6 -0
+        ot = p2m_mmio_write_dm;
+        nt = p2m_ram_rw;
+    }
+
+    gpfn_s = start >> PAGE_SHIFT;
+    gpfn_e = end >> PAGE_SHIFT;

Considering that the first really ought to be PFN_DOWN() - is the latter

really

Quoted 2 lines

correct? I'd rather expect that to be PFN_UP()...

Quoted 7 lines

I think the latter one is also should be PFN_DOWN. Considering a range is [0x0000, 0x1000],
then only the first pfn 0 should be changed. Note that the follow loop:
for (gpfn = gpfn_s; gpfn <= gpfn_e; gpfn++)
The gpfn_e is included. If PFN_UP is to calculate the gpfn_e, there's wrong p2m type change
to pfn 1.

But then (i.e. if the range is inclusive) it still needs to be PFN_UP(end - 1).

Jan

#18 lore raw

Re: [PATCH v3 2/2] ioreq-server: write protected range and forwarding

Quoted 51 lines
Quoted 43 lines
Quoted 32 lines · On 10.09.14 at 11:19, <[email protected]> wrote:
Quoted 1 line · On 10.09.14 at 07:32, <[email protected]> wrote:

-----Original Message-----
From: [email protected] [mailto:xen-devel- [email protected]] On Behalf Of Jan Beulich
Sent: Wednesday, September 3, 2014 9:17 PM
To: Ye, Wei
Cc: Tian, Kevin; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Dugger, Donald D; [email protected]; [email protected]; Lv, Zhiyuan; Zhang, Yang Z
Subject: Re: [Xen-devel] [PATCH v3 2/2] ioreq-server: write protected range and forwarding

Quoted 2 lines
Quoted 1 line
Quoted 1 line · On 03.09.14 at 23:53, <[email protected]> wrote:

+static int hvm_change_p2m_type_ioreq_server(struct domain *d,

uint16_t set,

Quoted 13 lines
xen/arch/x86/hvm/hvm.c +13 -0
+                                            uint64_t start, uint64_t
+end) {
+    int rc = -EINVAL;
+    uint64_t gpfn_s, gpfn_e, gpfn;
+    p2m_type_t ot, nt;
+
+    if ( set )
+    {
+        ot = p2m_ram_rw;
+        nt = p2m_mmio_write_dm;
+    }
+    else
+    {
Quoted 8 lines

xen/arch/x86/hvm/hvm.c static int hvm_get_ioreq_server_info(struct domain *d, ioservid_t id, · line 1161

Quoted 6 lines
xen/arch/x86/hvm/hvm.c +6 -0
+        ot = p2m_mmio_write_dm;
+        nt = p2m_ram_rw;
+    }
+
+    gpfn_s = start >> PAGE_SHIFT;
+    gpfn_e = end >> PAGE_SHIFT;

Considering that the first really ought to be PFN_DOWN() - is the latter

really

Quoted 2 lines

correct? I'd rather expect that to be PFN_UP()...

Quoted 7 lines

I think the latter one is also should be PFN_DOWN. Considering a range is [0x0000, 0x1000],
then only the first pfn 0 should be changed. Note that the follow loop:
for (gpfn = gpfn_s; gpfn <= gpfn_e; gpfn++)
The gpfn_e is included. If PFN_UP is to calculate the gpfn_e, there's wrong p2m type change
to pfn 1.

Quoted 1 line

But then (i.e. if the range is inclusive) it still needs to be

Of course this should read "... is not inclusive) ..."

Jan

Quoted 3 lines

PFN_UP(end - 1).

Jan