Skip to contentSkip to conversation

Add support for multi-page vm_event ring buffer

PATCH In review Petre Pircalabu 6 participants 19 messages Sep 13, 2018 – Sep 24, 2018

lore
Share & tools
lore
#0 lore raw

cover Add support for multi-page vm_event ring buffer

This series enables the vm_event ring to use buffers larger than 4K.

Petre Pircalabu (4):
x86_emulator: Add PHONY uninstall target tools/libxc: Define VM_EVENT type
x86: Add map_domain_pages_global
vm_event: Add support for multi-page ring buffer

 tools/libxc/include/xenctrl.h       |   2 +
 tools/libxc/xc_monitor.c            |   9 +-
 tools/libxc/xc_private.h            |   9 +-
 tools/libxc/xc_vm_event.c           | 139 ++++++++++++++-----
 tools/tests/x86_emulator/Makefile   |   3 +
 tools/tests/xen-access/xen-access.c |  33 +++--
 xen/arch/x86/domain_page.c          |  21 +++
 xen/arch/x86/mm.c                   |  14 ++
 xen/common/vm_event.c               | 258 +++++++++++++++++++++++++++---------
 xen/include/public/domctl.h         |  22 ++-
 xen/include/public/memory.h         |   1 +
 xen/include/xen/domain_page.h       |   9 ++
 xen/include/xen/sched.h             |   5 +-
 xen/include/xen/vm_event.h          |   4 +
 14 files changed, 414 insertions(+), 115 deletions(-)
Signature · 7 lines
2.7.4


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#1 lore raw

PATCH 1/4 x86_emulator: Add PHONY uninstall target

The missing uninstall target breaks the top 'tools' uninstall target.

---
 tools/tests/x86_emulator/Makefile | 3 +++
 1 file changed, 3 insertions(+)
tools/tests/x86_emulator/Makefile +3 -0
index e8a3e90..8696a65 100644
@@ -126,6 +126,9 @@ distclean: clean
 .PHONY: install uninstall
 install uninstall:
 
+.PHONY: uninstall
+uninstall:
+
 x86_emulate:
 	[ -L $@ ] || ln -sf $(XEN_ROOT)/xen/arch/x86/$@
 
Signature · 7 lines
2.7.4


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Acked-by on v1 1/4
#2 lore raw

Re: [PATCH 1/4] x86_emulator: Add PHONY uninstall target

I think the patch subject prefix should be changed to tools/tests.

Quoted 3 lines · Petre Pircalabu

The missing uninstall target breaks the top 'tools' uninstall target.

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#3 lore raw

Re: [PATCH 1/4] x86_emulator: Add PHONY uninstall target

In reply to Petre Pircalabu
Quoted 2 lines
Quoted 1 line
Quoted 1 line · On 13.09.18 at 17:01, <[email protected]> wrote:

The missing uninstall target breaks the top 'tools' uninstall target.

Missing?

tools/tests/x86_emulator/Makefile distclean: clean · line 130

Quoted 8 lines
tools/tests/x86_emulator/Makefile +2 -0
@@ -126,6 +126,9 @@ distclean: clean
 .PHONY: install uninstall
 install uninstall:
 
+.PHONY: uninstall
+uninstall:

Please look at the patch context above your change. It was added a couple of weeks ago (09613d3b5f), and you really should have noticed this the latest when re-basing on top of that change.

Jan

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#4 lore raw

Re: [PATCH 1/4] x86_emulator: Add PHONY uninstall target

Quoted 21 lines · Jan Beulich
Quoted 6 lines
Quoted 4 lines
Quoted 4 lines · On 13.09.18 at 17:01, <[email protected]> wrote:

The missing uninstall target breaks the top 'tools' uninstall target.

Missing?

Quoted 9 lines
tools/tests/x86_emulator/Makefile +0 -0
@@ -126,6 +126,9 @@ distclean: clean

 .PHONY: install uninstall
 install uninstall:

+.PHONY: uninstall
+uninstall:

Please look at the patch context above your change. It was added a couple of weeks ago (09613d3b5f), and you really should have noticed this the latest when re-basing on top of that change.

Quoted 2 lines

Jan

Thank-you very much for pointing it out. I have missed it completely. I will completely remove this patch in the next patchset iteration.

//Petre

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#5 lore raw

PATCH 2/4 tools/libxc: Define VM_EVENT type

In reply to Petre Pircalabu

Define the type for each of the supported vm_event rings (paging, monitor and sharing) and replace the ring param field with this type.

---
 tools/libxc/xc_monitor.c    |  2 +-
 tools/libxc/xc_private.h    |  6 +--
 tools/libxc/xc_vm_event.c   | 90 +++++++++++++++++++++++++++++----------------
 xen/include/public/domctl.h | 21 +++++++----
 4 files changed, 77 insertions(+), 42 deletions(-)
tools/libxc/xc_monitor.c +1 -1
index 4ac823e..15e6a0e 100644
@@ -24,7 +24,7 @@
 
 void *xc_monitor_enable(xc_interface *xch, uint32_t domain_id, uint32_t *port)
 {
-    return xc_vm_event_enable(xch, domain_id, HVM_PARAM_MONITOR_RING_PFN,
+    return xc_vm_event_enable(xch, domain_id, XEN_VM_EVENT_TYPE_MONITOR,
                               port);
 }
 
tools/libxc/xc_private.h +3 -3
index 705eaa9..be22986 100644
@@ -430,10 +430,10 @@ int xc_ffs64(uint64_t x);
 int xc_vm_event_control(xc_interface *xch, uint32_t domain_id, unsigned int op,
                         unsigned int mode, uint32_t *port);
 /*
- * Enables vm_event and returns the mapped ring page indicated by param.
- * param can be HVM_PARAM_PAGING/ACCESS/SHARING_RING_PFN
+ * Enables vm_event and returns the mapped ring page indicated by type.
+ * type can be XEN_VM_EVENT_TYPE_(PAGING/MONITOR/SHARING)
  */
-void *xc_vm_event_enable(xc_interface *xch, uint32_t domain_id, int param,
+void *xc_vm_event_enable(xc_interface *xch, uint32_t domain_id, int type,
                          uint32_t *port);
 
 int do_dm_op(xc_interface *xch, uint32_t domid, unsigned int nr_bufs, ...);
tools/libxc/xc_vm_event.c +59 -31

109 lines not on this page · raw message

xen/include/public/domctl.h +14 -7
index 82b6967..ac4ced2 100644
@@ -757,10 +757,17 @@ struct xen_domctl_gdbsx_domstatus {
 
 /*
  * There are currently three rings available for VM events:
- * sharing, monitor and paging. This hypercall allows one to
- * control these rings (enable/disable), as well as to signal
- * to the hypervisor to pull responses (resume) from the given
- * ring.
+ * sharing, monitor and paging
+ */
+
+#define XEN_VM_EVENT_TYPE_PAGING 1
+#define XEN_VM_EVENT_TYPE_MONITOR 2
+#define XEN_VM_EVENT_TYPE_SHARING 3
+
+/*
+ * This hypercall allows one to control the vm_event rings (enable/disable),
+ * as well as to signal to the hypervisor to pull responses (resume) from
+ * the given ring.
  */
 #define XEN_VM_EVENT_ENABLE               0
 #define XEN_VM_EVENT_DISABLE              1
@@ -780,7 +787,7 @@ struct xen_domctl_gdbsx_domstatus {
  * EXDEV  - guest has PoD enabled
  * EBUSY  - guest has or had paging enabled, ring buffer still active
  */
-#define XEN_DOMCTL_VM_EVENT_OP_PAGING            1
+#define XEN_DOMCTL_VM_EVENT_OP_PAGING XEN_VM_EVENT_TYPE_PAGING
 
 /*
  * Monitor helper.
@@ -804,7 +811,7 @@ struct xen_domctl_gdbsx_domstatus {
  * EBUSY  - guest has or had access enabled, ring buffer still active
  *
  */
-#define XEN_DOMCTL_VM_EVENT_OP_MONITOR           2
+#define XEN_DOMCTL_VM_EVENT_OP_MONITOR XEN_VM_EVENT_TYPE_MONITOR
 
 /*
  * Sharing ENOMEM helper.
@@ -819,7 +826,7 @@ struct xen_domctl_gdbsx_domstatus {
  * Note that shring can be turned on (as per the domctl below)
  * *without* this ring being setup.
  */
-#define XEN_DOMCTL_VM_EVENT_OP_SHARING           3
+#define XEN_DOMCTL_VM_EVENT_OP_SHARING XEN_VM_EVENT_TYPE_SHARING
 
 /* Use for teardown/setup of helper<->hypervisor interface for paging, 
  * access and sharing.*/
Signature · 7 lines
2.7.4


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#6 lore raw

Re: [PATCH 2/4] tools/libxc: Define VM_EVENT type

Quoted 13 lines
Quoted 1 line
Quoted 1 line · On 13.09.18 at 17:01, <[email protected]> wrote:
xen/include/public/domctl.h +2 -4
@@ -757,10 +757,17 @@ struct xen_domctl_gdbsx_domstatus {
 
 /*
  * There are currently three rings available for VM events:
- * sharing, monitor and paging. This hypercall allows one to
- * control these rings (enable/disable), as well as to signal
- * to the hypervisor to pull responses (resume) from the given
- * ring.
+ * sharing, monitor and paging
+ */

xen/include/public/domctl.h struct xen_domctl_gdbsx_domstatus { · line 771

Quoted 10 lines
xen/include/public/domctl.h +9 -0
+
+#define XEN_VM_EVENT_TYPE_PAGING 1
+#define XEN_VM_EVENT_TYPE_MONITOR 2
+#define XEN_VM_EVENT_TYPE_SHARING 3
+
+/*
+ * This hypercall allows one to control the vm_event rings (enable/disable),
+ * as well as to signal to the hypervisor to pull responses (resume) from
+ * the given ring.
  */

What's the reason to modify the comment, the more with a style violation (malformed single line comment) as the result?

Quoted 19 lines
xen/include/public/domctl.h +2 -2
@@ -780,7 +787,7 @@ struct xen_domctl_gdbsx_domstatus {
  * EXDEV  - guest has PoD enabled
  * EBUSY  - guest has or had paging enabled, ring buffer still active
  */
-#define XEN_DOMCTL_VM_EVENT_OP_PAGING            1
+#define XEN_DOMCTL_VM_EVENT_OP_PAGING XEN_VM_EVENT_TYPE_PAGING
 
 /*
  * Monitor helper.
@@ -804,7 +811,7 @@ struct xen_domctl_gdbsx_domstatus {
  * EBUSY  - guest has or had access enabled, ring buffer still active
  *
  */
-#define XEN_DOMCTL_VM_EVENT_OP_MONITOR           2
+#define XEN_DOMCTL_VM_EVENT_OP_MONITOR XEN_VM_EVENT_TYPE_MONITOR
 
 /*
  * Sharing ENOMEM helper.

xen/include/public/domctl.h struct xen_domctl_gdbsx_domstatus { · line 829

Quoted 5 lines
xen/include/public/domctl.h +1 -1
@@ -819,7 +826,7 @@ struct xen_domctl_gdbsx_domstatus {
  * Note that shring can be turned on (as per the domctl below)
  * *without* this ring being setup.
  */
-#define XEN_DOMCTL_VM_EVENT_OP_SHARING           3
+#define XEN_DOMCTL_VM_EVENT_OP_SHARING XEN_VM_EVENT_TYPE_SHARING

And what's the reason to retain these (now redundant) XEN_DOMCTL_VM_EVENT_OP_* definitions? Either they're independent (in which case they shouldn't resolve to XEN_VM_EVENT_TYPE_*) or they're true aliases (tolerating arbitrary future changes to XEN_VM_EVENT_TYPE_* without further adjustment here), and then unnecessary to retain.

Jan

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#7 lore raw

Re: [PATCH 2/4] tools/libxc: Define VM_EVENT type

Quoted 22 lines · Jan Beulich
Quoted 22 lines
Quoted 4 lines
Quoted 4 lines · On 13.09.18 at 17:01, <[email protected]> wrote:
xen/include/public/domctl.h +0 -0
@@ -757,10 +757,17 @@ struct xen_domctl_gdbsx_domstatus {


 /*
  * There are currently three rings available for VM events:
- * sharing, monitor and paging. This hypercall allows one to
- * control these rings (enable/disable), as well as to signal
- * to the hypervisor to pull responses (resume) from the given
- * ring.
+ * sharing, monitor and paging
+ */

xen/include/public/domctl.h +6 -0
+
+#define XEN_VM_EVENT_TYPE_PAGING 1
+#define XEN_VM_EVENT_TYPE_MONITOR 2
+#define XEN_VM_EVENT_TYPE_SHARING 3
+
+/*
Quoted 9 lines
Quoted 6 lines

+ * This hypercall allows one to control the vm_event rings (enable/disable),
+ * as well as to signal to the hypervisor to pull responses (resume) from
+ * the given ring.
  */

What's the reason to modify the comment, the more with a style violation (malformed single line comment) as the result?

I have split the comment in 2 parts. The first ("There are ... sharing, monitor and paging ") describes the 3 XEN_VM_EVENT_TYPE_* rings, and the second describes the vm_event hypercalls ( XEN_VM_EVENT_ENABLE .... ). Other than the missing period at the end of "paging" (which I will fix in the next iteration) I cannot identify other coding style violations.

Quoted 34 lines
Quoted 22 lines
xen/include/public/domctl.h +0 -0
@@ -780,7 +787,7 @@ struct xen_domctl_gdbsx_domstatus {
   * EXDEV  - guest has PoD enabled
   * EBUSY  - guest has or had paging enabled, ring buffer still

active
  */

xen/include/public/domctl.h +1 -1
-#define XEN_DOMCTL_VM_EVENT_OP_PAGING            1
+#define XEN_DOMCTL_VM_EVENT_OP_PAGING XEN_VM_EVENT_TYPE_PAGING


 /*
  * Monitor helper.

xen/include/public/domctl.h +0 -0
@@ -804,7 +811,7 @@ struct xen_domctl_gdbsx_domstatus {
   * EBUSY  - guest has or had access enabled, ring buffer still

active
  *
  */

xen/include/public/domctl.h +1 -1
-#define XEN_DOMCTL_VM_EVENT_OP_MONITOR           2
+#define XEN_DOMCTL_VM_EVENT_OP_MONITOR XEN_VM_EVENT_TYPE_MONITOR


 /*
  * Sharing ENOMEM helper.

xen/include/public/domctl.h struct xen_domctl_gdbsx_domstatus { · line 829

Quoted 5 lines
xen/include/public/domctl.h +1 -1
@@ -819,7 +826,7 @@ struct xen_domctl_gdbsx_domstatus {
   * Note that shring can be turned on (as per the domctl below)
   * *without* this ring being setup.
   */
-#define XEN_DOMCTL_VM_EVENT_OP_SHARING           3
+#define XEN_DOMCTL_VM_EVENT_OP_SHARING XEN_VM_EVENT_TYPE_SHARING

And what's the reason to retain these (now redundant) XEN_DOMCTL_VM_EVENT_OP_* definitions? Either they're independent (in which case they shouldn't resolve to XEN_VM_EVENT_TYPE_*) or they're true aliases (tolerating arbitrary future changes to XEN_VM_EVENT_TYPE_* without further adjustment here), and then unnecessary to retain.

Quoted 3 lines

Jan

I kept them despite the redundancy because the domctl.h header is public and I didn't wanted to break any existent (external) clients of this interface.
However, if you think removing them altogether it's best, I will replace them with the XEN_VM_EVENT_TYPE_* and increment XEN_DOMCTL_INTERFACE_VERSION.

//Petre

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#8 lore raw

Re: [PATCH 2/4] tools/libxc: Define VM_EVENT type

Quoted 30 lines
Quoted 1 line
Quoted 1 line · On 14.09.18 at 13:11, <[email protected]> wrote:
Quoted 19 lines · Jan Beulich
Quoted 19 lines
Quoted 1 line
Quoted 1 line · On 13.09.18 at 17:01, <[email protected]> wrote:
xen/include/public/domctl.h +8 -4
@@ -757,10 +757,17 @@ struct xen_domctl_gdbsx_domstatus {
 
 /*
  * There are currently three rings available for VM events:
- * sharing, monitor and paging. This hypercall allows one to
- * control these rings (enable/disable), as well as to signal
- * to the hypervisor to pull responses (resume) from the given
- * ring.
+ * sharing, monitor and paging
+ */
+
+#define XEN_VM_EVENT_TYPE_PAGING 1
+#define XEN_VM_EVENT_TYPE_MONITOR 2
+#define XEN_VM_EVENT_TYPE_SHARING 3
+
+/*
Quoted 9 lines
Quoted 6 lines
xen/include/public/domctl.h +2 -0
+ * This hypercall allows one to control the vm_event rings
 (enable/disable),
+ * as well as to signal to the hypervisor to pull responses

(resume) from
+ * the given ring.
*/

What's the reason to modify the comment, the more with a style violation (malformed single line comment) as the result?

Quoted 1 line

I have split the comment in 2 parts.

I realize you did this, but you still don't really clarify why.

Quoted 6 lines

The first ("There are ... sharing,
monitor and paging ") describes the 3 XEN_VM_EVENT_TYPE_* rings, and the second describes the vm_event hypercalls ( XEN_VM_EVENT_ENABLE .... ). Other than the missing period at the end of "paging" (which I will fix in the next iteration) I cannot identify other coding style violations.

Well, I've already told you: The first of the two resulting comments is now supposed to be all on a single line.

Quoted 33 lines
Quoted 18 lines
Quoted 18 lines
xen/include/public/domctl.h +0 -0
@@ -780,7 +787,7 @@ struct xen_domctl_gdbsx_domstatus {
  * EXDEV  - guest has PoD enabled
  * EBUSY  - guest has or had paging enabled, ring buffer still

active
*/

xen/include/public/domctl.h +1 -1
-#define XEN_DOMCTL_VM_EVENT_OP_PAGING            1
+#define XEN_DOMCTL_VM_EVENT_OP_PAGING XEN_VM_EVENT_TYPE_PAGING
 
 /*
  * Monitor helper.
@@ -804,7 +811,7 @@ struct xen_domctl_gdbsx_domstatus {
  * EBUSY  - guest has or had access enabled, ring buffer still

active
*
*/

xen/include/public/domctl.h +1 -1
-#define XEN_DOMCTL_VM_EVENT_OP_MONITOR           2
+#define XEN_DOMCTL_VM_EVENT_OP_MONITOR XEN_VM_EVENT_TYPE_MONITOR
Quoted 15 lines

xen/include/public/domctl.h struct xen_domctl_gdbsx_domstatus { · line 829

Quoted 8 lines

/*
* Sharing ENOMEM helper.

xen/include/public/domctl.h +1 -1
@@ -819,7 +826,7 @@ struct xen_domctl_gdbsx_domstatus {
  * Note that shring can be turned on (as per the domctl below)
  * *without* this ring being setup.
  */
-#define XEN_DOMCTL_VM_EVENT_OP_SHARING           3
+#define XEN_DOMCTL_VM_EVENT_OP_SHARING XEN_VM_EVENT_TYPE_SHARING

And what's the reason to retain these (now redundant) XEN_DOMCTL_VM_EVENT_OP_* definitions? Either they're independent (in which case they shouldn't resolve to XEN_VM_EVENT_TYPE_*) or they're true aliases (tolerating arbitrary future changes to XEN_VM_EVENT_TYPE_* without further adjustment here), and then unnecessary to retain.

Quoted 6 lines

I kept them despite the redundancy because the domctl.h header is public and I didn't wanted to break any existent (external) clients of this interface.
However, if you think removing them altogether it's best, I will replace them with the XEN_VM_EVENT_TYPE_* and increment XEN_DOMCTL_INTERFACE_VERSION.

Yes, that's what I think should be happening, unless there were other reasons.

Jan

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#9 lore raw

PATCH 3/4 x86: Add map_domain_pages_global

In reply to Petre Pircalabu

Create a single mapping for multiple domain pages.

---
 tools/libxc/xc_vm_event.c     |  2 +-
 xen/arch/x86/domain_page.c    | 22 ++++++++++++++++++++++
 xen/include/xen/domain_page.h |  9 +++++++++
 3 files changed, 32 insertions(+), 1 deletion(-)
tools/libxc/xc_vm_event.c +1 -1
index dd34cec..de37ca5 100644
@@ -74,7 +74,7 @@ static int xc_vm_event_domctl(int type, unsigned int *param)
     {
     case XEN_VM_EVENT_TYPE_PAGING:
         *param = XEN_DOMCTL_VM_EVENT_OP_PAGING;
-	break;
+        break;
 
     case XEN_VM_EVENT_TYPE_MONITOR:
         *param = XEN_DOMCTL_VM_EVENT_OP_MONITOR;
xen/arch/x86/domain_page.c +22 -0
index 4a07cfb..0d23e52 100644
@@ -317,6 +317,28 @@ void *map_domain_page_global(mfn_t mfn)
     return vmap(&mfn, 1);
 }
 
+void *map_domain_pages_global(const mfn_t *mfn, unsigned int nr)
+{
+    ASSERT(!in_irq() &&
+           ((system_state >= SYS_STATE_boot &&
+             system_state < SYS_STATE_active) ||
+            local_irq_is_enabled()));
+
+    return vmap(mfn, nr);
+}
+
+void *__map_domain_pages_global(const struct page_info *pg, unsigned int nr)
+{
+    mfn_t mfn[nr];
+    int i;
+    struct page_info *cur_pg = (struct page_info *)&pg[0];
+
+    for (i = 0; i < nr; i++)
+        mfn[i] = page_to_mfn(cur_pg++);
+
+    return map_domain_pages_global(mfn, nr);
+}
+
 void unmap_domain_page_global(const void *ptr)
 {
     unsigned long va = (unsigned long)ptr;
xen/include/xen/domain_page.h +9 -0
index 32669a3..76422f9 100644
@@ -42,6 +42,7 @@ mfn_t domain_page_map_to_mfn(const void *va);
  * mappings can also be unmapped from any context.
  */
 void *map_domain_page_global(mfn_t mfn);
+void *map_domain_pages_global(const mfn_t *mfn, unsigned int nr);
 void unmap_domain_page_global(const void *va);
 
 #define __map_domain_page(pg)        map_domain_page(page_to_mfn(pg))
@@ -51,6 +52,8 @@ static inline void *__map_domain_page_global(const struct page_info *pg)
     return map_domain_page_global(page_to_mfn(pg));
 }
 
+void *__map_domain_pages_global(const struct page_info *pg, unsigned int nr);
+
 #else /* !CONFIG_DOMAIN_PAGE */
 
 #define map_domain_page(mfn)                __mfn_to_virt(mfn_x(mfn))
@@ -68,6 +71,12 @@ static inline void *__map_domain_page_global(const struct page_info *pg)
     return page_to_virt(pg);
 }
 
+static inline void *__map_domain_pages_global(const struct page_info *pg,
+                                              unsigned int nr)
+{
+    return __map_domain_page_global(pg);
+}
+
 static inline void unmap_domain_page_global(const void *va) {};
 
 #endif /* !CONFIG_DOMAIN_PAGE */
Signature · 7 lines
2.7.4


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#10 lore raw

Re: [PATCH 3/4] x86: Add map_domain_pages_global

Quoted 2 lines
Quoted 1 line
Quoted 1 line · On 13.09.18 at 17:01, <[email protected]> wrote:

Create a single mapping for multiple domain pages.

At least as far as map_domain_pages_global() goes, you will want to justify why what we have isn't enough.

tools/libxc/xc_vm_event.c static int xc_vm_event_domctl(int type, unsigned int *param) · line 80

Quoted 11 lines
tools/libxc/xc_vm_event.c +1 -1
@@ -74,7 +74,7 @@ static int xc_vm_event_domctl(int type, unsigned int *param)
     {
     case XEN_VM_EVENT_TYPE_PAGING:
         *param = XEN_DOMCTL_VM_EVENT_OP_PAGING;
-	break;
+        break;
 
     case XEN_VM_EVENT_TYPE_MONITOR:
         *param = XEN_DOMCTL_VM_EVENT_OP_MONITOR;

Clearly not something you want to do in an otherwise hypervisor-only patch.

Quoted 1 line

+void *__map_domain_pages_global(const struct page_info *pg, unsigned int nr)

I we really think we need this function, no new name space violations please (here: no undue leading underscores).

tools/libxc/xc_vm_event.c

Quoted 2 lines
tools/libxc/xc_vm_event.c +2 -0
+{
+    mfn_t mfn[nr];

I think we'd like to avoid any (new) VLAs on the stack.

Quoted 1 line

+ int i;

unsigned int

tools/libxc/xc_vm_event.c static inline void *__map_domain_page_global(const struct page_info *pg) · line 78

Quoted 9 lines
tools/libxc/xc_vm_event.c +5 -0
@@ -68,6 +71,12 @@ static inline void *__map_domain_page_global(const struct page_info *pg)
     return page_to_virt(pg);
 }
 
+static inline void *__map_domain_pages_global(const struct page_info *pg,
+                                              unsigned int nr)
+{
+    return __map_domain_page_global(pg);
+}

Even if functionally this might be correct, you shouldn't do so, as you drop nr without any explanation.

Overall, considering the other comment you've got on the last patch, assuming contiguous physical pages here is likely the wrong route anyway.

Jan

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#11 lore raw

Re: [PATCH 3/4] x86: Add map_domain_pages_global

In reply to Petre Pircalabu

Hi,

Quoted 68 lines · Petre Pircalabu

Create a single mapping for multiple domain pages.

---
  tools/libxc/xc_vm_event.c     |  2 +-
  xen/arch/x86/domain_page.c    | 22 ++++++++++++++++++++++
  xen/include/xen/domain_page.h |  9 +++++++++
  3 files changed, 32 insertions(+), 1 deletion(-)
tools/libxc/xc_vm_event.c +1 -1
index dd34cec..de37ca5 100644
@@ -74,7 +74,7 @@ static int xc_vm_event_domctl(int type, unsigned int *param)
      {
      case XEN_VM_EVENT_TYPE_PAGING:
          *param = XEN_DOMCTL_VM_EVENT_OP_PAGING;
-	break;
+        break;
 
      case XEN_VM_EVENT_TYPE_MONITOR:
          *param = XEN_DOMCTL_VM_EVENT_OP_MONITOR;
xen/arch/x86/domain_page.c +22 -0
index 4a07cfb..0d23e52 100644
@@ -317,6 +317,28 @@ void *map_domain_page_global(mfn_t mfn)
      return vmap(&mfn, 1);
  }
 
+void *map_domain_pages_global(const mfn_t *mfn, unsigned int nr)
+{
+    ASSERT(!in_irq() &&
+           ((system_state >= SYS_STATE_boot &&
+             system_state < SYS_STATE_active) ||
+            local_irq_is_enabled()));
+
+    return vmap(mfn, nr);
+}
+
+void *__map_domain_pages_global(const struct page_info *pg, unsigned int nr)
+{
+    mfn_t mfn[nr];
+    int i;
+    struct page_info *cur_pg = (struct page_info *)&pg[0];
+
+    for (i = 0; i < nr; i++)
+        mfn[i] = page_to_mfn(cur_pg++);
+
+    return map_domain_pages_global(mfn, nr);
+}
+
  void unmap_domain_page_global(const void *ptr)
  {
      unsigned long va = (unsigned long)ptr;
xen/include/xen/domain_page.h +1 -0
index 32669a3..76422f9 100644
@@ -42,6 +42,7 @@ mfn_t domain_page_map_to_mfn(const void *va);
   * mappings can also be unmapped from any context.
   */
  void *map_domain_page_global(mfn_t mfn);
+void *map_domain_pages_global(const mfn_t *mfn, unsigned int nr);
  void unmap_domain_page_global(const void *va);
 
  #define __map_domain_page(pg)        map_domain_page(page_to_mfn(pg))

xen/include/xen/domain_page.h static inline void *__map_domain_page_global(const struct page_info *pg) · line 55

Quoted 4 lines
xen/include/xen/domain_page.h +1 -0
@@ -51,6 +52,8 @@ static inline void *__map_domain_page_global(const struct page_info *pg)
      return map_domain_page_global(page_to_mfn(pg));
  }
 
+void *__map_domain_pages_global(const struct page_info *pg, unsigned int nr);

This would require an implementation for Arm, assuming this is going ahead.

Quoted 8 lines
xen/include/xen/domain_page.h +1 -0
+
  #else /* !CONFIG_DOMAIN_PAGE */
 
  #define map_domain_page(mfn)                __mfn_to_virt(mfn_x(mfn))
@@ -68,6 +71,12 @@ static inline void *__map_domain_page_global(const struct page_info *pg)
      return page_to_virt(pg);
  }

xen/include/xen/domain_page.h static inline void *__map_domain_page_global(const struct page_info *pg) · line 82

Quoted 10 lines
xen/include/xen/domain_page.h +6 -0
 
+static inline void *__map_domain_pages_global(const struct page_info *pg,
+                                              unsigned int nr)
+{
+    return __map_domain_page_global(pg);
+}
+
  static inline void unmap_domain_page_global(const void *va) {};
 
  #endif /* !CONFIG_DOMAIN_PAGE */

Cheers,

Signature · 6 lines
Julien Grall

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#12 lore raw

PATCH 4/4 vm_event: Add support for multi-page ring buffer

In reply to Petre Pircalabu

In high throughput introspection scenarios where lots of monitor vm_events are generated, the ring buffer can fill up before the monitor application gets a chance to handle all the requests thus blocking other vcpus which will have to wait for a slot to become available.

This patch adds support for extending the ring buffer by allocating a number of pages from domheap and mapping them to the monitor application's domain using the foreignmemory_map_resource interface. Unlike the current implementation, the ring buffer pages are not part of the introspected DomU, so they will not be reclaimed when the monitor is disabled.

---
 tools/libxc/include/xenctrl.h       |   2 +
 tools/libxc/xc_monitor.c            |   7 +
 tools/libxc/xc_private.h            |   3 +
 tools/libxc/xc_vm_event.c           |  49 +++++++
 tools/tests/xen-access/xen-access.c |  33 +++--
 xen/arch/x86/domain_page.c          |   3 +-
 xen/arch/x86/mm.c                   |  14 ++
 xen/common/vm_event.c               | 258 +++++++++++++++++++++++++++---------
 xen/include/public/domctl.h         |   1 +
 xen/include/public/memory.h         |   1 +
 xen/include/xen/sched.h             |   5 +-
 xen/include/xen/vm_event.h          |   4 +
 12 files changed, 305 insertions(+), 75 deletions(-)
tools/libxc/include/xenctrl.h +2 -0
index bb75bcc..4f91ee9 100644
@@ -2005,6 +2005,8 @@ int xc_get_mem_access(xc_interface *xch, uint32_t domain_id,
  * Caller has to unmap this page when done.
  */
 void *xc_monitor_enable(xc_interface *xch, uint32_t domain_id, uint32_t *port);
+void *xc_monitor_enable_ex(xc_interface *xch, uint32_t domain_id,
+                           int order, uint32_t *port);
 int xc_monitor_disable(xc_interface *xch, uint32_t domain_id);
 int xc_monitor_resume(xc_interface *xch, uint32_t domain_id);
 /*
tools/libxc/xc_monitor.c +7 -0
index 15e6a0e..5188835 100644
@@ -28,6 +28,13 @@ void *xc_monitor_enable(xc_interface *xch, uint32_t domain_id, uint32_t *port)
                               port);
 }
 
+void *xc_monitor_enable_ex(xc_interface *xch, uint32_t domain_id, int order,
+                           uint32_t *port)
+{
+    return xc_vm_event_enable_ex(xch, domain_id, XEN_VM_EVENT_TYPE_MONITOR,
+                                 order, port);
+}
+
 int xc_monitor_disable(xc_interface *xch, uint32_t domain_id)
 {
     return xc_vm_event_control(xch, domain_id,
tools/libxc/xc_private.h +3 -0
index be22986..03d9460 100644
@@ -436,6 +436,9 @@ int xc_vm_event_control(xc_interface *xch, uint32_t domain_id, unsigned int op,
 void *xc_vm_event_enable(xc_interface *xch, uint32_t domain_id, int type,
                          uint32_t *port);
 
+void *xc_vm_event_enable_ex(xc_interface *xch, uint32_t domain_id, int type,
+                            int order, uint32_t *port);
+
 int do_dm_op(xc_interface *xch, uint32_t domid, unsigned int nr_bufs, ...);
 
 #endif /* __XC_PRIVATE_H__ */
tools/libxc/xc_vm_event.c +49 -0

61 lines not on this page · raw message

tools/tests/xen-access/xen-access.c +24 -9
index 6aaee16..f4c4eda 100644
@@ -68,7 +68,8 @@ typedef struct vm_event {
     int port;
     vm_event_back_ring_t back_ring;
     uint32_t evtchn_port;
-    void *ring_page;
+    void *ring_buffer;
+    unsigned int ring_page_count;
 } vm_event_t;
 
 typedef struct xenaccess {
@@ -136,8 +137,9 @@ int xenaccess_teardown(xc_interface *xch, xenaccess_t *xenaccess)
         return 0;
 
     /* Tear down domain xenaccess in Xen */
-    if ( xenaccess->vm_event.ring_page )
-        munmap(xenaccess->vm_event.ring_page, XC_PAGE_SIZE);
+    if ( xenaccess->vm_event.ring_buffer )
+        munmap(xenaccess->vm_event.ring_buffer,
+               xenaccess->vm_event.ring_page_count * XC_PAGE_SIZE );
 
     if ( mem_access_enable )
     {
@@ -210,12 +212,25 @@ xenaccess_t *xenaccess_init(xc_interface **xch_r, domid_t domain_id)
     /* Set domain id */
     xenaccess->vm_event.domain_id = domain_id;
 
-    /* Enable mem_access */
-    xenaccess->vm_event.ring_page =
+    /* Ring buffer page count */
+    xenaccess->vm_event.ring_page_count = 2;
+
+    xenaccess->vm_event.ring_buffer = xc_monitor_enable_ex(
+        xenaccess->xc_handle,
+        xenaccess->vm_event.domain_id,
+        xenaccess->vm_event.ring_page_count,
+        &xenaccess->vm_event.evtchn_port);
+
+    if (xenaccess->vm_event.ring_buffer == NULL && errno == EOPNOTSUPP)
+    {
+        xenaccess->vm_event.ring_page_count = 1;
+        xenaccess->vm_event.ring_buffer =
             xc_monitor_enable(xenaccess->xc_handle,
                               xenaccess->vm_event.domain_id,
                               &xenaccess->vm_event.evtchn_port);
-    if ( xenaccess->vm_event.ring_page == NULL )
+    }
+
+    if ( xenaccess->vm_event.ring_buffer == NULL )
     {
         switch ( errno ) {
             case EBUSY:
@@ -254,10 +269,10 @@ xenaccess_t *xenaccess_init(xc_interface **xch_r, domid_t domain_id)
     xenaccess->vm_event.port = rc;
 
     /* Initialise ring */
-    SHARED_RING_INIT((vm_event_sring_t *)xenaccess->vm_event.ring_page);
+    SHARED_RING_INIT((vm_event_sring_t *)xenaccess->vm_event.ring_buffer);
     BACK_RING_INIT(&xenaccess->vm_event.back_ring,
-                   (vm_event_sring_t *)xenaccess->vm_event.ring_page,
-                   XC_PAGE_SIZE);
+                   (vm_event_sring_t *)xenaccess->vm_event.ring_buffer,
+                   XC_PAGE_SIZE * xenaccess->vm_event.ring_page_count );
 
     /* Get max_gpfn */
     rc = xc_domain_maximum_gpfn(xenaccess->xc_handle,
xen/arch/x86/domain_page.c +1 -2
index 0d23e52..2a9cbf3 100644
@@ -331,10 +331,9 @@ void *__map_domain_pages_global(const struct page_info *pg, unsigned int nr)
 {
     mfn_t mfn[nr];
     int i;
-    struct page_info *cur_pg = (struct page_info *)&pg[0];
 
     for (i = 0; i < nr; i++)
-        mfn[i] = page_to_mfn(cur_pg++);
+        mfn[i] = page_to_mfn(pg++);
 
     return map_domain_pages_global(mfn, nr);
 }
xen/arch/x86/mm.c +14 -0
index baea2f5..bec09d0 100644
@@ -103,6 +103,7 @@
 #include <xen/efi.h>
 #include <xen/grant_table.h>
 #include <xen/hypercall.h>
+#include <xen/vm_event.h>
 #include <asm/paging.h>
 #include <asm/shadow.h>
 #include <asm/page.h>
@@ -4415,6 +4416,19 @@ int arch_acquire_resource(struct domain *d, unsigned int type,
     }
 #endif
 
+    case XENMEM_resource_vm_event:
+    {
+        rc = vm_event_get_ring_frames(d, id, frame, nr_frames, mfn_list);
+        if ( rc )
+            break;
+        /*
+         * The frames will have been assigned to the domain that created
+         * the ioreq server.
+         */
+        *flags |= XENMEM_rsrc_acq_caller_owned;
+        break;
+    }
+
     default:
         rc = -EOPNOTSUPP;
         break;
xen/common/vm_event.c +196 -62

363 lines not on this page · raw message

xen/include/public/domctl.h +1 -0
index ac4ced2..066d4da 100644
@@ -772,6 +772,7 @@ struct xen_domctl_gdbsx_domstatus {
 #define XEN_VM_EVENT_ENABLE               0
 #define XEN_VM_EVENT_DISABLE              1
 #define XEN_VM_EVENT_RESUME               2
+#define XEN_VM_EVENT_GET_PORT             3
 
 /*
  * Domain memory paging
xen/include/public/memory.h +1 -0
index 8fc27ce..09400f5 100644
@@ -612,6 +612,7 @@ struct xen_mem_acquire_resource {
 
 #define XENMEM_resource_ioreq_server 0
 #define XENMEM_resource_grant_table 1
+#define XENMEM_resource_vm_event 2
 
     /*
      * IN - a type-specific resource identifier, which must be zero
xen/include/xen/sched.h +3 -2
index 0ba80cb..be01f93 100644
@@ -286,9 +286,10 @@ struct vm_event_domain
     /* The ring has 64 entries */
     unsigned char foreign_producers;
     unsigned char target_producers;
-    /* shared ring page */
-    void *ring_page;
+    /* shared ring pages */
+    void *ring_buffer;
     struct page_info *ring_pg_struct;
+    unsigned int ring_order;
     /* front-end ring */
     vm_event_front_ring_t front_ring;
     /* event channel port (vcpu0 only) */
xen/include/xen/vm_event.h +4 -0
index 2ff6e1c..d9c5e93 100644
@@ -80,6 +80,10 @@ void vm_event_set_registers(struct vcpu *v, vm_event_response_t *rsp);
 
 void vm_event_monitor_next_interrupt(struct vcpu *v);
 
+int vm_event_get_ring_frames(struct domain *d, unsigned int id,
+                             unsigned long frame, unsigned int nr_frames,
+                             xen_pfn_t mfn_list[]);
+
 #endif /* __VM_EVENT_H__ */
 
 /*
Signature · 7 lines
2.7.4


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#13 lore raw

Re: [PATCH 4/4] vm_event: Add support for multi-page ring buffer

Quoted 6 lines · Petre Pircalabu

In high throughput introspection scenarios where lots of monitor vm_events are generated, the ring buffer can fill up before the monitor application gets a chance to handle all the requests thus blocking other vcpus which will have to wait for a slot to become available.

Quoted 8 lines

This patch adds support for extending the ring buffer by allocating a number of pages from domheap and mapping them to the monitor application's domain using the foreignmemory_map_resource interface. Unlike the current implementation, the ring buffer pages are not part of the introspected DomU, so they will not be reclaimed when the monitor is disabled.

Thanks for this addition, it has been on the TODO for a long while now. Could you also please push the patches as a git branch somewhere?

Quoted 20 lines
---
 tools/libxc/include/xenctrl.h       |   2 +
 tools/libxc/xc_monitor.c            |   7 +
 tools/libxc/xc_private.h            |   3 +
 tools/libxc/xc_vm_event.c           |  49 +++++++
 tools/tests/xen-access/xen-access.c |  33 +++--
 xen/arch/x86/domain_page.c          |   3 +-
 xen/arch/x86/mm.c                   |  14 ++
 xen/common/vm_event.c               | 258 +++++++++++++++++++++++++++---------
 xen/include/public/domctl.h         |   1 +
 xen/include/public/memory.h         |   1 +
 xen/include/xen/sched.h             |   5 +-
 xen/include/xen/vm_event.h          |   4 +
 12 files changed, 305 insertions(+), 75 deletions(-)
tools/libxc/include/xenctrl.h +0 -0
index bb75bcc..4f91ee9 100644

tools/libxc/include/xenctrl.h int xc_get_mem_access(xc_interface *xch, uint32_t domain_id, · line 2009

Quoted 5 lines
tools/libxc/include/xenctrl.h +2 -0
@@ -2005,6 +2005,8 @@ int xc_get_mem_access(xc_interface *xch, uint32_t domain_id,
  * Caller has to unmap this page when done.
  */
 void *xc_monitor_enable(xc_interface *xch, uint32_t domain_id, uint32_t *port);
+void *xc_monitor_enable_ex(xc_interface *xch, uint32_t domain_id,
+                           int order, uint32_t *port);

I don't think there is value in keeping both version of the API around. As libxc is not considered to be a stable API extending the existing API would be the route I would prefer. Also, perhaps instead of "order" name the input variable "page_count" to make it more clear what it's about.

Quoted 134 lines
tools/libxc/include/xenctrl.h +0 -0
 int xc_monitor_disable(xc_interface *xch, uint32_t domain_id);
 int xc_monitor_resume(xc_interface *xch, uint32_t domain_id);
 /*
tools/libxc/xc_monitor.c +7 -0
index 15e6a0e..5188835 100644
@@ -28,6 +28,13 @@ void *xc_monitor_enable(xc_interface *xch, uint32_t domain_id, uint32_t *port)
                               port);
 }
 
+void *xc_monitor_enable_ex(xc_interface *xch, uint32_t domain_id, int order,
+                           uint32_t *port)
+{
+    return xc_vm_event_enable_ex(xch, domain_id, XEN_VM_EVENT_TYPE_MONITOR,
+                                 order, port);
+}
+
 int xc_monitor_disable(xc_interface *xch, uint32_t domain_id)
 {
     return xc_vm_event_control(xch, domain_id,
tools/libxc/xc_private.h +3 -0
index be22986..03d9460 100644
@@ -436,6 +436,9 @@ int xc_vm_event_control(xc_interface *xch, uint32_t domain_id, unsigned int op,
 void *xc_vm_event_enable(xc_interface *xch, uint32_t domain_id, int type,
                          uint32_t *port);
 
+void *xc_vm_event_enable_ex(xc_interface *xch, uint32_t domain_id, int type,
+                            int order, uint32_t *port);
+
 int do_dm_op(xc_interface *xch, uint32_t domid, unsigned int nr_bufs, ...);
 
 #endif /* __XC_PRIVATE_H__ */
tools/libxc/xc_vm_event.c +49 -0
index de37ca5..216bbe2 100644
@@ -21,6 +21,7 @@
  */
 
 #include "xc_private.h"
+#include "xenforeignmemory.h"
 
 int xc_vm_event_control(xc_interface *xch, uint32_t domain_id, unsigned int op,
                         unsigned int mode, uint32_t *port)
@@ -184,6 +185,54 @@ void *xc_vm_event_enable(xc_interface *xch, uint32_t domain_id, int type,
     return ring_page;
 }
 
+void *xc_vm_event_enable_ex(xc_interface *xch, uint32_t domain_id, int type,
+                            int order, uint32_t *port)
+{
+    xenforeignmemory_resource_handle *fres = NULL;
+    int saved_errno;
+    void *ring_buffer = NULL;
+
+    if ( !port )
+    {
+        errno = EINVAL;
+        return NULL;
+    }
+
+    /* Pause the domain for ring page setup */
+    if ( xc_domain_pause(xch, domain_id) )
+    {
+        PERROR("Unable to pause domain\n");
+        return NULL;
+    }
+
+    fres = xenforeignmemory_map_resource(xch->fmem, domain_id,
+                                         XENMEM_resource_vm_event, type, 0,
+                                         order, &ring_buffer,
+                                         PROT_READ | PROT_WRITE, 0);
+    if ( !fres )
+    {
+        PERROR("Unable to map vm_event ring pages resource\n");
+        goto out;
+    }
+
+    if ( xc_vm_event_control(xch, domain_id, XEN_VM_EVENT_GET_PORT, type, port) )
+        PERROR("Unable to get vm_event channel port\n");
+
+out:
+    saved_errno = errno;
+    if ( xc_domain_unpause(xch, domain_id) != 0 )
+    {
+        if (fres)
+            saved_errno = errno;
+        PERROR("Unable to unpause domain");
+    }
+
+    free(fres);
+    errno = saved_errno;
+    return ring_buffer;
+}
+
+
 /*
  * Local variables:
  * mode: C
tools/tests/xen-access/xen-access.c +5 -5
index 6aaee16..f4c4eda 100644
@@ -68,7 +68,8 @@ typedef struct vm_event {
     int port;
     vm_event_back_ring_t back_ring;
     uint32_t evtchn_port;
-    void *ring_page;
+    void *ring_buffer;
+    unsigned int ring_page_count;
 } vm_event_t;
 
 typedef struct xenaccess {
@@ -136,8 +137,9 @@ int xenaccess_teardown(xc_interface *xch, xenaccess_t *xenaccess)
         return 0;
 
     /* Tear down domain xenaccess in Xen */
-    if ( xenaccess->vm_event.ring_page )
-        munmap(xenaccess->vm_event.ring_page, XC_PAGE_SIZE);
+    if ( xenaccess->vm_event.ring_buffer )
+        munmap(xenaccess->vm_event.ring_buffer,
+               xenaccess->vm_event.ring_page_count * XC_PAGE_SIZE );
 
     if ( mem_access_enable )
     {
@@ -210,12 +212,25 @@ xenaccess_t *xenaccess_init(xc_interface **xch_r, domid_t domain_id)
     /* Set domain id */
     xenaccess->vm_event.domain_id = domain_id;
 
-    /* Enable mem_access */
-    xenaccess->vm_event.ring_page =

tools/tests/xen-access/xen-access.c xenaccess_t *xenaccess_init(xc_interface **xch_r, domid_t domain_id) · line 224

Quoted 10 lines
tools/tests/xen-access/xen-access.c +10 -0
+    /* Ring buffer page count */
+    xenaccess->vm_event.ring_page_count = 2;
+
+    xenaccess->vm_event.ring_buffer = xc_monitor_enable_ex(
+        xenaccess->xc_handle,
+        xenaccess->vm_event.domain_id,
+        xenaccess->vm_event.ring_page_count,
+        &xenaccess->vm_event.evtchn_port);
+
+    if (xenaccess->vm_event.ring_buffer == NULL && errno == EOPNOTSUPP)

How would this situation ever arise? If there is a chance that you can't setup multi-page rings, perhaps adding a hypercall that would tell the user how many pages are max available for the ring is the better route. This just seems like guessing right now.

tools/tests/xen-access/xen-access.c xenaccess_t *xenaccess_init(xc_interface **xch_r, domid_t domain_id) · line 230

Quoted 6 lines
tools/tests/xen-access/xen-access.c +3 -0
+    {
+        xenaccess->vm_event.ring_page_count = 1;
+        xenaccess->vm_event.ring_buffer =
             xc_monitor_enable(xenaccess->xc_handle,
                               xenaccess->vm_event.domain_id,
                               &xenaccess->vm_event.evtchn_port);

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#14 lore raw

Re: [PATCH 4/4] vm_event: Add support for multi-page ring buffer

Quoted 21 lines · Tamas K Lengyel
Quoted 19 lines · Petre Pircalabu

In high throughput introspection scenarios where lots of monitor vm_events are generated, the ring buffer can fill up before the monitor
application gets a chance to handle all the requests thus blocking other vcpus which will have to wait for a slot to become available.

This patch adds support for extending the ring buffer by allocating a
number of pages from domheap and mapping them to the monitor application's domain using the foreignmemory_map_resource interface.
Unlike the current implementation, the ring buffer pages are not part of
the introspected DomU, so they will not be reclaimed when the monitor is
disabled.

Quoted 4 lines
Quoted 1 line

Thanks for this addition, it has been on the TODO for a long while now. Could you also please push the patches as a git branch somewhere?

I've pushed it to my github repository (branch :
multi_page_ring_buffer/devel_new)
https://github.com/petrepircalabu/xen/tree/multi_page_ring_buffer/devel
_new

Quoted 14 lines
Quoted 14 lines

---
 tools/libxc/include/xenctrl.h       |   2 + tools/libxc/xc_monitor.c            |   7 + tools/libxc/xc_private.h            |   3 + tools/libxc/xc_vm_event.c           |  49 +++++++
+        xenaccess->vm_event.domain_id,
+        xenaccess->vm_event.ring_page_count,
+        &xenaccess->vm_event.evtchn_port);
+
+    if (xenaccess->vm_event.ring_buffer == NULL && errno == EOPNOTSUPP)

Quoted 5 lines

How would this situation ever arise? If there is a chance that you can't setup multi-page rings, perhaps adding a hypercall that would tell the user how many pages are max available for the ring is the better route. This just seems like guessing right now.

The multi page ring buffer is mapped using xenforeignmemory_map_resource which uses IOCTL_PRIVCMD_MMAP_RESOURCE. This ioctl was added in kernel 4.18.1, which is a relatively new kernel. If the monitor domain doesnt't recognize this hypercall it sets errno to EOPNOTSUPP.

Quoted 7 lines
Quoted 7 lines

+    {
+        xenaccess->vm_event.ring_page_count = 1;
+        xenaccess->vm_event.ring_buffer = xc_monitor_enable(xenaccess->xc_handle, xenaccess->vm_event.domain_id, &xenaccess->vm_event.evtchn_port);

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#15 lore raw

Re: [PATCH 4/4] vm_event: Add support for multi-page ring buffer

In reply to Petre Pircalabu
Quoted 5 lines · Petre Pircalabu

In high throughput introspection scenarios where lots of monitor vm_events are generated, the ring buffer can fill up before the monitor application gets a chance to handle all the requests thus blocking other vcpus which will have to wait for a slot to become available.

Quoted 8 lines

This patch adds support for extending the ring buffer by allocating a number of pages from domheap and mapping them to the monitor application's domain using the foreignmemory_map_resource interface. Unlike the current implementation, the ring buffer pages are not part of the introspected DomU, so they will not be reclaimed when the monitor is disabled.

What about the slotted format for the synchronous events?  While this is fine for the async bits, I don't think we want to end up changing the mapping API twice.

Simply increasing the size of the ring puts more pressure on the

xen/arch/x86/domain_page.c void *__map_domain_pages_global(const struct page_info *pg, unsigned int nr) · line 336

Quoted 13 lines
xen/arch/x86/domain_page.c +1 -2
index 0d23e52..2a9cbf3 100644
@@ -331,10 +331,9 @@ void *__map_domain_pages_global(const struct page_info *pg, unsigned int nr)
 {
     mfn_t mfn[nr];
     int i;
-    struct page_info *cur_pg = (struct page_info *)&pg[0];
 
     for (i = 0; i < nr; i++)
-        mfn[i] = page_to_mfn(cur_pg++);
+        mfn[i] = page_to_mfn(pg++);

This hunk looks like it should be in the previous patch?  That said...

Quoted 12 lines
xen/arch/x86/domain_page.c +0 -0
     return map_domain_pages_global(mfn, nr);
 }
xen/common/vm_event.c +1 -0
index 4793aac..faece3c 100644
@@ -39,16 +39,66 @@
 #define vm_event_ring_lock(_ved)       spin_lock(&(_ved)->ring_lock)
 #define vm_event_ring_unlock(_ved)     spin_unlock(&(_ved)->ring_lock)
 
+#define XEN_VM_EVENT_ALLOC_FROM_DOMHEAP 0xFFFFFFFF

xen/common/vm_event.c line 52

Quoted 10 lines
xen/common/vm_event.c +10 -0
+
+static int vm_event_alloc_ring(struct domain *d, struct vm_event_domain *ved)
+{
+    struct page_info *page;
+    void *va = NULL;
+    int i, rc = -ENOMEM;
+
+    page = alloc_domheap_pages(d, ved->ring_order, MEMF_no_refcount);
+    if ( !page )
+        return -ENOMEM;

... what is wrong with vzalloc()?

You don't want to be making a ring_order allocation, especially as the order grows.  All you need are some mappings which are virtually contiguous, not physically contiguous.

~Andrew

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#16 lore raw

Re: [PATCH 4/4] vm_event: Add support for multi-page ring buffer

Quoted 18 lines · Andrew Cooper
Quoted 17 lines · Petre Pircalabu

In high throughput introspection scenarios where lots of monitor vm_events are generated, the ring buffer can fill up before the monitor
application gets a chance to handle all the requests thus blocking other vcpus which will have to wait for a slot to become available.

This patch adds support for extending the ring buffer by allocating a
number of pages from domheap and mapping them to the monitor application's domain using the foreignmemory_map_resource interface.
Unlike the current implementation, the ring buffer pages are not part of
the introspected DomU, so they will not be reclaimed when the monitor is
disabled.

Quoted 8 lines
Quoted 1 line

What about the slotted format for the synchronous events? While this is
fine for the async bits, I don't think we want to end up changing the mapping API twice.

Simply increasing the size of the ring puts more pressure on the

I'm currently investigating this approach and I will send an inplementation proposal with the next version of the patch.

Quoted 13 lines
Quoted 13 lines
xen/arch/x86/domain_page.c +0 -1
index 0d23e52..2a9cbf3 100644
@@ -331,10 +331,9 @@ void *__map_domain_pages_global(const struct
 page_info *pg, unsigned int nr)
 {
     mfn_t mfn[nr];
     int i;
-    struct page_info *cur_pg = (struct page_info *)&pg[0];
 
Quoted 6 lines

xen/arch/x86/domain_page.c void *__map_domain_pages_global(const struct page_info *pg, unsigned int nr) · line 336

Quoted 3 lines
xen/arch/x86/domain_page.c +1 -1
     for (i = 0; i < nr; i++)
-        mfn[i] = page_to_mfn(cur_pg++);
+        mfn[i] = page_to_mfn(pg++);

This hunk looks like it should be in the previous patch? That said...

Yep. I've completely missed it. This piece of code will be removed along with the map_domain_pages_global patch.

Quoted 30 lines
Quoted 17 lines
xen/arch/x86/domain_page.c +0 -0
     return map_domain_pages_global(mfn, nr);
 }
xen/common/vm_event.c +3 -0
index 4793aac..faece3c 100644
@@ -39,16 +39,66 @@
 #define vm_event_ring_lock(_ved)       spin_lock(&(_ved)-
 >ring_lock)
 #define vm_event_ring_unlock(_ved)     spin_unlock(&(_ved)-
 >ring_lock)
 
+#define XEN_VM_EVENT_ALLOC_FROM_DOMHEAP 0xFFFFFFFF
+
+static int vm_event_alloc_ring(struct domain *d, struct

xen/common/vm_event.c line 56

Quoted 10 lines
xen/common/vm_event.c +8 -0
 vm_event_domain *ved)
+{
+    struct page_info *page;
+    void *va = NULL;
+    int i, rc = -ENOMEM;
+
+    page = alloc_domheap_pages(d, ved->ring_order,
 MEMF_no_refcount);
+    if ( !page )
+        return -ENOMEM;

... what is wrong with vzalloc()?

Quoted 4 lines

You don't want to be making a ring_order allocation, especially as the
order grows. All you need are some mappings which are virtually contiguous, not physically contiguous.

Unfortunately, vzalloc doesn't work here (the acquire_resource succeeds, but the subsequent mapping fails:
(XEN) mm.c:1024:d0v5 pg_owner d0 l1e_owner d0, but real_pg_owner d-1 (XEN) mm.c:1100:d0v5 Error getting mfn dd24d (pfn ffffffffffffffff) from L1 entry 80000000dd24d227 for l1e_owner d0, pg_owner d0 (XEN) mm.c:1024:d0v5 pg_owner d0 l1e_owner d0, but real_pg_owner d-1 (XEN) mm.c:1100:d0v5 Error getting mfn dd24c (pfn ffffffffffffffff) from L1 entry 80000000dd24c227 for l1e_owner d0, pg_owner d0

However, allocating each page with alloc_domheap_page and then mapping them using vmap does the trick.
Until the next version is ready (slotted format for synchronous events)
I have pushed an intermediate version which addresses the issues
signaled by you an Jan to my github fork of the xen repository:
https://github.com/petrepircalabu/xen/tree/multi_page_ring_buffer/devel
_new

Quoted 1 line

~Andrew

Many thanks for your support,
Petre

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#17 lore raw

Re: [PATCH 4/4] vm_event: Add support for multi-page ring buffer

In reply to Petre Pircalabu
Quoted 5 lines
Quoted 1 line
Quoted 1 line · On 13.09.18 at 17:02, <[email protected]> wrote:
xen/arch/x86/domain_page.c +0 -0
@@ -331,10 +331,9 @@ void *__map_domain_pages_global(const struct page_info *pg, unsigned int nr)
 {

xen/arch/x86/domain_page.c void *__map_domain_pages_global(const struct page_info *pg, unsigned int nr) · line 339

Quoted 10 lines
xen/arch/x86/domain_page.c +1 -2
     mfn_t mfn[nr];
     int i;
-    struct page_info *cur_pg = (struct page_info *)&pg[0];
 
     for (i = 0; i < nr; i++)
-        mfn[i] = page_to_mfn(cur_pg++);
+        mfn[i] = page_to_mfn(pg++);
 
     return map_domain_pages_global(mfn, nr);
 }

Please could you avoid having such random unrelated changes in your patches?

Quoted 6 lines
xen/arch/x86/domain_page.c +2 -0
@@ -4415,6 +4416,19 @@ int arch_acquire_resource(struct domain *d, unsigned int type,
     }
 #endif
 
+    case XENMEM_resource_vm_event:
+    {

xen/arch/x86/domain_page.c int arch_acquire_resource(struct domain *d, unsigned int type, · line 4430

Quoted 10 lines
xen/arch/x86/domain_page.c +10 -0
+        rc = vm_event_get_ring_frames(d, id, frame, nr_frames, mfn_list);
+        if ( rc )
+            break;
+        /*
+         * The frames will have been assigned to the domain that created
+         * the ioreq server.
+         */
+        *flags |= XENMEM_rsrc_acq_caller_owned;
+        break;
+    }

Isn't it wrong to assume that the monitor application will run in the same domain as the / one ioreq server?

xen/common/vm_event.c line 42

Quoted 7 lines
xen/common/vm_event.c +1 -0
@@ -39,16 +39,66 @@
 #define vm_event_ring_lock(_ved)       spin_lock(&(_ved)->ring_lock)
 #define vm_event_ring_unlock(_ved)     spin_unlock(&(_ved)->ring_lock)
 
+#define XEN_VM_EVENT_ALLOC_FROM_DOMHEAP 0xFFFFFFFF

Note this constant's type vs ...

xen/common/vm_event.c line 96

Quoted 5 lines
xen/common/vm_event.c +1 -1
 static int vm_event_enable(
     struct domain *d,
-    struct xen_domctl_vm_event_op *vec,
     struct vm_event_domain **ved,
+    unsigned long param,

... the function parameter type here. I don't see why this can't be "unsigned int".

xen/common/vm_event.c static int vm_event_enable( · line 154

Quoted 6 lines
xen/common/vm_event.c +1 -1
@@ -88,12 +151,12 @@ static int vm_event_enable(
     if ( rc < 0 )
         goto err;
 
-    (*ved)->xen_port = vec->port = rc;
+    (*ved)->xen_port =  rc;

Yet another unrelated change? It looks to be replaced by code further down (in the callers), but it's not clear to me why the change is needed (here). Perhaps worth splitting up the patch a little?

Jan

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#18 lore raw

Re: [PATCH 4/4] vm_event: Add support for multi-page ring buffer

Quoted 15 lines · Jan Beulich
Quoted 15 lines
Quoted 1 line
Quoted 1 line · On 13.09.18 at 17:02, <[email protected]> wrote:
xen/arch/x86/domain_page.c +1 -2
@@ -331,10 +331,9 @@ void *__map_domain_pages_global(const struct
 page_info *pg, unsigned int nr)
 {
     mfn_t mfn[nr];
     int i;
-    struct page_info *cur_pg = (struct page_info *)&pg[0];
 
     for (i = 0; i < nr; i++)
-        mfn[i] = page_to_mfn(cur_pg++);
+        mfn[i] = page_to_mfn(pg++);
 
Quoted 6 lines
Quoted 2 lines
xen/arch/x86/domain_page.c +0 -0
     return map_domain_pages_global(mfn, nr);

}

Please could you avoid having such random unrelated changes in your patches?

Thank-uy very much for noticing it, I've completely missed it. However I drop the map_domain_pages functionality altoghether as Andrew pointed out is not necessary to have a physically contiguous area.

Quoted 15 lines
Quoted 15 lines
xen/arch/x86/domain_page.c +7 -0
@@ -4415,6 +4416,19 @@ int arch_acquire_resource(struct domain *d,
 unsigned int type,
     }
 #endif
 
+    case XENMEM_resource_vm_event:
+    {
+        rc = vm_event_get_ring_frames(d, id, frame, nr_frames,
 mfn_list);
+        if ( rc )
+            break;
+        /*
+         * The frames will have been assigned to the domain that

created
+ * the ioreq server.

Quoted 8 lines

xen/arch/x86/domain_page.c

Quoted 4 lines
xen/arch/x86/domain_page.c +4 -0
+         */
+        *flags |= XENMEM_rsrc_acq_caller_owned;
+        break;
+    }

Isn't it wrong to assume that the monitor application will run in the same
domain as the / one ioreq server?

I will remove the ioreq server comment as it's misleading. This call is not using a ioreq server, but only allocates the memory for the vm_event's ring buffer.

Quoted 12 lines

xen/common/vm_event.c line 44

Quoted 9 lines
xen/common/vm_event.c +1 -0
@@ -39,16 +39,66 @@
 #define vm_event_ring_lock(_ved)       spin_lock(&(_ved)-
 >ring_lock)
 #define vm_event_ring_unlock(_ved)     spin_unlock(&(_ved)-
 >ring_lock)
 
+#define XEN_VM_EVENT_ALLOC_FROM_DOMHEAP 0xFFFFFFFF

Note this constant's type vs ...

Quoted 8 lines

xen/common/vm_event.c line 96

Quoted 5 lines
xen/common/vm_event.c +1 -1
 static int vm_event_enable(
     struct domain *d,
-    struct xen_domctl_vm_event_op *vec,
     struct vm_event_domain **ved,
+    unsigned long param,

... the function parameter type here. I don't see why this can't be "unsigned int".

Fixed.

Quoted 5 lines
Quoted 5 lines
xen/common/vm_event.c +0 -0
@@ -88,12 +151,12 @@ static int vm_event_enable(
     if ( rc < 0 )
         goto err;
 
Quoted 7 lines

xen/common/vm_event.c static int vm_event_enable( · line 154

Quoted 2 lines
xen/common/vm_event.c +1 -1
-    (*ved)->xen_port = vec->port = rc;
+    (*ved)->xen_port =  rc;

Yet another unrelated change? It looks to be replaced by code further down (in the callers), but it's not clear to me why the change is needed
(here). Perhaps worth splitting up the patch a little?

vm_event_enabled can be called from both vm_event_domctl and vm_event_get_ring_frames (acquire_resource call path). I have removed the vec parameter (struct xen_domctl_vm_event_op) to make the function caller agnostic.

Many thank for your comments,
Petre

Quoted 4 lines

Jan

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel