Skip to contentSkip to conversation

xenguest optimisations

PATCH Superseded Frediano Ziglio 5 participants 36 messages Jun 3 – Jun 9

Subsystems (3)

v4 of 10 Superseded Latest is v10 →

lore
Share & tools
lore
#0 lore raw

v4 cover xenguest optimisations

From: Frediano Ziglio <[email protected]>

Reduce some number of parts passed to writev. Avoid possible allocation sending data with writev. Reduce number of allocations sending memory state.

Implement and use new Xen and Linux kernel ABI to copy foreign memory. This new ABI allows to replace theexpensive map/copy/unmap sequence with a single call.

Changes since v1:
- add commit to cache up to 4 pages in hypercall;
- add other 2 commits reducing chunks passed to write/writev.

Changes since v2:
- update patches commit prefixes;
- add other 2 optisations.

Changes since v3:
- address some comments;
- add patches for foreign copy optimisation.

Edwin Török (3):
libs/guest: allocate various migration arrays just once libs/call: cache up to 4 pages in hypercall bounce buffers
PoC: libs/guest: use foreign copy during migration

Frediano Ziglio (12):
libs/guest: Reduce number of parts in write_split_record libs/guest: Reduce number of I/O vectors in write_batch libs/guest: Reduce number of I/O vectors in write_batch libs/guest: Use a single write_exact in write_headers libs/guest: avoids using 2 indexes
libs/guest: fill directly iov structure
libs/ctrl: Allows writev_exact to change iov array libs/guest: add xg_foreignmemory_copy_{from,to}
xen: implement new foreign copy hypercall
privcmd: Add definition for new Linux privcmd to access new Xen
hypercall
libs/guest: use new hypercall if available libs/guest: finalize PoC

 public-abi/memory.h                   |  40 ++++-
 tools/include/xen-sys/Linux/privcmd.h |  10 ++
 tools/libs/call/buffer.c              |  28 ++--
 tools/libs/call/core.c                |   3 +-
 tools/libs/call/private.h             |   8 +-
 tools/libs/ctrl/xc_private.c          |  26 +--
 tools/libs/ctrl/xc_private.h          |   2 +-
 tools/libs/guest/xg_sr_common.c       |  95 ++++++++++-
 tools/libs/guest/xg_sr_common.h       |  21 +++
 tools/libs/guest/xg_sr_restore.c      | 100 ++++++------
 tools/libs/guest/xg_sr_save.c         | 225 +++++++++++---------------
 xen/common/memory.c                   | 133 +++++++++++++++
 xen/include/public/memory.h           |  40 ++++-
 13 files changed, 512 insertions(+), 219 deletions(-)
Signature · 1 line
2.54.0
#1 lore raw

PATCH v4 1/16 libs/guest: Reduce number of parts in write_split_record

From: Frediano Ziglio <[email protected]>

Small optimization.
There's no much sense to split the header in 2 pieces, it will just take more time and space to reassemble them in the final buffer.
This also avoids truncating combined_length to 32 bit in case of 64 bit machines potentially avoiding following record_length check (it could still be truncated writing it in xc_sr_rhdr structure but the following check will catch it).
The function become more coherent with following read_record function.

--
Changes since v2:
- change prefix in subject.

Changes since v3:
- clarify commit message.

---
 tools/libs/guest/xg_sr_common.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
tools/libs/guest/xg_sr_common.c +3 -3
index 7ccdc3b1f6..86c148c62f 100644
@@ -59,11 +59,11 @@ int write_split_record(struct xc_sr_context *ctx, struct xc_sr_record *rec,
     static const char zeroes[(1u << REC_ALIGN_ORDER) - 1] = { 0 };
 
     xc_interface *xch = ctx->xch;
-    typeof(rec->length) combined_length = rec->length + sz;
+    size_t combined_length = rec->length + sz;
     size_t record_length = ROUNDUP(combined_length, REC_ALIGN_ORDER);
+    struct xc_sr_rhdr rhdr = { rec->type, combined_length };
     struct iovec parts[] = {
-        { &rec->type,       sizeof(rec->type) },
-        { &combined_length, sizeof(combined_length) },
+        { &rhdr,            sizeof(rhdr) },
         { rec->data,        rec->length },
         { buf,              sz },
         { (void *)zeroes,   record_length - combined_length },
Signature · 1 line
2.54.0
Reviewed-by on v4 1/16
#2 lore raw

Re: [PATCH v4 01/16] libs/guest: Reduce number of parts in write_split_record

Quoted 8 lines · Frediano Ziglio

From: Frediano Ziglio <[email protected]>

Small optimization.
There's no much sense to split the header in 2 pieces, it will just take more time and space to reassemble them in the final buffer.
This also avoids truncating combined_length to 32 bit in case of 64 bit machines potentially avoiding following record_length check

I'm not sure I understand the sentence above: rec->length is a fixed width type uint32_t, and hence it will always be 32bit, regardless of whether it's build in 32 or 64 bit modes.

I do get the truncation part, and that using size_t is indeed better.

Quoted 6 lines

(it could still be truncated writing it in xc_sr_rhdr structure but the following check will catch it).
The function become more coherent with following read_record function.

With the commit message possibly clarified.

Thanks, Roger.

#3 lore raw

Re: [PATCH v4 01/16] libs/guest: Reduce number of parts in write_split_record

Quoted 11 lines · Roger Pau Monné
Quoted 8 lines · Frediano Ziglio

From: Frediano Ziglio <[email protected]>

Small optimization.
There's no much sense to split the header in 2 pieces, it will just take more time and space to reassemble them in the final buffer.
This also avoids truncating combined_length to 32 bit in case of 64 bit machines potentially avoiding following record_length check

Quoted 6 lines

I'm not sure I understand the sentence above: rec->length is a fixed width type uint32_t, and hence it will always be 32bit, regardless of whether it's build in 32 or 64 bit modes.

I do get the truncation part, and that using size_t is indeed better.

I improved the commit message as

This also avoids truncating combined_length to 32 bit in case of 64 bit machines potentially avoiding following record_length check (it could still be truncated writing it in xc_sr_rhdr structure but the following check will catch it).

Quoted 5 lines
Quoted 5 lines

(it could still be truncated writing it in xc_sr_rhdr structure but the following check will catch it).
The function become more coherent with following read_record function.

Quoted 7 lines
Quoted 1 line

With the commit message possibly clarified.

Thanks, Roger.

Frediano

#4 lore raw

PATCH v4 2/16 libs/guest: Reduce number of I/O vectors in write_batch

In reply to Frediano Ziglio

From: Frediano Ziglio <[email protected]>

Small optimization.
Reduce number of pieces passed to writev.

--
Changes since v2:
- change prefix in subject.

---
 tools/libs/guest/xg_sr_save.c | 35 ++++++++++++++++-------------------
 1 file changed, 16 insertions(+), 19 deletions(-)
tools/libs/guest/xg_sr_save.c +16 -19
index 3b2c5222e4..1700d81905 100644
@@ -97,9 +97,12 @@ static int write_batch(struct xc_sr_context *ctx)
     void *page, *orig_page;
     uint64_t *rec_pfns = NULL;
     struct iovec *iov = NULL; int iovcnt = 0;
-    struct xc_sr_rec_page_data_header hdr = { 0 };
-    struct xc_sr_record rec = {
-        .type = REC_TYPE_PAGE_DATA,
+    struct {
+        struct xc_sr_rhdr rec;
+        struct xc_sr_rec_page_data_header page_data;
+    } hdrs = {
+        { .type = REC_TYPE_PAGE_DATA },
+        { 0 },
     };
 
     assert(nr_pfns != 0);
@@ -115,7 +118,7 @@ static int write_batch(struct xc_sr_context *ctx)
     /* Pointers to locally allocated pages.  Need freeing. */
     local_pages = calloc(nr_pfns, sizeof(*local_pages));
     /* iovec[] for writev(). */
-    iov = malloc((nr_pfns + 4) * sizeof(*iov));
+    iov = malloc((nr_pfns + 2) * sizeof(*iov));
 
     if ( !mfns || !types || !errors || !guest_data || !local_pages || !iov )
     {
@@ -216,28 +219,22 @@ static int write_batch(struct xc_sr_context *ctx)
         goto err;
     }
 
-    hdr.count = nr_pfns;
+    hdrs.rec.length = sizeof(hdrs.page_data);
+    hdrs.rec.length += nr_pfns * sizeof(*rec_pfns);
+    hdrs.rec.length += nr_pages * PAGE_SIZE;
 
-    rec.length = sizeof(hdr);
-    rec.length += nr_pfns * sizeof(*rec_pfns);
-    rec.length += nr_pages * PAGE_SIZE;
+    hdrs.page_data.count = nr_pfns;
 
     for ( i = 0; i < nr_pfns; ++i )
         rec_pfns[i] = ((uint64_t)(types[i]) << 32) | ctx->save.batch_pfns[i];
 
-    iov[0].iov_base = &rec.type;
-    iov[0].iov_len = sizeof(rec.type);
+    iov[0].iov_base = &hdrs;
+    iov[0].iov_len = sizeof(hdrs);
 
-    iov[1].iov_base = &rec.length;
-    iov[1].iov_len = sizeof(rec.length);
+    iov[1].iov_base = rec_pfns;
+    iov[1].iov_len = nr_pfns * sizeof(*rec_pfns);
 
-    iov[2].iov_base = &hdr;
-    iov[2].iov_len = sizeof(hdr);
-
-    iov[3].iov_base = rec_pfns;
-    iov[3].iov_len = nr_pfns * sizeof(*rec_pfns);
-
-    iovcnt = 4;
+    iovcnt = 2;
 
     if ( nr_pages )
     {
Signature · 1 line
2.54.0
Reviewed-by on v4 2/16
#5 lore raw

Re: [PATCH v4 02/16] libs/guest: Reduce number of I/O vectors in write_batch

Quoted 6 lines · Frediano Ziglio

From: Frediano Ziglio <[email protected]>

Small optimization.
Reduce number of pieces passed to writev.

Quoted 12 lines

--
Changes since v2:
- change prefix in subject.

---
 tools/libs/guest/xg_sr_save.c | 35 ++++++++++++++++-------------------
 1 file changed, 16 insertions(+), 19 deletions(-)
tools/libs/guest/xg_sr_save.c +0 -0
index 3b2c5222e4..1700d81905 100644

tools/libs/guest/xg_sr_save.c static int write_batch(struct xc_sr_context *ctx) · line 102

Quoted 9 lines
tools/libs/guest/xg_sr_save.c +3 -3
@@ -97,9 +97,12 @@ static int write_batch(struct xc_sr_context *ctx)
     void *page, *orig_page;
     uint64_t *rec_pfns = NULL;
     struct iovec *iov = NULL; int iovcnt = 0;
-    struct xc_sr_rec_page_data_header hdr = { 0 };
-    struct xc_sr_record rec = {
-        .type = REC_TYPE_PAGE_DATA,
+    struct {
+        struct xc_sr_rhdr rec;
+        struct xc_sr_rec_page_data_header page_data;

Is there a possible worry that the compiler (for another architecture) will introduce non-zero padding between those two structs?

tools/libs/guest/xg_sr_save.c static int write_batch(struct xc_sr_context *ctx) · line 105

Quoted 3 lines
tools/libs/guest/xg_sr_save.c +3 -0
+    } hdrs = {
+        { .type = REC_TYPE_PAGE_DATA },
+        { 0 },

Do you need the explicit initialization to 0 here? All unspecified fields in the initialization will already be set to 0.

tools/libs/guest/xg_sr_save.c static int write_batch(struct xc_sr_context *ctx) · line 121

Quoted 9 lines
tools/libs/guest/xg_sr_save.c +1 -1
     };
 
     assert(nr_pfns != 0);
@@ -115,7 +118,7 @@ static int write_batch(struct xc_sr_context *ctx)
     /* Pointers to locally allocated pages.  Need freeing. */
     local_pages = calloc(nr_pfns, sizeof(*local_pages));
     /* iovec[] for writev(). */
-    iov = malloc((nr_pfns + 4) * sizeof(*iov));
+    iov = malloc((nr_pfns + 2) * sizeof(*iov));

It would seem more natural to use calloc() here, but it would also do a zeroing that we don't care.

Thanks, Roger.

#6 lore raw

Re: [PATCH v4 02/16] libs/guest: Reduce number of I/O vectors in write_batch

Quoted 27 lines · Roger Pau Monné
Quoted 6 lines · Frediano Ziglio

From: Frediano Ziglio <[email protected]>

Small optimization.
Reduce number of pieces passed to writev.

Quoted 16 lines

--
Changes since v2:
- change prefix in subject.

---
 tools/libs/guest/xg_sr_save.c | 35 ++++++++++++++++-------------------
 1 file changed, 16 insertions(+), 19 deletions(-)
tools/libs/guest/xg_sr_save.c +0 -1
index 3b2c5222e4..1700d81905 100644
@@ -97,9 +97,12 @@ static int write_batch(struct xc_sr_context *ctx)
     void *page, *orig_page;
     uint64_t *rec_pfns = NULL;
     struct iovec *iov = NULL; int iovcnt = 0;
-    struct xc_sr_rec_page_data_header hdr = { 0 };
Quoted 9 lines

tools/libs/guest/xg_sr_save.c static int write_batch(struct xc_sr_context *ctx) · line 102

Quoted 5 lines
tools/libs/guest/xg_sr_save.c +3 -2
-    struct xc_sr_record rec = {
-        .type = REC_TYPE_PAGE_DATA,
+    struct {
+        struct xc_sr_rhdr rec;
+        struct xc_sr_rec_page_data_header page_data;

Is there a possible worry that the compiler (for another architecture) will introduce non-zero padding between those two structs?

No, the first structure is aligned to 32 bits (we don't support 16 bits systems), the second 32 or 64 bits.
In both cases the final structure won't need padding in the middle.

Quoted 7 lines

tools/libs/guest/xg_sr_save.c static int write_batch(struct xc_sr_context *ctx) · line 105

Quoted 3 lines
tools/libs/guest/xg_sr_save.c +3 -0
+    } hdrs = {
+        { .type = REC_TYPE_PAGE_DATA },
+        { 0 },

Do you need the explicit initialization to 0 here? All unspecified fields in the initialization will already be set to 0.

I'll remove it.

Quoted 13 lines

tools/libs/guest/xg_sr_save.c static int write_batch(struct xc_sr_context *ctx) · line 121

Quoted 9 lines
tools/libs/guest/xg_sr_save.c +1 -1
     };
 
     assert(nr_pfns != 0);
@@ -115,7 +118,7 @@ static int write_batch(struct xc_sr_context *ctx)
     /* Pointers to locally allocated pages.  Need freeing. */
     local_pages = calloc(nr_pfns, sizeof(*local_pages));
     /* iovec[] for writev(). */
-    iov = malloc((nr_pfns + 4) * sizeof(*iov));
+    iov = malloc((nr_pfns + 2) * sizeof(*iov));

It would seem more natural to use calloc() here, but it would also do a zeroing that we don't care.

Quoted 1 line

Thanks, Roger.

Frediano

#7 lore raw

PATCH v4 3/16 libs/guest: Reduce number of I/O vectors in write_batch

In reply to Frediano Ziglio

From: Frediano Ziglio <[email protected]>

Each page was sent using a different iovec item. This potentially exceed Linux maximum (1024).
Also some implementation (MiniOS) emulate writev with multiple write calls.

--
Changes since v2:
- change prefix in subject.

---
 tools/libs/guest/xg_sr_save.c | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)
tools/libs/guest/xg_sr_save.c +9 -2
index 1700d81905..62a39dfecc 100644
@@ -240,13 +240,20 @@ static int write_batch(struct xc_sr_context *ctx)
     {
         for ( i = 0; i < nr_pfns; ++i )
         {
-            if ( guest_data[i] )
+            if ( !guest_data[i] )
+                continue;
+
+            if ( iov[iovcnt-1].iov_base + iov[iovcnt-1].iov_len != guest_data[i] )
             {
                 iov[iovcnt].iov_base = guest_data[i];
                 iov[iovcnt].iov_len = PAGE_SIZE;
                 iovcnt++;
-                --nr_pages;
             }
+            else
+            {
+                iov[iovcnt-1].iov_len += PAGE_SIZE;
+            }
+            --nr_pages;
         }
     }
 
Signature · 1 line
2.54.0
Reviewed-by on v4 3/16
#8 lore raw

Re: [PATCH v4 03/16] libs/guest: Reduce number of I/O vectors in write_batch

Quoted 5 lines · Frediano Ziglio

From: Frediano Ziglio <[email protected]>

Each page was sent using a different iovec item. This potentially exceed Linux maximum (1024).
Also some implementation (MiniOS) emulate writev with multiple write calls.

I would add to the commit message:

"Coalesce adjacent IO vector elements to attempt to reduce the number of overall IO vectors for each operation."

Quoted 1 line
Quoted 12 lines

--
Changes since v2:
- change prefix in subject.

---
 tools/libs/guest/xg_sr_save.c | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)
tools/libs/guest/xg_sr_save.c +0 -0
index 1700d81905..62a39dfecc 100644

tools/libs/guest/xg_sr_save.c static int write_batch(struct xc_sr_context *ctx) · line 246

Quoted 8 lines
tools/libs/guest/xg_sr_save.c +4 -1
@@ -240,13 +240,20 @@ static int write_batch(struct xc_sr_context *ctx)
     {
         for ( i = 0; i < nr_pfns; ++i )
         {
-            if ( guest_data[i] )
+            if ( !guest_data[i] )
+                continue;
+
+            if ( iov[iovcnt-1].iov_base + iov[iovcnt-1].iov_len != guest_data[i] )

Nit: space between subtraction and operands: iovcnt - 1.

Thanks, Roger.

#9 lore raw

PATCH v4 4/16 libs/guest: Use a single write_exact in write_headers

In reply to Frediano Ziglio

From: Frediano Ziglio <[email protected]>

Reduce number of syscalls.

--
Changes since v2:
- change prefix in subject.

---
 tools/libs/guest/xg_sr_save.c | 37 +++++++++++++++++------------------
 1 file changed, 18 insertions(+), 19 deletions(-)
tools/libs/guest/xg_sr_save.c +18 -19
index 62a39dfecc..8c4e760f8d 100644
@@ -10,17 +10,22 @@ static int write_headers(struct xc_sr_context *ctx, uint16_t guest_type)
 {
     xc_interface *xch = ctx->xch;
     int32_t xen_version = xc_version(xch, XENVER_version, NULL);
-    struct xc_sr_ihdr ihdr = {
-        .marker  = IHDR_MARKER,
-        .id      = htonl(IHDR_ID),
-        .version = htonl(3),
-        .options = htons(IHDR_OPT_LITTLE_ENDIAN),
-    };
-    struct xc_sr_dhdr dhdr = {
-        .type       = guest_type,
-        .page_shift = XC_PAGE_SHIFT,
-        .xen_major  = (xen_version >> 16) & 0xffff,
-        .xen_minor  = (xen_version)       & 0xffff,
+    struct {
+        struct xc_sr_ihdr ihdr;
+        struct xc_sr_dhdr dhdr;
+    } hdrs = {
+        {
+            .marker  = IHDR_MARKER,
+            .id      = htonl(IHDR_ID),
+            .version = htonl(3),
+            .options = htons(IHDR_OPT_LITTLE_ENDIAN),
+        },
+        {
+            .type       = guest_type,
+            .page_shift = XC_PAGE_SHIFT,
+            .xen_major  = (xen_version >> 16) & 0xffff,
+            .xen_minor  = (xen_version)       & 0xffff,
+        },
     };
 
     if ( xen_version < 0 )
@@ -29,15 +34,9 @@ static int write_headers(struct xc_sr_context *ctx, uint16_t guest_type)
         return -1;
     }
 
-    if ( write_exact(ctx->fd, &ihdr, sizeof(ihdr)) )
-    {
-        PERROR("Unable to write Image Header to stream");
-        return -1;
-    }
-
-    if ( write_exact(ctx->fd, &dhdr, sizeof(dhdr)) )
+    if ( write_exact(ctx->fd, &hdrs, sizeof(hdrs)) )
     {
-        PERROR("Unable to write Domain Header to stream");
+        PERROR("Unable to write Headers to stream");
         return -1;
     }
 
Signature · 1 line
2.54.0
Reviewed-by on v4 4/16
#10 lore raw

Re: [PATCH v4 04/16] libs/guest: Use a single write_exact in write_headers

Quoted 3 lines · Frediano Ziglio

From: Frediano Ziglio <[email protected]>

Reduce number of syscalls.

... by coalescing the image and the domain headers into a single IO vector array.

Quoted 2 lines
Quoted 30 lines

--
Changes since v2:
- change prefix in subject.

---
 tools/libs/guest/xg_sr_save.c | 37 +++++++++++++++++------------------
 1 file changed, 18 insertions(+), 19 deletions(-)
tools/libs/guest/xg_sr_save.c +4 -11
index 62a39dfecc..8c4e760f8d 100644
@@ -10,17 +10,22 @@ static int write_headers(struct xc_sr_context *ctx, uint16_t guest_type)
 {
     xc_interface *xch = ctx->xch;
     int32_t xen_version = xc_version(xch, XENVER_version, NULL);
-    struct xc_sr_ihdr ihdr = {
-        .marker  = IHDR_MARKER,
-        .id      = htonl(IHDR_ID),
-        .version = htonl(3),
-        .options = htons(IHDR_OPT_LITTLE_ENDIAN),
-    };
-    struct xc_sr_dhdr dhdr = {
-        .type       = guest_type,
-        .page_shift = XC_PAGE_SHIFT,
-        .xen_major  = (xen_version >> 16) & 0xffff,
-        .xen_minor  = (xen_version)       & 0xffff,
+    struct {
+        struct xc_sr_ihdr ihdr;
+        struct xc_sr_dhdr dhdr;
+    } hdrs = {

tools/libs/guest/xg_sr_save.c static int write_headers(struct xc_sr_context *ctx, uint16_t guest_type) · line 26

Quoted 10 lines
tools/libs/guest/xg_sr_save.c +10 -0
+        {
+            .marker  = IHDR_MARKER,
+            .id      = htonl(IHDR_ID),
+            .version = htonl(3),
+            .options = htons(IHDR_OPT_LITTLE_ENDIAN),
+        },
+        {
+            .type       = guest_type,
+            .page_shift = XC_PAGE_SHIFT,
+            .xen_major  = (xen_version >> 16) & 0xffff,

You don't strictly need the mask here AFAICT?

Thanks, Roger.

#11 lore raw

PATCH v4 5/16 libs/guest: allocate various migration arrays just once

In reply to Frediano Ziglio

From: Edwin Török <[email protected]>

Allocate these array just once at the start of migration, using the maximum batch size, and free them at the end.

--
Changes since v2:
- change prefix in subject.

Changes since v3:
- fix comment style

---
 tools/libs/guest/xg_sr_common.h | 13 +++++++
 tools/libs/guest/xg_sr_save.c   | 66 +++++++++++++--------------------
 2 files changed, 39 insertions(+), 40 deletions(-)
tools/libs/guest/xg_sr_common.h +13 -0
index f1573aefcb..82549b5589 100644
@@ -209,6 +209,18 @@ static inline int update_blob(struct xc_sr_blob *blob,
     return 0;
 }
 
+struct xc_sr_context_save_buffers
+{
+    xen_pfn_t batch_pfns[MAX_BATCH_SIZE];
+    xen_pfn_t mfns[MAX_BATCH_SIZE];
+    xen_pfn_t types[MAX_BATCH_SIZE];
+    int errors[MAX_BATCH_SIZE];
+    void *guest_data[MAX_BATCH_SIZE];
+    void *local_pages[MAX_BATCH_SIZE];
+    struct iovec iov[MAX_BATCH_SIZE + 2]; /* Headers + data. */
+    uint64_t rec_pfns[MAX_BATCH_SIZE];
+};
+
 struct xc_sr_context
 {
     xc_interface *xch;
@@ -244,6 +256,7 @@ struct xc_sr_context
             unsigned long *deferred_pages;
             unsigned long nr_deferred_pages;
             xc_hypercall_buffer_t dirty_bitmap_hbuf;
+            struct xc_sr_context_save_buffers *buffers;
         } save;
 
         struct /* Restore data. */
tools/libs/guest/xg_sr_save.c +26 -40

124 lines not on this page · raw message

Signature · 1 line
2.54.0
#12 lore raw

Re: [PATCH v4 05/16] libs/guest: allocate various migration arrays just once

Quoted 5 lines · Frediano Ziglio

From: Edwin Török <[email protected]>

Allocate these array just once at the start of migration, using the maximum batch size, and free them at the end.

Quoted 8 lines

--
Changes since v2:
- change prefix in subject.

Changes since v3:
- fix comment style

These are intentionally freed/reallocated so valgrind can find overflows.

~Andrew

#13 lore raw

Re: [PATCH v4 05/16] libs/guest: allocate various migration arrays just once

Quoted 12 lines · Andrew Cooper
Quoted 11 lines · Frediano Ziglio

From: Edwin Török <[email protected]>

Allocate these array just once at the start of migration, using the maximum batch size, and free them at the end.

--
Changes since v2:
- change prefix in subject.

Quoted 4 lines
Quoted 2 lines

Changes since v3:
- fix comment style

These are intentionally freed/reallocated so valgrind can find overflows.

Hm, then we likely need to keep the current behavior for CONFIG_DEBUG builds of the library?

Thanks, Roger.

#14 lore raw

Re: [PATCH v4 05/16] libs/guest: allocate various migration arrays just once

In reply to Andrew Cooper
Quoted 13 lines · Andrew Cooper
Quoted 11 lines · Frediano Ziglio

From: Edwin Török <[email protected]>

Allocate these array just once at the start of migration, using the maximum batch size, and free them at the end.

--
Changes since v2:
- change prefix in subject.

Quoted 6 lines
Quoted 2 lines

Changes since v3:
- fix comment style

These are intentionally freed/reallocated so valgrind can find overflows.

~Andrew

I understand the value of checking for possible overflow. But this should not make the programs less efficient.

What about something like this (not tested, just to get the gist):

#if CONFIG_DEBUG && valgrind && whatever
#define MEM_NOACCESS_BUFFER(name, size) uint8_t name[size]; #define MEM_NOACCESS_INIT(field) VALGRIND_MAKE_MEM_NOACCESS(field, sizeof(field))
#define MEM_UNDEFINED_INIT(field) VALGRIND_MAKE_MEM_UNDEFINED(field, sizeof(field))
#else
#define MEM_NOACCESS_BUFFER(name, size)
#define MEM_NOACCESS_INIT(field) do {} while(0)
#define MEM_UNDEFINED_INIT(field) do {} while(0)
#endif
...

struct xc_sr_context_save_buffers

{
    MEM_NOACCESS_BUFFER(na0, 16)
    xen_pfn_t batch_pfns[MAX_BATCH_SIZE];
    MEM_NOACCESS_BUFFER(na1, 16)
    xen_pfn_t mfns[MAX_BATCH_SIZE];
    MEM_NOACCESS_BUFFER(na2, 16)
    xen_pfn_t types[MAX_BATCH_SIZE];
    MEM_NOACCESS_BUFFER(na3, 16)
    void *local_pages[MAX_BATCH_SIZE];
    MEM_NOACCESS_BUFFER(na4, 16)
    struct iovec iov[MAX_BATCH_SIZE + 2]; /* Headers + data. */
    MEM_NOACCESS_BUFFER(na5, 16)
    uint64_t rec_pfns[MAX_BATCH_SIZE];
    MEM_NOACCESS_BUFFER(na6, 16)
};

...

    ctx->save.buffers = calloc(1, sizeof(*ctx->save.buffers));

    if ( !dirty_bitmap || !ctx->save.deferred_pages ||
!ctx->save.buffers || !dest_buf )
    {
        ERROR("Unable to allocate memory for dirty bitmaps, deferred pages"
              " and various batch buffers");
        rc = -1;
        errno = ENOMEM;
        goto err;
    }
    ctx->save.batch_pfns = ctx->save.buffers->batch_pfns;
    MEM_NOACCESS_INIT(ctx->save.buffers->na0);
    MEM_NOACCESS_INIT(ctx->save.buffers->na1);
    MEM_NOACCESS_INIT(ctx->save.buffers->na2);
    MEM_NOACCESS_INIT(ctx->save.buffers->na3);
    MEM_NOACCESS_INIT(ctx->save.buffers->na4);
    MEM_NOACCESS_INIT(ctx->save.buffers->na5);
    MEM_NOACCESS_INIT(ctx->save.buffers->na6);

...

    /* Mfns of the batch pfns. */
    MEM_UNDEFINED_INIT(ctx->save.buffers->mfns);
    mfns = ctx->save.buffers->mfns;
    /* Types of the batch pfns. */
    MEM_UNDEFINED_INIT(ctx->save.buffers->types);
    types = ctx->save.buffers->types;
    /* Pointers to locally allocated pages.  Need freeing. */
    MEM_UNDEFINED_INIT(ctx->save.buffers->local_pages);
    local_pages = ctx->save.buffers->local_pages;
    memset(local_pages, 0, sizeof(*local_pages) * nr_pfns);

...

Frediano

#15 lore raw

Re: [PATCH v4 05/16] libs/guest: allocate various migration arrays just once

In reply to Frediano Ziglio
Quoted 7 lines · Frediano Ziglio

From: Edwin Török <[email protected]>

Allocate these array just once at the start of migration, using the maximum batch size, and free them at the end.

Jan made a comment about this patch (and how it related to a still pending patch of his):

https://lore.kernel.org/xen-devel/[email protected]/

That is still unresolved AFAICT.

Quoted 16 lines

--
Changes since v2:
- change prefix in subject.

Changes since v3:
- fix comment style

---
 tools/libs/guest/xg_sr_common.h | 13 +++++++
 tools/libs/guest/xg_sr_save.c   | 66 +++++++++++++--------------------
 2 files changed, 39 insertions(+), 40 deletions(-)
tools/libs/guest/xg_sr_common.h +0 -0
index f1573aefcb..82549b5589 100644

tools/libs/guest/xg_sr_common.h static inline int update_blob(struct xc_sr_blob *blob, · line 217

Quoted 9 lines
tools/libs/guest/xg_sr_common.h +6 -0
@@ -209,6 +209,18 @@ static inline int update_blob(struct xc_sr_blob *blob,
     return 0;
 }
 
+struct xc_sr_context_save_buffers
+{
+    xen_pfn_t batch_pfns[MAX_BATCH_SIZE];
+    xen_pfn_t mfns[MAX_BATCH_SIZE];
+    xen_pfn_t types[MAX_BATCH_SIZE];
+    int errors[MAX_BATCH_SIZE];

FWIW: I would possibly place errors at the end of the structure. It seems more natural and is the only array that has 4 byte alignment instead of 8 (on 64bits at least).

Quoted 58 lines
tools/libs/guest/xg_sr_common.h +7 -0
+    void *guest_data[MAX_BATCH_SIZE];
+    void *local_pages[MAX_BATCH_SIZE];
+    struct iovec iov[MAX_BATCH_SIZE + 2]; /* Headers + data. */
+    uint64_t rec_pfns[MAX_BATCH_SIZE];
+};
+
 struct xc_sr_context
 {
     xc_interface *xch;
@@ -244,6 +256,7 @@ struct xc_sr_context
             unsigned long *deferred_pages;
             unsigned long nr_deferred_pages;
             xc_hypercall_buffer_t dirty_bitmap_hbuf;
+            struct xc_sr_context_save_buffers *buffers;
         } save;
 
         struct /* Restore data. */
tools/libs/guest/xg_sr_save.c +10 -8
index 8c4e760f8d..7d8055a3f9 100644
@@ -86,16 +86,16 @@ static int write_checkpoint_record(struct xc_sr_context *ctx)
 static int write_batch(struct xc_sr_context *ctx)
 {
     xc_interface *xch = ctx->xch;
-    xen_pfn_t *mfns = NULL, *types = NULL;
+    xen_pfn_t *mfns, *types;
     void *guest_mapping = NULL;
-    void **guest_data = NULL;
-    void **local_pages = NULL;
-    int *errors = NULL, rc = -1;
+    void **guest_data;
+    void **local_pages;
+    int *errors, rc = -1;
     unsigned int i, p, nr_pages = 0, nr_pages_mapped = 0;
     unsigned int nr_pfns = ctx->save.nr_batch_pfns;
     void *page, *orig_page;
-    uint64_t *rec_pfns = NULL;
-    struct iovec *iov = NULL; int iovcnt = 0;
+    uint64_t *rec_pfns;
+    struct iovec *iov; int iovcnt = 0;
     struct {
         struct xc_sr_rhdr rec;
         struct xc_sr_rec_page_data_header page_data;
@@ -105,26 +105,24 @@ static int write_batch(struct xc_sr_context *ctx)
     };
 
     assert(nr_pfns != 0);
+    assert(nr_pfns <= MAX_BATCH_SIZE);
+    assert(ctx->save.buffers);
 
     /* Mfns of the batch pfns. */
-    mfns = malloc(nr_pfns * sizeof(*mfns));
+    mfns = ctx->save.buffers->mfns;
     /* Types of the batch pfns. */
-    types = malloc(nr_pfns * sizeof(*types));
+    types = ctx->save.buffers->types;
     /* Errors from attempting to map the gfns. */

tools/libs/guest/xg_sr_save.c static int write_batch(struct xc_sr_context *ctx) · line 122

Quoted 10 lines
tools/libs/guest/xg_sr_save.c +5 -3
-    errors = malloc(nr_pfns * sizeof(*errors));
+    errors = ctx->save.buffers->errors;
     /* Pointers to page data to send.  Mapped gfns or local allocations. */
-    guest_data = calloc(nr_pfns, sizeof(*guest_data));
+    guest_data = ctx->save.buffers->guest_data;
+    memset(guest_data, 0, sizeof(*guest_data) * nr_pfns);
     /* Pointers to locally allocated pages.  Need freeing. */
-    local_pages = calloc(nr_pfns, sizeof(*local_pages));
+    local_pages = ctx->save.buffers->local_pages;
+    memset(local_pages, 0, sizeof(*local_pages) * nr_pfns);

See below - I think it's possible to avoid the memset() and keep the same guarantees.

Quoted 36 lines
tools/libs/guest/xg_sr_save.c +2 -17
     /* iovec[] for writev(). */
-    iov = malloc((nr_pfns + 2) * sizeof(*iov));
-
-    if ( !mfns || !types || !errors || !guest_data || !local_pages || !iov )
-    {
-        ERROR("Unable to allocate arrays for a batch of %u pages",
-              nr_pfns);
-        goto err;
-    }
+    iov = ctx->save.buffers->iov;
+    rec_pfns = ctx->save.buffers->rec_pfns;
 
     for ( i = 0; i < nr_pfns; ++i )
     {
@@ -210,14 +208,6 @@ static int write_batch(struct xc_sr_context *ctx)
         }
     }
 
-    rec_pfns = malloc(nr_pfns * sizeof(*rec_pfns));
-    if ( !rec_pfns )
-    {
-        ERROR("Unable to allocate %zu bytes of memory for page data pfn list",
-              nr_pfns * sizeof(*rec_pfns));
-        goto err;
-    }
-
     hdrs.rec.length = sizeof(hdrs.page_data);
     hdrs.rec.length += nr_pfns * sizeof(*rec_pfns);
     hdrs.rec.length += nr_pages * PAGE_SIZE;
@@ -267,17 +257,13 @@ static int write_batch(struct xc_sr_context *ctx)
     rc = ctx->save.nr_batch_pfns = 0;
 
  err:
-    free(rec_pfns);
     if ( guest_mapping )
         xenforeignmemory_unmap(xch->fmem, guest_mapping, nr_pages_mapped);

tools/libs/guest/xg_sr_save.c static int write_batch(struct xc_sr_context *ctx) · line 265

Quoted 10 lines
tools/libs/guest/xg_sr_save.c +2 -6
     for ( i = 0; local_pages && i < nr_pfns; ++i )
+    {
         free(local_pages[i]);
-    free(iov);
-    free(local_pages);
-    free(guest_data);
-    free(errors);
-    free(types);
-    free(mfns);
+        local_pages[i] = NULL;

If you are doing this cleanup here, you could also do guest_data[i] = NULL and avoid the memset, since at the start of each write_batch() the arrays will already be zeroed (either because they are allocated with calloc() on the first call, and always cleaned up in write_batch() after usage.

Thanks, Roger.

#16 lore raw

Re: [PATCH v4 05/16] libs/guest: allocate various migration arrays just once

Quoted 6 lines · Roger Pau Monné
Quoted 5 lines · Frediano Ziglio

From: Edwin Török <[email protected]>

Allocate these array just once at the start of migration, using the maximum batch size, and free them at the end.

Quoted 8 lines
Quoted 2 lines

Jan made a comment about this patch (and how it related to a still pending patch of his):

https://lore.kernel.org/xen-devel/[email protected]/

That is still unresolved AFAICT.

That patch is not pending.  It, along with it's series, is wholly nacked.

I have explained in public and in private what it would take to retract my nack, but until such time as the feedback is actioned, that old series is going nowhere, and has no baring on other changes.

~Andrew

#17 lore raw

Re: [PATCH v4 05/16] libs/guest: allocate various migration arrays just once

In reply to Roger Pau Monné
Quoted 36 lines · Roger Pau Monné
Quoted 7 lines · Frediano Ziglio

From: Edwin Török <[email protected]>

Allocate these array just once at the start of migration, using the maximum batch size, and free them at the end.

Jan made a comment about this patch (and how it related to a still pending patch of his):

https://lore.kernel.org/xen-devel/[email protected]/

That is still unresolved AFAICT.

Quoted 19 lines

--
Changes since v2:
- change prefix in subject.

Changes since v3:
- fix comment style

---
 tools/libs/guest/xg_sr_common.h | 13 +++++++
 tools/libs/guest/xg_sr_save.c   | 66 +++++++++++++--------------------
 2 files changed, 39 insertions(+), 40 deletions(-)
tools/libs/guest/xg_sr_common.h +0 -0
index f1573aefcb..82549b5589 100644
@@ -209,6 +209,18 @@ static inline int update_blob(struct xc_sr_blob *blob,
     return 0;
 }
 
Quoted 11 lines

tools/libs/guest/xg_sr_common.h static inline int update_blob(struct xc_sr_blob *blob, · line 217

Quoted 6 lines
tools/libs/guest/xg_sr_common.h +6 -0
+struct xc_sr_context_save_buffers
+{
+    xen_pfn_t batch_pfns[MAX_BATCH_SIZE];
+    xen_pfn_t mfns[MAX_BATCH_SIZE];
+    xen_pfn_t types[MAX_BATCH_SIZE];
+    int errors[MAX_BATCH_SIZE];

FWIW: I would possibly place errors at the end of the structure. It seems more natural and is the only array that has 4 byte alignment instead of 8 (on 64bits at least).

Moved. It disappears later in "libs/guest: finalize PoC" but won't hurt.

Quoted 63 lines
Quoted 63 lines
tools/libs/guest/xg_sr_common.h +7 -0
+    void *guest_data[MAX_BATCH_SIZE];
+    void *local_pages[MAX_BATCH_SIZE];
+    struct iovec iov[MAX_BATCH_SIZE + 2]; /* Headers + data. */
+    uint64_t rec_pfns[MAX_BATCH_SIZE];
+};
+
 struct xc_sr_context
 {
     xc_interface *xch;
@@ -244,6 +256,7 @@ struct xc_sr_context
             unsigned long *deferred_pages;
             unsigned long nr_deferred_pages;
             xc_hypercall_buffer_t dirty_bitmap_hbuf;
+            struct xc_sr_context_save_buffers *buffers;
         } save;
 
         struct /* Restore data. */
tools/libs/guest/xg_sr_save.c +12 -10
index 8c4e760f8d..7d8055a3f9 100644
@@ -86,16 +86,16 @@ static int write_checkpoint_record(struct xc_sr_context *ctx)
 static int write_batch(struct xc_sr_context *ctx)
 {
     xc_interface *xch = ctx->xch;
-    xen_pfn_t *mfns = NULL, *types = NULL;
+    xen_pfn_t *mfns, *types;
     void *guest_mapping = NULL;
-    void **guest_data = NULL;
-    void **local_pages = NULL;
-    int *errors = NULL, rc = -1;
+    void **guest_data;
+    void **local_pages;
+    int *errors, rc = -1;
     unsigned int i, p, nr_pages = 0, nr_pages_mapped = 0;
     unsigned int nr_pfns = ctx->save.nr_batch_pfns;
     void *page, *orig_page;
-    uint64_t *rec_pfns = NULL;
-    struct iovec *iov = NULL; int iovcnt = 0;
+    uint64_t *rec_pfns;
+    struct iovec *iov; int iovcnt = 0;
     struct {
         struct xc_sr_rhdr rec;
         struct xc_sr_rec_page_data_header page_data;
@@ -105,26 +105,24 @@ static int write_batch(struct xc_sr_context *ctx)
     };
 
     assert(nr_pfns != 0);
+    assert(nr_pfns <= MAX_BATCH_SIZE);
+    assert(ctx->save.buffers);
 
     /* Mfns of the batch pfns. */
-    mfns = malloc(nr_pfns * sizeof(*mfns));
+    mfns = ctx->save.buffers->mfns;
     /* Types of the batch pfns. */
-    types = malloc(nr_pfns * sizeof(*types));
+    types = ctx->save.buffers->types;
     /* Errors from attempting to map the gfns. */
-    errors = malloc(nr_pfns * sizeof(*errors));
+    errors = ctx->save.buffers->errors;
     /* Pointers to page data to send.  Mapped gfns or local allocations. */
-    guest_data = calloc(nr_pfns, sizeof(*guest_data));
+    guest_data = ctx->save.buffers->guest_data;
Quoted 9 lines

tools/libs/guest/xg_sr_save.c static int write_batch(struct xc_sr_context *ctx) · line 122

Quoted 5 lines
tools/libs/guest/xg_sr_save.c +3 -1
+    memset(guest_data, 0, sizeof(*guest_data) * nr_pfns);
     /* Pointers to locally allocated pages.  Need freeing. */
-    local_pages = calloc(nr_pfns, sizeof(*local_pages));
+    local_pages = ctx->save.buffers->local_pages;
+    memset(local_pages, 0, sizeof(*local_pages) * nr_pfns);

See below - I think it's possible to avoid the memset() and keep the same guarantees.

What's the issue with memset?
It's more consistent with the old "calloc" in the code and in the compilers we use (gcc/clang) memset is a builtin that's optimized better than setting as single fields.
Note that "guest_data" disappears in "libs/guest: fill directly iov structure" commit.

Quoted 44 lines
Quoted 44 lines
tools/libs/guest/xg_sr_save.c +3 -22
     /* iovec[] for writev(). */
-    iov = malloc((nr_pfns + 2) * sizeof(*iov));
-
-    if ( !mfns || !types || !errors || !guest_data || !local_pages || !iov )
-    {
-        ERROR("Unable to allocate arrays for a batch of %u pages",
-              nr_pfns);
-        goto err;
-    }
+    iov = ctx->save.buffers->iov;
+    rec_pfns = ctx->save.buffers->rec_pfns;
 
     for ( i = 0; i < nr_pfns; ++i )
     {
@@ -210,14 +208,6 @@ static int write_batch(struct xc_sr_context *ctx)
         }
     }
 
-    rec_pfns = malloc(nr_pfns * sizeof(*rec_pfns));
-    if ( !rec_pfns )
-    {
-        ERROR("Unable to allocate %zu bytes of memory for page data pfn list",
-              nr_pfns * sizeof(*rec_pfns));
-        goto err;
-    }
-
     hdrs.rec.length = sizeof(hdrs.page_data);
     hdrs.rec.length += nr_pfns * sizeof(*rec_pfns);
     hdrs.rec.length += nr_pages * PAGE_SIZE;
@@ -267,17 +257,13 @@ static int write_batch(struct xc_sr_context *ctx)
     rc = ctx->save.nr_batch_pfns = 0;
 
  err:
-    free(rec_pfns);
     if ( guest_mapping )
         xenforeignmemory_unmap(xch->fmem, guest_mapping, nr_pages_mapped);
     for ( i = 0; local_pages && i < nr_pfns; ++i )
+    {
         free(local_pages[i]);
-    free(iov);
-    free(local_pages);
-    free(guest_data);
-    free(errors);
-    free(types);
Quoted 9 lines

tools/libs/guest/xg_sr_save.c static int write_batch(struct xc_sr_context *ctx) · line 265

Quoted 2 lines
tools/libs/guest/xg_sr_save.c +1 -1
-    free(mfns);
+        local_pages[i] = NULL;

If you are doing this cleanup here, you could also do guest_data[i] = NULL and avoid the memset, since at the start of each write_batch() the arrays will already be zeroed (either because they are allocated with calloc() on the first call, and always cleaned up in write_batch() after usage.

Given that there's a patch removing "guest_data" entirely, maybe I would remove the "memset" for "local_pages" in another commit, maybe "libs/guest: finalize PoC".

Quoted 1 line

Thanks, Roger.

Frediano

#18 lore raw

PATCH v4 6/16 libs/call: cache up to 4 pages in hypercall bounce buffers

In reply to Frediano Ziglio

From: Edwin Török <[email protected]>

During migration there are a lot of mmap/munmap calls, because xc_get_pfn_type_batch exceeds the default hypercall bounce buffer cache size, and needs to allocate every time it is called.

munmap is slow, especially in a PV Dom0 (takes an emulation fault), so is best avoided.

Eventually it'd be good if the memory pool from xmalloc_tlsf.c was reused here, but for now make it handle the commonly encountered sizes (so far up to 4 pages).

--
Changes since v2:
- change prefix in subject.

---
 tools/libs/call/buffer.c  | 28 +++++++++++++++++-----------
 tools/libs/call/core.c    |  3 ++-
 tools/libs/call/private.h |  8 +++++---
 3 files changed, 24 insertions(+), 15 deletions(-)
tools/libs/call/buffer.c +17 -11
index 2579b8c719..15d2f98a6b 100644
@@ -56,13 +56,13 @@ static void *cache_alloc(xencall_handle *xcall, size_t nr_pages)
     if ( xcall->buffer_current_allocations > xcall->buffer_maximum_allocations )
         xcall->buffer_maximum_allocations = xcall->buffer_current_allocations;
 
-    if ( nr_pages > 1 )
+    if ( nr_pages > ARRAY_SIZE(xcall->buffer_cache) )
     {
         xcall->buffer_cache_toobig++;
     }
-    else if ( xcall->buffer_cache_nr > 0 )
+    else if ( xcall->buffer_cache_nr[nr_pages-1] > 0 )
     {
-        p = xcall->buffer_cache[--xcall->buffer_cache_nr];
+        p = xcall->buffer_cache[nr_pages-1][--xcall->buffer_cache_nr[nr_pages-1]];
         xcall->buffer_cache_hits++;
     }
     else
@@ -84,10 +84,10 @@ static int cache_free(xencall_handle *xcall, void *p, size_t nr_pages)
     xcall->buffer_total_releases++;
     xcall->buffer_current_allocations--;
 
-    if ( nr_pages == 1 &&
-         xcall->buffer_cache_nr < BUFFER_CACHE_SIZE )
+    if ( nr_pages && nr_pages < ARRAY_SIZE(xcall->buffer_cache) &&
+         xcall->buffer_cache_nr[nr_pages-1] < BUFFER_CACHE_SIZE )
     {
-        xcall->buffer_cache[xcall->buffer_cache_nr++] = p;
+        xcall->buffer_cache[nr_pages-1][xcall->buffer_cache_nr[nr_pages-1]++] = p;
         rc = 1;
     }
 
@@ -108,17 +108,23 @@ void buffer_release_cache(xencall_handle *xcall)
     DBGPRINTF("current allocations:%d maximum allocations:%d",
               xcall->buffer_current_allocations,
               xcall->buffer_maximum_allocations);
-    DBGPRINTF("cache current size:%d",
-              xcall->buffer_cache_nr);
+    for ( unsigned i = 0; i < ARRAY_SIZE(xcall->buffer_cache_nr); ++i )
+    {
+        DBGPRINTF("cache current size[%u pages]:%d", i+1,
+                xcall->buffer_cache_nr[i]);
+    }
     DBGPRINTF("cache hits:%d misses:%d toobig:%d",
               xcall->buffer_cache_hits,
               xcall->buffer_cache_misses,
               xcall->buffer_cache_toobig);
 
-    while ( xcall->buffer_cache_nr > 0 )
+    for ( unsigned i = 0; i < ARRAY_SIZE(xcall->buffer_cache_nr); ++i )
     {
-        p = xcall->buffer_cache[--xcall->buffer_cache_nr];
-        osdep_free_pages(xcall, p, 1);
+        while ( xcall->buffer_cache_nr[i] > 0 )
+        {
+            p = xcall->buffer_cache[i][--xcall->buffer_cache_nr[i]];
+            osdep_free_pages(xcall, p, i + 1);
+        }
     }
 
     cache_unlock(xcall);
tools/libs/call/core.c +2 -1
index 02c4f8e1ae..dd8877c1a0 100644
@@ -14,6 +14,7 @@
  */
 
 #include <stdlib.h>
+#include <string.h>
 
 #include "private.h"
 
@@ -44,7 +45,7 @@ xencall_handle *xencall_open(xentoollog_logger *logger, unsigned open_flags)
     xentoolcore__register_active_handle(&xcall->tc_ah);
 
     xcall->flags = open_flags;
-    xcall->buffer_cache_nr = 0;
+    memset(xcall->buffer_cache_nr, 0, sizeof(xcall->buffer_cache_nr));
 
     xcall->buffer_total_allocations = 0;
     xcall->buffer_total_releases = 0;
tools/libs/call/private.h +5 -3
index 9c3aa432ef..8e6a208975 100644
@@ -31,13 +31,15 @@ struct xencall_handle {
     Xentoolcore__Active_Handle tc_ah;
 
     /*
-     * A simple cache of unused, single page, hypercall buffers
+     * A simple cache of unused, small, hypercall buffers
+     * buffer_cache[i]'s size is (i+1) pages
      *
      * Protected by a global lock.
      */
 #define BUFFER_CACHE_SIZE 4
-    int buffer_cache_nr;
-    void *buffer_cache[BUFFER_CACHE_SIZE];
+#define BUFFER_CACHE_NRPAGES 4
+    int buffer_cache_nr[BUFFER_CACHE_NRPAGES];
+    void *buffer_cache[BUFFER_CACHE_NRPAGES][BUFFER_CACHE_SIZE];
 
     /*
      * Hypercall buffer statistics. All protected by the global
Signature · 1 line
2.54.0
#19 lore raw

PATCH v4 7/16 libs/guest: avoids using 2 indexes

In reply to Frediano Ziglio

From: Frediano Ziglio <[email protected]>

Simplify code, after the first scan of the various arrays we don't need to keep original types and PFNs but only the ones having data.

---
 tools/libs/guest/xg_sr_restore.c | 26 +++++++++++++-------------
 1 file changed, 13 insertions(+), 13 deletions(-)
tools/libs/guest/xg_sr_restore.c +13 -13
index e148fc594a..fb46142d87 100644
@@ -260,9 +260,7 @@ static int process_page_data(struct xc_sr_context *ctx, unsigned int count,
     int *map_errs = malloc(count * sizeof(*map_errs));
     int rc;
     void *mapping = NULL, *guest_page = NULL;
-    unsigned int i, /* i indexes the pfns from the record. */
-        j,          /* j indexes the subset of pfns we decide to map. */
-        nr_pages = 0;
+    unsigned nr_pages;
 
     if ( !mfns || !map_errs )
     {
@@ -279,12 +277,18 @@ static int process_page_data(struct xc_sr_context *ctx, unsigned int count,
         goto err;
     }
 
-    for ( i = 0; i < count; ++i )
+    nr_pages = 0;
+    for ( unsigned i = 0; i < count; ++i )
     {
         ctx->restore.ops.set_page_type(ctx, pfns[i], types[i]);
 
-        if ( page_type_has_stream_data(types[i]) )
-            mfns[nr_pages++] = ctx->restore.ops.pfn_to_gfn(ctx, pfns[i]);
+        if ( !page_type_has_stream_data(types[i]) )
+            continue;
+
+        mfns[nr_pages] = ctx->restore.ops.pfn_to_gfn(ctx, pfns[i]);
+        pfns[nr_pages] = pfns[i];
+        types[nr_pages] = types[i];
+        nr_pages++;
     }
 
     /* Nothing to do? */
@@ -302,16 +306,13 @@ static int process_page_data(struct xc_sr_context *ctx, unsigned int count,
         goto err;
     }
 
-    for ( i = 0, j = 0; i < count; ++i )
+    for ( unsigned i = 0; i < nr_pages; ++i )
     {
-        if ( !page_type_has_stream_data(types[i]) )
-            continue;
-
-        if ( map_errs[j] )
+        if ( map_errs[i] )
         {
             rc = -1;
             ERROR("Mapping pfn %#"PRIpfn" (mfn %#"PRIpfn", type %#"PRIx32") failed with %d",
-                  pfns[i], mfns[j], types[i], map_errs[j]);
+                  pfns[i], mfns[i], types[i], map_errs[i]);
             goto err;
         }
 
@@ -337,7 +338,6 @@ static int process_page_data(struct xc_sr_context *ctx, unsigned int count,
             memcpy(guest_page, page_data, PAGE_SIZE);
         }
 
-        ++j;
         guest_page += PAGE_SIZE;
         page_data += PAGE_SIZE;
     }
Signature · 1 line
2.54.0
#21 lore raw

PATCH v4 8/16 libs/guest: fill directly iov structure

In reply to Frediano Ziglio

From: Frediano Ziglio <[email protected]>

Instead of storing page pointers into an array and lately adding to iov vector add the pages directly to iov to avoid "guest_data" array.

---
 tools/libs/guest/xg_sr_common.h |  1 -
 tools/libs/guest/xg_sr_save.c   | 62 ++++++++++++---------------------
 2 files changed, 22 insertions(+), 41 deletions(-)
tools/libs/guest/xg_sr_common.h +0 -1
index 82549b5589..e8452746e4 100644
@@ -215,7 +215,6 @@ struct xc_sr_context_save_buffers
     xen_pfn_t mfns[MAX_BATCH_SIZE];
     xen_pfn_t types[MAX_BATCH_SIZE];
     int errors[MAX_BATCH_SIZE];
-    void *guest_data[MAX_BATCH_SIZE];
     void *local_pages[MAX_BATCH_SIZE];
     struct iovec iov[MAX_BATCH_SIZE + 2]; /* Headers + data. */
     uint64_t rec_pfns[MAX_BATCH_SIZE];
tools/libs/guest/xg_sr_save.c +22 -40

104 lines not on this page · raw message

Signature · 1 line
2.54.0
#22 lore raw

PATCH v4 9/16 libs/ctrl: Allows writev_exact to change iov array

In reply to Frediano Ziglio

From: Frediano Ziglio <[email protected]>

Avoid having to allocate and copy the array if a partial write happens.
The implementation in tools/libs/store/xs.c already use this signature and method.

--
Changes since v2:
- change prefix in subject.

---
 tools/libs/ctrl/xc_private.c | 26 +++++---------------------
 tools/libs/ctrl/xc_private.h |  2 +-
 2 files changed, 6 insertions(+), 22 deletions(-)
tools/libs/ctrl/xc_private.c +5 -21
index bb0f81d6f3..946fc307aa 100644
@@ -635,7 +635,7 @@ int write_exact(int fd, const void *data, size_t size)
 /*
  * MiniOS's libc doesn't know about writev(). Implement it as multiple write()s.
  */
-int writev_exact(int fd, const struct iovec *iov, int iovcnt)
+int writev_exact(int fd, struct iovec *iov, int iovcnt)
 {
     int rc, i;
 
@@ -649,9 +649,8 @@ int writev_exact(int fd, const struct iovec *iov, int iovcnt)
     return 0;
 }
 #else
-int writev_exact(int fd, const struct iovec *iov, int iovcnt)
+int writev_exact(int fd, struct iovec *iov, int iovcnt)
 {
-    struct iovec *local_iov = NULL;
     int rc = 0, iov_idx = 0, saved_errno = 0;
     ssize_t len;
 
@@ -686,23 +685,9 @@ int writev_exact(int fd, const struct iovec *iov, int iovcnt)
                 len -= iov[iov_idx++].iov_len;
             else
             {
-                /* Partial write of iov[iov_idx]. Copy iov so we can adjust
-                 * element iov_idx and resubmit the rest. */
-                if ( !local_iov )
-                {
-                    local_iov = malloc(iovcnt * sizeof(*iov));
-                    if ( !local_iov )
-                    {
-                        saved_errno = ENOMEM;
-                        rc = -1;
-                        goto out;
-                    }
-
-                    iov = memcpy(local_iov, iov, iovcnt * sizeof(*iov));
-                }
-
-                local_iov[iov_idx].iov_base += len;
-                local_iov[iov_idx].iov_len  -= len;
+                /* Partial write of iov[iov_idx]. */
+                iov[iov_idx].iov_base += len;
+                iov[iov_idx].iov_len  -= len;
                 break;
             }
         }
@@ -711,7 +696,6 @@ int writev_exact(int fd, const struct iovec *iov, int iovcnt)
     saved_errno = 0;
 
  out:
-    free(local_iov);
     errno = saved_errno;
     return rc;
 }
tools/libs/ctrl/xc_private.h +1 -1
index b5892ae8dc..3af996e900 100644
@@ -383,7 +383,7 @@ int xc_flush_mmu_updates(xc_interface *xch, struct xc_mmu *mmu);
 /* Return 0 on success; -1 on error setting errno. */
 int read_exact(int fd, void *data, size_t size); /* EOF => -1, errno=0 */
 int write_exact(int fd, const void *data, size_t size);
-int writev_exact(int fd, const struct iovec *iov, int iovcnt);
+int writev_exact(int fd, struct iovec *iov, int iovcnt);
 
 int xc_ffs8(uint8_t x);
 int xc_ffs16(uint16_t x);
Signature · 1 line
2.54.0
#23 lore raw

PATCH v4 10/16 libs/guest: add xg_foreignmemory_copy_{from,to}

In reply to Frediano Ziglio

From: Frediano Ziglio <[email protected]>

This change prepare code to use a new "foreign copy" hypercall. The new hypercall will copy memory from/to a foreign domain. The new hypercall can be emulated with a sequence of:
- map foreign memory;
- copy memory;
- unmap foreign memory.

---
 tools/libs/guest/xg_sr_common.c | 60 +++++++++++++++++++++++++++++++++
 tools/libs/guest/xg_sr_common.h |  8 +++++
 2 files changed, 68 insertions(+)
tools/libs/guest/xg_sr_common.c +60 -0
index 86c148c62f..a94e9dfbff 100644
@@ -156,6 +156,66 @@ static void __attribute__((unused)) build_assertions(void)
     BUILD_BUG_ON(sizeof(struct xc_sr_rec_hvm_params)        != 8);
 }
 
+enum {
+    foreigncopy_from,
+    foreigncopy_to
+};
+
+static int xg_foreignmemory_copy(xc_interface *xch, domid_t domid,
+                                 int dir, size_t nr_pages, void *buffer,
+                                 const xen_pfn_t foreign_pfns[nr_pages])
+{
+    if ( nr_pages == 0 )
+        return 0;
+
+    if ( !buffer || !foreign_pfns )
+    {
+        errno = EINVAL;
+        return -1;
+    }
+
+    int err[nr_pages];
+    const int prot = (dir == foreigncopy_from) ? PROT_READ : PROT_READ|PROT_WRITE;
+
+    void *p = xenforeignmemory_map(xch->fmem, domid, prot, nr_pages, foreign_pfns, err);
+    if ( !p )
+    {
+        errno = EINVAL;
+        return -1;
+    }
+
+    for ( size_t n = 0; n < nr_pages; ++n )
+        if ( err[n] )
+        {
+            xenforeignmemory_unmap(xch->fmem, p, nr_pages);
+            errno = -err[n];
+            return -1;
+        }
+
+    if ( dir == foreigncopy_from )
+        memcpy(buffer, p, nr_pages * XC_PAGE_SIZE);
+    else
+        memcpy(p, buffer, nr_pages * XC_PAGE_SIZE);
+
+    return xenforeignmemory_unmap(xch->fmem, p, nr_pages);
+}
+
+int xg_foreignmemory_copy_from(xc_interface *xch, domid_t dom,
+                               size_t nr_pages, void *dest,
+                               const xen_pfn_t source[nr_pages])
+{
+    return xg_foreignmemory_copy(xch, dom, foreigncopy_from,
+                                 nr_pages, dest, source);
+}
+
+int xg_foreignmemory_copy_to(xc_interface *xch, domid_t dom,
+                             size_t nr_pages, const xen_pfn_t dest[nr_pages],
+                             const void *source)
+{
+    return xg_foreignmemory_copy(xch, dom, foreigncopy_to,
+                                 nr_pages, (void *) source, dest);
+}
+
 /*
  * Local variables:
  * mode: C
tools/libs/guest/xg_sr_common.h +8 -0
index e8452746e4..72c9511f38 100644
@@ -553,6 +553,14 @@ static inline bool page_type_has_stream_data(uint32_t type)
     }
 }
 
+int xg_foreignmemory_copy_from(xc_interface *xch, domid_t dom,
+                               size_t nr_pages, void *dest,
+                               const xen_pfn_t source[nr_pages]);
+
+int xg_foreignmemory_copy_to(xc_interface *xch, domid_t dom,
+                             size_t nr_pages, const xen_pfn_t dest[nr_pages],
+                             const void *source);
+
 #endif
 /*
  * Local variables:
Signature · 1 line
2.54.0
#24 lore raw

PATCH v4 11/16 PoC: libs/guest: use foreign copy during migration

In reply to Frediano Ziglio

From: Edwin Török <[email protected]>

ministat confirms the improvement:

```
x baseline
+ foreigncopy
    N           Min           Max        Median           Avg        Stddev
x  20     1.1306997     1.1447931     1.1356569     1.1365742   0.003242175
+  20     0.4311504    0.44180303    0.43616705    0.43600089  0.0031094689
Difference at 95.0% confidence
	-0.700573 +/- 0.00203311
	-61.639% +/- 0.133355%
	(Student's t, pooled s = 0.00317652)
```

The tests pass too, which means that it has correctly migrated all guest memory.

Frediano: This PoC was adapted to be included in a final series.

---
 tools/libs/guest/xg_sr_common.h  |  1 +
 tools/libs/guest/xg_sr_restore.c | 42 +++--------------
 tools/libs/guest/xg_sr_save.c    | 81 +++++++++-----------------------
 3 files changed, 30 insertions(+), 94 deletions(-)
tools/libs/guest/xg_sr_common.h +1 -0
index 72c9511f38..0e0e279ae1 100644
@@ -256,6 +256,7 @@ struct xc_sr_context
             unsigned long nr_deferred_pages;
             xc_hypercall_buffer_t dirty_bitmap_hbuf;
             struct xc_sr_context_save_buffers *buffers;
+            void *dest_buf;
         } save;
 
         struct /* Restore data. */
tools/libs/guest/xg_sr_restore.c +6 -36

69 lines not on this page · raw message

tools/libs/guest/xg_sr_save.c +23 -58

122 lines not on this page · raw message

Signature · 1 line
2.54.0
#25 lore raw

Re: [PATCH v4 11/16] PoC: libs/guest: use foreign copy during migration

Quoted 16 lines · Frediano Ziglio

From: Edwin Török <[email protected]>

ministat confirms the improvement:

```
x baseline
+ foreigncopy
    N           Min           Max        Median           Avg        Stddev
x  20     1.1306997     1.1447931     1.1356569     1.1365742   0.003242175
+  20     0.4311504    0.44180303    0.43616705    0.43600089  0.0031094689
Difference at 95.0% confidence
	-0.700573 +/- 0.00203311
	-61.639% +/- 0.133355%
	(Student's t, pooled s = 0.00317652)
```
Quoted 7 lines

The tests pass too, which means that it has correctly migrated all guest memory.

Frediano: This PoC was adapted to be included in a final series.

This patch is definitely out of order, seeing as you don't introduce the new hypercall until the following patch.

But there's also absolutely no information what those stats are.  From memory, I think it was wallclock time of migrating a VM, but there are no units or sizes of the VM presented, so those are just random numbers.

It also doesn't state whether it's measured from a PV or a PVH guest. However bad PV is (and it is bad), I think the improvement will be better in a PVH guest, because the foreign map/unmap operations being replaced are even more expensive in PVH.

There's another area in libxenguest which would likely benefit; domain construction.  Even with kernels and initrds in the MB range, foreign copy is probalby a win, and it surely will be for e.g. the ACPI tables which are a few kB.

~Andrew

#26 lore raw

Re: [PATCH v4 11/16] PoC: libs/guest: use foreign copy during migration

Quoted 21 lines · Andrew Cooper
Quoted 19 lines · Frediano Ziglio

From: Edwin Török <[email protected]>

ministat confirms the improvement:

```
x baseline
+ foreigncopy
    N           Min           Max        Median           Avg        Stddev
x  20     1.1306997     1.1447931     1.1356569     1.1365742   0.003242175
+  20     0.4311504    0.44180303    0.43616705    0.43600089  0.0031094689
Difference at 95.0% confidence
      -0.700573 +/- 0.00203311
      -61.639% +/- 0.133355%
      (Student's t, pooled s = 0.00317652)
```

The tests pass too, which means that it has correctly migrated all guest memory.

Quoted 8 lines
Quoted 4 lines

Frediano: This PoC was adapted to be included in a final series.

This patch is definitely out of order, seeing as you don't introduce the new hypercall until the following patch.

Yes and no. In the original PoC it was, as it was trying to check performance difference using the new hypercall. But to prove that code was working the new hypercalls were "emulated" using functions doing map/copy/unmap. This also makes sure no performance regressions using the new wrappers. So the code works with or without the new hypercalls.

Quoted 4 lines

But there's also absolutely no information what those stats are. From memory, I think it was wallclock time of migrating a VM, but there are no units or sizes of the VM presented, so those are just random numbers.

I had the same though. I quickly searched if Edwin reappeared on ML but it seems not at the moment.

Quoted 5 lines

It also doesn't state whether it's measured from a PV or a PVH guest. However bad PV is (and it is bad), I think the improvement will be better in a PVH guest, because the foreign map/unmap operations being replaced are even more expensive in PVH.

The PoC was simply stripping out PV support so surely they were PVH/HVM statistics.

Quoted 6 lines

There's another area in libxenguest which would likely benefit; domain construction. Even with kernels and initrds in the MB range, foreign copy is probalby a win, and it surely will be for e.g. the ACPI tables which are a few kB.

~Andrew

Frediano

#27 lore raw

PATCH v4 12/16 xen: implement new foreign copy hypercall

In reply to Frediano Ziglio

From: Frediano Ziglio <[email protected]>

Add a sub hypercall to __HYPERVISOR_memory_op to allow to read/write memory from/to a foreign domain.

---
 xen/common/memory.c         | 133 ++++++++++++++++++++++++++++++++++++
 xen/include/public/memory.h |  40 ++++++++++-
 2 files changed, 172 insertions(+), 1 deletion(-)
xen/common/memory.c +133 -0
index 3672bda025..6a2d9c3190 100644
@@ -1545,6 +1545,132 @@ static int acquire_resource(
     return rc;
 }
 
+/*
+ * The "noinline" qualifier avoid the compiler to create a large function
+ * consuming quite a lot of stack.
+ */
+static int noinline mem_foreigncopy(
+    XEN_GUEST_HANDLE_PARAM(xen_foreigncopy_t) arg)
+{
+    struct domain *d, *const currd = current->domain;
+    xen_foreigncopy_t copy;
+    int rc, direction;
+
+    if ( !arch_acquire_resource_check(currd) )
+        return -EACCES;
+
+    if ( copy_from_guest(&copy, arg, 1) )
+        return -EFAULT;
+
+    if ( copy.flags & ~1u )
+        return -EINVAL;
+
+    direction = copy.flags & XENMEM_foreigncopy_direction;
+
+    if ( copy.nr_frames == 0 )
+        return 0;
+
+    rc = rcu_lock_remote_domain_by_id(copy.domid, &d);
+    if ( rc )
+        return rc;
+
+    /*
+     * Check we are allowed to map and access these foreign pages.
+     */
+    rc = xsm_map_gmfn_foreign(XSM_TARGET, currd, d);
+    if ( rc )
+        goto out;
+
+    do {
+        /*
+         * Arbitrary size.  Not too much stack space, and a reasonable stride
+         * for continuation checks.
+         */
+        xen_pfn_t gfn_list[32];
+        unsigned int todo = MIN(ARRAY_SIZE(gfn_list), copy.nr_frames);
+
+        rc = -EFAULT;
+        if ( copy_from_guest(gfn_list, copy.frame_list, todo) )
+            goto out;
+
+        for ( unsigned i = 0; i < todo; i++ )
+        {
+            struct page_info *foreign_page;
+            void *foreign;
+            p2m_type_t p2mt;
+
+            foreign_page = get_page_from_gfn(d, gfn_list[i], &p2mt, P2M_ALLOC);
+
+            if ( unlikely(p2mt != p2m_ram_rw
+#ifdef CONFIG_X86
+                 && p2mt != p2m_ram_logdirty
+#endif
+                 ) && foreign_page )
+            {
+                put_page(foreign_page);
+                foreign_page = NULL;
+            }
+            if ( unlikely(!foreign_page) )
+            {
+                gdprintk(XENLOG_WARNING,
+                         "Error accessing foreign mfn %" PRI_mfn "\n",
+                         gfn_list[i]);
+                rc = -EINVAL;
+                copy.nr_frames -= i;
+                guest_handle_add_offset(copy.frame_list, i);
+                goto out;
+            }
+
+            /* A page is dirtied when it's being copied to. */
+            if ( direction == XENMEM_foreigncopy_to )
+                paging_mark_dirty(d, page_to_mfn(foreign_page));
+
+            foreign = map_domain_page(page_to_mfn(foreign_page));
+            if ( direction == XENMEM_foreigncopy_from )
+                rc = copy_to_guest(copy.buffer, foreign, PAGE_SIZE);
+            else
+                rc = copy_from_guest(foreign, copy.buffer, PAGE_SIZE);
+            unmap_domain_page(foreign);
+            put_page(foreign_page);
+
+            if ( unlikely(rc) )
+            {
+                gdprintk(XENLOG_WARNING,
+                         "Error copying to mfn %" PRI_mfn "\n", gfn_list[i]);
+                copy.nr_frames -= i;
+                guest_handle_add_offset(copy.frame_list, i);
+                goto out;
+            }
+
+            guest_handle_add_offset(copy.buffer, PAGE_SIZE);
+        }
+
+        copy.nr_frames -= todo;
+        guest_handle_add_offset(copy.frame_list, todo);
+
+        if ( copy.nr_frames && hypercall_preempt_check() )
+        {
+            rc = hypercall_create_continuation(
+                __HYPERVISOR_memory_op, "lh", XENMEM_foreigncopy, arg);
+            goto out;
+        }
+    } while ( copy.nr_frames );
+
+    rc = 0;
+
+ out:
+    rcu_unlock_domain(d);
+
+    /* Update in all cases, it allows the caller to know how many
+     * frames were successfully copied and the continuation to
+     * continue correctly.
+     */
+    if ( copy_to_guest(arg, &copy, 1) )
+        rc = -EFAULT;
+
+    return rc;
+}
+
 long do_memory_op(unsigned long cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
 {
     struct domain *d, *curr_d = current->domain;
@@ -2012,6 +2138,13 @@ long do_memory_op(unsigned long cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
             start_extent);
         break;
 
+    case XENMEM_foreigncopy:
+        if ( unlikely(start_extent) )
+            return -EINVAL;
+
+        rc = mem_foreigncopy(guest_handle_cast(arg, xen_foreigncopy_t));
+        break;
+
     default:
         rc = arch_memory_op(cmd, arg);
         break;
xen/include/public/memory.h +39 -1
index bd9fc37b52..b48d1f378f 100644
@@ -740,7 +740,45 @@ struct xen_vnuma_topology_info {
 typedef struct xen_vnuma_topology_info xen_vnuma_topology_info_t;
 DEFINE_XEN_GUEST_HANDLE(xen_vnuma_topology_info_t);
 
-/* Next available subop number is 29 */
+/*
+ * Copy memory from/to a given domain.
+ */
+#define XENMEM_foreigncopy 29
+struct xen_foreigncopy {
+    /* IN - The domain whose resource is to be copied. */
+    domid_t domid;
+
+    /* IN - Flags. */
+#define XENMEM_foreigncopy_from 0
+#define XENMEM_foreigncopy_to 1
+#define XENMEM_foreigncopy_direction 1
+    uint16_t flags;
+
+    /*
+     * IN
+     *
+     * As an IN parameter number of frames of the domain to be copied.
+     */
+    uint32_t nr_frames;
+
+    /*
+     * IN
+     *
+     * Frames to be copied.
+     */
+    XEN_GUEST_HANDLE(xen_pfn_t) frame_list;
+
+    /*
+     * IN/OUT
+     *
+     * Userspace buffer to read/write from.
+     */
+    XEN_GUEST_HANDLE(uint8) buffer;
+};
+typedef struct xen_foreigncopy xen_foreigncopy_t;
+DEFINE_XEN_GUEST_HANDLE(xen_foreigncopy_t);
+
+/* Next available subop number is 30 */
 
 #endif /* __XEN_PUBLIC_MEMORY_H__ */
 
Signature · 1 line
2.54.0
#28 lore raw

Re: [PATCH v4 12/16] xen: implement new foreign copy hypercall

Quoted 9 lines · Frediano Ziglio
xen/common/memory.c +3 -0
@@ -1545,6 +1545,132 @@ static int acquire_resource(
     return rc;
 }
 
+/*
+ * The "noinline" qualifier avoid the compiler to create a large function
+ * consuming quite a lot of stack.

xen/common/memory.c static int acquire_resource( · line 1560

Quoted 10 lines
xen/common/memory.c +10 -0
+ */
+static int noinline mem_foreigncopy(
+    XEN_GUEST_HANDLE_PARAM(xen_foreigncopy_t) arg)
+{
+    struct domain *d, *const currd = current->domain;
+    xen_foreigncopy_t copy;
+    int rc, direction;
+
+    if ( !arch_acquire_resource_check(currd) )
+        return -EACCES;

How is the new sub-op related to acquire-resource? And aren't you instead lacking an XSM check?

xen/common/memory.c static int acquire_resource( · line 1565

Quoted 4 lines
xen/common/memory.c +4 -0
+    if ( copy_from_guest(&copy, arg, 1) )
+        return -EFAULT;
+
+    if ( copy.flags & ~1u )

If a suffix is needed here in the first place, please use an upper-case one. Misra only demands L suffixes to be upper-case, but iirc we decided that then we want all suffixes that way.

xen/common/memory.c static int acquire_resource( · line 1573

Quoted 8 lines
xen/common/memory.c +8 -0
+        return -EINVAL;
+
+    direction = copy.flags & XENMEM_foreigncopy_direction;
+
+    if ( copy.nr_frames == 0 )
+        return 0;
+
+    rc = rcu_lock_remote_domain_by_id(copy.domid, &d);

Why not rcu_lock_domain_by_any_id()? IOW why would a self-copy need prohibiting?

Quoted 25 lines
xen/common/memory.c +25 -0
+    if ( rc )
+        return rc;
+
+    /*
+     * Check we are allowed to map and access these foreign pages.
+     */
+    rc = xsm_map_gmfn_foreign(XSM_TARGET, currd, d);
+    if ( rc )
+        goto out;
+
+    do {
+        /*
+         * Arbitrary size.  Not too much stack space, and a reasonable stride
+         * for continuation checks.
+         */
+        xen_pfn_t gfn_list[32];
+        unsigned int todo = MIN(ARRAY_SIZE(gfn_list), copy.nr_frames);
+
+        rc = -EFAULT;
+        if ( copy_from_guest(gfn_list, copy.frame_list, todo) )
+            goto out;
+
+        for ( unsigned i = 0; i < todo; i++ )
+        {
+            struct page_info *foreign_page;

xen/common/memory.c static int acquire_resource( · line 1608

Quoted 10 lines
xen/common/memory.c +10 -0
+            void *foreign;
+            p2m_type_t p2mt;
+
+            foreign_page = get_page_from_gfn(d, gfn_list[i], &p2mt, P2M_ALLOC);
+
+            if ( unlikely(p2mt != p2m_ram_rw
+#ifdef CONFIG_X86
+                 && p2mt != p2m_ram_logdirty
+#endif
+                 ) && foreign_page )

This is ugly formatting wise, and the use of unlikely() isn't very likely to have the effect you intend: As long as the compiler can't translate the && expression to something involving only a single conditional branch, which of the branches is it that is unlikely to be taken?

xen/common/memory.c static int acquire_resource( · line 1617

Quoted 9 lines
xen/common/memory.c +9 -0
+            {
+                put_page(foreign_page);
+                foreign_page = NULL;
+            }
+            if ( unlikely(!foreign_page) )
+            {
+                gdprintk(XENLOG_WARNING,
+                         "Error accessing foreign mfn %" PRI_mfn "\n",
+                         gfn_list[i]);

As per get_page_from_gfn() and gfn_list[] it's a GFN, not an MFN.

Quoted 12 lines
xen/common/memory.c +12 -0
+                rc = -EINVAL;
+                copy.nr_frames -= i;
+                guest_handle_add_offset(copy.frame_list, i);
+                goto out;
+            }
+
+            /* A page is dirtied when it's being copied to. */
+            if ( direction == XENMEM_foreigncopy_to )
+                paging_mark_dirty(d, page_to_mfn(foreign_page));
+
+            foreign = map_domain_page(page_to_mfn(foreign_page));
+            if ( direction == XENMEM_foreigncopy_from )

xen/common/memory.c static int acquire_resource( · line 1639

Quoted 10 lines
xen/common/memory.c +10 -0
+                rc = copy_to_guest(copy.buffer, foreign, PAGE_SIZE);
+            else
+                rc = copy_from_guest(foreign, copy.buffer, PAGE_SIZE);
+            unmap_domain_page(foreign);
+            put_page(foreign_page);
+
+            if ( unlikely(rc) )
+            {
+                gdprintk(XENLOG_WARNING,
+                         "Error copying to mfn %" PRI_mfn "\n", gfn_list[i]);

"to" isn't always correct here, and the problem wasn't with the GFN (not MFN) anyway, but with the buffer.

Quoted 19 lines
xen/common/memory.c +19 -0
+                copy.nr_frames -= i;
+                guest_handle_add_offset(copy.frame_list, i);
+                goto out;
+            }
+
+            guest_handle_add_offset(copy.buffer, PAGE_SIZE);
+        }
+
+        copy.nr_frames -= todo;
+        guest_handle_add_offset(copy.frame_list, todo);
+
+        if ( copy.nr_frames && hypercall_preempt_check() )
+        {
+            rc = hypercall_create_continuation(
+                __HYPERVISOR_memory_op, "lh", XENMEM_foreigncopy, arg);
+            goto out;
+        }
+    } while ( copy.nr_frames );
+

xen/common/memory.c static int acquire_resource( · line 1668

Quoted 10 lines
xen/common/memory.c +10 -0
+    rc = 0;
+
+ out:
+    rcu_unlock_domain(d);
+
+    /* Update in all cases, it allows the caller to know how many
+     * frames were successfully copied and the continuation to
+     * continue correctly.
+     */
+    if ( copy_to_guest(arg, &copy, 1) )

Since you already used copy_from_guest() one the way in, __copy_to_guest() will do here.

xen/common/memory.c static int acquire_resource( · line 1672

Quoted 4 lines
xen/common/memory.c +4 -0
+        rc = -EFAULT;
+
+    return rc;
+}

Before looking at the implementation in yet more detail: This is quite a bit of new code. Did you at least consider extending MMUEXT_COPY_PAGE along the lines of what c6b8bdfe3b47 ("x86: extend mmu_update hypercall to allow update of foreign pagetables") did, allowing two domains to be specified in the foreigndom argument?

xen/common/memory.c long do_memory_op(unsigned long cmd, XEN_GUEST_HANDLE_PARAM(void) arg) · line 2143

Quoted 7 lines
xen/common/memory.c +3 -0
@@ -2012,6 +2138,13 @@ long do_memory_op(unsigned long cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
             start_extent);
         break;
 
+    case XENMEM_foreigncopy:
+        if ( unlikely(start_extent) )
+            return -EINVAL;

Why make this different from other continuable sub-ops?

Quoted 32 lines
xen/include/public/memory.h +25 -1
@@ -740,7 +740,45 @@ struct xen_vnuma_topology_info {
 typedef struct xen_vnuma_topology_info xen_vnuma_topology_info_t;
 DEFINE_XEN_GUEST_HANDLE(xen_vnuma_topology_info_t);
 
-/* Next available subop number is 29 */
+/*
+ * Copy memory from/to a given domain.
+ */
+#define XENMEM_foreigncopy 29
+struct xen_foreigncopy {
+    /* IN - The domain whose resource is to be copied. */
+    domid_t domid;
+
+    /* IN - Flags. */
+#define XENMEM_foreigncopy_from 0
+#define XENMEM_foreigncopy_to 1
+#define XENMEM_foreigncopy_direction 1
+    uint16_t flags;
+
+    /*
+     * IN
+     *
+     * As an IN parameter number of frames of the domain to be copied.
+     */
+    uint32_t nr_frames;
+
+    /*
+     * IN
+     *
+     * Frames to be copied.

xen/include/public/memory.h struct xen_vnuma_topology_info { · line 777

Quoted 10 lines
xen/include/public/memory.h +10 -0
+     */
+    XEN_GUEST_HANDLE(xen_pfn_t) frame_list;
+
+    /*
+     * IN/OUT
+     *
+     * Userspace buffer to read/write from.
+     */
+    XEN_GUEST_HANDLE(uint8) buffer;
+};

Seeing the two handles - what about compat_memory_op()?

Jan

#29 lore raw

Re: [PATCH v4 12/16] xen: implement new foreign copy hypercall

Quoted 32 lines · Jan Beulich
Quoted 31 lines · Frediano Ziglio
xen/common/memory.c +31 -0
+    if ( rc )
+        return rc;
+
+    /*
+     * Check we are allowed to map and access these foreign pages.
+     */
+    rc = xsm_map_gmfn_foreign(XSM_TARGET, currd, d);
+    if ( rc )
+        goto out;
+
+    do {
+        /*
+         * Arbitrary size.  Not too much stack space, and a reasonable stride
+         * for continuation checks.
+         */
+        xen_pfn_t gfn_list[32];
+        unsigned int todo = MIN(ARRAY_SIZE(gfn_list), copy.nr_frames);
+
+        rc = -EFAULT;
+        if ( copy_from_guest(gfn_list, copy.frame_list, todo) )
+            goto out;
+
+        for ( unsigned i = 0; i < todo; i++ )
+        {
+            struct page_info *foreign_page;
+            void *foreign;
+            p2m_type_t p2mt;
+
+            foreign_page = get_page_from_gfn(d, gfn_list[i], &p2mt, P2M_ALLOC);
+
+            if ( unlikely(p2mt != p2m_ram_rw
Quoted 8 lines

xen/common/memory.c static int acquire_resource( · line 1608

Quoted 4 lines
xen/common/memory.c +4 -0
+#ifdef CONFIG_X86
+                 && p2mt != p2m_ram_logdirty
+#endif
+                 ) && foreign_page )

This is ugly formatting wise, and the use of unlikely() isn't very likely to have the effect you intend: As long as the compiler can't translate the && expression to something involving only a single conditional branch, which of the branches is it that is unlikely to be taken?

Irrespective of what the compiler thinks or may do, Eclair will hard reject it because unlikely() is a macro.

~Andrew

#30 lore raw

Re: [PATCH v4 12/16] xen: implement new foreign copy hypercall

In reply to Frediano Ziglio

Le 03/06/2026 à 15:08, Frediano Ziglio a écrit :

Quoted 182 lines

From: Frediano Ziglio <[email protected]>

Add a sub hypercall to __HYPERVISOR_memory_op to allow to read/write memory from/to a foreign domain.

---
  xen/common/memory.c         | 133 ++++++++++++++++++++++++++++++++++++
  xen/include/public/memory.h |  40 ++++++++++-
  2 files changed, 172 insertions(+), 1 deletion(-)
xen/common/memory.c +133 -0
index 3672bda025..6a2d9c3190 100644
@@ -1545,6 +1545,132 @@ static int acquire_resource(
      return rc;
  }
 
+/*
+ * The "noinline" qualifier avoid the compiler to create a large function
+ * consuming quite a lot of stack.
+ */
+static int noinline mem_foreigncopy(
+    XEN_GUEST_HANDLE_PARAM(xen_foreigncopy_t) arg)
+{
+    struct domain *d, *const currd = current->domain;
+    xen_foreigncopy_t copy;
+    int rc, direction;
+
+    if ( !arch_acquire_resource_check(currd) )
+        return -EACCES;
+
+    if ( copy_from_guest(&copy, arg, 1) )
+        return -EFAULT;
+
+    if ( copy.flags & ~1u )
+        return -EINVAL;
+
+    direction = copy.flags & XENMEM_foreigncopy_direction;
+
+    if ( copy.nr_frames == 0 )
+        return 0;
+
+    rc = rcu_lock_remote_domain_by_id(copy.domid, &d);
+    if ( rc )
+        return rc;
+
+    /*
+     * Check we are allowed to map and access these foreign pages.
+     */
+    rc = xsm_map_gmfn_foreign(XSM_TARGET, currd, d);
+    if ( rc )
+        goto out;
+
+    do {
+        /*
+         * Arbitrary size.  Not too much stack space, and a reasonable stride
+         * for continuation checks.
+         */
+        xen_pfn_t gfn_list[32];
+        unsigned int todo = MIN(ARRAY_SIZE(gfn_list), copy.nr_frames);
+
+        rc = -EFAULT;
+        if ( copy_from_guest(gfn_list, copy.frame_list, todo) )
+            goto out;
+
+        for ( unsigned i = 0; i < todo; i++ )
+        {
+            struct page_info *foreign_page;
+            void *foreign;
+            p2m_type_t p2mt;
+
+            foreign_page = get_page_from_gfn(d, gfn_list[i], &p2mt, P2M_ALLOC);
+
+            if ( unlikely(p2mt != p2m_ram_rw
+#ifdef CONFIG_X86
+                 && p2mt != p2m_ram_logdirty
+#endif
+                 ) && foreign_page )
+            {
+                put_page(foreign_page);
+                foreign_page = NULL;
+            }
+            if ( unlikely(!foreign_page) )
+            {
+                gdprintk(XENLOG_WARNING,
+                         "Error accessing foreign mfn %" PRI_mfn "\n",
+                         gfn_list[i]);
+                rc = -EINVAL;
+                copy.nr_frames -= i;
+                guest_handle_add_offset(copy.frame_list, i);
+                goto out;
+            }
+
+            /* A page is dirtied when it's being copied to. */
+            if ( direction == XENMEM_foreigncopy_to )
+                paging_mark_dirty(d, page_to_mfn(foreign_page));
+
+            foreign = map_domain_page(page_to_mfn(foreign_page));
+            if ( direction == XENMEM_foreigncopy_from )
+                rc = copy_to_guest(copy.buffer, foreign, PAGE_SIZE);
+            else
+                rc = copy_from_guest(foreign, copy.buffer, PAGE_SIZE);
+            unmap_domain_page(foreign);
+            put_page(foreign_page);
+
+            if ( unlikely(rc) )
+            {
+                gdprintk(XENLOG_WARNING,
+                         "Error copying to mfn %" PRI_mfn "\n", gfn_list[i]);
+                copy.nr_frames -= i;
+                guest_handle_add_offset(copy.frame_list, i);
+                goto out;
+            }
+
+            guest_handle_add_offset(copy.buffer, PAGE_SIZE);
+        }
+
+        copy.nr_frames -= todo;
+        guest_handle_add_offset(copy.frame_list, todo);
+
+        if ( copy.nr_frames && hypercall_preempt_check() )
+        {
+            rc = hypercall_create_continuation(
+                __HYPERVISOR_memory_op, "lh", XENMEM_foreigncopy, arg);
+            goto out;
+        }
+    } while ( copy.nr_frames );
+
+    rc = 0;
+
+ out:
+    rcu_unlock_domain(d);
+
+    /* Update in all cases, it allows the caller to know how many
+     * frames were successfully copied and the continuation to
+     * continue correctly.
+     */
+    if ( copy_to_guest(arg, &copy, 1) )
+        rc = -EFAULT;
+
+    return rc;
+}
+
  long do_memory_op(unsigned long cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
  {
      struct domain *d, *curr_d = current->domain;
@@ -2012,6 +2138,13 @@ long do_memory_op(unsigned long cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
              start_extent);
          break;
 
+    case XENMEM_foreigncopy:
+        if ( unlikely(start_extent) )
+            return -EINVAL;
+
+        rc = mem_foreigncopy(guest_handle_cast(arg, xen_foreigncopy_t));
+        break;
+
      default:
          rc = arch_memory_op(cmd, arg);
          break;
xen/include/public/memory.h +11 -1
index bd9fc37b52..b48d1f378f 100644
@@ -740,7 +740,45 @@ struct xen_vnuma_topology_info {
  typedef struct xen_vnuma_topology_info xen_vnuma_topology_info_t;
  DEFINE_XEN_GUEST_HANDLE(xen_vnuma_topology_info_t);
 
-/* Next available subop number is 29 */
+/*
+ * Copy memory from/to a given domain.
+ */
+#define XENMEM_foreigncopy 29
+struct xen_foreigncopy {
+    /* IN - The domain whose resource is to be copied. */
+    domid_t domid;
+
+    /* IN - Flags. */
+#define XENMEM_foreigncopy_from 0
+#define XENMEM_foreigncopy_to 1

xen/include/public/memory.h struct xen_vnuma_topology_info { · line 763

Quoted 10 lines
xen/include/public/memory.h +10 -0
+#define XENMEM_foreigncopy_direction 1
+    uint16_t flags;
+
+    /*
+     * IN
+     *
+     * As an IN parameter number of frames of the domain to be copied.
+     */
+    uint32_t nr_frames;
+

The interface only allows copies to be made at page granularity, while that can be ok, that probably wants to be stated.

Quoted 4 lines
xen/include/public/memory.h +4 -0
+    /*
+     * IN
+     *
+     * Frames to be copied.

xen/include/public/memory.h struct xen_vnuma_topology_info { · line 777

Quoted 10 lines
xen/include/public/memory.h +10 -0
+     */
+    XEN_GUEST_HANDLE(xen_pfn_t) frame_list;
+
+    /*
+     * IN/OUT
+     *
+     * Userspace buffer to read/write from.
+     */
+    XEN_GUEST_HANDLE(uint8) buffer;
+};

The interface looks a bit asymetric, on one hand, it takes pfns and on the other hand, it takes a guest virtual address.

Though, using guest pointers is not great for PVH domains, as it requires expensive pagewalks (especially for a lot of pages).

Would it be preferable to have a list of gmfn for both sides ?

xen/include/public/memory.h struct xen_vnuma_topology_info { · line 784

Quoted 7 lines
xen/include/public/memory.h +4 -0
+typedef struct xen_foreigncopy xen_foreigncopy_t;
+DEFINE_XEN_GUEST_HANDLE(xen_foreigncopy_t);
+
+/* Next available subop number is 30 */
 
  #endif /* __XEN_PUBLIC_MEMORY_H__ */
 

Teddy

#31 lore raw

Re: [PATCH v4 12/16] xen: implement new foreign copy hypercall

Quoted 216 lines · Teddy Astie
Quoted 182 lines · Frediano Ziglio

From: Frediano Ziglio <[email protected]>

Add a sub hypercall to __HYPERVISOR_memory_op to allow to read/write memory from/to a foreign domain.

---
  xen/common/memory.c         | 133 ++++++++++++++++++++++++++++++++++++
  xen/include/public/memory.h |  40 ++++++++++-
  2 files changed, 172 insertions(+), 1 deletion(-)
xen/common/memory.c +133 -0
index 3672bda025..6a2d9c3190 100644
@@ -1545,6 +1545,132 @@ static int acquire_resource(
      return rc;
  }
 
+/*
+ * The "noinline" qualifier avoid the compiler to create a large function
+ * consuming quite a lot of stack.
+ */
+static int noinline mem_foreigncopy(
+    XEN_GUEST_HANDLE_PARAM(xen_foreigncopy_t) arg)
+{
+    struct domain *d, *const currd = current->domain;
+    xen_foreigncopy_t copy;
+    int rc, direction;
+
+    if ( !arch_acquire_resource_check(currd) )
+        return -EACCES;
+
+    if ( copy_from_guest(&copy, arg, 1) )
+        return -EFAULT;
+
+    if ( copy.flags & ~1u )
+        return -EINVAL;
+
+    direction = copy.flags & XENMEM_foreigncopy_direction;
+
+    if ( copy.nr_frames == 0 )
+        return 0;
+
+    rc = rcu_lock_remote_domain_by_id(copy.domid, &d);
+    if ( rc )
+        return rc;
+
+    /*
+     * Check we are allowed to map and access these foreign pages.
+     */
+    rc = xsm_map_gmfn_foreign(XSM_TARGET, currd, d);
+    if ( rc )
+        goto out;
+
+    do {
+        /*
+         * Arbitrary size.  Not too much stack space, and a reasonable stride
+         * for continuation checks.
+         */
+        xen_pfn_t gfn_list[32];
+        unsigned int todo = MIN(ARRAY_SIZE(gfn_list), copy.nr_frames);
+
+        rc = -EFAULT;
+        if ( copy_from_guest(gfn_list, copy.frame_list, todo) )
+            goto out;
+
+        for ( unsigned i = 0; i < todo; i++ )
+        {
+            struct page_info *foreign_page;
+            void *foreign;
+            p2m_type_t p2mt;
+
+            foreign_page = get_page_from_gfn(d, gfn_list[i], &p2mt, P2M_ALLOC);
+
+            if ( unlikely(p2mt != p2m_ram_rw
+#ifdef CONFIG_X86
+                 && p2mt != p2m_ram_logdirty
+#endif
+                 ) && foreign_page )
+            {
+                put_page(foreign_page);
+                foreign_page = NULL;
+            }
+            if ( unlikely(!foreign_page) )
+            {
+                gdprintk(XENLOG_WARNING,
+                         "Error accessing foreign mfn %" PRI_mfn "\n",
+                         gfn_list[i]);
+                rc = -EINVAL;
+                copy.nr_frames -= i;
+                guest_handle_add_offset(copy.frame_list, i);
+                goto out;
+            }
+
+            /* A page is dirtied when it's being copied to. */
+            if ( direction == XENMEM_foreigncopy_to )
+                paging_mark_dirty(d, page_to_mfn(foreign_page));
+
+            foreign = map_domain_page(page_to_mfn(foreign_page));
+            if ( direction == XENMEM_foreigncopy_from )
+                rc = copy_to_guest(copy.buffer, foreign, PAGE_SIZE);
+            else
+                rc = copy_from_guest(foreign, copy.buffer, PAGE_SIZE);
+            unmap_domain_page(foreign);
+            put_page(foreign_page);
+
+            if ( unlikely(rc) )
+            {
+                gdprintk(XENLOG_WARNING,
+                         "Error copying to mfn %" PRI_mfn "\n", gfn_list[i]);
+                copy.nr_frames -= i;
+                guest_handle_add_offset(copy.frame_list, i);
+                goto out;
+            }
+
+            guest_handle_add_offset(copy.buffer, PAGE_SIZE);
+        }
+
+        copy.nr_frames -= todo;
+        guest_handle_add_offset(copy.frame_list, todo);
+
+        if ( copy.nr_frames && hypercall_preempt_check() )
+        {
+            rc = hypercall_create_continuation(
+                __HYPERVISOR_memory_op, "lh", XENMEM_foreigncopy, arg);
+            goto out;
+        }
+    } while ( copy.nr_frames );
+
+    rc = 0;
+
+ out:
+    rcu_unlock_domain(d);
+
+    /* Update in all cases, it allows the caller to know how many
+     * frames were successfully copied and the continuation to
+     * continue correctly.
+     */
+    if ( copy_to_guest(arg, &copy, 1) )
+        rc = -EFAULT;
+
+    return rc;
+}
+
  long do_memory_op(unsigned long cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
  {
      struct domain *d, *curr_d = current->domain;
@@ -2012,6 +2138,13 @@ long do_memory_op(unsigned long cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
              start_extent);
          break;
 
+    case XENMEM_foreigncopy:
+        if ( unlikely(start_extent) )
+            return -EINVAL;
+
+        rc = mem_foreigncopy(guest_handle_cast(arg, xen_foreigncopy_t));
+        break;
+
      default:
          rc = arch_memory_op(cmd, arg);
          break;
xen/include/public/memory.h +11 -1
index bd9fc37b52..b48d1f378f 100644
@@ -740,7 +740,45 @@ struct xen_vnuma_topology_info {
  typedef struct xen_vnuma_topology_info xen_vnuma_topology_info_t;
  DEFINE_XEN_GUEST_HANDLE(xen_vnuma_topology_info_t);
 
-/* Next available subop number is 29 */
+/*
+ * Copy memory from/to a given domain.
+ */
+#define XENMEM_foreigncopy 29
+struct xen_foreigncopy {
+    /* IN - The domain whose resource is to be copied. */
+    domid_t domid;
+
+    /* IN - Flags. */
+#define XENMEM_foreigncopy_from 0
+#define XENMEM_foreigncopy_to 1

xen/include/public/memory.h struct xen_vnuma_topology_info { · line 763

Quoted 10 lines
xen/include/public/memory.h +10 -0
+#define XENMEM_foreigncopy_direction 1
+    uint16_t flags;
+
+    /*
+     * IN
+     *
+     * As an IN parameter number of frames of the domain to be copied.
+     */
+    uint32_t nr_frames;
+

The interface only allows copies to be made at page granularity, while that can be ok, that probably wants to be stated.

Quoted 4 lines
xen/include/public/memory.h +4 -0
+    /*
+     * IN
+     *
+     * Frames to be copied.

xen/include/public/memory.h struct xen_vnuma_topology_info { · line 777

Quoted 10 lines
xen/include/public/memory.h +10 -0
+     */
+    XEN_GUEST_HANDLE(xen_pfn_t) frame_list;
+
+    /*
+     * IN/OUT
+     *
+     * Userspace buffer to read/write from.
+     */
+    XEN_GUEST_HANDLE(uint8) buffer;
+};

The interface looks a bit asymetric, on one hand, it takes pfns and on the other hand, it takes a guest virtual address.

Quoted 5 lines

Though, using guest pointers is not great for PVH domains, as it requires expensive pagewalks (especially for a lot of pages).

Would it be preferable to have a list of gmfn for both sides ?

All this was considered and the interface could accept either virtual pointer or GFNs.
However, the code was more complicated and I think to resolve this correctly is out of scope here.
How is the structure passed using the hypercall? With a virtual pointer? Why not complain also about that pointer? It would be good for PVH not to use virtual addresses at all.
The PFN/virtual should be solved more generically.

Quoted 9 lines

xen/include/public/memory.h struct xen_vnuma_topology_info { · line 784

Quoted 7 lines
xen/include/public/memory.h +4 -0
+typedef struct xen_foreigncopy xen_foreigncopy_t;
+DEFINE_XEN_GUEST_HANDLE(xen_foreigncopy_t);
+
+/* Next available subop number is 30 */
 
  #endif /* __XEN_PUBLIC_MEMORY_H__ */
 

Teddy

Frediano

#32 lore raw

PATCH v4 13/16 privcmd: Add definition for new Linux privcmd to access new Xen hypercall

In reply to Frediano Ziglio

From: Frediano Ziglio <[email protected]>

Userspace should use new ioctl to access new hypercall.

---
 tools/include/xen-sys/Linux/privcmd.h | 10 ++++++++++
 1 file changed, 10 insertions(+)
tools/include/xen-sys/Linux/privcmd.h +10 -0
index 607dfa2287..4b80eeae06 100644
@@ -100,6 +100,14 @@ typedef struct privcmd_pcidev_get_gsi {
 	__u32 gsi;
 } privcmd_pcidev_get_gsi_t;
 
+typedef struct privcmd_foreigncopy {
+	domid_t dom;          /* Foreign domain. */
+	__u16 dir;            /* Direction,  0 from, 1 to. */
+	__u32 num;            /* Number of pages to copy. */
+	const xen_pfn_t __user *pfns; /* Array of pfns. */
+	void __user *buffer;  /* Buffer to copy to/from, must be page aligned. */
+} privcmd_foreigncopy_t;
+
 /*
  * @cmd: IOCTL_PRIVCMD_HYPERCALL
  * @arg: &privcmd_hypercall_t
@@ -121,6 +129,8 @@ typedef struct privcmd_pcidev_get_gsi {
 	_IOC(_IOC_NONE, 'P', 7, sizeof(privcmd_mmap_resource_t))
 #define IOCTL_PRIVCMD_PCIDEV_GET_GSI			\
 	_IOC(_IOC_NONE, 'P', 10, sizeof(privcmd_pcidev_get_gsi_t))
+#define IOCTL_PRIVCMD_FOREIGNCOPY				\
+	_IOC(_IOC_NONE, 'P', 11, sizeof(privcmd_foreigncopy_t))
 #define IOCTL_PRIVCMD_UNIMPLEMENTED				\
 	_IOC(_IOC_NONE, 'P', 0xFF, 0)
 
Signature · 1 line
2.54.0
#33 lore raw

PATCH v4 14/16 libs/guest: use new hypercall if available

In reply to Frediano Ziglio

From: Frediano Ziglio <[email protected]>

Use new hypercall if available, otherwise fall back to map+copy+unmap sequence.

---
 tools/libs/guest/xg_sr_common.c | 47 ++++++++++++++++++++++++++-------
 1 file changed, 38 insertions(+), 9 deletions(-)
tools/libs/guest/xg_sr_common.c +38 -9
index a94e9dfbff..cb3e6a5658 100644
@@ -156,11 +156,6 @@ static void __attribute__((unused)) build_assertions(void)
     BUILD_BUG_ON(sizeof(struct xc_sr_rec_hvm_params)        != 8);
 }
 
-enum {
-    foreigncopy_from,
-    foreigncopy_to
-};
-
 static int xg_foreignmemory_copy(xc_interface *xch, domid_t domid,
                                  int dir, size_t nr_pages, void *buffer,
                                  const xen_pfn_t foreign_pfns[nr_pages])
@@ -174,8 +169,42 @@ static int xg_foreignmemory_copy(xc_interface *xch, domid_t domid,
         return -1;
     }
 
+    /*
+     * If foreign copy is supported, -1 not initialized, 0 not supported,
+     * 1 supported.
+     */
+    static char foreign_copy_supported = -1;
+
+    if ( foreign_copy_supported )
+    {
+        int rc;
+        privcmd_foreigncopy_t copy = {
+            .dom = domid,
+            .dir = dir,
+            .num = nr_pages,
+            .buffer = buffer,
+        };
+        DECLARE_HYPERCALL_BOUNCE_IN(foreign_pfns, nr_pages * sizeof(xen_pfn_t));
+
+        if ( xc_hypercall_bounce_pre(xch, foreign_pfns) )
+            return -1;
+
+        copy.pfns = foreign_pfns;
+
+        rc = ioctl(xencall_fd(xch->xcall), IOCTL_PRIVCMD_FOREIGNCOPY, &copy);
+        if ( foreign_copy_supported < 0 )
+            foreign_copy_supported =
+                (!rc || (errno != ENOTTY && errno != ENOSYS));
+
+        xc_hypercall_bounce_post(xch, foreign_pfns);
+
+        if ( foreign_copy_supported )
+            return rc;
+    }
+
+    /* Fallback, emulate. */
     int err[nr_pages];
-    const int prot = (dir == foreigncopy_from) ? PROT_READ : PROT_READ|PROT_WRITE;
+    const int prot = (dir == XENMEM_foreigncopy_from) ? PROT_READ : PROT_READ|PROT_WRITE;
 
     void *p = xenforeignmemory_map(xch->fmem, domid, prot, nr_pages, foreign_pfns, err);
     if ( !p )
@@ -192,7 +221,7 @@ static int xg_foreignmemory_copy(xc_interface *xch, domid_t domid,
             return -1;
         }
 
-    if ( dir == foreigncopy_from )
+    if ( dir == XENMEM_foreigncopy_from )
         memcpy(buffer, p, nr_pages * XC_PAGE_SIZE);
     else
         memcpy(p, buffer, nr_pages * XC_PAGE_SIZE);
@@ -204,7 +233,7 @@ int xg_foreignmemory_copy_from(xc_interface *xch, domid_t dom,
                                size_t nr_pages, void *dest,
                                const xen_pfn_t source[nr_pages])
 {
-    return xg_foreignmemory_copy(xch, dom, foreigncopy_from,
+    return xg_foreignmemory_copy(xch, dom, XENMEM_foreigncopy_from,
                                  nr_pages, dest, source);
 }
 
@@ -212,7 +241,7 @@ int xg_foreignmemory_copy_to(xc_interface *xch, domid_t dom,
                              size_t nr_pages, const xen_pfn_t dest[nr_pages],
                              const void *source)
 {
-    return xg_foreignmemory_copy(xch, dom, foreigncopy_to,
+    return xg_foreignmemory_copy(xch, dom, XENMEM_foreigncopy_to,
                                  nr_pages, (void *) source, dest);
 }
 
Signature · 1 line
2.54.0
#34 lore raw

PATCH v4 15/16 libs/guest: finalize PoC

In reply to Frediano Ziglio

From: Frediano Ziglio <[email protected]>

Remove now unused map_errs array.
Test and restore verification code.
Report correctly errors from writev_exact.
Allocate verification buffer using hypercall buffer to avoid errors using hypercall.

---
 tools/libs/guest/xg_sr_common.h  |  4 +-
 tools/libs/guest/xg_sr_restore.c | 48 +++++++++++++++---
 tools/libs/guest/xg_sr_save.c    | 83 +++++++++++++++++++++-----------
 3 files changed, 100 insertions(+), 35 deletions(-)
tools/libs/guest/xg_sr_common.h +2 -2
index 0e0e279ae1..cd562f028a 100644
@@ -214,7 +214,6 @@ struct xc_sr_context_save_buffers
     xen_pfn_t batch_pfns[MAX_BATCH_SIZE];
     xen_pfn_t mfns[MAX_BATCH_SIZE];
     xen_pfn_t types[MAX_BATCH_SIZE];
-    int errors[MAX_BATCH_SIZE];
     void *local_pages[MAX_BATCH_SIZE];
     struct iovec iov[MAX_BATCH_SIZE + 2]; /* Headers + data. */
     uint64_t rec_pfns[MAX_BATCH_SIZE];
@@ -255,8 +254,8 @@ struct xc_sr_context
             unsigned long *deferred_pages;
             unsigned long nr_deferred_pages;
             xc_hypercall_buffer_t dirty_bitmap_hbuf;
+            xc_hypercall_buffer_t dest_buf;
             struct xc_sr_context_save_buffers *buffers;
-            void *dest_buf;
         } save;
 
         struct /* Restore data. */
@@ -267,6 +266,7 @@ struct xc_sr_context
             int send_back_fd;
             unsigned long p2m_size;
             xc_hypercall_buffer_t dirty_bitmap_hbuf;
+            xc_hypercall_buffer_t verify_buf;
 
             /* From Image Header. */
             uint32_t format_version;
tools/libs/guest/xg_sr_restore.c +42 -6

92 lines not on this page · raw message

tools/libs/guest/xg_sr_save.c +56 -27

141 lines not on this page · raw message

Signature · 1 line
2.54.0
#35 lore raw

PATCH v4 16/16 xen/privcmd: Add new ABI to allow copying foreign memory

In reply to Frediano Ziglio

From: Frediano Ziglio <[email protected]>

This new ABI allows to copy foreign domain memory to/from a buffer. This avoids having to map/copy/unmap foreign memory which is expensive.
This operation is done particularly when migrating VMs.

---
 arch/x86/include/asm/xen/interface.h |  1 +
 drivers/xen/privcmd.c                | 51 ++++++++++++++++++++++++++++
 include/uapi/xen/privcmd.h           | 10 ++++++
 include/xen/interface/memory.h       | 37 ++++++++++++++++++++
 4 files changed, 99 insertions(+)
arch/x86/include/asm/xen/interface.h +1 -0
index a078a2b0f032..bac3c3bc60fd 100644
@@ -91,6 +91,7 @@ DEFINE_GUEST_HANDLE(int);
 DEFINE_GUEST_HANDLE(void);
 DEFINE_GUEST_HANDLE(uint64_t);
 DEFINE_GUEST_HANDLE(uint32_t);
+DEFINE_GUEST_HANDLE(uint8_t);
 DEFINE_GUEST_HANDLE(xen_pfn_t);
 DEFINE_GUEST_HANDLE(xen_ulong_t);
 #endif
drivers/xen/privcmd.c +51 -0

69 lines not on this page · raw message

include/uapi/xen/privcmd.h +10 -0
index 8e2c8fd44764..786d769ad4f8 100644
@@ -131,6 +131,14 @@ struct privcmd_pcidev_get_gsi {
 	__u32 gsi;
 };
 
+struct privcmd_foreigncopy {
+	domid_t dom;		/* foreign domain */
+	__u16 dir;		/* direction,  0 from, 1 to */
+	__u32 num;		/* number of pages to copy */
+	const xen_pfn_t __user *pfns;	/* array of pfns */
+	void __user *buffer;	/* buffer to copy to/from */
+};
+
 /*
  * @cmd: IOCTL_PRIVCMD_HYPERCALL
  * @arg: &privcmd_hypercall_t
@@ -164,5 +172,7 @@ struct privcmd_pcidev_get_gsi {
 	_IOW('P', 9, struct privcmd_ioeventfd)
 #define IOCTL_PRIVCMD_PCIDEV_GET_GSI				\
 	_IOC(_IOC_NONE, 'P', 10, sizeof(struct privcmd_pcidev_get_gsi))
+#define IOCTL_PRIVCMD_FOREIGNCOPY				\
+	_IOC(_IOC_NONE, 'P', 11, sizeof(struct privcmd_foreigncopy))
 
 #endif /* __LINUX_PUBLIC_PRIVCMD_H__ */
include/xen/interface/memory.h +37 -0
index 1a371a825c55..5981402fccde 100644
@@ -325,4 +325,41 @@ struct xen_mem_acquire_resource {
 };
 DEFINE_GUEST_HANDLE_STRUCT(xen_mem_acquire_resource);
 
+/*
+ * Copy memory from/to a given domain.
+ */
+#define XENMEM_foreigncopy 29
+struct xen_foreigncopy {
+    /* IN - The domain whose resource is to be copied */
+    domid_t domid;
+
+    /* IN - Flags */
+#define XENMEM_foreigncopy_from 0
+#define XENMEM_foreigncopy_to 1
+#define XENMEM_foreigncopy_direction 1
+    uint16_t flags;
+
+    /*
+     * IN
+     *
+     * As an IN parameter number of frames of the domain to be copied.
+     */
+    uint32_t nr_frames;
+
+    /*
+     * IN
+     *
+     * Frames to be copied.
+     */
+    GUEST_HANDLE(xen_pfn_t) frame_list;
+
+    /*
+     * IN/OUT
+     *
+     * Userspace buffer to read/write from.
+     */
+    GUEST_HANDLE(uint8_t) buffer;
+};
+DEFINE_GUEST_HANDLE_STRUCT(xen_foreigncopy);
+
 #endif /* __XEN_PUBLIC_MEMORY_H__ */
Signature · 1 line
2.54.0