Skip to contentSkip to conversation

Minor xenguest optimisations

PATCH Superseded Frediano Ziglio 3 participants 13 messages May 26 – Jun 3

v3 of 10 Superseded Latest is v10 →

lore
Share & tools
lore
#0 lore raw

v3 cover Minor xenguest optimisations

From: Frediano Ziglio <[email protected]>

Reduce some number of parts passed to writev. Avoid possible allocation sending data with writev. Reduce number of allocations sending memory state.

Changes since v1:
- add commit to cache up to 4 pages in hypercall;
- add other 2 commits reducing chunks passed to write/writev.

Changes since v2:
- update patches commit prefixes;
- add other 2 optisations.

Edwin Török (2):
libs/guest: allocate various migration arrays just once libs/call: cache up to 4 pages in hypercall bounce buffers

Frediano Ziglio (7):
libs/guest: Reduce number of parts in write_split_record libs/guest: Reduce number of I/O vectors in write_batch libs/guest: Reduce number of I/O vectors in write_batch libs/guest: Use a single write_exact in write_headers libs/guest: avoids using 2 indexes
libs/guest: fill directly iov structure libs/ctrl: Allows writev_exact to change iov array

 tools/libs/call/buffer.c         |  28 +++--
 tools/libs/call/core.c           |   3 +-
 tools/libs/call/private.h        |   8 +-
 tools/libs/ctrl/xc_private.c     |  26 +----
 tools/libs/ctrl/xc_private.h     |   2 +-
 tools/libs/guest/xg_sr_common.c  |   6 +-
 tools/libs/guest/xg_sr_common.h  |  12 +++
 tools/libs/guest/xg_sr_restore.c |  26 ++---
 tools/libs/guest/xg_sr_save.c    | 169 +++++++++++++------------------
 9 files changed, 128 insertions(+), 152 deletions(-)
Signature · 1 line
2.54.0
#1 lore raw

PATCH v3 1/9 libs/guest: Reduce number of parts in write_split_record

From: Frediano Ziglio <[email protected]>

Small optimization.
There's no much sense to split the header in 2 pieces, it will just take more time and space to reassemble them in the final buffer.
This also avoids truncating combined_length to 32 bit in case of 64 bit machines potentially avoiding following record_length check. The function become more coherent with following read_record function.

--
Changes since v2:
- change prefix in subject.

---
 tools/libs/guest/xg_sr_common.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
tools/libs/guest/xg_sr_common.c +3 -3
index 7ccdc3b1f6aa..86c148c62f3e 100644
@@ -59,11 +59,11 @@ int write_split_record(struct xc_sr_context *ctx, struct xc_sr_record *rec,
     static const char zeroes[(1u << REC_ALIGN_ORDER) - 1] = { 0 };
 
     xc_interface *xch = ctx->xch;
-    typeof(rec->length) combined_length = rec->length + sz;
+    size_t combined_length = rec->length + sz;
     size_t record_length = ROUNDUP(combined_length, REC_ALIGN_ORDER);
+    struct xc_sr_rhdr rhdr = { rec->type, combined_length };
     struct iovec parts[] = {
-        { &rec->type,       sizeof(rec->type) },
-        { &combined_length, sizeof(combined_length) },
+        { &rhdr,            sizeof(rhdr) },
         { rec->data,        rec->length },
         { buf,              sz },
         { (void *)zeroes,   record_length - combined_length },
Signature · 1 line
2.54.0
#2 lore raw

Re: [PATCH v3 1/9] libs/guest: Reduce number of parts in write_split_record

Quoted 6 lines · Frediano Ziglio

From: Frediano Ziglio <[email protected]>

Small optimization.
There's no much sense to split the header in 2 pieces, it will just take more time and space to reassemble them in the final buffer.

This part is fine.

Quoted 4 lines

This also avoids truncating combined_length to 32 bit in case of 64 bit machines potentially avoiding following record_length check. The function become more coherent with following read_record function.

This isn't.  The truncation just happens in a different place now, when constructing rhdr.  The length is strictly 32 bits.

I'd just drop this paragraph.  The largest record is marginally over 4M, and callers won't be passing in anything larger.

~Andrew

#3 lore raw

Re: [PATCH v3 1/9] libs/guest: Reduce number of parts in write_split_record

Quoted 11 lines · Andrew Cooper
Quoted 6 lines · Frediano Ziglio

From: Frediano Ziglio <[email protected]>

Small optimization.
There's no much sense to split the header in 2 pieces, it will just take more time and space to reassemble them in the final buffer.

This part is fine.

Quoted 8 lines
Quoted 4 lines

This also avoids truncating combined_length to 32 bit in case of 64 bit machines potentially avoiding following record_length check. The function become more coherent with following read_record function.

This isn't. The truncation just happens in a different place now, when constructing rhdr. The length is strictly 32 bits.

Yes, there's still the truncation but the below check

if ( record_length > REC_LENGTH_MAX )

is now catching it.

Quoted 4 lines

I'd just drop this paragraph. The largest record is marginally over 4M, and callers won't be passing in anything larger.

~Andrew

Frediano

#4 lore raw

PATCH v3 2/9 libs/guest: Reduce number of I/O vectors in write_batch

In reply to Frediano Ziglio

From: Frediano Ziglio <[email protected]>

Small optimization.
Reduce number of pieces passed to writev.

--
Changes since v2:
- change prefix in subject.

---
 tools/libs/guest/xg_sr_save.c | 35 ++++++++++++++++-------------------
 1 file changed, 16 insertions(+), 19 deletions(-)
tools/libs/guest/xg_sr_save.c +16 -19
index 3b2c5222e429..1700d819051a 100644
@@ -97,9 +97,12 @@ static int write_batch(struct xc_sr_context *ctx)
     void *page, *orig_page;
     uint64_t *rec_pfns = NULL;
     struct iovec *iov = NULL; int iovcnt = 0;
-    struct xc_sr_rec_page_data_header hdr = { 0 };
-    struct xc_sr_record rec = {
-        .type = REC_TYPE_PAGE_DATA,
+    struct {
+        struct xc_sr_rhdr rec;
+        struct xc_sr_rec_page_data_header page_data;
+    } hdrs = {
+        { .type = REC_TYPE_PAGE_DATA },
+        { 0 },
     };
 
     assert(nr_pfns != 0);
@@ -115,7 +118,7 @@ static int write_batch(struct xc_sr_context *ctx)
     /* Pointers to locally allocated pages.  Need freeing. */
     local_pages = calloc(nr_pfns, sizeof(*local_pages));
     /* iovec[] for writev(). */
-    iov = malloc((nr_pfns + 4) * sizeof(*iov));
+    iov = malloc((nr_pfns + 2) * sizeof(*iov));
 
     if ( !mfns || !types || !errors || !guest_data || !local_pages || !iov )
     {
@@ -216,28 +219,22 @@ static int write_batch(struct xc_sr_context *ctx)
         goto err;
     }
 
-    hdr.count = nr_pfns;
+    hdrs.rec.length = sizeof(hdrs.page_data);
+    hdrs.rec.length += nr_pfns * sizeof(*rec_pfns);
+    hdrs.rec.length += nr_pages * PAGE_SIZE;
 
-    rec.length = sizeof(hdr);
-    rec.length += nr_pfns * sizeof(*rec_pfns);
-    rec.length += nr_pages * PAGE_SIZE;
+    hdrs.page_data.count = nr_pfns;
 
     for ( i = 0; i < nr_pfns; ++i )
         rec_pfns[i] = ((uint64_t)(types[i]) << 32) | ctx->save.batch_pfns[i];
 
-    iov[0].iov_base = &rec.type;
-    iov[0].iov_len = sizeof(rec.type);
+    iov[0].iov_base = &hdrs;
+    iov[0].iov_len = sizeof(hdrs);
 
-    iov[1].iov_base = &rec.length;
-    iov[1].iov_len = sizeof(rec.length);
+    iov[1].iov_base = rec_pfns;
+    iov[1].iov_len = nr_pfns * sizeof(*rec_pfns);
 
-    iov[2].iov_base = &hdr;
-    iov[2].iov_len = sizeof(hdr);
-
-    iov[3].iov_base = rec_pfns;
-    iov[3].iov_len = nr_pfns * sizeof(*rec_pfns);
-
-    iovcnt = 4;
+    iovcnt = 2;
 
     if ( nr_pages )
     {
Signature · 1 line
2.54.0
#5 lore raw

PATCH v3 3/9 libs/guest: Reduce number of I/O vectors in write_batch

In reply to Frediano Ziglio

From: Frediano Ziglio <[email protected]>

Each page was sent using a different iovec item. This potentially exceed Linux maximum (1024).
Also some implementation (MiniOS) emulate writev with multiple write calls.

--
Changes since v2:
- change prefix in subject.

---
 tools/libs/guest/xg_sr_save.c | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)
tools/libs/guest/xg_sr_save.c +9 -2
index 1700d819051a..62a39dfecc7a 100644
@@ -240,13 +240,20 @@ static int write_batch(struct xc_sr_context *ctx)
     {
         for ( i = 0; i < nr_pfns; ++i )
         {
-            if ( guest_data[i] )
+            if ( !guest_data[i] )
+                continue;
+
+            if ( iov[iovcnt-1].iov_base + iov[iovcnt-1].iov_len != guest_data[i] )
             {
                 iov[iovcnt].iov_base = guest_data[i];
                 iov[iovcnt].iov_len = PAGE_SIZE;
                 iovcnt++;
-                --nr_pages;
             }
+            else
+            {
+                iov[iovcnt-1].iov_len += PAGE_SIZE;
+            }
+            --nr_pages;
         }
     }
 
Signature · 1 line
2.54.0
#6 lore raw

PATCH v3 4/9 libs/guest: Use a single write_exact in write_headers

In reply to Frediano Ziglio

From: Frediano Ziglio <[email protected]>

Reduce number of syscalls.

--
Changes since v2:
- change prefix in subject.

---
 tools/libs/guest/xg_sr_save.c | 37 +++++++++++++++++------------------
 1 file changed, 18 insertions(+), 19 deletions(-)
tools/libs/guest/xg_sr_save.c +18 -19
index 62a39dfecc7a..8c4e760f8d0a 100644
@@ -10,17 +10,22 @@ static int write_headers(struct xc_sr_context *ctx, uint16_t guest_type)
 {
     xc_interface *xch = ctx->xch;
     int32_t xen_version = xc_version(xch, XENVER_version, NULL);
-    struct xc_sr_ihdr ihdr = {
-        .marker  = IHDR_MARKER,
-        .id      = htonl(IHDR_ID),
-        .version = htonl(3),
-        .options = htons(IHDR_OPT_LITTLE_ENDIAN),
-    };
-    struct xc_sr_dhdr dhdr = {
-        .type       = guest_type,
-        .page_shift = XC_PAGE_SHIFT,
-        .xen_major  = (xen_version >> 16) & 0xffff,
-        .xen_minor  = (xen_version)       & 0xffff,
+    struct {
+        struct xc_sr_ihdr ihdr;
+        struct xc_sr_dhdr dhdr;
+    } hdrs = {
+        {
+            .marker  = IHDR_MARKER,
+            .id      = htonl(IHDR_ID),
+            .version = htonl(3),
+            .options = htons(IHDR_OPT_LITTLE_ENDIAN),
+        },
+        {
+            .type       = guest_type,
+            .page_shift = XC_PAGE_SHIFT,
+            .xen_major  = (xen_version >> 16) & 0xffff,
+            .xen_minor  = (xen_version)       & 0xffff,
+        },
     };
 
     if ( xen_version < 0 )
@@ -29,15 +34,9 @@ static int write_headers(struct xc_sr_context *ctx, uint16_t guest_type)
         return -1;
     }
 
-    if ( write_exact(ctx->fd, &ihdr, sizeof(ihdr)) )
-    {
-        PERROR("Unable to write Image Header to stream");
-        return -1;
-    }
-
-    if ( write_exact(ctx->fd, &dhdr, sizeof(dhdr)) )
+    if ( write_exact(ctx->fd, &hdrs, sizeof(hdrs)) )
     {
-        PERROR("Unable to write Domain Header to stream");
+        PERROR("Unable to write Headers to stream");
         return -1;
     }
 
Signature · 1 line
2.54.0
#7 lore raw

PATCH v3 5/9 libs/guest: allocate various migration arrays just once

In reply to Frediano Ziglio

From: Edwin Török <[email protected]>

Allocate these array just once at the start of migration, using the maximum batch size, and free them at the end.

--
Changes since v2:
- change prefix in subject.

---
 tools/libs/guest/xg_sr_common.h | 13 +++++++
 tools/libs/guest/xg_sr_save.c   | 66 +++++++++++++--------------------
 2 files changed, 39 insertions(+), 40 deletions(-)
tools/libs/guest/xg_sr_common.h +13 -0
index f1573aefcbff..77312bae9cf5 100644
@@ -209,6 +209,18 @@ static inline int update_blob(struct xc_sr_blob *blob,
     return 0;
 }
 
+struct xc_sr_context_save_buffers
+{
+    xen_pfn_t batch_pfns[MAX_BATCH_SIZE];
+    xen_pfn_t mfns[MAX_BATCH_SIZE];
+    xen_pfn_t types[MAX_BATCH_SIZE];
+    int errors[MAX_BATCH_SIZE];
+    void *guest_data[MAX_BATCH_SIZE];
+    void *local_pages[MAX_BATCH_SIZE];
+    struct iovec iov[MAX_BATCH_SIZE + 2]; /* headers + data */
+    uint64_t rec_pfns[MAX_BATCH_SIZE];
+};
+
 struct xc_sr_context
 {
     xc_interface *xch;
@@ -244,6 +256,7 @@ struct xc_sr_context
             unsigned long *deferred_pages;
             unsigned long nr_deferred_pages;
             xc_hypercall_buffer_t dirty_bitmap_hbuf;
+            struct xc_sr_context_save_buffers *buffers;
         } save;
 
         struct /* Restore data. */
tools/libs/guest/xg_sr_save.c +26 -40

124 lines not on this page · raw message

Signature · 1 line
2.54.0
#8 lore raw

PATCH v3 6/9 libs/call: cache up to 4 pages in hypercall bounce buffers

In reply to Frediano Ziglio

From: Edwin Török <[email protected]>

During migration there are a lot of mmap/munmap calls, because xc_get_pfn_type_batch exceeds the default hypercall bounce buffer cache size, and needs to allocate every time it is called.

munmap is slow, especially in a PV Dom0 (takes an emulation fault), so is best avoided.

Eventually it'd be good if the memory pool from xmalloc_tlsf.c was reused here, but for now make it handle the commonly encountered sizes (so far up to 4 pages).

--
Changes since v2:
- change prefix in subject.

---
 tools/libs/call/buffer.c  | 28 +++++++++++++++++-----------
 tools/libs/call/core.c    |  3 ++-
 tools/libs/call/private.h |  8 +++++---
 3 files changed, 24 insertions(+), 15 deletions(-)
tools/libs/call/buffer.c +17 -11
index 2579b8c71986..15d2f98a6b71 100644
@@ -56,13 +56,13 @@ static void *cache_alloc(xencall_handle *xcall, size_t nr_pages)
     if ( xcall->buffer_current_allocations > xcall->buffer_maximum_allocations )
         xcall->buffer_maximum_allocations = xcall->buffer_current_allocations;
 
-    if ( nr_pages > 1 )
+    if ( nr_pages > ARRAY_SIZE(xcall->buffer_cache) )
     {
         xcall->buffer_cache_toobig++;
     }
-    else if ( xcall->buffer_cache_nr > 0 )
+    else if ( xcall->buffer_cache_nr[nr_pages-1] > 0 )
     {
-        p = xcall->buffer_cache[--xcall->buffer_cache_nr];
+        p = xcall->buffer_cache[nr_pages-1][--xcall->buffer_cache_nr[nr_pages-1]];
         xcall->buffer_cache_hits++;
     }
     else
@@ -84,10 +84,10 @@ static int cache_free(xencall_handle *xcall, void *p, size_t nr_pages)
     xcall->buffer_total_releases++;
     xcall->buffer_current_allocations--;
 
-    if ( nr_pages == 1 &&
-         xcall->buffer_cache_nr < BUFFER_CACHE_SIZE )
+    if ( nr_pages && nr_pages < ARRAY_SIZE(xcall->buffer_cache) &&
+         xcall->buffer_cache_nr[nr_pages-1] < BUFFER_CACHE_SIZE )
     {
-        xcall->buffer_cache[xcall->buffer_cache_nr++] = p;
+        xcall->buffer_cache[nr_pages-1][xcall->buffer_cache_nr[nr_pages-1]++] = p;
         rc = 1;
     }
 
@@ -108,17 +108,23 @@ void buffer_release_cache(xencall_handle *xcall)
     DBGPRINTF("current allocations:%d maximum allocations:%d",
               xcall->buffer_current_allocations,
               xcall->buffer_maximum_allocations);
-    DBGPRINTF("cache current size:%d",
-              xcall->buffer_cache_nr);
+    for ( unsigned i = 0; i < ARRAY_SIZE(xcall->buffer_cache_nr); ++i )
+    {
+        DBGPRINTF("cache current size[%u pages]:%d", i+1,
+                xcall->buffer_cache_nr[i]);
+    }
     DBGPRINTF("cache hits:%d misses:%d toobig:%d",
               xcall->buffer_cache_hits,
               xcall->buffer_cache_misses,
               xcall->buffer_cache_toobig);
 
-    while ( xcall->buffer_cache_nr > 0 )
+    for ( unsigned i = 0; i < ARRAY_SIZE(xcall->buffer_cache_nr); ++i )
     {
-        p = xcall->buffer_cache[--xcall->buffer_cache_nr];
-        osdep_free_pages(xcall, p, 1);
+        while ( xcall->buffer_cache_nr[i] > 0 )
+        {
+            p = xcall->buffer_cache[i][--xcall->buffer_cache_nr[i]];
+            osdep_free_pages(xcall, p, i + 1);
+        }
     }
 
     cache_unlock(xcall);
tools/libs/call/core.c +2 -1
index 02c4f8e1aefa..dd8877c1a0bd 100644
@@ -14,6 +14,7 @@
  */
 
 #include <stdlib.h>
+#include <string.h>
 
 #include "private.h"
 
@@ -44,7 +45,7 @@ xencall_handle *xencall_open(xentoollog_logger *logger, unsigned open_flags)
     xentoolcore__register_active_handle(&xcall->tc_ah);
 
     xcall->flags = open_flags;
-    xcall->buffer_cache_nr = 0;
+    memset(xcall->buffer_cache_nr, 0, sizeof(xcall->buffer_cache_nr));
 
     xcall->buffer_total_allocations = 0;
     xcall->buffer_total_releases = 0;
tools/libs/call/private.h +5 -3
index 9c3aa432efe2..8e6a20897534 100644
@@ -31,13 +31,15 @@ struct xencall_handle {
     Xentoolcore__Active_Handle tc_ah;
 
     /*
-     * A simple cache of unused, single page, hypercall buffers
+     * A simple cache of unused, small, hypercall buffers
+     * buffer_cache[i]'s size is (i+1) pages
      *
      * Protected by a global lock.
      */
 #define BUFFER_CACHE_SIZE 4
-    int buffer_cache_nr;
-    void *buffer_cache[BUFFER_CACHE_SIZE];
+#define BUFFER_CACHE_NRPAGES 4
+    int buffer_cache_nr[BUFFER_CACHE_NRPAGES];
+    void *buffer_cache[BUFFER_CACHE_NRPAGES][BUFFER_CACHE_SIZE];
 
     /*
      * Hypercall buffer statistics. All protected by the global
Signature · 1 line
2.54.0
#9 lore raw

PATCH v3 7/9 libs/guest: avoids using 2 indexes

In reply to Frediano Ziglio

From: Frediano Ziglio <[email protected]>

Simplify code, after the first scan of the various arrays we don't need to keep original types and PFNs but only the ones having data.

---
 tools/libs/guest/xg_sr_restore.c | 26 +++++++++++++-------------
 1 file changed, 13 insertions(+), 13 deletions(-)
tools/libs/guest/xg_sr_restore.c +13 -13
index e148fc594a73..fb46142d8729 100644
@@ -260,9 +260,7 @@ static int process_page_data(struct xc_sr_context *ctx, unsigned int count,
     int *map_errs = malloc(count * sizeof(*map_errs));
     int rc;
     void *mapping = NULL, *guest_page = NULL;
-    unsigned int i, /* i indexes the pfns from the record. */
-        j,          /* j indexes the subset of pfns we decide to map. */
-        nr_pages = 0;
+    unsigned nr_pages;
 
     if ( !mfns || !map_errs )
     {
@@ -279,12 +277,18 @@ static int process_page_data(struct xc_sr_context *ctx, unsigned int count,
         goto err;
     }
 
-    for ( i = 0; i < count; ++i )
+    nr_pages = 0;
+    for ( unsigned i = 0; i < count; ++i )
     {
         ctx->restore.ops.set_page_type(ctx, pfns[i], types[i]);
 
-        if ( page_type_has_stream_data(types[i]) )
-            mfns[nr_pages++] = ctx->restore.ops.pfn_to_gfn(ctx, pfns[i]);
+        if ( !page_type_has_stream_data(types[i]) )
+            continue;
+
+        mfns[nr_pages] = ctx->restore.ops.pfn_to_gfn(ctx, pfns[i]);
+        pfns[nr_pages] = pfns[i];
+        types[nr_pages] = types[i];
+        nr_pages++;
     }
 
     /* Nothing to do? */
@@ -302,16 +306,13 @@ static int process_page_data(struct xc_sr_context *ctx, unsigned int count,
         goto err;
     }
 
-    for ( i = 0, j = 0; i < count; ++i )
+    for ( unsigned i = 0; i < nr_pages; ++i )
     {
-        if ( !page_type_has_stream_data(types[i]) )
-            continue;
-
-        if ( map_errs[j] )
+        if ( map_errs[i] )
         {
             rc = -1;
             ERROR("Mapping pfn %#"PRIpfn" (mfn %#"PRIpfn", type %#"PRIx32") failed with %d",
-                  pfns[i], mfns[j], types[i], map_errs[j]);
+                  pfns[i], mfns[i], types[i], map_errs[i]);
             goto err;
         }
 
@@ -337,7 +338,6 @@ static int process_page_data(struct xc_sr_context *ctx, unsigned int count,
             memcpy(guest_page, page_data, PAGE_SIZE);
         }
 
-        ++j;
         guest_page += PAGE_SIZE;
         page_data += PAGE_SIZE;
     }
Signature · 1 line
2.54.0
#10 lore raw

PATCH v3 8/9 libs/guest: fill directly iov structure

In reply to Frediano Ziglio

From: Frediano Ziglio <[email protected]>

Instead of storing page pointers into an array and lately adding to iov vector add the pages directly to iov to avoid "guest_data" array.

---
 tools/libs/guest/xg_sr_common.h |  1 -
 tools/libs/guest/xg_sr_save.c   | 62 ++++++++++++---------------------
 2 files changed, 22 insertions(+), 41 deletions(-)
tools/libs/guest/xg_sr_common.h +0 -1
index 77312bae9cf5..797b4d1a7da1 100644
@@ -215,7 +215,6 @@ struct xc_sr_context_save_buffers
     xen_pfn_t mfns[MAX_BATCH_SIZE];
     xen_pfn_t types[MAX_BATCH_SIZE];
     int errors[MAX_BATCH_SIZE];
-    void *guest_data[MAX_BATCH_SIZE];
     void *local_pages[MAX_BATCH_SIZE];
     struct iovec iov[MAX_BATCH_SIZE + 2]; /* headers + data */
     uint64_t rec_pfns[MAX_BATCH_SIZE];
tools/libs/guest/xg_sr_save.c +22 -40

104 lines not on this page · raw message

Signature · 1 line
2.54.0
#11 lore raw

PATCH v3 9/9 libs/ctrl: Allows writev_exact to change iov array

In reply to Frediano Ziglio

From: Frediano Ziglio <[email protected]>

Avoid having to allocate and copy the array if a partial write happens.
The implementation in tools/libs/store/xs.c already use this signature and method.

--
Changes since v2:
- change prefix in subject.

---
 tools/libs/ctrl/xc_private.c | 26 +++++---------------------
 tools/libs/ctrl/xc_private.h |  2 +-
 2 files changed, 6 insertions(+), 22 deletions(-)
tools/libs/ctrl/xc_private.c +5 -21
index bb0f81d6f389..946fc307aabd 100644
@@ -635,7 +635,7 @@ int write_exact(int fd, const void *data, size_t size)
 /*
  * MiniOS's libc doesn't know about writev(). Implement it as multiple write()s.
  */
-int writev_exact(int fd, const struct iovec *iov, int iovcnt)
+int writev_exact(int fd, struct iovec *iov, int iovcnt)
 {
     int rc, i;
 
@@ -649,9 +649,8 @@ int writev_exact(int fd, const struct iovec *iov, int iovcnt)
     return 0;
 }
 #else
-int writev_exact(int fd, const struct iovec *iov, int iovcnt)
+int writev_exact(int fd, struct iovec *iov, int iovcnt)
 {
-    struct iovec *local_iov = NULL;
     int rc = 0, iov_idx = 0, saved_errno = 0;
     ssize_t len;
 
@@ -686,23 +685,9 @@ int writev_exact(int fd, const struct iovec *iov, int iovcnt)
                 len -= iov[iov_idx++].iov_len;
             else
             {
-                /* Partial write of iov[iov_idx]. Copy iov so we can adjust
-                 * element iov_idx and resubmit the rest. */
-                if ( !local_iov )
-                {
-                    local_iov = malloc(iovcnt * sizeof(*iov));
-                    if ( !local_iov )
-                    {
-                        saved_errno = ENOMEM;
-                        rc = -1;
-                        goto out;
-                    }
-
-                    iov = memcpy(local_iov, iov, iovcnt * sizeof(*iov));
-                }
-
-                local_iov[iov_idx].iov_base += len;
-                local_iov[iov_idx].iov_len  -= len;
+                /* Partial write of iov[iov_idx]. */
+                iov[iov_idx].iov_base += len;
+                iov[iov_idx].iov_len  -= len;
                 break;
             }
         }
@@ -711,7 +696,6 @@ int writev_exact(int fd, const struct iovec *iov, int iovcnt)
     saved_errno = 0;
 
  out:
-    free(local_iov);
     errno = saved_errno;
     return rc;
 }
tools/libs/ctrl/xc_private.h +1 -1
index b5892ae8dc1e..3af996e9002e 100644
@@ -383,7 +383,7 @@ int xc_flush_mmu_updates(xc_interface *xch, struct xc_mmu *mmu);
 /* Return 0 on success; -1 on error setting errno. */
 int read_exact(int fd, void *data, size_t size); /* EOF => -1, errno=0 */
 int write_exact(int fd, const void *data, size_t size);
-int writev_exact(int fd, const struct iovec *iov, int iovcnt);
+int writev_exact(int fd, struct iovec *iov, int iovcnt);
 
 int xc_ffs8(uint8_t x);
 int xc_ffs16(uint16_t x);
Signature · 1 line
2.54.0
#12 lore raw
In reply to Frediano Ziglio
Quoted 10 lines · Frediano Ziglio

From: Frediano Ziglio <[email protected]>

Reduce some number of parts passed to writev. Avoid possible allocation sending data with writev. Reduce number of allocations sending memory state.

Changes since v1:
- add commit to cache up to 4 pages in hypercall;
- add other 2 commits reducing chunks passed to write/writev.

Quoted 6 lines

Changes since v2:
- update patches commit prefixes;
- add other 2 optisations.

Edwin Török (2):
libs/guest: allocate various migration arrays just once

Btw, in particular related to this one, see also
https://lists.xen.org/archives/html/xen-devel/2022-04/msg02037.html (and maybe one or two others in the same series).

Jan

Quoted 22 lines

libs/call: cache up to 4 pages in hypercall bounce buffers

Frediano Ziglio (7):
libs/guest: Reduce number of parts in write_split_record libs/guest: Reduce number of I/O vectors in write_batch libs/guest: Reduce number of I/O vectors in write_batch libs/guest: Use a single write_exact in write_headers libs/guest: avoids using 2 indexes
libs/guest: fill directly iov structure libs/ctrl: Allows writev_exact to change iov array

 tools/libs/call/buffer.c         |  28 +++--
 tools/libs/call/core.c           |   3 +-
 tools/libs/call/private.h        |   8 +-
 tools/libs/ctrl/xc_private.c     |  26 +----
 tools/libs/ctrl/xc_private.h     |   2 +-
 tools/libs/guest/xg_sr_common.c  |   6 +-
 tools/libs/guest/xg_sr_common.h  |  12 +++
 tools/libs/guest/xg_sr_restore.c |  26 ++---
 tools/libs/guest/xg_sr_save.c    | 169 +++++++++++++------------------
 9 files changed, 128 insertions(+), 152 deletions(-)