xenguest optimisations
v9 cover xenguest optimisations
Reduce number of allocations sending memory state.
Changes since v1:
- add commit to cache up to 4 pages in hypercall;
- add other 2 commits reducing chunks passed to write/writev.
Changes since v2:
- update patches commit prefixes;
- add other 2 optisations.
Changes since v3:
- address some comments;
- add patches for foreign copy optimisation.
Changes since v4:
- added Reviewed-by;
- improved commit messages;
- other minor fixes, see individual commits.
Changes since v5:
- avoids potential buffer underflow if nr_pages is 0 calling cache_alloc;
- do not overwrite errno if xenforeignmemory_map fails;
- lot of changes to "implement new foreign copy hypercall", see specific commit.
Changes since v6:
- removed merged patch;
- keep only optimization commits for now;
- improve comments;
- merged "fill directly iov structure collapsing them" and moved it;
- split "allocate various migration arrays just once";
- add a commit for memory checks using Valgrind.
Changes since v7:
- removed merged commits;
- minor style fixes.
Changes since v8:
- added Reviewed-by;
- remove useless check;
- remove useless memset;
- initialize variables while declaring them.
Edwin Török (2):
libs/call: cache up to 4 pages in hypercall bounce buffers libs/guest: allocate various migration arrays just once
Frediano Ziglio (2):
libs/guest: move batch_pfns into a separate structure libs/guest: use Valgrind to detect various buffer overflows
tools/libs/call/buffer.c | 34 ++++++++----- tools/libs/call/core.c | 3 +- tools/libs/call/private.h | 8 ++-- tools/libs/ctrl/xc_private.h | 8 ++++ tools/libs/guest/xg_sr_common.h | 19 +++++++- tools/libs/guest/xg_sr_save.c | 84 ++++++++++++++++----------------- 6 files changed, 98 insertions(+), 58 deletions(-)
Signature · 1 line
2.43.0
PATCH v9 1/4 libs/call: cache up to 4 pages in hypercall bounce buffers
From: Edwin Török <[email protected]>
During migration there are a lot of mmap/munmap calls, because xc_get_pfn_type_batch() exceeds the default hypercall bounce buffer cache size, and needs to allocate every time it is called.
munmap() is slow, especially in a PV Dom0 (takes an emulation fault), so is best avoided.
Eventually it'd be good if the memory pool from xmalloc_tlsf.c was reused here, but for now make it handle the commonly encountered sizes (so far up to 4 pages).
---
Changes since v2:
- change prefix in subject.
Changes since v4:
- fix off-by-one bug.
Changes since v5:
- avoids potential buffer underflow if nr_pages is 0 calling cache_alloc.
Changes since v6:
- align changes made to cache_alloc to cache_free.
Changes since v7:
- use "unsigned int" instead of "unsigned".
Changes since v8:
- added Reviewed-by.
--- tools/libs/call/buffer.c | 34 +++++++++++++++++++++++----------- tools/libs/call/core.c | 3 ++- tools/libs/call/private.h | 8 +++++--- 3 files changed, 30 insertions(+), 15 deletions(-)
tools/libs/call/buffer.c +23 -11
@@ -49,6 +49,9 @@ static void *cache_alloc(xencall_handle *xcall, size_t nr_pages) {void *p =NULL ; +if ( nr_pages ==0 ) +return NULL ; +cache_lock (xcall); xcall->buffer_total_allocations++;
@@ -56,13 +59,13 @@ static void *cache_alloc(xencall_handle *xcall, size_t nr_pages)if ( xcall->buffer_current_allocations > xcall->buffer_maximum_allocations ) xcall->buffer_maximum_allocations = xcall->buffer_current_allocations; -if ( nr_pages >1 ) +if ( nr_pages >ARRAY_SIZE (xcall->buffer_cache) ) { xcall->buffer_cache_toobig++; } -else if ( xcall->buffer_cache_nr >0 ) +else if ( xcall->buffer_cache_nr[nr_pages-1 ] >0 ) { - p = xcall->buffer_cache[--xcall->buffer_cache_nr]; + p = xcall->buffer_cache[nr_pages-1 ][--xcall->buffer_cache_nr[nr_pages-1 ]]; xcall->buffer_cache_hits++; }else
@@ -79,15 +82,18 @@ static int cache_free(xencall_handle *xcall, void *p, size_t nr_pages) {int rc =0 ; +if ( nr_pages ==0 ) +return 0 ; +cache_lock (xcall); xcall->buffer_total_releases++; xcall->buffer_current_allocations--; -if ( nr_pages ==1 && - xcall->buffer_cache_nr <BUFFER_CACHE_SIZE ) +if ( nr_pages && nr_pages <=ARRAY_SIZE (xcall->buffer_cache) && + xcall->buffer_cache_nr[nr_pages-1 ] <BUFFER_CACHE_SIZE ) { - xcall->buffer_cache[xcall->buffer_cache_nr++] = p; + xcall->buffer_cache[nr_pages-1 ][xcall->buffer_cache_nr[nr_pages-1 ]++] = p; rc =1 ; }
@@ -108,17 +114,23 @@ void buffer_release_cache(xencall_handle *xcall)DBGPRINTF ("current allocations:%d maximum allocations:%d" , xcall->buffer_current_allocations, xcall->buffer_maximum_allocations); -DBGPRINTF ("cache current size:%d" , - xcall->buffer_cache_nr); +for (unsigned int i =0 ; i <ARRAY_SIZE (xcall->buffer_cache_nr); ++i ) + { +DBGPRINTF ("cache current size[%u pages]:%d" , i+1 , + xcall->buffer_cache_nr[i]); + }DBGPRINTF ("cache hits:%d misses:%d toobig:%d" , xcall->buffer_cache_hits, xcall->buffer_cache_misses, xcall->buffer_cache_toobig); -while ( xcall->buffer_cache_nr >0 ) +for (unsigned int i =0 ; i <ARRAY_SIZE (xcall->buffer_cache_nr); ++i ) { - p = xcall->buffer_cache[--xcall->buffer_cache_nr]; -osdep_free_pages (xcall, p,1 ); +while ( xcall->buffer_cache_nr[i] >0 ) + { + p = xcall->buffer_cache[i][--xcall->buffer_cache_nr[i]]; +osdep_free_pages (xcall, p, i +1 ); + } }cache_unlock (xcall);
tools/libs/call/core.c +2 -1
@@ -14,6 +14,7 @@*/ #include <stdlib.h> +#include <string.h> #include "private.h"
@@ -44,7 +45,7 @@ xencall_handle *xencall_open(xentoollog_logger *logger, unsigned open_flags)xentoolcore__register_active_handle (&xcall->tc_ah); xcall->flags = open_flags; - xcall->buffer_cache_nr =0 ; +memset (xcall->buffer_cache_nr,0 ,sizeof (xcall->buffer_cache_nr)); xcall->buffer_total_allocations =0 ; xcall->buffer_total_releases =0 ;
tools/libs/call/private.h +5 -3
@@ -31,13 +31,15 @@ struct xencall_handle {Xentoolcore__Active_Handle tc_ah;/* -* A simple cache of unused, single page, hypercall buffers +* A simple cache of unused, small, hypercall buffers +* buffer_cache[i]'s size is (i+1) pages * * Protected by a global lock. */ #define BUFFER_CACHE_SIZE 4 -int buffer_cache_nr; -void *buffer_cache[BUFFER_CACHE_SIZE ]; +#define BUFFER_CACHE_NRPAGES 4 +int buffer_cache_nr[BUFFER_CACHE_NRPAGES ]; +void *buffer_cache[BUFFER_CACHE_NRPAGES ][BUFFER_CACHE_SIZE ];/* * Hypercall buffer statistics. All protected by the global
Signature · 1 line
2.43.0
PATCH v9 2/4 libs/guest: move batch_pfns into a separate structure
In reply to Frediano ZiglioPreparation for a followup patch "libs/guest: allocate various migration arrays just once".
---
Changes since v6:
- split from "libs/guest: allocate various migration arrays just once".
Changes since v7:
- initialize "batch_pfns" on declaration.
Changes since v8:
- remove useless check;
- added Reviewed-by.
--- tools/libs/guest/xg_sr_common.h | 5 ++++- tools/libs/guest/xg_sr_save.c | 28 ++++++++++++++-------------- 2 files changed, 18 insertions(+), 15 deletions(-)
tools/libs/guest/xg_sr_common.h +4 -1
@@ -239,11 +239,14 @@ struct xc_sr_contextstruct precopy_stats stats; -xen_pfn_t *batch_pfns;unsigned int nr_batch_pfns;unsigned long *deferred_pages;unsigned long nr_deferred_pages;xc_hypercall_buffer_t dirty_bitmap_hbuf; +struct xc_sr_context_save_buffers + { +xen_pfn_t batch_pfns[MAX_BATCH_SIZE ]; + } *buffers; } save; struct /* Restore data. */
tools/libs/guest/xg_sr_save.c +14 -14
@@ -75,7 +75,7 @@ static int write_checkpoint_record(struct xc_sr_context *ctx)/* * Writes a batch of memory as a PAGE_DATA record into the stream. The batch -* is constructed in ctx->save.batch_pfns. +* is constructed in ctx->save.buffers->batch_pfns. * * This function: * - gets the types for each pfn in the batch.
@@ -95,6 +95,7 @@ static int write_batch(struct xc_sr_context *ctx)void *page, *orig_page;uint64_t *rec_pfns =NULL ;struct iovec *iov =NULL ;int iovcnt =0 ; +xen_pfn_t *const batch_pfns = ctx->save.buffers->batch_pfns;struct {struct xc_sr_rhdr rec;struct xc_sr_rec_page_data_header page_data;
@@ -110,6 +111,7 @@ static int write_batch(struct xc_sr_context *ctx) };assert (nr_pfns !=0 ); +assert (nr_pfns <=MAX_BATCH_SIZE );/* Mfns of the batch pfns. */ mfns =malloc (nr_pfns *sizeof (*mfns));
@@ -141,13 +143,12 @@ static int write_batch(struct xc_sr_context *ctx)for ( i =0 ; i < nr_pfns; ++i ) { - types[i] = mfns[i] = ctx->save.ops.pfn_to_gfn (ctx, - ctx->save.batch_pfns[i]); + types[i] = mfns[i] = ctx->save.ops.pfn_to_gfn (ctx, batch_pfns[i]);/* Likely a ballooned page. */ if ( mfns[i] ==INVALID_MFN ) { -set_bit (ctx->save.batch_pfns[i], ctx->save.deferred_pages); +set_bit (batch_pfns[i], ctx->save.deferred_pages); ++ctx->save.nr_deferred_pages; } }
@@ -193,7 +194,7 @@ static int write_batch(struct xc_sr_context *ctx)if ( errors[p] ) {ERROR ("Mapping of pfn %#" PRIpfn" (mfn %#" PRIpfn") failed %d" , - ctx->save.batch_pfns[i], mfns[p], errors[p]); + batch_pfns[i], mfns[p], errors[p]); goto err; }
@@ -207,7 +208,7 @@ static int write_batch(struct xc_sr_context *ctx) {if ( rc ==-1 && errno ==EAGAIN ) { -set_bit (ctx->save.batch_pfns[i], ctx->save.deferred_pages); +set_bit (batch_pfns[i], ctx->save.deferred_pages); ++ctx->save.nr_deferred_pages; types[i] =XEN_DOMCTL_PFINFO_XTAB ; --nr_pages;
@@ -235,7 +236,7 @@ static int write_batch(struct xc_sr_context *ctx) hdrs.rec.length += nr_pages *PAGE_SIZE ;for ( i =0 ; i < nr_pfns; ++i ) - rec_pfns[i] = ((uint64_t )(types[i]) <<32 ) | ctx->save.batch_pfns[i]; + rec_pfns[i] = ((uint64_t )(types[i]) <<32 ) | batch_pfns[i];if (writev_exact (ctx->fd, iov, iovcnt) ) {
@@ -274,9 +275,9 @@ static int flush_batch(struct xc_sr_context *ctx)if ( !rc ) { -VALGRIND_MAKE_MEM_UNDEFINED (ctx->save.batch_pfns, +VALGRIND_MAKE_MEM_UNDEFINED (ctx->save.buffers->batch_pfns,MAX_BATCH_SIZE * -sizeof (*ctx->save.batch_pfns)); +sizeof (*ctx->save.buffers->batch_pfns)); }return rc;
@@ -293,7 +294,7 @@ static int add_to_batch(struct xc_sr_context *ctx, xen_pfn_t pfn) rc =flush_batch (ctx);if ( rc ==0 ) - ctx->save.batch_pfns[ctx->save.nr_batch_pfns++] = pfn; + ctx->save.buffers->batch_pfns[ctx->save.nr_batch_pfns++] = pfn;return rc; }
@@ -784,11 +785,10 @@ static int setup(struct xc_sr_context *ctx) dirty_bitmap =xc_hypercall_buffer_alloc_pages ( xch, dirty_bitmap,NRPAGES (bitmap_size (ctx->save.p2m_size))); - ctx->save.batch_pfns =malloc (MAX_BATCH_SIZE * -sizeof (*ctx->save.batch_pfns)); ctx->save.deferred_pages =bitmap_alloc (ctx->save.p2m_size); + ctx->save.buffers =calloc (1 ,sizeof (*ctx->save.buffers)); -if ( !ctx->save.batch_pfns || !dirty_bitmap || !ctx->save.deferred_pages ) +if ( !ctx->save.buffers || !dirty_bitmap || !ctx->save.deferred_pages ) {ERROR ("Unable to allocate memory for dirty bitmaps, batch pfns and" " deferred pages" );
@@ -819,7 +819,7 @@ static void cleanup(struct xc_sr_context *ctx)xc_hypercall_buffer_free_pages (xch, dirty_bitmap,NRPAGES (bitmap_size (ctx->save.p2m_size)));free (ctx->save.deferred_pages); -free (ctx->save.batch_pfns); +free (ctx->save.buffers); }/*
Signature · 1 line
2.43.0
PATCH v9 3/4 libs/guest: allocate various migration arrays just once
In reply to Frediano ZiglioFrom: Edwin Török <[email protected]>
Allocate these array just once at the start of migration, using the maximum batch size, and free them at the end.
---
Changes since v2:
- change prefix in subject.
Changes since v3:
- fix comment style
Changes since v4:
- change order of fields in structure.
Changes since v6:
- split preparation commit.
Changes since v8:
- remove useless memset;
- initialize variables while declaring them.
--- tools/libs/guest/xg_sr_common.h | 6 +++++ tools/libs/guest/xg_sr_save.c | 45 ++++++++++++--------------------- 2 files changed, 22 insertions(+), 29 deletions(-)
tools/libs/guest/xg_sr_common.h +6 -0
@@ -246,6 +246,12 @@ struct xc_sr_contextstruct xc_sr_context_save_buffers {xen_pfn_t batch_pfns[MAX_BATCH_SIZE ]; +xen_pfn_t mfns[MAX_BATCH_SIZE ]; +xen_pfn_t types[MAX_BATCH_SIZE ]; +void *local_pages[MAX_BATCH_SIZE ]; +struct iovec iov[MAX_BATCH_SIZE +2 ];/* Headers + data. */ +uint64_t rec_pfns[MAX_BATCH_SIZE ]; +int errors[MAX_BATCH_SIZE ]; } *buffers; } save;
tools/libs/guest/xg_sr_save.c +16 -29
81 lines not on this page · raw message
Signature · 1 line
2.43.0
Re: [PATCH v9 3/4] libs/guest: allocate various migration arrays just once
Quoted 7 lines · Frediano Ziglio
From: Edwin Török <[email protected]>
Allocate these array just once at the start of migration, using the maximum batch size, and free them at the end.
Signed-off-by Edwin Török <[email protected]>Signed-off-by Frediano Ziglio <[email protected]>
Thanks,
--
Anthony Perard | Vates XCP-ng Developer
XCP-ng & Xen Orchestra - Vates solutions
web: https://vates.tech
PATCH v9 4/4 libs/guest: use Valgrind to detect various buffer overflows
In reply to Frediano ZiglioPreviously this was done as buffers were allocated separately.
--- tools/libs/ctrl/xc_private.h | 8 ++++++++ tools/libs/guest/xg_sr_common.h | 8 ++++++++ tools/libs/guest/xg_sr_save.c | 13 +++++++++++++ 3 files changed, 29 insertions(+)
tools/libs/ctrl/xc_private.h +8 -0
@@ -45,8 +45,16 @@#if defined(HAVE_VALGRIND_MEMCHECK_H ) && !defined(NDEBUG ) && !defined(__MINIOS__)/* Compile in Valgrind client requests? */ #include <valgrind/memcheck.h> +#define MEM_NOACCESS_BUFFER (name, size) uint8_t name[size]; +#define MEM_NOACCESS_INIT (field) \ + VALGRIND_MAKE_MEM_NOACCESS(field, sizeof(field)) +#define MEM_UNDEFINED_INIT (field) \ + VALGRIND_MAKE_MEM_UNDEFINED(field, sizeof(field))#else #define VALGRIND_MAKE_MEM_UNDEFINED (addr, len)/* addr, len */ +#define MEM_NOACCESS_BUFFER (name, size) +#define MEM_NOACCESS_INIT (field) do {} while(0) +#define MEM_UNDEFINED_INIT (field) do {} while(0)#endif #if defined(__MINIOS__)
tools/libs/guest/xg_sr_common.h +8 -0
@@ -245,13 +245,21 @@ struct xc_sr_contextxc_hypercall_buffer_t dirty_bitmap_hbuf;struct xc_sr_context_save_buffers { + MEM_NOACCESS_BUFFER(na0, 16);xen_pfn_t batch_pfns[MAX_BATCH_SIZE ]; + MEM_NOACCESS_BUFFER(na1, 16);xen_pfn_t mfns[MAX_BATCH_SIZE ]; + MEM_NOACCESS_BUFFER(na2, 16);xen_pfn_t types[MAX_BATCH_SIZE ]; + MEM_NOACCESS_BUFFER(na3, 16);void *local_pages[MAX_BATCH_SIZE ]; + MEM_NOACCESS_BUFFER(na4, 16);struct iovec iov[MAX_BATCH_SIZE +2 ];/* Headers + data. */ + MEM_NOACCESS_BUFFER(na5, 16);uint64_t rec_pfns[MAX_BATCH_SIZE ]; + MEM_NOACCESS_BUFFER(na6, 16);int errors[MAX_BATCH_SIZE ]; + MEM_NOACCESS_BUFFER(na7, 16); } *buffers; } save;
tools/libs/guest/xg_sr_save.c +13 -0
@@ -123,6 +123,11 @@ static int write_batch(struct xc_sr_context *ctx)assert (nr_pfns !=0 );assert (nr_pfns <=MAX_BATCH_SIZE ); +MEM_UNDEFINED_INIT (ctx->save.buffers->mfns); +MEM_UNDEFINED_INIT (ctx->save.buffers->types); +MEM_UNDEFINED_INIT (ctx->save.buffers->iov); +MEM_UNDEFINED_INIT (ctx->save.buffers->rec_pfns); + iov[0 ].iov_base = &hdrs; iov[0 ].iov_len =sizeof (hdrs);
@@ -783,6 +788,14 @@ static int setup(struct xc_sr_context *ctx) errno =ENOMEM ; goto err; } +MEM_NOACCESS_INIT (ctx->save.buffers->na0); +MEM_NOACCESS_INIT (ctx->save.buffers->na1); +MEM_NOACCESS_INIT (ctx->save.buffers->na2); +MEM_NOACCESS_INIT (ctx->save.buffers->na3); +MEM_NOACCESS_INIT (ctx->save.buffers->na4); +MEM_NOACCESS_INIT (ctx->save.buffers->na5); +MEM_NOACCESS_INIT (ctx->save.buffers->na6); +MEM_NOACCESS_INIT (ctx->save.buffers->na7); rc =0 ;
Signature · 1 line
2.43.0
Re: [PATCH v9 4/4] libs/guest: use Valgrind to detect various buffer overflows
Quoted 6 lines · Frediano Ziglio
tools/libs/ctrl/xc_private.h +0
-0@@ -45,8 +45,16 @@#if defined(HAVE_VALGRIND_MEMCHECK_H ) && !defined(NDEBUG ) && !defined(__MINIOS__)/* Compile in Valgrind client requests? */ #include <valgrind/memcheck.h>
tools/libs/ctrl/xc_private.h line 57
Quoted 10 lines
tools/libs/ctrl/xc_private.h +8
-0+#define MEM_NOACCESS_BUFFER (name, size) uint8_t name[size]; +#define MEM_NOACCESS_INIT (field) \ + VALGRIND_MAKE_MEM_NOACCESS(field, sizeof(field)) +#define MEM_UNDEFINED_INIT (field) \ + VALGRIND_MAKE_MEM_UNDEFINED(field, sizeof(field))#else #define VALGRIND_MAKE_MEM_UNDEFINED (addr, len)/* addr, len */ +#define MEM_NOACCESS_BUFFER (name, size) +#define MEM_NOACCESS_INIT (field) do {} while(0) +#define MEM_UNDEFINED_INIT (field) do {} while(0)
Why the _INIT suffix in the macros? It looks like something is initialised, but that's not the case.
We could follow the valgrind naming and do, while using something different than "mem" to say we don't need a size:make_buffer_noaccess and make_buffer_undefined
Ormark_buffer_as_noaccessmark_buffer_as_undefined
tools/libs/guest/xg_sr_common.h struct xc_sr_context · line 248
Quoted 12 lines
tools/libs/ctrl/xc_private.h +0
-0#endif #if defined(__MINIOS__)tools/libs/guest/xg_sr_common.h +1
-0@@ -245,13 +245,21 @@ struct xc_sr_contextxc_hypercall_buffer_t dirty_bitmap_hbuf;struct xc_sr_context_save_buffers { + MEM_NOACCESS_BUFFER(na0, 16);
This first redzone buffer and the last (na7) one looks unnecessary, as they are before the beginning of the buffer, and after the end.
Quoted 22 lines
tools/libs/guest/xg_sr_common.h +7
-0xen_pfn_t batch_pfns[MAX_BATCH_SIZE ]; +MEM_NOACCESS_BUFFER (na1,16 );xen_pfn_t mfns[MAX_BATCH_SIZE ]; +MEM_NOACCESS_BUFFER (na2,16 );xen_pfn_t types[MAX_BATCH_SIZE ]; +MEM_NOACCESS_BUFFER (na3,16 );void *local_pages[MAX_BATCH_SIZE ]; +MEM_NOACCESS_BUFFER (na4,16 );struct iovec iov[MAX_BATCH_SIZE +2 ];/* Headers + data. */ +MEM_NOACCESS_BUFFER (na5,16 );uint64_t rec_pfns[MAX_BATCH_SIZE ]; +MEM_NOACCESS_BUFFER (na6,16 );int errors[MAX_BATCH_SIZE ]; +MEM_NOACCESS_BUFFER (na7,16 ); } *buffers; } save;tools/libs/guest/xg_sr_save.c +0
-0
tools/libs/guest/xg_sr_save.c static int write_batch(struct xc_sr_context *ctx) · line 129
Quoted 7 lines
tools/libs/guest/xg_sr_save.c +4
-0@@ -123,6 +123,11 @@ static int write_batch(struct xc_sr_context *ctx)assert (nr_pfns !=0 );assert (nr_pfns <=MAX_BATCH_SIZE ); +MEM_UNDEFINED_INIT (ctx->save.buffers->mfns); +MEM_UNDEFINED_INIT (ctx->save.buffers->types); +MEM_UNDEFINED_INIT (ctx->save.buffers->iov); +MEM_UNDEFINED_INIT (ctx->save.buffers->rec_pfns);
Why is errors not also marked as undefined?
To bad we can't really test this patch, beside check that it build.
Thanks,
--
Anthony Perard | Vates XCP-ng Developer
XCP-ng & Xen Orchestra - Vates solutions
web: https://vates.tech
Re: [PATCH v9 4/4] libs/guest: use Valgrind to detect various buffer overflows
Quoted 13 lines · Anthony PERARD
Quoted 11 lines · Frediano Ziglio
tools/libs/ctrl/xc_private.h +5
-0@@ -45,8 +45,16 @@#if defined(HAVE_VALGRIND_MEMCHECK_H ) && !defined(NDEBUG ) && !defined(__MINIOS__)/* Compile in Valgrind client requests? */ #include <valgrind/memcheck.h> +#define MEM_NOACCESS_BUFFER (name, size) uint8_t name[size]; +#define MEM_NOACCESS_INIT (field) \ + VALGRIND_MAKE_MEM_NOACCESS(field, sizeof(field)) +#define MEM_UNDEFINED_INIT (field) \ + VALGRIND_MAKE_MEM_UNDEFINED(field, sizeof(field))
Quoted 9 lines
tools/libs/ctrl/xc_private.h line 57
Quoted 5 lines
tools/libs/ctrl/xc_private.h +3
-0#else #define VALGRIND_MAKE_MEM_UNDEFINED (addr, len)/* addr, len */ +#define MEM_NOACCESS_BUFFER (name, size) +#define MEM_NOACCESS_INIT (field) do {} while(0) +#define MEM_UNDEFINED_INIT (field) do {} while(0)Why the
_INITsuffix in the macros? It looks like something is initialised, but that's not the case.
It initializes the memory to be not accessible.
Quoted 8 lines
We could follow the valgrind naming and do, while using something different than "mem" to say we don't need a size:
make_buffer_noaccessandmake_buffer_undefinedOr
mark_buffer_as_noaccessmark_buffer_as_undefined
It makes sense, but I wanted to be able to remove the buffers entirely if debug is not enabled.
Quoted 17 lines
tools/libs/guest/xg_sr_common.h struct xc_sr_context · line 248
Quoted 13 lines
tools/libs/ctrl/xc_private.h +0
-0#endif #if defined(__MINIOS__)tools/libs/guest/xg_sr_common.h +1
-0@@ -245,13 +245,21 @@ struct xc_sr_contextxc_hypercall_buffer_t dirty_bitmap_hbuf;struct xc_sr_context_save_buffers { + MEM_NOACCESS_BUFFER(na0, 16);This first redzone buffer and the last (na7) one looks unnecessary, as they are before the beginning of the buffer, and after the end.
Before for buffer underflow, after for buffer overflow, but I suppose you mean that Valgrind/ASAN already add them anyway so it's not really necessary.
Yes, I'll remove.
Quoted 25 lines
Quoted 25 lines
tools/libs/guest/xg_sr_common.h +7
-0xen_pfn_t batch_pfns[MAX_BATCH_SIZE ]; +MEM_NOACCESS_BUFFER (na1,16 );xen_pfn_t mfns[MAX_BATCH_SIZE ]; +MEM_NOACCESS_BUFFER (na2,16 );xen_pfn_t types[MAX_BATCH_SIZE ]; +MEM_NOACCESS_BUFFER (na3,16 );void *local_pages[MAX_BATCH_SIZE ]; +MEM_NOACCESS_BUFFER (na4,16 );struct iovec iov[MAX_BATCH_SIZE +2 ];/* Headers + data. */ +MEM_NOACCESS_BUFFER (na5,16 );uint64_t rec_pfns[MAX_BATCH_SIZE ]; +MEM_NOACCESS_BUFFER (na6,16 );int errors[MAX_BATCH_SIZE ]; +MEM_NOACCESS_BUFFER (na7,16 ); } *buffers; } save;tools/libs/guest/xg_sr_save.c +0
-0@@ -123,6 +123,11 @@ static int write_batch(struct xc_sr_context *ctx)assert (nr_pfns !=0 );assert (nr_pfns <=MAX_BATCH_SIZE );
Quoted 7 lines
tools/libs/guest/xg_sr_save.c static int write_batch(struct xc_sr_context *ctx) · line 129
Quoted 4 lines
tools/libs/guest/xg_sr_save.c +4
-0+MEM_UNDEFINED_INIT (ctx->save.buffers->mfns); +MEM_UNDEFINED_INIT (ctx->save.buffers->types); +MEM_UNDEFINED_INIT (ctx->save.buffers->iov); +MEM_UNDEFINED_INIT (ctx->save.buffers->rec_pfns);Why is errors not also marked as undefined?
I suppose a rebase mistake, later in the series errors field is removed.
Quoted 2 lines
To bad we can't really test this patch, beside check that it build.
Indeed, I realize Valgrind support is not that updated at the moment.
I'll add ASAN support as Andrew suggested.
Quoted 9 lines
Thanks,
--
Anthony Perard | Vates XCP-ng DeveloperXCP-ng & Xen Orchestra - Vates solutions
web: https://vates.tech
Frediano
Re: [PATCH v9 4/4] libs/guest: use Valgrind to detect various buffer overflows
In reply to Frediano ZiglioQuoted 13 lines · Frediano Ziglio
tools/libs/guest/xg_sr_common.h +3
-0@@ -245,13 +245,21 @@ struct xc_sr_contextxc_hypercall_buffer_t dirty_bitmap_hbuf;struct xc_sr_context_save_buffers { + MEM_NOACCESS_BUFFER(na0, 16);xen_pfn_t batch_pfns[MAX_BATCH_SIZE ]; + MEM_NOACCESS_BUFFER(na1, 16);xen_pfn_t mfns[MAX_BATCH_SIZE ]; + MEM_NOACCESS_BUFFER(na2, 16);
tools/libs/guest/xg_sr_common.h struct xc_sr_context · line 262
Quoted 10 lines
tools/libs/guest/xg_sr_common.h +5
-0xen_pfn_t types[MAX_BATCH_SIZE ]; +MEM_NOACCESS_BUFFER (na3,16 );void *local_pages[MAX_BATCH_SIZE ]; +MEM_NOACCESS_BUFFER (na4,16 );struct iovec iov[MAX_BATCH_SIZE +2 ];/* Headers + data. */ +MEM_NOACCESS_BUFFER (na5,16 );uint64_t rec_pfns[MAX_BATCH_SIZE ]; +MEM_NOACCESS_BUFFER (na6,16 );int errors[MAX_BATCH_SIZE ]; +MEM_NOACCESS_BUFFER (na7,16 );
If you're going to do this, use 64 not 16, and put iov[] at the end. It is good practice to misalign large buffers by one cacheline, not that I suspect the access pattern in this algorithm is relevant.
But, I'm not sure of the utility of doing this. By removing the dynamic allocations you also prevent ASAN from working.
We either care about them all (and keep the dynamic allocations), or don't care, and get rid of it all. This half-way-house serves no good purpose.
~Andrew
Re: [PATCH v9 4/4] libs/guest: use Valgrind to detect various buffer overflows
Quoted 21 lines · Andrew Cooper
Quoted 19 lines · Frediano Ziglio
tools/libs/guest/xg_sr_common.h +6
-0@@ -245,13 +245,21 @@ struct xc_sr_contextxc_hypercall_buffer_t dirty_bitmap_hbuf;struct xc_sr_context_save_buffers { + MEM_NOACCESS_BUFFER(na0, 16);xen_pfn_t batch_pfns[MAX_BATCH_SIZE ]; + MEM_NOACCESS_BUFFER(na1, 16);xen_pfn_t mfns[MAX_BATCH_SIZE ]; + MEM_NOACCESS_BUFFER(na2, 16);xen_pfn_t types[MAX_BATCH_SIZE ]; + MEM_NOACCESS_BUFFER(na3, 16);void *local_pages[MAX_BATCH_SIZE ]; + MEM_NOACCESS_BUFFER(na4, 16);struct iovec iov[MAX_BATCH_SIZE +2 ];/* Headers + data. */ + MEM_NOACCESS_BUFFER(na5, 16);
Quoted 9 lines
tools/libs/guest/xg_sr_common.h struct xc_sr_context · line 262
Quoted 4 lines
tools/libs/guest/xg_sr_common.h +2
-0uint64_t rec_pfns[MAX_BATCH_SIZE ]; +MEM_NOACCESS_BUFFER (na6,16 );int errors[MAX_BATCH_SIZE ]; +MEM_NOACCESS_BUFFER (na7,16 );If you're going to do this, use 64 not 16, and put iov[] at the end. It is good practice to misalign large buffers by one cacheline, not that I suspect the access pattern in this algorithm is relevant.
Easily doable. Why put "iov" at the end?
Quoted 3 lines
But, I'm not sure of the utility of doing this. By removing the dynamic allocations you also prevent ASAN from working.
It can be extended to ASAN using "sanitizer/asan_interface.h", specifically "ASAN_POISON_MEMORY_REGION" macro.
Quoted 5 lines
We either care about them all (and keep the dynamic allocations), or don't care, and get rid of it all. This half-way-house serves no good purpose.
~Andrew
Why is a half-way-house ?
Frediano