Multiple fixes to XENMEM_acquire_resource
PATCH 2/5 xen/gnttab: Rework resource acquisition
In reply to Andrew CooperThe existing logic doesn't function in the general case for mapping a guests grant table, due to arbitrary 32 frame limit, and the default grant table limit being 64.
In order to start addressing this, rework the existing grant table logic by implementing a single gnttab_acquire_resource(). This is far more efficient than the previous acquire_grant_table() in memory.c because it doesn't take the grant table write lock, and attempt to grow the table, for every single frame.
The new gnttab_acquire_resource() function subsumes the previous two gnttab_get_{shared,status}_frame() helpers.
No functional change.
---
CC: George Dunlap <[email protected]>
CC: Ian Jackson <[email protected]>
CC: Jan Beulich <[email protected]>
CC: Konrad Rzeszutek Wilk <[email protected]>
CC: Stefano Stabellini <[email protected]>
CC: Wei Liu <[email protected]>
CC: Julien Grall <[email protected]>
CC: Paul Durrant <[email protected]>
CC: Michał Leszczyński <[email protected]>
CC: Hubert Jasudowicz <[email protected]>
--- xen/common/grant_table.c | 93 ++++++++++++++++++++++++++++++------------- xen/common/memory.c | 42 +------------------ xen/include/xen/grant_table.h | 19 ++++----- 3 files changed, 75 insertions(+), 79 deletions(-)
xen/common/grant_table.c +66 -27
@@ -4013,6 +4013,72 @@ static int gnttab_get_shared_frame_mfn(struct domain *d,return 0 ; } +int gnttab_acquire_resource ( +struct domain *d,unsigned int id,unsigned long frame, +unsigned int nr_frames,xen_pfn_t mfn_list[]) +{ +struct grant_table *gt = d->grant_table; +unsigned int i = nr_frames, tot_frames; +void **vaddrs; +int rc =0 ; + +/* Input sanity. */ +if ( !nr_frames ) +return -EINVAL ; + +/* Overflow checks */ +if ( frame + nr_frames < frame ) +return -EINVAL ; + + tot_frames = frame + nr_frames; +if ( tot_frames != frame + nr_frames ) +return -EINVAL ; + +/* Grow table if necessary. */ +grant_write_lock (gt); +switch ( id ) + { +mfn_t tmp; + +case XENMEM_resource_grant_table_id_shared: + rc =gnttab_get_shared_frame_mfn (d, tot_frames -1 , &tmp); +break ; + +case XENMEM_resource_grant_table_id_status: +if ( gt->gt_version !=2 ) + { +default : + rc = -EINVAL ; +break ; + } + rc =gnttab_get_status_frame_mfn (d, tot_frames -1 , &tmp); +break ; + } + +/* Any errors from growing the table? */ +if ( rc ) + goto out; + +switch ( id ) + { +case XENMEM_resource_grant_table_id_shared: + vaddrs = gt->shared_raw; +break ; + +case XENMEM_resource_grant_table_id_status: + vaddrs = (void **)gt->status; +break ; + } + +for ( i =0 ; i < nr_frames; ++i ) + mfn_list[i] =virt_to_mfn (vaddrs[frame + i]); + + out: +grant_write_unlock (gt); + +return rc; +} +int gnttab_map_frame (struct domain *d,unsigned long idx,gfn_t gfn,mfn_t *mfn) {int rc =0 ;
@@ -4047,33 +4113,6 @@ int gnttab_map_frame(struct domain *d, unsigned long idx, gfn_t gfn, mfn_t *mfn)return rc; } -int gnttab_get_shared_frame (struct domain *d,unsigned long idx, -mfn_t *mfn) -{ -struct grant_table *gt = d->grant_table; -int rc; - -grant_write_lock (gt); - rc =gnttab_get_shared_frame_mfn (d, idx, mfn); -grant_write_unlock (gt); - -return rc; -} - -int gnttab_get_status_frame (struct domain *d,unsigned long idx, -mfn_t *mfn) -{ -struct grant_table *gt = d->grant_table; -int rc; - -grant_write_lock (gt); - rc = (gt->gt_version ==2 ) ? -gnttab_get_status_frame_mfn (d, idx, mfn) : -EINVAL ; -grant_write_unlock (gt); - -return rc; -} -static void gnttab_usage_print (struct domain *rd) {int first =1 ;
xen/common/memory.c +2 -40
54 lines not on this page · raw message
xen/include/xen/grant_table.h +7 -12
@@ -56,10 +56,10 @@ int mem_sharing_gref_to_gfn(struct grant_table *gt, grant_ref_t ref,int gnttab_map_frame (struct domain *d,unsigned long idx,gfn_t gfn,mfn_t *mfn); -int gnttab_get_shared_frame (struct domain *d,unsigned long idx, -mfn_t *mfn); -int gnttab_get_status_frame (struct domain *d,unsigned long idx, -mfn_t *mfn); + +int gnttab_acquire_resource ( +struct domain *d,unsigned int id,unsigned long frame, +unsigned int nr_frames,xen_pfn_t mfn_list[]);#else
@@ -93,14 +93,9 @@ static inline int gnttab_map_frame(struct domain *d, unsigned long idx,return -EINVAL ; } -static inline int gnttab_get_shared_frame (struct domain *d,unsigned long idx, -mfn_t *mfn) -{ -return -EINVAL ; -} - -static inline int gnttab_get_status_frame (struct domain *d,unsigned long idx, -mfn_t *mfn) +static inline int gnttab_acquire_resource ( +struct domain *d,unsigned int id,unsigned long frame, +unsigned int nr_frames,xen_pfn_t mfn_list[]) {return -EINVAL ; }
Signature · 1 line
2.11.0
PATCH 4/5 xen/memory: Fix acquire_resource size semantics
In reply to Andrew CooperCalling XENMEM_acquire_resource with a NULL frame_list is a request for the size of the resource, but the returned 32 is bogus.
If someone tries to follow it for XENMEM_resource_ioreq_server, the acquire call will fail as IOREQ servers currently top out at 2 frames, and it is only half the size of the default grant table limit for guests.
Also, no users actually request a resource size, because it was never wired up in the sole implemenation of resource aquisition in Linux.
Introduce a new resource_max_frames() to calculate the size of a resource, and implement it the IOREQ and grant subsystems.
It is impossible to guarentee that a mapping call following a successful size call will succedd (e.g. The target IOREQ server gets destroyed, or the domain switches from grant v2 to v1). Document the restriction, and use the flexibility to simplify the paths to be lockless.
---
CC: George Dunlap <[email protected]>
CC: Ian Jackson <[email protected]>
CC: Jan Beulich <[email protected]>
CC: Konrad Rzeszutek Wilk <[email protected]>
CC: Stefano Stabellini <[email protected]>
CC: Wei Liu <[email protected]>
CC: Julien Grall <[email protected]>
CC: Paul Durrant <[email protected]>
CC: Michał Leszczyński <[email protected]>
CC: Hubert Jasudowicz <[email protected]>
--- xen/arch/x86/mm.c | 20 ++++++++++++++++ xen/common/grant_table.c | 19 +++++++++++++++ xen/common/memory.c | 55 +++++++++++++++++++++++++++++++++---------- xen/include/asm-x86/mm.h | 3 +++ xen/include/public/memory.h | 16 +++++++++---- xen/include/xen/grant_table.h | 8 +++++++ xen/include/xen/mm.h | 6 +++++ 7 files changed, 110 insertions(+), 17 deletions(-)
xen/arch/x86/mm.c +20 -0
@@ -4600,6 +4600,26 @@ int xenmem_add_to_physmap_one(return rc; } +unsigned int arch_resource_max_frames ( +struct domain *d,unsigned int type,unsigned int id) +{ +unsigned int nr =0 ; + +switch ( type ) + { +#ifdef CONFIG_HVM +case XENMEM_resource_ioreq_server: +if ( !is_hvm_domain (d) ) +break ; +/* One frame for the buf-ioreq ring, and one frame per 128 vcpus. */ + nr =1 +DIV_ROUND_UP (d->max_vcpus *sizeof (struct ioreq ),PAGE_SIZE ); +break ; +#endif + } + +return nr; +} +int arch_acquire_resource (struct domain *d,unsigned int type,unsigned int id,unsigned long frame,unsigned int nr_frames,xen_pfn_t mfn_list[])
xen/common/grant_table.c +19 -0
@@ -4013,6 +4013,25 @@ static int gnttab_get_shared_frame_mfn(struct domain *d,return 0 ; } +unsigned int gnttab_resource_max_frames (struct domain *d,unsigned int id) +{ +unsigned int nr =0 ; + +/* Don't need the grant lock. This limit is fixed at domain create time. */ +switch ( id ) + { +case XENMEM_resource_grant_table_id_shared: + nr = d->grant_table->max_grant_frames; +break ; + +case XENMEM_resource_grant_table_id_status: + nr =grant_to_status_frames (d->grant_table->max_grant_frames); +break ; + } + +return nr; +} +int gnttab_acquire_resource (struct domain *d,unsigned int id,unsigned long frame,unsigned int nr_frames,xen_pfn_t mfn_list[])
xen/common/memory.c +42 -13
79 lines not on this page · raw message
xen/include/asm-x86/mm.h +3 -0
@@ -649,6 +649,9 @@ static inline bool arch_mfn_in_directmap(unsigned long mfn)return mfn <= (virt_to_mfn (eva -1 ) +1 ); } +unsigned int arch_resource_max_frames (struct domain *d, +unsigned int type,unsigned int id); +int arch_acquire_resource (struct domain *d,unsigned int type,unsigned int id,unsigned long frame,unsigned int nr_frames,xen_pfn_t mfn_list[]);
xen/include/public/memory.h +12 -4
@@ -639,10 +639,18 @@ struct xen_mem_acquire_resource {#define XENMEM_resource_grant_table_id_status 1/* -* IN/OUT - As an IN parameter number of frames of the resource -* to be mapped. However, if the specified value is 0 and -* frame_list is NULL then this field will be set to the -* maximum value supported by the implementation on return. +* IN/OUT +* +* As an IN parameter number of frames of the resource to be mapped. +* +* When frame_list is NULL and nr_frames is 0, this is interpreted as a +* request for the size of the resource, which shall be returned in the +* nr_frames field. +* +* The size of a resource will never be zero, but a nonzero result doesn't +* guarentee that a subsequent mapping request will be successful. There +* are further type/id specific constraints which may change between the +* two calls. */ uint32_t nr_frames;uint32_t pad;
xen/include/xen/grant_table.h +8 -0
@@ -57,6 +57,8 @@ int mem_sharing_gref_to_gfn(struct grant_table *gt, grant_ref_t ref,int gnttab_map_frame (struct domain *d,unsigned long idx,gfn_t gfn,mfn_t *mfn); +unsigned int gnttab_resource_max_frames (struct domain *d,unsigned int id); +int gnttab_acquire_resource (struct domain *d,unsigned int id,unsigned long frame,unsigned int nr_frames,xen_pfn_t mfn_list[]);
@@ -93,6 +95,12 @@ static inline int gnttab_map_frame(struct domain *d, unsigned long idx,return -EINVAL ; } +static inline unsigned int gnttab_resource_max_frames ( +struct domain *d,unsigned int id) +{ +return 0 ; +} +static inline int gnttab_acquire_resource (struct domain *d,unsigned int id,unsigned long frame,unsigned int nr_frames,xen_pfn_t mfn_list[])
xen/include/xen/mm.h +6 -0
@@ -686,6 +686,12 @@ static inline void put_page_alloc_ref(struct page_info *page) }#ifndef CONFIG_ARCH_ACQUIRE_RESOURCE +static inline unsigned int arch_resource_max_frames ( +struct domain *d,unsigned int type,unsigned int id) +{ +return 0 ; +} +static inline int arch_acquire_resource (struct domain *d,unsigned int type,unsigned int id,unsigned long frame,unsigned int nr_frames,xen_pfn_t mfn_list[])
Signature · 1 line
2.11.0
PATCH 5/5 tools/foreignmem: Support querying the size of a resource
In reply to Andrew CooperWith the Xen side of this interface fixed to return real sizes, userspace needs to be able to make the query.
Introduce xenforeignmemory_resource_size() for the purpose, bumping the library minor version and providing compatiblity for the non-Linux builds.
Its not possible to reuse the IOCTL_PRIVCMD_MMAP_RESOURCE infrastructure, because it depends on having already mmap()'d a suitably sized region before it will make an XENMEM_acquire_resource hypercall to Xen.
Instead, open a xencall handle and make an XENMEM_acquire_resource hypercall directly.
---
CC: Ian Jackson <[email protected]>
CC: Wei Liu <[email protected]>
CC: Paul Durrant <[email protected]>
CC: Michał Leszczyński <[email protected]>
CC: Hubert Jasudowicz <[email protected]>
--- tools/libs/foreignmemory/Makefile | 2 +- tools/libs/foreignmemory/core.c | 14 +++++++++ .../libs/foreignmemory/include/xenforeignmemory.h | 15 ++++++++++ tools/libs/foreignmemory/libxenforeignmemory.map | 4 +++ tools/libs/foreignmemory/linux.c | 35 ++++++++++++++++++++++ tools/libs/foreignmemory/private.h | 14 +++++++++ 6 files changed, 83 insertions(+), 1 deletion(-)
tools/libs/foreignmemory/Makefile +1 -1
@@ -2,7 +2,7 @@ XEN_ROOT = $(CURDIR)/../../.. include $(XEN_ROOT)/tools/Rules.mk MAJOR = 1 -MINOR = 3 +MINOR = 4 LIBNAME := foreignmemory USELIBS := toollog toolcore
tools/libs/foreignmemory/core.c +14 -0
@@ -53,6 +53,10 @@ xenforeignmemory_handle *xenforeignmemory_open(xentoollog_logger *logger,if (!fmem->logger) goto err; } + fmem->xcall =xencall_open (fmem->logger,0 ); +if ( !fmem->xcall ) + goto err; + rc =osdep_xenforeignmemory_open (fmem);if ( rc <0 ) goto err;
@@ -61,6 +65,7 @@ xenforeignmemory_handle *xenforeignmemory_open(xentoollog_logger *logger, err:xentoolcore__deregister_active_handle (&fmem->tc_ah);osdep_xenforeignmemory_close (fmem); +xencall_close (fmem->xcall);xtl_logger_destroy (fmem->logger_tofree);free (fmem);return NULL ;
@@ -75,6 +80,7 @@ int xenforeignmemory_close(xenforeignmemory_handle *fmem)xentoolcore__deregister_active_handle (&fmem->tc_ah); rc =osdep_xenforeignmemory_close (fmem); +xencall_close (fmem->xcall);xtl_logger_destroy (fmem->logger_tofree);free (fmem);return rc;
@@ -188,6 +194,14 @@ int xenforeignmemory_unmap_resource(return rc; } +int xenforeignmemory_resource_size ( +xenforeignmemory_handle *fmem,domid_t domid,unsigned int type, +unsigned int id,unsigned long *nr_frames) +{ +return osdep_xenforeignmemory_resource_size (fmem, domid, type, + id, nr_frames); +} +/* * Local variables: * mode: C
tools/libs/foreignmemory/include/xenforeignmemory.h +15 -0
@@ -179,6 +179,21 @@ xenforeignmemory_resource_handle *xenforeignmemory_map_resource(int xenforeignmemory_unmap_resource (xenforeignmemory_handle *fmem,xenforeignmemory_resource_handle *fres); +/** +* Determine the maximum size of a specific resource. +* +* @parm fmem handle to the open foreignmemory interface +* @parm domid the domain id +* @parm type the resource type +* @parm id the type-specific resource identifier +* +* Return 0 on success and fills in *nr_frames. Sets errno and return -1 on +* error. +*/ +int xenforeignmemory_resource_size ( +xenforeignmemory_handle *fmem,domid_t domid,unsigned int type, +unsigned int id,unsigned long *nr_frames); +#endif /*
tools/libs/foreignmemory/libxenforeignmemory.map +4 -0
@@ -19,3 +19,7 @@ VERS_1.3 { xenforeignmemory_map_resource; xenforeignmemory_unmap_resource; } VERS_1.2; +VERS_1.4 { + global: + xenforeignmemory_resource_size; +} VERS_1.3;
tools/libs/foreignmemory/linux.c +35 -0
@@ -28,6 +28,8 @@#include "private.h" +#include <xen/memory.h> +#define ROUNDUP (_x,_w) (((unsigned long)(_x)+(1UL<<(_w))-1) & ~((1UL<<(_w))-1))#ifndef O_CLOEXEC
@@ -340,6 +342,39 @@ int osdep_xenforeignmemory_map_resource(return 0 ; } +int osdep_xenforeignmemory_resource_size ( +xenforeignmemory_handle *fmem,domid_t domid,unsigned int type, +unsigned int id,unsigned long *nr_frames) +{ +int rc; +struct xen_mem_acquire_resource *xmar = +xencall_alloc_buffer (fmem->xcall,sizeof (*xmar)); + +if ( !xmar ) + { +PERROR ("Could not bounce memory for acquire_resource hypercall" ); +return -1 ; + } + + *xmar = (struct xen_mem_acquire_resource ){ + .domid = domid, + .type = type, + .id = id, + }; + + rc =xencall2 (fmem->xcall, __HYPERVISOR_memory_op, + XENMEM_acquire_resource, (uintptr_t )xmar); +if ( rc ) + goto out; + + *nr_frames = xmar->nr_frames; + + out: +xencall_free_buffer (fmem->xcall, xmar); + +return rc; +} +/* * Local variables: * mode: C
tools/libs/foreignmemory/private.h +14 -0
@@ -4,6 +4,7 @@#include <xentoollog.h> #include <xenforeignmemory.h> +#include <xencall.h> #include <xentoolcore_internal.h>
@@ -20,6 +21,7 @@struct xenforeignmemory_handle {xentoollog_logger *logger, *logger_tofree; +xencall_handle *xcall;unsigned flags;int fd;Xentoolcore__Active_Handle tc_ah;
@@ -74,6 +76,15 @@ static inline int osdep_xenforeignmemory_unmap_resource( {return 0 ; } + +static inline int osdep_xenforeignmemory_resource_size ( +xenforeignmemory_handle *fmem,domid_t domid,unsigned int type, +unsigned int id,unsigned long *nr_frames) +{ + errno =EOPNOTSUPP ; +return -1 ; +} +#else int osdep_xenforeignmemory_restrict (xenforeignmemory_handle *fmem,domid_t domid);
@@ -81,6 +92,9 @@ int osdep_xenforeignmemory_map_resource( xenforeignmemory_handle *fmem, xenforeignmemory_resource_handle *fres);int osdep_xenforeignmemory_unmap_resource (xenforeignmemory_handle *fmem,xenforeignmemory_resource_handle *fres); +int osdep_xenforeignmemory_resource_size ( +xenforeignmemory_handle *fmem,domid_t domid,unsigned int type, +unsigned int id,unsigned long *nr_frames);#endif #define PERROR(_f...) \
Signature · 1 line
2.11.0
PATCH 3/5 xen/memory: Fix compat XENMEM_acquire_resource for size requests
In reply to Andrew CooperCopy the nr_frames from the correct structure, so the caller doesn't unconditionally receive 0.
---
CC: George Dunlap <[email protected]>
CC: Ian Jackson <[email protected]>
CC: Jan Beulich <[email protected]>
CC: Konrad Rzeszutek Wilk <[email protected]>
CC: Stefano Stabellini <[email protected]>
CC: Wei Liu <[email protected]>
CC: Julien Grall <[email protected]>
CC: Paul Durrant <[email protected]>
CC: Michał Leszczyński <[email protected]>
CC: Hubert Jasudowicz <[email protected]>
--- xen/common/compat/memory.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
xen/common/compat/memory.c +1 -1
@@ -599,7 +599,7 @@ int compat_memory_op(unsigned int cmd, XEN_GUEST_HANDLE_PARAM(void) compat)if (__copy_field_to_guest (guest_handle_cast (compat, compat_mem_acquire_resource_t), - &cmp.mar, nr_frames) ) + nat.mar, nr_frames) )return -EFAULT ; }else
Signature · 1 line
2.11.0
cover Multiple fixes to XENMEM_acquire_resource
I thought this was going to be a very simple small bugfix for Michał's Processor Trace series. Serves me right for expecting it not to be full of bear traps...
The sole implementation of acquire_resource never asks for size, so its little surprise that Xen is broken for compat callers, and returns incorrect information for regular callers.
Patches 1-2 are cleanup with no net functional change. Patches 3-5 are fixes to the size side of this interface, and allow userspace to obtain the actual size of resources.
I'm still working on fixing the batch mapping support, which has further sharp corners, especially for compat callers.
This is sufficenet of a series so far to post for comments. A branch can be obtained from:
https://xenbits.xen.org/gitweb/?p=people/andrewcoop/xen.git;a=shortlog;h=refs/heads/xen-acquire-size
Andrew Cooper (5):
xen/memory: Introduce CONFIG_ARCH_ACQUIRE_RESOURCE xen/gnttab: Rework resource acquisition
xen/memory: Fix compat XENMEM_acquire_resource for size requests xen/memory: Fix acquire_resource size semantics tools/foreignmem: Support querying the size of a resource
tools/libs/foreignmemory/Makefile | 2 +- tools/libs/foreignmemory/core.c | 14 +++ .../libs/foreignmemory/include/xenforeignmemory.h | 15 +++ tools/libs/foreignmemory/libxenforeignmemory.map | 4 + tools/libs/foreignmemory/linux.c | 35 +++++++ tools/libs/foreignmemory/private.h | 14 +++ xen/arch/x86/Kconfig | 1 + xen/arch/x86/mm.c | 20 ++++ xen/common/Kconfig | 3 + xen/common/compat/memory.c | 2 +- xen/common/grant_table.c | 112 ++++++++++++++++----- xen/common/memory.c | 85 +++++++--------- xen/include/asm-arm/mm.h | 8 -- xen/include/asm-x86/mm.h | 3 + xen/include/public/memory.h | 16 ++- xen/include/xen/grant_table.h | 21 ++-- xen/include/xen/mm.h | 15 +++ 17 files changed, 273 insertions(+), 97 deletions(-)
Signature · 1 line
2.11.0
PATCH 1/5 xen/memory: Introduce CONFIG_ARCH_ACQUIRE_RESOURCE
New architectures shouldn't be forced to implement no-op stubs for unused functionality.
Introduce CONFIG_ARCH_ACQUIRE_RESOURCE which can be opted in to, and provide compatibility logic in xen/mm.h
No functional change.
---
CC: Jan Beulich <[email protected]>
CC: Wei Liu <[email protected]>
CC: Roger Pau Monné <[email protected]>
CC: Stefano Stabellini <[email protected]>
CC: Julien Grall <[email protected]>
CC: Volodymyr Babchuk <[email protected]>
CC: Paul Durrant <[email protected]>
CC: Michał Leszczyński <[email protected]>
CC: Hubert Jasudowicz <[email protected]>
--- xen/arch/x86/Kconfig | 1 + xen/common/Kconfig | 3 +++ xen/include/asm-arm/mm.h | 8 -------- xen/include/xen/mm.h | 9 +++++++++ 4 files changed, 13 insertions(+), 8 deletions(-)
xen/arch/x86/Kconfig +1 -0
@@ -6,6 +6,7 @@ config X86 select ACPI select ACPI_LEGACY_TABLES_LOOKUP select ARCH_SUPPORTS_INT128 + select ARCH_ACQUIRE_RESOURCE select COMPAT select CORE_PARKING select HAS_ALTERNATIVE
xen/common/Kconfig +3 -0
@@ -22,6 +22,9 @@ config GRANT_TABLE If unsure, say Y. +config ARCH_ACQUIRE_RESOURCE + bool + config HAS_ALTERNATIVE bool
xen/include/asm-arm/mm.h +0 -8
@@ -358,14 +358,6 @@ static inline void put_page_and_type(struct page_info *page)void clear_and_clean_page (struct page_info *page); -static inline -int arch_acquire_resource (struct domain *d,unsigned int type,unsigned int id, -unsigned long frame,unsigned int nr_frames, -xen_pfn_t mfn_list[]) -{ -return -EOPNOTSUPP ; -} -unsigned int arch_get_dma_bitsize (void );#endif /* __ARCH_ARM_MM__ */
xen/include/xen/mm.h +9 -0
@@ -685,4 +685,13 @@ static inline void put_page_alloc_ref(struct page_info *page) } } +#ifndef CONFIG_ARCH_ACQUIRE_RESOURCE +static inline int arch_acquire_resource ( +struct domain *d,unsigned int type,unsigned int id,unsigned long frame, +unsigned int nr_frames,xen_pfn_t mfn_list[]) +{ +return -EOPNOTSUPP ; +} +#endif /* !CONFIG_ARCH_ACQUIRE_RESOURCE */ +#endif /* __XEN_MM_H__ */
Signature · 1 line
2.11.0
Re: [PATCH 2/5] xen/gnttab: Rework resource acquisition
In reply to Andrew CooperQuoted 5 lines · Andrew Cooper
xen/common/grant_table.c +0
-0
xen/common/grant_table.c static int gnttab_get_shared_frame_mfn(struct domain *d, · line 4022
Quoted 10 lines
xen/common/grant_table.c +7
-0@@ -4013,6 +4013,72 @@ static int gnttab_get_shared_frame_mfn(struct domain *d,return 0 ; } +int gnttab_acquire_resource ( +struct domain *d,unsigned int id,unsigned long frame, +unsigned int nr_frames,xen_pfn_t mfn_list[]) +{ +struct grant_table *gt = d->grant_table; +unsigned int i = nr_frames, tot_frames; +void **vaddrs;
I've folded an "= NULL" here to fix the CentOS 6 build.
~Andrew
Re: [PATCH 5/5] tools/foreignmem: Support querying the size of a resource
In reply to Andrew CooperQuoted 13 lines · Andrew Cooper
With the Xen side of this interface fixed to return real sizes, userspace needs to be able to make the query.
Introduce
xenforeignmemory_resource_size()for the purpose, bumping the library minor version and providing compatiblity for the non-Linux builds.Its not possible to reuse the
IOCTL_PRIVCMD_MMAP_RESOURCEinfrastructure, because it depends on having already mmap()'d a suitably sized region before it will make anXENMEM_acquire_resourcehypercall to Xen.Instead, open a xencall handle and make an
XENMEM_acquire_resourcehypercall directly.
Quoted 7 lines
Signed-off-by Andrew Cooper <[email protected]>---
CC: Ian Jackson <[email protected]>
CC: Wei Liu <[email protected]>
CC: Paul Durrant <[email protected]>
CC: Michał Leszczyński <[email protected]>
CC: Hubert Jasudowicz <[email protected]>
I've folded:
tools/Rules.mk +0 -0
@@ -123,7 +123,7 @@ LDLIBS_libxencall = $(SHDEPS_libxencall)
$(XEN_LIBXENCALL)/libxencall$(libextens
SHLIB_libxencall = $(SHDEPS_libxencall) -Wl,-rpath-link=$(XEN_LIBXENCALL)
CFLAGS_libxenforeignmemory = -I$(XEN_LIBXENFOREIGNMEMORY)/include $(CFLAGS_xeninclude)
-SHDEPS_libxenforeignmemory = $(SHLIB_libxentoolcore) +SHDEPS_libxenforeignmemory = $(SHLIB_libxentoolcore) $(SHDEPS_libxencall) LDLIBS_libxenforeignmemory = $(SHDEPS_libxenforeignmemory) $(XEN_LIBXENFOREIGNMEMORY)/libxenforeignmemory$(libextension) SHLIB_libxenforeignmemory = $(SHDEPS_libxenforeignmemory) -Wl,-rpath-link=$(XEN_LIBXENFOREIGNMEMORY)
(quoted diff) +0 -0
b/tools/libs/foreignmemory/Makefile
index 8e07f92c59..f3a61e27c7 100644
tools/libs/foreignmemory/Makefile +0 -0
@@ -4,7 +4,7 @@ include $(XEN_ROOT)/tools/Rules.mk
MAJOR = 1
MINOR = 4
LIBNAME := foreignmemory
-USELIBS := toollog toolcore
+USELIBS := toollog toolcore call
SRCS-y += core.c
SRCS-$(CONFIG_Linux) += linux.c
to fix the build in certain containers.
~Andrew
Re: [PATCH 1/5] xen/memory: Introduce CONFIG_ARCH_ACQUIRE_RESOURCE
In reply to Andrew CooperQuoted 9 lines · Andrew Cooper
New architectures shouldn't be forced to implement no-op stubs for unused functionality.
Introduce
CONFIG_ARCH_ACQUIRE_RESOURCEwhich can be opted in to, and provide compatibility logic inxen/mm.hNo functional change.
Signed-off-by Andrew Cooper <[email protected]>
Re: [PATCH 2/5] xen/gnttab: Rework resource acquisition
In reply to Andrew CooperQuoted 9 lines · Andrew Cooper
The existing logic doesn't function in the general case for mapping a guests grant table, due to arbitrary 32 frame limit, and the default grant table limit being 64.
In order to start addressing this, rework the existing grant table logic by implementing a single
gnttab_acquire_resource(). This is far more efficient than the previousacquire_grant_table()inmemory.cbecause it doesn't take the grant table write lock, and attempt to grow the table, for every single frame.
Among the code you replace there is a comment "Iterate backwards in case table needs to grow" explaining why what you say about growing the grant table didn't actually happen.
Quoted 8 lines
xen/common/grant_table.c +2
-0@@ -4013,6 +4013,72 @@ static int gnttab_get_shared_frame_mfn(struct domain *d,return 0 ; } +int gnttab_acquire_resource( + struct domain *d, unsigned int id, unsigned long frame,
xen/common/grant_table.c static int gnttab_get_shared_frame_mfn(struct domain *d, · line 4027
Quoted 10 lines
xen/common/grant_table.c +10
-0+ unsigned int nr_frames, xen_pfn_t mfn_list[]) +{ +struct grant_table *gt = d->grant_table; +unsigned int i = nr_frames, tot_frames; +void **vaddrs; +int rc =0 ; + +/* Input sanity. */ +if ( !nr_frames ) +return -EINVAL ;
I can't seem to be able to find an equivalent of this in the old logic, and hence this looks like an unwarranted change in behavior to me. We have quite a few hypercall ops where some count being zero is simply a no-op, i.e. yielding success without doing anything.
xen/common/grant_table.c static int gnttab_get_shared_frame_mfn(struct domain *d, · line 4035
Quoted 7 lines
xen/common/grant_table.c +7
-0+/* Overflow checks */ +if ( frame + nr_frames < frame ) +return -EINVAL ; + + tot_frames = frame + nr_frames; +if ( tot_frames != frame + nr_frames ) +return -EINVAL ;
I find the naming here quite confusing. I realize part of this stems from the code you replace, but anyway: "unsigned long frame" typically represents a memory frame number of some sort, making the calculation look as if it was wrong. (Initially I merely meant to ask whether this check isn't redundant with the prior one, or vice versa.)
Quoted 23 lines
xen/common/grant_table.c +23
-0+/* Grow table if necessary. */ +grant_write_lock (gt); +switch ( id ) + { +mfn_t tmp; + +case XENMEM_resource_grant_table_id_shared: + rc =gnttab_get_shared_frame_mfn (d, tot_frames -1 , &tmp); +break ; + +case XENMEM_resource_grant_table_id_status: +if ( gt->gt_version !=2 ) + { +default : + rc = -EINVAL ; +break ; + } + rc =gnttab_get_status_frame_mfn (d, tot_frames -1 , &tmp); +break ; + } + +/* Any errors from growing the table? */ +if ( rc )
xen/common/grant_table.c static int gnttab_get_shared_frame_mfn(struct domain *d, · line 4069
Quoted 10 lines
xen/common/grant_table.c +10
-0+ goto out; + +switch ( id ) + { +case XENMEM_resource_grant_table_id_shared: + vaddrs = gt->shared_raw; +break ; + +case XENMEM_resource_grant_table_id_status: + vaddrs = (void **)gt->status;
Now this is the kind of cast that I consider really dangerous, and hence worth trying hard to avoid. With the code structure as is, I don't see an immediate solution though.
xen/common/grant_table.c static int gnttab_get_shared_frame_mfn(struct domain *d, · line 4069
Quoted 2 lines
xen/common/grant_table.c +2
-0+break ; + }
Worth having an ASSERT_UNREACHABLE() default case here?
Jan
Re: [PATCH 3/5] xen/memory: Fix compat XENMEM_acquire_resource for size requests
In reply to Andrew CooperQuoted 2 lines · Andrew Cooper
Copy the
nr_framesfrom the correct structure, so the caller doesn't unconditionally receive 0.
Well, no - it does get copied from the correct structure. It's just that the field doesn't get set properly up front. Otherwise you'll
(a) build in an unchecked assumption that the native and compat fields match in type and (b) set a bad example for people looking here and then cloning this code in perhaps a case where (a) is not even true. If you agree, the alternative change of setting cmp.mar.nr_frames from nat.mar->nr_frames before the call is
Jan
RE: [PATCH 1/5] xen/memory: Introduce CONFIG_ARCH_ACQUIRE_RESOURCE
In reply to Andrew CooperQuoted 10 lines
-----Original Message-----
From: Andrew Cooper <[email protected]>
Sent: 28 July 2020 12:37
To: Xen-devel <[email protected]>
Cc: Andrew Cooper <[email protected]>; Jan Beulich <[email protected]>; Wei Liu <[email protected]>; Roger Pau Monné <[email protected]>; Stefano Stabellini <[email protected]>; Julien Grall <[email protected]>; Volodymyr Babchuk <[email protected]>; Paul Durrant <[email protected]>; Michał Leszczyński <[email protected]>; Hubert Jasudowicz <[email protected]>
Subject: [PATCH 1/5] xen/memory: IntroduceCONFIG_ARCH_ACQUIRE_RESOURCE
Quoted 7 lines
New architectures shouldn't be forced to implement no-op stubs for unused functionality.
Introduce
CONFIG_ARCH_ACQUIRE_RESOURCEwhich can be opted in to, and provide compatibility logic inxen/mm.hNo functional change.
Code-wise, it looks fine, so...
...but ...
Quoted 24 lines
Signed-off-by Andrew Cooper <[email protected]>---
CC: Jan Beulich <[email protected]>
CC: Wei Liu <[email protected]>
CC: Roger Pau Monné <[email protected]>
CC: Stefano Stabellini <[email protected]>
CC: Julien Grall <[email protected]>
CC: Volodymyr Babchuk <[email protected]>
CC: Paul Durrant <[email protected]>
CC: Michał Leszczyński <[email protected]>
CC: Hubert Jasudowicz <[email protected]>--- xen/arch/x86/Kconfig | 1 + xen/common/Kconfig | 3 +++ xen/include/asm-arm/mm.h | 8 -------- xen/include/xen/mm.h | 9 +++++++++ 4 files changed, 13 insertions(+), 8 deletions(-)xen/arch/x86/Kconfig +0
-0
xen/arch/x86/Kconfig config X86 · line 9
Quoted 4 lines
xen/arch/x86/Kconfig +1
-0@@ -6,6 +6,7 @@ config X86 select ACPI select ACPI_LEGACY_TABLES_LOOKUP select ARCH_SUPPORTS_INT128 + select ARCH_ACQUIRE_RESOURCE
... I do wonder whether 'HAS_ACQUIRE_RESOURCE' is a better and more descriptive name.
Quoted 56 lines
xen/arch/x86/Kconfig +0
-0select COMPAT select CORE_PARKING select HAS_ALTERNATIVExen/common/Kconfig +3
-0@@ -22,6 +22,9 @@ config GRANT_TABLE If unsure, say Y. +config ARCH_ACQUIRE_RESOURCE + bool + config HAS_ALTERNATIVE boolxen/include/asm-arm/mm.h +0
-8@@ -358,14 +358,6 @@ static inline void put_page_and_type(struct page_info *page)void clear_and_clean_page (struct page_info *page); -static inline -int arch_acquire_resource (struct domain *d,unsigned int type,unsigned int id, -unsigned long frame,unsigned int nr_frames, -xen_pfn_t mfn_list[]) -{ -return -EOPNOTSUPP ; -} -unsigned int arch_get_dma_bitsize (void );#endif /* __ARCH_ARM_MM__ */ xen/include/xen/mm.h +9
-1@@ -685,4 +685,13 @@ static inline void put_page_alloc_ref(struct page_info *page) } } +#ifndef CONFIG_ARCH_ACQUIRE_RESOURCE +static inline int arch_acquire_resource ( +struct domain *d,unsigned int type,unsigned int id,unsigned long frame, +unsigned int nr_frames,xen_pfn_t mfn_list[]) +{ +return -EOPNOTSUPP ; +} +#endif /* !CONFIG_ARCH_ACQUIRE_RESOURCE */ +#endif /* __XEN_MM_H__ */ --2.11.0
RE: [PATCH 2/5] xen/gnttab: Rework resource acquisition
In reply to Andrew CooperQuoted 15 lines
-----Original Message-----
From: Andrew Cooper <[email protected]>
Sent: 28 July 2020 12:37
To: Xen-devel <[email protected]>
Cc: Andrew Cooper <[email protected]>; George Dunlap <[email protected]>; Ian Jackson <[email protected]>; Jan Beulich <[email protected]>; Konrad Rzeszutek Wilk <[email protected]>; Stefano Stabellini <[email protected]>; Wei Liu <[email protected]>; Julien Grall <[email protected]>; Paul Durrant <[email protected]>; Michał Leszczyński <[email protected]>; Hubert Jasudowicz <[email protected]>
Subject: [PATCH 2/5] xen/gnttab: Rework resource acquisitionThe existing logic doesn't function in the general case for mapping a guests grant table, due to arbitrary 32 frame limit, and the default grant table limit being 64.
Quoted 6 lines
In order to start addressing this, rework the existing grant table logic by implementing a single
gnttab_acquire_resource(). This is far more efficient than the previousacquire_grant_table()inmemory.cbecause it doesn't take the grant table write lock, and attempt to grow the table, for every single frame.
But that should not have happened before because the code deliberately iterates backwards, thereby starting with the last frame, thereby growing the table at most once. (I agree that dropping and re-acquiring the lock every time was sub-optimal).
Quoted 33 lines
The new
gnttab_acquire_resource()function subsumes the previous two gnttab_get_{shared,status}_frame() helpers.No functional change.
Signed-off-by Andrew Cooper <[email protected]>---
CC: George Dunlap <[email protected]>
CC: Ian Jackson <[email protected]>
CC: Jan Beulich <[email protected]>
CC: Konrad Rzeszutek Wilk <[email protected]>
CC: Stefano Stabellini <[email protected]>
CC: Wei Liu <[email protected]>
CC: Julien Grall <[email protected]>
CC: Paul Durrant <[email protected]>
CC: Michał Leszczyński <[email protected]>
CC: Hubert Jasudowicz <[email protected]>--- xen/common/grant_table.c | 93 ++++++++++++++++++++++++++++++------------- xen/common/memory.c | 42 +------------------ xen/include/xen/grant_table.h | 19 ++++----- 3 files changed, 75 insertions(+), 79 deletions(-)xen/common/grant_table.c +2
-0@@ -4013,6 +4013,72 @@ static int gnttab_get_shared_frame_mfn(struct domain *d,return 0 ; } +int gnttab_acquire_resource( + struct domain *d, unsigned int id, unsigned long frame,
xen/common/grant_table.c static int gnttab_get_shared_frame_mfn(struct domain *d, · line 4027
Quoted 10 lines
xen/common/grant_table.c +10
-0+ unsigned int nr_frames, xen_pfn_t mfn_list[]) +{ +struct grant_table *gt = d->grant_table; +unsigned int i = nr_frames, tot_frames; +void **vaddrs; +int rc =0 ; + +/* Input sanity. */ +if ( !nr_frames ) +return -EINVAL ;
This was not an error before. Does mapping 0 frames really need to be a failure?
xen/common/grant_table.c static int gnttab_get_shared_frame_mfn(struct domain *d, · line 4033
Quoted 6 lines
xen/common/grant_table.c +6
-0+ +/* Overflow checks */ +if ( frame + nr_frames < frame ) +return -EINVAL ; + + tot_frames = frame + nr_frames;
That name is confusing. 'last_frame' perhaps (and then make the -1 implicit)?
Quoted 26 lines
xen/common/grant_table.c +26
-0+if ( tot_frames != frame + nr_frames ) +return -EINVAL ; + +/* Grow table if necessary. */ +grant_write_lock (gt); +switch ( id ) + { +mfn_t tmp; + +case XENMEM_resource_grant_table_id_shared: + rc =gnttab_get_shared_frame_mfn (d, tot_frames -1 , &tmp); +break ; + +case XENMEM_resource_grant_table_id_status: +if ( gt->gt_version !=2 ) + { +default : + rc = -EINVAL ; +break ; + } + rc =gnttab_get_status_frame_mfn (d, tot_frames -1 , &tmp); +break ; + } + +/* Any errors from growing the table? */ +if ( rc )
xen/common/grant_table.c static int gnttab_get_shared_frame_mfn(struct domain *d, · line 4069
Quoted 10 lines
xen/common/grant_table.c +10
-0+ goto out; + +switch ( id ) + { +case XENMEM_resource_grant_table_id_shared: + vaddrs = gt->shared_raw; +break ; + +case XENMEM_resource_grant_table_id_status: + vaddrs = (void **)gt->status;
Erk. Could we avoid this casting nastiness by putting a loop in each switch case. I know it could be considered code duplication but this is a bit icky.
xen/common/grant_table.c static int gnttab_get_shared_frame_mfn(struct domain *d, · line 4077
Quoted 8 lines
xen/common/grant_table.c +8
-0+break ; + } + +for ( i =0 ; i < nr_frames; ++i ) + mfn_list[i] =virt_to_mfn (vaddrs[frame + i]); + + out: +grant_write_unlock (gt);
Since you deliberately grew the table first, could you not drop the write lock and acquire it a read lock before looping over the frames?
Paul
Quoted 141 lines
xen/common/grant_table.c +4
-27+ +return rc; +} +int gnttab_map_frame (struct domain *d,unsigned long idx,gfn_t gfn,mfn_t *mfn) {int rc =0 ;@@ -4047,33 +4113,6 @@ int gnttab_map_frame(struct domain *d, unsigned long idx, gfn_t gfn, mfn_t *mfn)return rc; } -int gnttab_get_shared_frame (struct domain *d,unsigned long idx, -mfn_t *mfn) -{ -struct grant_table *gt = d->grant_table; -int rc; - -grant_write_lock (gt); - rc =gnttab_get_shared_frame_mfn (d, idx, mfn); -grant_write_unlock (gt); - -return rc; -} - -int gnttab_get_status_frame (struct domain *d,unsigned long idx, -mfn_t *mfn) -{ -struct grant_table *gt = d->grant_table; -int rc; - -grant_write_lock (gt); - rc = (gt->gt_version ==2 ) ? -gnttab_get_status_frame_mfn (d, idx, mfn) : -EINVAL ; -grant_write_unlock (gt); - -return rc; -} -static void gnttab_usage_print (struct domain *rd) {int first = 1;xen/common/memory.c +2
-40@@ -1007,44 +1007,6 @@ static long xatp_permission_check(struct domain *d, unsigned int space)return xsm_add_to_physmap (XSM_TARGET , current->domain, d); } -static int acquire_grant_table (struct domain *d,unsigned int id, -unsigned long frame, -unsigned int nr_frames, -xen_pfn_t mfn_list[]) -{ -unsigned int i = nr_frames; - -/* Iterate backwards in case table needs to grow */ -while ( i-- !=0 ) - { -mfn_t mfn =INVALID_MFN ; -int rc; - -switch ( id ) - { -case XENMEM_resource_grant_table_id_shared: - rc =gnttab_get_shared_frame (d, frame + i, &mfn); -break ; - -case XENMEM_resource_grant_table_id_status: - rc =gnttab_get_status_frame (d, frame + i, &mfn); -break ; - -default : - rc = -EINVAL ; -break ; - } - -if ( rc ) -return rc; - -ASSERT (!mfn_eq (mfn,INVALID_MFN )); - mfn_list[i] =mfn_x (mfn); - } - -return 0 ; -} -static int acquire_resource (XEN_GUEST_HANDLE_PARAM (xen_mem_acquire_resource_t) arg) {@@ -1091,8 +1053,8 @@ static int acquire_resource(switch ( xmar.type ) {case XENMEM_resource_grant_table: - rc =acquire_grant_table (d, xmar.id, xmar.frame, xmar.nr_frames, - mfn_list); + rc =gnttab_acquire_resource (d, xmar.id, xmar.frame, xmar.nr_frames, + mfn_list);break ;default :xen/include/xen/grant_table.h +7
-13@@ -56,10 +56,10 @@ int mem_sharing_gref_to_gfn(struct grant_table *gt, grant_ref_t ref,int gnttab_map_frame (struct domain *d,unsigned long idx,gfn_t gfn,mfn_t *mfn); -int gnttab_get_shared_frame (struct domain *d,unsigned long idx, -mfn_t *mfn); -int gnttab_get_status_frame (struct domain *d,unsigned long idx, -mfn_t *mfn); + +int gnttab_acquire_resource ( +struct domain *d,unsigned int id,unsigned long frame, +unsigned int nr_frames,xen_pfn_t mfn_list[]);#else @@ -93,14 +93,9 @@ static inline int gnttab_map_frame(struct domain *d, unsigned long idx,return -EINVAL ; } -static inline int gnttab_get_shared_frame (struct domain *d,unsigned long idx, -mfn_t *mfn) -{ -return -EINVAL ; -} - -static inline int gnttab_get_status_frame (struct domain *d,unsigned long idx, -mfn_t *mfn) +static inline int gnttab_acquire_resource ( +struct domain *d,unsigned int id,unsigned long frame, +unsigned int nr_frames,xen_pfn_t mfn_list[]) {return -EINVAL ; } --2.11.0
RE: [PATCH 3/5] xen/memory: Fix compat XENMEM_acquire_resource for size requests
In reply to Andrew CooperQuoted 11 lines
-----Original Message-----
From: Andrew Cooper <[email protected]>
Sent: 28 July 2020 12:37
To: Xen-devel <[email protected]>
Cc: Andrew Cooper <[email protected]>; George Dunlap <[email protected]>; Ian Jackson <[email protected]>; Jan Beulich <[email protected]>; Konrad Rzeszutek Wilk <[email protected]>; Stefano Stabellini <[email protected]>; Wei Liu <[email protected]>; Julien Grall <[email protected]>; Paul Durrant <[email protected]>; Michał Leszczyński <[email protected]>; Hubert Jasudowicz <[email protected]>
Subject: [PATCH 3/5] xen/memory: Fix compatXENMEM_acquire_resourcefor size requests
Quoted 4 lines
Copy the
nr_framesfrom the correct structure, so the caller doesn't unconditionally receive 0.Signed-off-by Andrew Cooper <[email protected]>
Quoted 30 lines
---
CC: George Dunlap <[email protected]>
CC: Ian Jackson <[email protected]>
CC: Jan Beulich <[email protected]>
CC: Konrad Rzeszutek Wilk <[email protected]>
CC: Stefano Stabellini <[email protected]>
CC: Wei Liu <[email protected]>
CC: Julien Grall <[email protected]>
CC: Paul Durrant <[email protected]>
CC: Michał Leszczyński <[email protected]>
CC: Hubert Jasudowicz <[email protected]>--- xen/common/compat/memory.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)xen/common/compat/memory.c +1
-2@@ -599,7 +599,7 @@ int compat_memory_op(unsigned int cmd, XEN_GUEST_HANDLE_PARAM(void) compat)if (__copy_field_to_guest (guest_handle_cast (compat, compat_mem_acquire_resource_t), - &cmp.mar, nr_frames) ) + nat.mar, nr_frames) )return -EFAULT ; }else --2.11.0
RE: [PATCH 4/5] xen/memory: Fix acquire_resource size semantics
In reply to Andrew CooperQuoted 18 lines
-----Original Message-----
From: Xen-devel <[email protected]> On Behalf Of Andrew Cooper
Sent: 28 July 2020 12:37
To: Xen-devel <[email protected]>
Cc: Hubert Jasudowicz <[email protected]>; Stefano Stabellini <[email protected]>; Julien Grall <[email protected]>; Wei Liu <[email protected]>; Konrad Rzeszutek Wilk <[email protected]>; George Dunlap <[email protected]>; Andrew Cooper <[email protected]>; Paul Durrant <[email protected]>; Jan Beulich <[email protected]>; Michał Leszczyński <[email protected]>; Ian Jackson <[email protected]>
Subject: [PATCH 4/5] xen/memory: Fixacquire_resourcesize semanticsCalling
XENMEM_acquire_resourcewith a NULLframe_listis a request for the size of the resource, but the returned 32 is bogus.If someone tries to follow it for
XENMEM_resource_ioreq_server, the acquire call will fail as IOREQ servers currently top out at 2 frames, and it is only half the size of the default grant table limit for guests.
Quoted 7 lines
Also, no users actually request a resource size, because it was never wired up in the sole implemenation of resource aquisition in Linux.
Introduce a new
resource_max_frames()to calculate the size of a resource, and implement it the IOREQ and grant subsystems.It is impossible to guarentee that a mapping call following a successful size
s/guarantee/guarantee
Quoted 1 line
call will succedd (e.g. The target IOREQ server gets destroyed, or the domain
s/succedd/succeed
Quoted 67 lines
switches from grant v2 to v1). Document the restriction, and use the flexibility to simplify the paths to be lockless.
Signed-off-by Andrew Cooper <[email protected]>---
CC: George Dunlap <[email protected]>
CC: Ian Jackson <[email protected]>
CC: Jan Beulich <[email protected]>
CC: Konrad Rzeszutek Wilk <[email protected]>
CC: Stefano Stabellini <[email protected]>
CC: Wei Liu <[email protected]>
CC: Julien Grall <[email protected]>
CC: Paul Durrant <[email protected]>
CC: Michał Leszczyński <[email protected]>
CC: Hubert Jasudowicz <[email protected]>--- xen/arch/x86/mm.c | 20 ++++++++++++++++ xen/common/grant_table.c | 19 +++++++++++++++ xen/common/memory.c | 55 +++++++++++++++++++++++++++++++++---------- xen/include/asm-x86/mm.h | 3 +++ xen/include/public/memory.h | 16 +++++++++---- xen/include/xen/grant_table.h | 8 +++++++ xen/include/xen/mm.h | 6 +++++ 7 files changed, 110 insertions(+), 17 deletions(-)xen/arch/x86/mm.c +20
-0@@ -4600,6 +4600,26 @@ int xenmem_add_to_physmap_one(return rc; } +unsigned int arch_resource_max_frames ( +struct domain *d,unsigned int type,unsigned int id) +{ +unsigned int nr =0 ; + +switch ( type ) + { +#ifdef CONFIG_HVM +case XENMEM_resource_ioreq_server: +if ( !is_hvm_domain (d) ) +break ; +/* One frame for the buf-ioreq ring, and one frame per 128 vcpus. */ + nr =1 +DIV_ROUND_UP (d->max_vcpus *sizeof (struct ioreq ),PAGE_SIZE ); +break ; +#endif + } + +return nr; +} +int arch_acquire_resource (struct domain *d,unsigned int type,unsigned int id,unsigned long frame,unsigned int nr_frames,xen_pfn_t mfn_list[])xen/common/grant_table.c +3
-0@@ -4013,6 +4013,25 @@ static int gnttab_get_shared_frame_mfn(struct domain *d,return 0 ; } +unsigned int gnttab_resource_max_frames (struct domain *d,unsigned int id) +{ +unsigned int nr =0 ;
xen/common/grant_table.c static int gnttab_get_shared_frame_mfn(struct domain *d, · line 4028
Quoted 10 lines
xen/common/grant_table.c +10
-0+ +/* Don't need the grant lock. This limit is fixed at domain create time. */ +switch ( id ) + { +case XENMEM_resource_grant_table_id_shared: + nr = d->grant_table->max_grant_frames; +break ; + +case XENMEM_resource_grant_table_id_status: + nr =grant_to_status_frames (d->grant_table->max_grant_frames);
Two uses of d->grant_table, so perhaps define a stack variable for it? Also, should you not make sure 0 is returned in the case of a v1 table?
Quoted 14 lines
xen/common/grant_table.c +6
-0+break ; + } + +return nr; +} +int gnttab_acquire_resource (struct domain *d,unsigned int id,unsigned long frame,unsigned int nr_frames,xen_pfn_t mfn_list[])xen/common/memory.c +0
-0
xen/common/memory.c static long xatp_permission_check(struct domain *d, unsigned int space) · line 1014
Quoted 8 lines
xen/common/memory.c +5
-0@@ -1007,6 +1007,26 @@ static long xatp_permission_check(struct domain *d, unsigned int space)return xsm_add_to_physmap (XSM_TARGET , current->domain, d); } +/* +* Return 0 on any kind of error. Caller converts to -EINVAL. +* +* All nonzero values should be repeatable (i.e. derived from some fixed +* proerty of the domain), and describe the full resource (i.e. mapping the
s/property/property
xen/common/memory.c static long xatp_permission_check(struct domain *d, unsigned int space) · line 1015
Quoted 1 line
xen/common/memory.c +1
-0+* result of this call will be the entire resource).
This precludes dynamically adding a resource to a running domain. Do we really want to bake in that restriction?
Quoted 99 lines
xen/common/memory.c +36
-13+*/ +static unsigned int resource_max_frames (struct domain *d, +unsigned int type,unsigned int id) +{ +switch ( type ) + { +case XENMEM_resource_grant_table: +return gnttab_resource_max_frames (d, id); + +default : +return arch_resource_max_frames (d, type, id); + } +} +static int acquire_resource (XEN_GUEST_HANDLE_PARAM (xen_mem_acquire_resource_t) arg) {@@ -1018,6 +1038,7 @@ static int acquire_resource(* use-cases then per-CPU arrays or heap allocations may be required. */ xen_pfn_t mfn_list[32 ]; +unsigned int max_frames;int rc;if (copy_from_guest (&xmar, arg,1 ) )@@ -1026,19 +1047,6 @@ static int acquire_resource(if ( xmar.pad !=0 )return -EINVAL ; -if (guest_handle_is_null (xmar.frame_list) ) - { -if ( xmar.nr_frames ) -return -EINVAL ; - - xmar.nr_frames =ARRAY_SIZE (mfn_list); - -if (__copy_field_to_guest (arg, &xmar, nr_frames) ) -return -EFAULT ; - -return 0 ; - } -if ( xmar.nr_frames >ARRAY_SIZE (mfn_list) )return -E2BIG ;@@ -1050,6 +1058,27 @@ static int acquire_resource(if ( rc ) goto out; + max_frames =resource_max_frames (d, xmar.type, xmar.id); + + rc = -EINVAL ; +if ( !max_frames ) + goto out; + +if (guest_handle_is_null (xmar.frame_list) ) + { +if ( xmar.nr_frames ) + goto out; + + xmar.nr_frames = max_frames; + + rc = -EFAULT ; +if (__copy_field_to_guest (arg, &xmar, nr_frames) ) + goto out; + + rc =0 ; + goto out; + } +switch ( xmar.type ) {case XENMEM_resource_grant_table:xen/include/asm-x86/mm.h +3
-0@@ -649,6 +649,9 @@ static inline bool arch_mfn_in_directmap(unsigned long mfn)return mfn <= (virt_to_mfn (eva -1 ) +1 ); } +unsigned int arch_resource_max_frames (struct domain *d, +unsigned int type,unsigned int id); +int arch_acquire_resource (struct domain *d,unsigned int type,unsigned int id,unsigned long frame,unsigned int nr_frames,xen_pfn_t mfn_list[]);xen/include/public/memory.h +0
-4@@ -639,10 +639,18 @@ struct xen_mem_acquire_resource {#define XENMEM_resource_grant_table_id_status 1/* -* IN/OUT - As an IN parameter number of frames of the resource -* to be mapped. However, if the specified value is 0 and -* frame_list is NULL then this field will be set to the -* maximum value supported by the implementation on return.
xen/include/public/memory.h struct xen_mem_acquire_resource { · line 651
Quoted 10 lines
xen/include/public/memory.h +10
-0+* IN/OUT +* +* As an IN parameter number of frames of the resource to be mapped. +* +* When frame_list is NULL and nr_frames is 0, this is interpreted as a +* request for the size of the resource, which shall be returned in the +* nr_frames field. +* +* The size of a resource will never be zero, but a nonzero result doesn't +* guarentee that a subsequent mapping request will be successful. There
s/guarantee/guarantee
Paul
Quoted 51 lines
xen/include/public/memory.h +2
-0+* are further type/id specific constraints which may change between the +* two calls. */ uint32_t nr_frames;uint32_t pad;xen/include/xen/grant_table.h +8
-0@@ -57,6 +57,8 @@ int mem_sharing_gref_to_gfn(struct grant_table *gt, grant_ref_t ref,int gnttab_map_frame (struct domain *d,unsigned long idx,gfn_t gfn,mfn_t *mfn); +unsigned int gnttab_resource_max_frames (struct domain *d,unsigned int id); +int gnttab_acquire_resource (struct domain *d,unsigned int id,unsigned long frame,unsigned int nr_frames,xen_pfn_t mfn_list[]);@@ -93,6 +95,12 @@ static inline int gnttab_map_frame(struct domain *d, unsigned long idx,return -EINVAL ; } +static inline unsigned int gnttab_resource_max_frames ( +struct domain *d,unsigned int id) +{ +return 0 ; +} +static inline int gnttab_acquire_resource (struct domain *d,unsigned int id,unsigned long frame,unsigned int nr_frames,xen_pfn_t mfn_list[])xen/include/xen/mm.h +6
-1@@ -686,6 +686,12 @@ static inline void put_page_alloc_ref(struct page_info *page) }#ifndef CONFIG_ARCH_ACQUIRE_RESOURCE +static inline unsigned int arch_resource_max_frames ( +struct domain *d,unsigned int type,unsigned int id) +{ +return 0 ; +} +static inline int arch_acquire_resource (struct domain *d,unsigned int type,unsigned int id,unsigned long frame,unsigned int nr_frames,xen_pfn_t mfn_list[]) --2.11.0
RE: [PATCH 5/5] tools/foreignmem: Support querying the size of a resource
In reply to Andrew CooperQuoted 15 lines
-----Original Message-----
From: Andrew Cooper <[email protected]>
Sent: 28 July 2020 12:37
To: Xen-devel <[email protected]>
Cc: Andrew Cooper <[email protected]>; Ian Jackson <[email protected]>; Wei Liu <[email protected]>; Paul Durrant <[email protected]>; Michał Leszczyński <[email protected]>; Hubert Jasudowicz <[email protected]>
Subject: [PATCH 5/5]tools/foreignmem: Support querying the size of a resourceWith the Xen side of this interface fixed to return real sizes, userspace needs to be able to make the query.
Introduce
xenforeignmemory_resource_size()for the purpose, bumping the library minor version and providing compatiblity for the non-Linux builds.
Quoted 6 lines
Its not possible to reuse the
IOCTL_PRIVCMD_MMAP_RESOURCEinfrastructure, because it depends on having already mmap()'d a suitably sized region before it will make anXENMEM_acquire_resourcehypercall to Xen.Instead, open a xencall handle and make an
XENMEM_acquire_resourcehypercall directly.
Shame we have to do that but, as you say, it's the only option.
Quoted 2 lines
Signed-off-by Andrew Cooper <[email protected]>
Quoted 215 lines
---
CC: Ian Jackson <[email protected]>
CC: Wei Liu <[email protected]>
CC: Paul Durrant <[email protected]>
CC: Michał Leszczyński <[email protected]>
CC: Hubert Jasudowicz <[email protected]>--- tools/libs/foreignmemory/Makefile | 2 +- tools/libs/foreignmemory/core.c | 14 +++++++++ .../libs/foreignmemory/include/xenforeignmemory.h | 15 ++++++++++ tools/libs/foreignmemory/libxenforeignmemory.map | 4 +++ tools/libs/foreignmemory/linux.c | 35 ++++++++++++++++++++++ tools/libs/foreignmemory/private.h | 14 +++++++++ 6 files changed, 83 insertions(+), 1 deletion(-)tools/libs/foreignmemory/Makefile +1
-1@@ -2,7 +2,7 @@ XEN_ROOT = $(CURDIR)/../../.. include $(XEN_ROOT)/tools/Rules.mk MAJOR = 1 -MINOR = 3 +MINOR = 4 LIBNAME := foreignmemory USELIBS := toollog toolcoretools/libs/foreignmemory/core.c +14
-0@@ -53,6 +53,10 @@ xenforeignmemory_handle *xenforeignmemory_open(xentoollog_logger *logger,if (!fmem->logger) goto err; } + fmem->xcall =xencall_open (fmem->logger,0 ); +if ( !fmem->xcall ) + goto err; + rc =osdep_xenforeignmemory_open (fmem);if ( rc <0 ) goto err;@@ -61,6 +65,7 @@ xenforeignmemory_handle *xenforeignmemory_open(xentoollog_logger *logger, err:xentoolcore__deregister_active_handle (&fmem->tc_ah);osdep_xenforeignmemory_close (fmem); +xencall_close (fmem->xcall);xtl_logger_destroy (fmem->logger_tofree);free (fmem);return NULL ;@@ -75,6 +80,7 @@ int xenforeignmemory_close(xenforeignmemory_handle *fmem)xentoolcore__deregister_active_handle (&fmem->tc_ah); rc =osdep_xenforeignmemory_close (fmem); +xencall_close (fmem->xcall);xtl_logger_destroy (fmem->logger_tofree);free (fmem);return rc;@@ -188,6 +194,14 @@ int xenforeignmemory_unmap_resource(return rc; } +int xenforeignmemory_resource_size ( +xenforeignmemory_handle *fmem,domid_t domid,unsigned int type, +unsigned int id,unsigned long *nr_frames) +{ +return osdep_xenforeignmemory_resource_size (fmem, domid, type, + id, nr_frames); +} +/* * Local variables: * mode: C tools/libs/foreignmemory/include/xenforeignmemory.h +15
-0@@ -179,6 +179,21 @@ xenforeignmemory_resource_handle *xenforeignmemory_map_resource(int xenforeignmemory_unmap_resource (xenforeignmemory_handle *fmem,xenforeignmemory_resource_handle *fres); +/** +* Determine the maximum size of a specific resource. +* +* @parm fmem handle to the open foreignmemory interface +* @parm domid the domain id +* @parm type the resource type +* @parm id the type-specific resource identifier +* +* Return 0 on success and fills in *nr_frames. Sets errno and return -1 on +* error. +*/ +int xenforeignmemory_resource_size ( +xenforeignmemory_handle *fmem,domid_t domid,unsigned int type, +unsigned int id,unsigned long *nr_frames); +#endif /* tools/libs/foreignmemory/libxenforeignmemory.map +4
-0@@ -19,3 +19,7 @@ VERS_1.3 { xenforeignmemory_map_resource; xenforeignmemory_unmap_resource; } VERS_1.2; +VERS_1.4 { + global: + xenforeignmemory_resource_size; +} VERS_1.3;tools/libs/foreignmemory/linux.c +35
-0@@ -28,6 +28,8 @@#include "private.h" +#include <xen/memory.h> +#define ROUNDUP (_x,_w) (((unsigned long)(_x)+(1UL<<(_w))-1) & ~((1UL<<(_w))-1))#ifndef O_CLOEXEC @@ -340,6 +342,39 @@ int osdep_xenforeignmemory_map_resource(return 0 ; } +int osdep_xenforeignmemory_resource_size ( +xenforeignmemory_handle *fmem,domid_t domid,unsigned int type, +unsigned int id,unsigned long *nr_frames) +{ +int rc; +struct xen_mem_acquire_resource *xmar = +xencall_alloc_buffer (fmem->xcall,sizeof (*xmar)); + +if ( !xmar ) + { +PERROR ("Could not bounce memory for acquire_resource hypercall" ); +return -1 ; + } + + *xmar = (struct xen_mem_acquire_resource ){ + .domid = domid, + .type = type, + .id = id, + }; + + rc =xencall2 (fmem->xcall, __HYPERVISOR_memory_op, + XENMEM_acquire_resource, (uintptr_t )xmar); +if ( rc ) + goto out; + + *nr_frames = xmar->nr_frames; + + out: +xencall_free_buffer (fmem->xcall, xmar); + +return rc; +} +/* * Local variables: * mode: C tools/libs/foreignmemory/private.h +14
-1@@ -4,6 +4,7 @@#include <xentoollog.h> #include <xenforeignmemory.h> +#include <xencall.h> #include <xentoolcore_internal.h> @@ -20,6 +21,7 @@struct xenforeignmemory_handle {xentoollog_logger *logger, *logger_tofree; +xencall_handle *xcall;unsigned flags;int fd;Xentoolcore__Active_Handle tc_ah;@@ -74,6 +76,15 @@ static inline int osdep_xenforeignmemory_unmap_resource( {return 0 ; } + +static inline int osdep_xenforeignmemory_resource_size ( +xenforeignmemory_handle *fmem,domid_t domid,unsigned int type, +unsigned int id,unsigned long *nr_frames) +{ + errno =EOPNOTSUPP ; +return -1 ; +} +#else int osdep_xenforeignmemory_restrict (xenforeignmemory_handle *fmem,domid_t domid);@@ -81,6 +92,9 @@ int osdep_xenforeignmemory_map_resource( xenforeignmemory_handle *fmem, xenforeignmemory_resource_handle *fres);int osdep_xenforeignmemory_unmap_resource (xenforeignmemory_handle *fmem,xenforeignmemory_resource_handle *fres); +int osdep_xenforeignmemory_resource_size ( +xenforeignmemory_handle *fmem,domid_t domid,unsigned int type, +unsigned int id,unsigned long *nr_frames);#endif #define PERROR(_f...) \ --2.11.0
Re: [PATCH 1/5] xen/memory: Introduce CONFIG_ARCH_ACQUIRE_RESOURCE
In reply to Andrew CooperHi Andrew,
Quoted 9 lines · Andrew Cooper
New architectures shouldn't be forced to implement no-op stubs for unused functionality.
Introduce
CONFIG_ARCH_ACQUIRE_RESOURCEwhich can be opted in to, and provide compatibility logic inxen/mm.hNo functional change.
Signed-off-by Andrew Cooper <[email protected]>
With one question below:
xen/include/xen/mm.h static inline void put_page_alloc_ref(struct page_info *page) · line 691
Quoted 12 lines
xen/include/xen/mm.h +4
-0@@ -685,4 +685,13 @@ static inline void put_page_alloc_ref(struct page_info *page) } } +#ifndef CONFIG_ARCH_ACQUIRE_RESOURCE +static inline int arch_acquire_resource ( +struct domain *d,unsigned int type,unsigned int id,unsigned long frame, +unsigned int nr_frames,xen_pfn_t mfn_list[])
Any reason to change the way we indent the arguments?
xen/include/xen/mm.h static inline void put_page_alloc_ref(struct page_info *page) · line 697
Quoted 7 lines
xen/include/xen/mm.h +5
-0+{ +return -EOPNOTSUPP ; +} +#endif /* !CONFIG_ARCH_ACQUIRE_RESOURCE */ +#endif /* __XEN_MM_H__ */
Cheers,
Signature · 1 line
Julien Grall
Re: [PATCH 2/5] xen/gnttab: Rework resource acquisition
In reply to Andrew CooperHi Andrew,
Quoted 38 lines · Andrew Cooper
The existing logic doesn't function in the general case for mapping a guests grant table, due to arbitrary 32 frame limit, and the default grant table limit being 64.
In order to start addressing this, rework the existing grant table logic by implementing a single
gnttab_acquire_resource(). This is far more efficient than the previousacquire_grant_table()inmemory.cbecause it doesn't take the grant table write lock, and attempt to grow the table, for every single frame.The new
gnttab_acquire_resource()function subsumes the previous two gnttab_get_{shared,status}_frame() helpers.No functional change.
Signed-off-by Andrew Cooper <[email protected]>---
CC: George Dunlap <[email protected]>
CC: Ian Jackson <[email protected]>
CC: Jan Beulich <[email protected]>
CC: Konrad Rzeszutek Wilk <[email protected]>
CC: Stefano Stabellini <[email protected]>
CC: Wei Liu <[email protected]>
CC: Julien Grall <[email protected]>
CC: Paul Durrant <[email protected]>
CC: Michał Leszczyński <[email protected]>
CC: Hubert Jasudowicz <[email protected]>--- xen/common/grant_table.c | 93 ++++++++++++++++++++++++++++++------------- xen/common/memory.c | 42 +------------------ xen/include/xen/grant_table.h | 19 ++++----- 3 files changed, 75 insertions(+), 79 deletions(-)xen/common/grant_table.c +0
-0
xen/common/grant_table.c static int gnttab_get_shared_frame_mfn(struct domain *d, · line 4018
Quoted 6 lines
xen/common/grant_table.c +3
-0@@ -4013,6 +4013,72 @@ static int gnttab_get_shared_frame_mfn(struct domain *d,return 0 ; } +int gnttab_acquire_resource ( +struct domain *d,unsigned int id,unsigned long frame, +unsigned int nr_frames,xen_pfn_t mfn_list[])
Any reasons for changing the way we usually indent parameters?
xen/common/grant_table.c static int gnttab_get_shared_frame_mfn(struct domain *d, · line 4023
Quoted 5 lines
xen/common/grant_table.c +5
-0+{ +struct grant_table *gt = d->grant_table; +unsigned int i = nr_frames, tot_frames; +void **vaddrs; +int rc =0 ;
AFAICT rc will always be initialized when used. So is it necessary?
Quoted 19 lines
xen/common/grant_table.c +19
-0+ +/* Input sanity. */ +if ( !nr_frames ) +return -EINVAL ; + +/* Overflow checks */ +if ( frame + nr_frames < frame ) +return -EINVAL ; + + tot_frames = frame + nr_frames; +if ( tot_frames != frame + nr_frames ) +return -EINVAL ; + +/* Grow table if necessary. */ +grant_write_lock (gt); +switch ( id ) + { +mfn_t tmp; +
xen/common/grant_table.c static int gnttab_get_shared_frame_mfn(struct domain *d, · line 4052
Quoted 10 lines
xen/common/grant_table.c +10
-0+case XENMEM_resource_grant_table_id_shared: + rc =gnttab_get_shared_frame_mfn (d, tot_frames -1 , &tmp); +break ; + +case XENMEM_resource_grant_table_id_status: +if ( gt->gt_version !=2 ) + { +default : + rc = -EINVAL ; +break ;
I am aware this is valid C code, but I find this construct confusing. Could you just duplicate the 2 lines and have a separate default case?
Quoted 10 lines
xen/common/grant_table.c +10
-0+ } + rc =gnttab_get_status_frame_mfn (d, tot_frames -1 , &tmp); +break ; + } + +/* Any errors from growing the table? */ +if ( rc ) + goto out; + + switch ( id )
xen/common/grant_table.c static int gnttab_get_shared_frame_mfn(struct domain *d, · line 4072
Quoted 10 lines
xen/common/grant_table.c +10
-0+ { +case XENMEM_resource_grant_table_id_shared: + vaddrs = gt->shared_raw; +break ; + +case XENMEM_resource_grant_table_id_status: + vaddrs = (void **)gt->status; +break ; + } +
NIT: Would it make sense to check vaddrs? This would avoid NULL dereference pointers if something went wrong.
xen/common/grant_table.c static int gnttab_get_shared_frame_mfn(struct domain *d, · line 4081
Quoted 9 lines
xen/common/grant_table.c +9
-0+for ( i =0 ; i < nr_frames; ++i ) + mfn_list[i] =virt_to_mfn (vaddrs[frame + i]); + + out: +grant_write_unlock (gt); + +return rc; +} +
Cheers,
Signature · 1 line
Julien Grall
Re: [PATCH 4/5] xen/memory: Fix acquire_resource size semantics
In reply to Paul DurrantHi Paul,
Quoted 16 lines · Paul Durrant
xen/common/memory.c static long xatp_permission_check(struct domain *d, unsigned int space) · line 1014
Quoted 13 lines
xen/common/memory.c +5
-0@@ -1007,6 +1007,26 @@ static long xatp_permission_check(struct domain *d, unsigned int space)return xsm_add_to_physmap (XSM_TARGET , current->domain, d); } +/* +* Return 0 on any kind of error. Caller converts to -EINVAL. +* +* All nonzero values should be repeatable (i.e. derived from some fixed +* proerty of the domain), and describe the full resource (i.e. mapping the s/property/property
Quoted 3 lines
xen/common/memory.c static long xatp_permission_check(struct domain *d, unsigned int space) · line 1015
Quoted 1 line
xen/common/memory.c +1
-0+* result of this call will be the entire resource). This precludes dynamically adding a resource to a running domain. Do we really want to bake in that restriction?
AFAICT, this restriction is not documented in the ABI. In particular, it is written:
"
The size of a resource will never be zero, but a nonzero result doesn't guarentee that a subsequent mapping request will be successful. There are further type/id specific constraints which may change between the two calls.
"
So I think a domain couldn't rely on this behavior. Although, it might be good to clarify in the comment on top of resource_max_frames that this an implementation decision and not part of the ABI.
Cheers,
Signature · 1 line
Julien Grall
Re: [PATCH 1/5] xen/memory: Introduce CONFIG_ARCH_ACQUIRE_RESOURCE
In reply to Julien GrallQuoted 11 lines · Julien Grall
Hi Andrew,
Quoted 8 lines · Andrew Cooper
New architectures shouldn't be forced to implement no-op stubs for unused
functionality.Introduce
CONFIG_ARCH_ACQUIRE_RESOURCEwhich can be opted in to, and provide
compatibility logic inxen/mm.h
Quoted 7 lines
Quoted 3 lines
No functional change.
Signed-off-by Andrew Cooper <[email protected]>With one question below:
Acked-by Julien Grall <[email protected]>
Thanks,
Quoted 17 lines
Quoted 10 lines
xen/include/xen/mm.h +0
-0@@ -685,4 +685,13 @@ static inline void put_page_alloc_ref(struct page_info *page) } }Quoted 5 lines
+#ifndef
CONFIG_ARCH_ACQUIRE_RESOURCE
+static inline int arch_acquire_resource(
+ struct domain *d, unsigned int type, unsigned int id, unsigned long frame,
+ unsigned intnr_frames,xen_pfn_tmfn_list[])Any reason to change the way we indent the arguments?
So its not all squashed on the right hand side.
~Andrew
Re: [PATCH 1/5] xen/memory: Introduce CONFIG_ARCH_ACQUIRE_RESOURCE
In reply to Paul DurrantQuoted 13 lines · Paul Durrant
Quoted 13 lines
-----Original Message-----
From: Andrew Cooper <[email protected]>
Sent: 28 July 2020 12:37
To: Xen-devel <[email protected]>
Cc: Andrew Cooper <[email protected]>; Jan Beulich <[email protected]>; Wei Liu <[email protected]>; Roger Pau Monné <[email protected]>; Stefano Stabellini <[email protected]>; Julien Grall <[email protected]>; Volodymyr Babchuk <[email protected]>; Paul Durrant <[email protected]>; Michał Leszczyński <[email protected]>; Hubert Jasudowicz <[email protected]>
Subject: [PATCH 1/5] xen/memory: IntroduceCONFIG_ARCH_ACQUIRE_RESOURCENew architectures shouldn't be forced to implement no-op stubs for unused functionality.
Quoted 7 lines
Quoted 4 lines
Introduce
CONFIG_ARCH_ACQUIRE_RESOURCEwhich can be opted in to, and provide compatibility logic inxen/mm.hNo functional change.
Code-wise, it looks fine, so...
Reviewed-by Paul Durrant <[email protected]>
Thanks,
Quoted 21 lines
...but ...
Quoted 18 lines
Signed-off-by Andrew Cooper <[email protected]>---
CC: Jan Beulich <[email protected]>
CC: Wei Liu <[email protected]>
CC: Roger Pau Monné <[email protected]>
CC: Stefano Stabellini <[email protected]>
CC: Julien Grall <[email protected]>
CC: Volodymyr Babchuk <[email protected]>
CC: Paul Durrant <[email protected]>
CC: Michał Leszczyński <[email protected]>
CC: Hubert Jasudowicz <[email protected]>--- xen/arch/x86/Kconfig | 1 + xen/common/Kconfig | 3 +++ xen/include/asm-arm/mm.h | 8 -------- xen/include/xen/mm.h | 9 +++++++++ 4 files changed, 13 insertions(+), 8 deletions(-)
Quoted 10 lines
xen/arch/x86/Kconfig config X86 · line 9
Quoted 9 lines
xen/arch/x86/Kconfig +1
-0@@ -6,6 +6,7 @@ config X86 select ACPI select ACPI_LEGACY_TABLES_LOOKUP select ARCH_SUPPORTS_INT128 + select ARCH_ACQUIRE_RESOURCE... I do wonder whether '
HAS_ACQUIRE_RESOURCE' is a better and more descriptive name.
We don't have a coherent policy for how to categorise these things. I can change the name if you insist, but I'm not sure it makes a useful difference.
~Andrew
RE: [PATCH 1/5] xen/memory: Introduce CONFIG_ARCH_ACQUIRE_RESOURCE
Quoted 68 lines
-----Original Message-----
From: Andrew Cooper <[email protected]>
Sent: 30 July 2020 18:34
To: [email protected]; 'Xen-devel' <[email protected]>
Cc: 'Jan Beulich' <[email protected]>; 'Wei Liu' <[email protected]>; 'Roger Pau Monné' <[email protected]>; 'Stefano Stabellini' <[email protected]>; 'Julien Grall' <[email protected]>; 'Volodymyr Babchuk' <[email protected]>; 'Michał Leszczyński' <[email protected]>; 'Hubert Jasudowicz' <[email protected]>
Subject: Re: [PATCH 1/5] xen/memory: IntroduceCONFIG_ARCH_ACQUIRE_RESOURCEQuoted 5 lines · Paul Durrant
Quoted 5 lines
-----Original Message-----
From: Andrew Cooper <[email protected]>
Sent: 28 July 2020 12:37
To: Xen-devel <[email protected]>
Cc: Andrew Cooper <[email protected]>; Jan Beulich <[email protected]>; Wei LiuQuoted 2 lines
Quoted 2 lines
Roger Pau Monné <[email protected]>; Stefano Stabellini <[email protected]>; Julien Grall <[email protected]>; Volodymyr Babchuk <[email protected]>; Paul Durrant <[email protected]>;
Michał
Quoted 6 lines
Quoted 6 lines
Leszczyński <[email protected]>; Hubert Jasudowicz <[email protected]>
Subject: [PATCH 1/5] xen/memory: IntroduceCONFIG_ARCH_ACQUIRE_RESOURCENew architectures shouldn't be forced to implement no-op stubs for unused functionality.
Quoted 7 lines
Quoted 4 lines
Introduce
CONFIG_ARCH_ACQUIRE_RESOURCEwhich can be opted in to, and provide compatibility logic inxen/mm.hNo functional change.
Code-wise, it looks fine, so...
Reviewed-by Paul Durrant <[email protected]>Thanks,
Quoted 21 lines
...but ...
Quoted 18 lines
Signed-off-by Andrew Cooper <[email protected]>---
CC: Jan Beulich <[email protected]>
CC: Wei Liu <[email protected]>
CC: Roger Pau Monné <[email protected]>
CC: Stefano Stabellini <[email protected]>
CC: Julien Grall <[email protected]>
CC: Volodymyr Babchuk <[email protected]>
CC: Paul Durrant <[email protected]>
CC: Michał Leszczyński <[email protected]>
CC: Hubert Jasudowicz <[email protected]>--- xen/arch/x86/Kconfig | 1 + xen/common/Kconfig | 3 +++ xen/include/asm-arm/mm.h | 8 -------- xen/include/xen/mm.h | 9 +++++++++ 4 files changed, 13 insertions(+), 8 deletions(-)Quoted 10 lines
xen/arch/x86/Kconfig config X86 · line 9
Quoted 9 lines
xen/arch/x86/Kconfig +1
-0@@ -6,6 +6,7 @@ config X86 select ACPI select ACPI_LEGACY_TABLES_LOOKUP select ARCH_SUPPORTS_INT128 + select ARCH_ACQUIRE_RESOURCE... I do wonder whether '
HAS_ACQUIRE_RESOURCE' is a better and more descriptive name.
Quoted 4 lines
We don't have a coherent policy for how to categorise these things. I can change the name if you insist, but I'm not sure it makes a useful difference.
Ok, it's fine. My R-b stands.
Paul
Quoted 1 line
~Andrew
Re: [PATCH 1/5] xen/memory: Introduce CONFIG_ARCH_ACQUIRE_RESOURCE
In reply to Andrew CooperQuoted 40 lines · Andrew Cooper
Quoted 11 lines · Julien Grall
Hi Andrew,
Quoted 8 lines · Andrew Cooper
New architectures shouldn't be forced to implement no-op stubs for unused
functionality.Introduce
CONFIG_ARCH_ACQUIRE_RESOURCEwhich can be opted in to, and provide
compatibility logic inxen/mm.hQuoted 7 lines
Quoted 3 lines
No functional change.
Signed-off-by Andrew Cooper <[email protected]>With one question below:
Acked-by Julien Grall <[email protected]>Thanks,
Quoted 17 lines
Quoted 13 lines
xen/include/xen/mm.h +2
-0@@ -685,4 +685,13 @@ static inline void put_page_alloc_ref(struct page_info *page) } } +#ifndef CONFIG_ARCH_ACQUIRE_RESOURCE +static inline int arch_acquire_resource ( + struct domain *d, unsigned int type, unsigned int id, unsignedQuoted 2 lines
long frame,
+ unsigned intnr_frames,xen_pfn_tmfn_list[])Any reason to change the way we indent the arguments?
Quoted 1 line
So its not all squashed on the right hand side.
Fair enough. I have asked the same question on a follow-up patch. Feel free to ignore it :).
Cheers,
Signature · 1 line
Julien Grall
Re: [PATCH 3/5] xen/memory: Fix compat XENMEM_acquire_resource for size requests
In reply to Jan BeulichQuoted 6 lines · Jan Beulich
Quoted 2 lines · Andrew Cooper
Copy the
nr_framesfrom the correct structure, so the caller doesn't unconditionally receive 0.Well, no - it does get copied from the correct structure. It's just that the field doesn't get set properly up front.
You appear to be objecting to my use of the term "correct".
There are two structures. One contains the correct value, and one contains the wrong value, which happens to always be 0.
I stand by sentence as currently written.
Quoted 3 lines
Otherwise you'll
(a) build in an unchecked assumption that the native and compat fields match in type
Did you actually check? Because I did before embarking on this course of action.
In file included from /local/xen.git/xen/include/xen/guest_access.h:10:0, from compat/memory.c:5:
/local/xen.git/xen/include/asm/guest_access.h:152:28: error: comparison of distinct pointer types lacks a cast [-Werror] (void)(&(hnd).p->field == _s); \
^compat/memory.c:628:22: note: in expansion of macro ‘__copy_field_to_guest’ if ( __copy_field_to_guest( ^~~~~~~~~~~~~~~~~~~~~
This is what the compiler thinks of the code, when nr_frames is changed from uint32_t to unsigned long.
Quoted 2 lines
and (b) set a bad example for people looking
here
This entire function is a massive set of bad examples; the worst IMO being the fact that there isn't a single useful comment anywhere in it concerning how the higher level loop structure works.
I'm constantly annoyed that I need to reverse engineer it from scratch every time I look at it, despite having a better-than-most understanding of what it is trying to achieve, and how it is supposed to work.
I realise this is noones fault in particular, but it is not fair/reasonable to claim that this change is the thing setting a bad example in this file.
Quoted 3 lines
and then cloning this code in perhaps a case where (a) is not even true. If you agree, the alternative change of setting cmp.mar.nr_frames from
nat.mar->nr_framesbefore the call is
Is there more to this sentence?
While this example could be implemented (at even higher overhead) by copying nat back to cmp before passing it back to the guest, the same is not true for the changes required to fix batching (which is another series the same size as this).
~Andrew
Re: [PATCH 4/5] xen/memory: Fix acquire_resource size semantics
In reply to Paul DurrantQuoted 94 lines · Paul Durrant
Quoted 18 lines
-----Original Message-----
From: Xen-devel <[email protected]> On Behalf Of Andrew Cooper
Sent: 28 July 2020 12:37
To: Xen-devel <[email protected]>
Cc: Hubert Jasudowicz <[email protected]>; Stefano Stabellini <[email protected]>; Julien Grall <[email protected]>; Wei Liu <[email protected]>; Konrad Rzeszutek Wilk <[email protected]>; George Dunlap <[email protected]>; Andrew Cooper <[email protected]>; Paul Durrant <[email protected]>; Jan Beulich <[email protected]>; Michał Leszczyński <[email protected]>; Ian Jackson <[email protected]>
Subject: [PATCH 4/5] xen/memory: Fixacquire_resourcesize semanticsCalling
XENMEM_acquire_resourcewith a NULLframe_listis a request for the size of the resource, but the returned 32 is bogus.If someone tries to follow it for
XENMEM_resource_ioreq_server, the acquire call will fail as IOREQ servers currently top out at 2 frames, and it is only half the size of the default grant table limit for guests.Quoted 7 lines
Also, no users actually request a resource size, because it was never wired up in the sole implemenation of resource aquisition in Linux.
Introduce a new
resource_max_frames()to calculate the size of a resource, and implement it the IOREQ and grant subsystems.It is impossible to guarentee that a mapping call following a successful size
s/guarantee/guarantee
Quoted 1 line
call will succedd (e.g. The target IOREQ server gets destroyed, or the domain
s/succedd/succeed
Quoted 64 lines
switches from grant v2 to v1). Document the restriction, and use the flexibility to simplify the paths to be lockless.
Signed-off-by Andrew Cooper <[email protected]>---
CC: George Dunlap <[email protected]>
CC: Ian Jackson <[email protected]>
CC: Jan Beulich <[email protected]>
CC: Konrad Rzeszutek Wilk <[email protected]>
CC: Stefano Stabellini <[email protected]>
CC: Wei Liu <[email protected]>
CC: Julien Grall <[email protected]>
CC: Paul Durrant <[email protected]>
CC: Michał Leszczyński <[email protected]>
CC: Hubert Jasudowicz <[email protected]>--- xen/arch/x86/mm.c | 20 ++++++++++++++++ xen/common/grant_table.c | 19 +++++++++++++++ xen/common/memory.c | 55 +++++++++++++++++++++++++++++++++---------- xen/include/asm-x86/mm.h | 3 +++ xen/include/public/memory.h | 16 +++++++++---- xen/include/xen/grant_table.h | 8 +++++++ xen/include/xen/mm.h | 6 +++++ 7 files changed, 110 insertions(+), 17 deletions(-)xen/arch/x86/mm.c +20
-0@@ -4600,6 +4600,26 @@ int xenmem_add_to_physmap_one(return rc; } +unsigned int arch_resource_max_frames ( +struct domain *d,unsigned int type,unsigned int id) +{ +unsigned int nr =0 ; + +switch ( type ) + { +#ifdef CONFIG_HVM +case XENMEM_resource_ioreq_server: +if ( !is_hvm_domain (d) ) +break ; +/* One frame for the buf-ioreq ring, and one frame per 128 vcpus. */ + nr =1 +DIV_ROUND_UP (d->max_vcpus *sizeof (struct ioreq ),PAGE_SIZE ); +break ; +#endif + } + +return nr; +} +int arch_acquire_resource (struct domain *d,unsigned int type,unsigned int id,unsigned long frame,unsigned int nr_frames,xen_pfn_t mfn_list[])xen/common/grant_table.c +0
-0@@ -4013,6 +4013,25 @@ static int gnttab_get_shared_frame_mfn(struct domain *d,return 0 ; }
Quoted 14 lines
xen/common/grant_table.c static int gnttab_get_shared_frame_mfn(struct domain *d, · line 4028
Quoted 13 lines
xen/common/grant_table.c +13
-0+unsigned int gnttab_resource_max_frames (struct domain *d,unsigned int id) +{ +unsigned int nr =0 ; + +/* Don't need the grant lock. This limit is fixed at domain create time. */ +switch ( id ) + { +case XENMEM_resource_grant_table_id_shared: + nr = d->grant_table->max_grant_frames; +break ; + +case XENMEM_resource_grant_table_id_status: + nr =grant_to_status_frames (d->grant_table->max_grant_frames);Two uses of
d->grant_table, so perhaps define a stack variable for it?
Can do.
Quoted 1 line
Also, should you not make sure 0 is returned in the case of a v1 table?
This was the case specifically discussed in the commit message, but perhaps it needs expanding.
Doing so would be buggy.
Some utility is going to query the resource size, and then try to map it (if it doesn't blindly know the size and/or subset it cares about already).
In between these two hypercalls from the utility, the guest can do a v1=>v2 or v2=>v1 switch and make the resource spontaneously appear or disappear.
The only case where we can know for certain whether the resource is available is when we're in the map hypercall. Therefore, userspace has to be able to get to the map call if there is potentially a resource available.
The semantics of the size call are really "this resource might exist, and if it does, this is how large it is".
As for the grant status frames specifically, I think making them a mappable resource might have been a poor choice in hind sight.
Only the guest can switch between grant versions. GNTTABOP_set_version strictly operates on current, unlike most of the other grant hypercalls which take a domid and let dom0 specify something other than DOMID_SELF.
There is GNTTABOP_get_version, but it is racy to use in the same way as described above, and if some utility does successfully map the status frames, what will happen in practice is that a guest attempting to switch from v2 back to v1 will have the set_version hypercall fail due to outstanding refs on the frames.
~Andrew
Re: [PATCH 4/5] xen/memory: Fix acquire_resource size semantics
In reply to Julien GrallQuoted 37 lines · Julien Grall
Hi Paul,
Quoted 19 lines · Paul Durrant
Quoted 13 lines
xen/common/memory.c +4
-0@@ -1007,6 +1007,26 @@ static long xatp_permission_check(struct domain *d, unsigned int space) return xsm_add_to_physmap(XSM_TARGET, current->domain, d); } +/* +* Return 0 on any kind of error. Caller converts to -EINVAL. +* +* All nonzero values should be repeatable (i.e. derived from some Quoted 3 lines
fixed
+ * proerty of the domain), and describe the full resource (i.e. mapping thes/property/property
Quoted 4 lines
Quoted 1 line
+ * result of this call will be the entire resource).
This precludes dynamically adding a resource to a running domain. Do we really want to bake in that restriction?
AFAICT, this restriction is not documented in the ABI. In particular, it is written:
"
The size of a resource will never be zero, but a nonzero result doesn't guarentee that a subsequent mapping request will be successful. There are further type/id specific constraints which may change between the two calls.
"
Quoted 3 lines
So I think a domain couldn't rely on this behavior. Although, it might be good to clarify in the comment on top of
resource_max_framesthat this an implementation decision and not part of the ABI.
There are two aspects here.
First, yes - I deliberately didn't state it in the ABI, just in case we might want to use it in the future. I could theoretically foresee using -EBUSY for the purpose.
That said however, we are currently deliberately taking dynamic resources out of Xen, because they've proved to be unnecessary in practice and a fertile source of complexity and security bugs.
I don't foresee accepting new dynamic resources, but that's not to say that someone can't theoretically come up with a sufficiently compelling counterexample.
~Andrew
Re: [PATCH 4/5] xen/memory: Fix acquire_resource size semantics
In reply to Andrew CooperHi Andrew,
Quoted 4 lines · Andrew Cooper
On 30/07/2020 09:31, Paul Durrant wrote:
In between these two hypercalls from the utility, the guest can do a v1=>v2 or v2=>v1 switch and make the resource spontaneously appear or disappear.
This can only happen on platform where grant-table v2 is enabled. Where this is not enabled (e.g Arm), then I think we want to return 0 as there is nothing to map.
Cheers,
Signature · 1 line
Julien Grall
Re: [PATCH 3/5] xen/memory: Fix compat XENMEM_acquire_resource for size requests
In reply to Andrew CooperQuoted 8 lines · Andrew Cooper
Quoted 6 lines · Jan Beulich
Quoted 2 lines · Andrew Cooper
Copy the
nr_framesfrom the correct structure, so the caller doesn't unconditionally receive 0.Well, no - it does get copied from the correct structure. It's just that the field doesn't get set properly up front.
Quoted 6 lines
You appear to be objecting to my use of the term "correct".
There are two structures. One contains the correct value, and one contains the wrong value, which happens to always be 0.
I stand by sentence as currently written.
At the risk of splitting hair, what you copy from is a field holding the correct value, but not the correct field. This only works correctly because of the way __copy_field_{from,to}_guest() happen to be implemented; there are possible alternative implementations where this would break, despite ...
Quoted 17 lines
Quoted 3 lines
Otherwise you'll
(a) build in an unchecked assumption that the native and compat fields match in typeDid you actually check? Because I did before embarking on this course of action.
In file included from /local/xen.git/xen/include/xen/guest_access.h:10:0, from
compat/memory.c:5:
/local/xen.git/xen/include/asm/guest_access.h:152:28: error: comparison of distinct pointer types lacks a cast [-Werror] (void)(&(hnd).p->field == _s); \
^compat/memory.c:628:22: note: in expansion of macro ‘__copy_field_to_guest’ if ( __copy_field_to_guest( ^~~~~~~~~~~~~~~~~~~~~
Quoted 2 lines
This is what the compiler thinks of the code, when
nr_framesis changed fromuint32_tto unsigned long.
... this type safety check (which, I admit, I didn't consider when writing my reply). I continue to think that handle and struct should match up not just for {,__}copy_{from,to}_guest() but also for {,__}copy_field_{from,to}_guest().
Quoted 7 lines
Quoted 2 lines
and (b) set a bad example for people looking
hereThis entire function is a massive set of bad examples; the worst IMO being the fact that there isn't a single useful comment anywhere in it concerning how the higher level loop structure works.
Quoted 7 lines
I'm constantly annoyed that I need to reverse engineer it from scratch every time I look at it, despite having a better-than-most understanding of what it is trying to achieve, and how it is supposed to work.
I realise this is noones fault in particular, but it is not fair/reasonable to claim that this change is the thing setting a bad example in this file.
I'd be happy to see "bad examples" be corrected. As stated at various occasions, at the time I first implemented the compat layer this seemed like the most reasonable approach to me. If you see room for improvement, then I'm all for it.
Quoted 5 lines
Quoted 3 lines
and then cloning this code in perhaps a case where (a) is not even true. If you agree, the alternative change of setting cmp.mar.nr_frames from
nat.mar->nr_framesbefore the call isIs there more to this sentence?
I guess I can't figure what you mean here.
Quoted 4 lines
While this example could be implemented (at even higher overhead) by copying nat back to cmp before passing it back to the guest, the same is not true for the changes required to fix batching (which is another series the same size as this).
I'll see when you post this, but I think we will want the principle outlined above to continue to hold.
Jan
Re: [PATCH 4/5] xen/memory: Fix acquire_resource size semantics
In reply to Andrew CooperQuoted 41 lines · Andrew Cooper
Quoted 9 lines · Paul Durrant
Quoted 9 lines
From: Xen-devel <[email protected]> On Behalf Of Andrew Cooper
Sent: 28 July 2020 12:37xen/common/grant_table.c +0
-0@@ -4013,6 +4013,25 @@ static int gnttab_get_shared_frame_mfn(struct domain *d,return 0 ; }Quoted 14 lines
xen/common/grant_table.c static int gnttab_get_shared_frame_mfn(struct domain *d, · line 4028
Quoted 13 lines
xen/common/grant_table.c +13
-0+unsigned int gnttab_resource_max_frames (struct domain *d,unsigned int id) +{ +unsigned int nr =0 ; + +/* Don't need the grant lock. This limit is fixed at domain create time. */ +switch ( id ) + { +case XENMEM_resource_grant_table_id_shared: + nr = d->grant_table->max_grant_frames; +break ; + +case XENMEM_resource_grant_table_id_status: + nr =grant_to_status_frames (d->grant_table->max_grant_frames);Two uses of
d->grant_table, so perhaps define a stack variable for it?Can do.
Quoted 1 line
Also, should you not make sure 0 is returned in the case of a v1 table?
This was the case specifically discussed in the commit message, but perhaps it needs expanding.
Doing so would be buggy.
Some utility is going to query the resource size, and then try to map it (if it doesn't blindly know the size and/or subset it cares about already).
In between these two hypercalls from the utility, the guest can do a v1=>v2 or v2=>v1 switch and make the resource spontaneously appear or disappear.
Quoted 7 lines
The only case where we can know for certain whether the resource is available is when we're in the map hypercall. Therefore, userspace has to be able to get to the map call if there is potentially a resource available.
The semantics of the size call are really "this resource might exist, and if it does, this is how large it is".
With you deriving from d->grant_table->max_grant_frames, this approach would imply that by obtaining a mapping the grant tables will get grown to their permitted maximum, no matter whether as much is actually needed by the guest. If this is indeed the intention, then we could as well set up maximum grant structures right at domain creation. Not something I would favor, but anyway...
Jan
Re: [PATCH 4/5] xen/memory: Fix acquire_resource size semantics
In reply to Andrew CooperQuoted 9 lines · Andrew Cooper
xen/common/memory.c +0
-5@@ -1026,19 +1047,6 @@ static int acquire_resource(if ( xmar.pad !=0 )return -EINVAL ; -if (guest_handle_is_null (xmar.frame_list) ) - { -if ( xmar.nr_frames ) -return -EINVAL ; -
xen/common/memory.c static int acquire_resource( · line 1051
Quoted 10 lines
xen/common/memory.c +0
-8- xmar.nr_frames =ARRAY_SIZE (mfn_list); - -if (__copy_field_to_guest (arg, &xmar, nr_frames) ) -return -EFAULT ; - -return 0 ; - } -if ( xmar.nr_frames >ARRAY_SIZE (mfn_list) )return -E2BIG ;
While arguably minor, the error code in the null-handle case would imo better be the same, no matter how big xmar.nr_frames is.
Jan
Re: [PATCH 4/5] xen/memory: Fix acquire_resource size semantics
Quoted 15 lines · Jan Beulich
Quoted 14 lines · Andrew Cooper
xen/common/memory.c +0
-10@@ -1026,19 +1047,6 @@ static int acquire_resource(if ( xmar.pad !=0 )return -EINVAL ; -if (guest_handle_is_null (xmar.frame_list) ) - { -if ( xmar.nr_frames ) -return -EINVAL ; - - xmar.nr_frames =ARRAY_SIZE (mfn_list); - -if (__copy_field_to_guest (arg, &xmar, nr_frames) ) -return -EFAULT ; -
Quoted 8 lines
xen/common/memory.c static int acquire_resource( · line 1051
Quoted 5 lines
xen/common/memory.c +0
-3-return 0 ; - } -if ( xmar.nr_frames >ARRAY_SIZE (mfn_list) )return -E2BIG ;While arguably minor, the error code in the null-handle case would imo better be the same, no matter how big xmar.nr_frames is.
This clause doesn't survive the fixes to batching.
Given how broken this infrastructure is, I'm not concerned with transient differences in error codes for users which will ultimately fail anyway.
~Andrew
Wei Liu
Acked-byRe: [PATCH 5/5] tools/foreignmem: Support querying the size of a resource
In reply to Andrew CooperQuoted 56 lines · Andrew Cooper
Quoted 13 lines · Andrew Cooper
With the Xen side of this interface fixed to return real sizes, userspace needs to be able to make the query.
Introduce
xenforeignmemory_resource_size()for the purpose, bumping the library minor version and providing compatiblity for the non-Linux builds.Its not possible to reuse the
IOCTL_PRIVCMD_MMAP_RESOURCEinfrastructure, because it depends on having already mmap()'d a suitably sized region before it will make anXENMEM_acquire_resourcehypercall to Xen.Instead, open a xencall handle and make an
XENMEM_acquire_resourcehypercall directly.Quoted 7 lines
Signed-off-by Andrew Cooper <[email protected]>---
CC: Ian Jackson <[email protected]>
CC: Wei Liu <[email protected]>
CC: Paul Durrant <[email protected]>
CC: Michał Leszczyński <[email protected]>
CC: Hubert Jasudowicz <[email protected]>I've folded:
tools/Rules.mk +0
-0@@ -123,7 +123,7 @@ LDLIBS_libxencall = $(SHDEPS_libxencall)$(XEN_LIBXENCALL)/libxencall$(libextens
SHLIB_libxencall = $(SHDEPS_libxencall) -Wl,-rpath-link=$(XEN_LIBXENCALL)
CFLAGS_libxenforeignmemory = -I$(XEN_LIBXENFOREIGNMEMORY)/include $(CFLAGS_xeninclude)tools/Rules.mk +1
-1-SHDEPS_libxenforeignmemory = $(SHLIB_libxentoolcore) +SHDEPS_libxenforeignmemory = $(SHLIB_libxentoolcore) $(SHDEPS_libxencall)LDLIBS_libxenforeignmemory = $(SHDEPS_libxenforeignmemory) $(XEN_LIBXENFOREIGNMEMORY)/libxenforeignmemory$(libextension) SHLIB_libxenforeignmemory = $(SHDEPS_libxenforeignmemory) -Wl,-rpath-link=$(XEN_LIBXENFOREIGNMEMORY)
tools/libs/foreignmemory/Makefile +0
-0@@ -4,7 +4,7 @@ include $(XEN_ROOT)/tools/Rules.mkMAJOR = 1
MINOR = 4
LIBNAME := foreignmemorytools/libs/foreignmemory/Makefile +1
-1-USELIBS := toollog toolcore +USELIBS := toollog toolcore call
SRCS-y +=core.c
SRCS-$(CONFIG_Linux) +=linux.c
Quoted 1 line
to fix the build in certain containers.
Re: [PATCH 2/5] xen/gnttab: Rework resource acquisition
In reply to Jan BeulichQuoted 17 lines · Jan Beulich
Quoted 6 lines · Andrew Cooper
The existing logic doesn't function in the general case for mapping a guests
grant table, due to arbitrary 32 frame limit, and the default grant table
limit being 64.Quoted 9 lines
In order to start addressing this, rework the existing grant table logic by
implementing a single gnttab_acquire_resource(). This is far more efficient
than the previousacquire_grant_table()inmemory.cbecause it doesn't take
the grant table write lock, and attempt to grow the table, for every single
frame.
Quoted 3 lines
Among the code you replace there is a comment "Iterate backwards in case table needs to grow" explaining why what you say about growing the grant table didn't actually happen.
What I have said is accurate.
Iterating backwards caused it to actually grow once, but every iteration still takes the lock and attempts to grow the table.
Quoted 16 lines
Quoted 16 lines
xen/common/grant_table.c +0
-0@@ -4013,6 +4013,72 @@ static int gnttab_get_shared_frame_mfn(struct domain *d, return 0; }+int gnttab_acquire_resource(
+ struct domain *d, unsigned int id, unsigned long frame, + unsigned intnr_frames,xen_pfn_tmfn_list[])xen/common/grant_table.c +6
-0+{ + struct grant_table *gt = d->grant_table; + unsigned int i = nr_frames, tot_frames; + void **vaddrs; + int rc = 0; +
Quoted 8 lines
Quoted 3 lines
+ /* Input sanity. */
+ if ( !nr_frames )
+ return-EINVAL;I can't seem to be able to find an equivalent of this in the old logic, and hence this looks like an unwarranted change in behavior to me. We have quite a few hypercall ops where some count being zero is simply a no-op, i.e. yielding success without doing anything.
There is no possible circumstance when getting here requesting 0 is legitimate.
Tolerating a zero input for a mapping request is a very expensive way of hiding caller bugs.
Most importantly however, the correctness of the logic does depends on nr_frames being nonzero.
Quoted 9 lines
Quoted 8 lines
+ /* Overflow checks */
+ if ( frame +nr_frames< frame )
+ return-EINVAL;
+
+tot_frames= frame +nr_frames;
+ if (tot_frames!= frame +nr_frames)
+ return-EINVAL;
Quoted 5 lines
I find the naming here quite confusing. I realize part of this stems from the code you replace, but anyway: "unsigned long frame" typically represents a memory frame number of some sort, making the calculation look as if it was wrong. (Initially I merely meant to ask whether this check isn't redundant with the prior one, or vice versa.)
Both checks are strictly necessary. The top one checks 64bit wrapping, while the second is checking 32bit truncation.
This code is horrible, both in terms of the acquire_resource API, and the grant API being off by one compared to normal, and needing the max_frame index, not the number of frames.
~Andrew
Re: [PATCH 2/5] xen/gnttab: Rework resource acquisition
In reply to Paul DurrantQuoted 15 lines · Paul Durrant
Quoted 15 lines
-----Original Message-----
From: Andrew Cooper <[email protected]>
Sent: 28 July 2020 12:37
To: Xen-devel <[email protected]>
Cc: Andrew Cooper <[email protected]>; George Dunlap <[email protected]>; Ian Jackson <[email protected]>; Jan Beulich <[email protected]>; Konrad Rzeszutek Wilk <[email protected]>; Stefano Stabellini <[email protected]>; Wei Liu <[email protected]>; Julien Grall <[email protected]>; Paul Durrant <[email protected]>; Michał Leszczyński <[email protected]>; Hubert Jasudowicz <[email protected]>
Subject: [PATCH 2/5] xen/gnttab: Rework resource acquisitionThe existing logic doesn't function in the general case for mapping a guests grant table, due to arbitrary 32 frame limit, and the default grant table limit being 64.
Quoted 7 lines
Quoted 6 lines
In order to start addressing this, rework the existing grant table logic by implementing a single
gnttab_acquire_resource(). This is far more efficient than the previousacquire_grant_table()inmemory.cbecause it doesn't take the grant table write lock, and attempt to grow the table, for every single frame.But that should not have happened before because the code deliberately iterates backwards, thereby starting with the last frame, thereby growing the table at most once. (I agree that dropping and re-acquiring the lock every time was sub-optimal).
It still attempts to grow on every iteration. Its just growing to a smaller size than already succeeded.
Quoted 32 lines
Quoted 32 lines
The new
gnttab_acquire_resource()function subsumes the previous two gnttab_get_{shared,status}_frame() helpers.No functional change.
Signed-off-by Andrew Cooper <[email protected]>---
CC: George Dunlap <[email protected]>
CC: Ian Jackson <[email protected]>
CC: Jan Beulich <[email protected]>
CC: Konrad Rzeszutek Wilk <[email protected]>
CC: Stefano Stabellini <[email protected]>
CC: Wei Liu <[email protected]>
CC: Julien Grall <[email protected]>
CC: Paul Durrant <[email protected]>
CC: Michał Leszczyński <[email protected]>
CC: Hubert Jasudowicz <[email protected]>--- xen/common/grant_table.c | 93 ++++++++++++++++++++++++++++++------------- xen/common/memory.c | 42 +------------------ xen/include/xen/grant_table.h | 19 ++++----- 3 files changed, 75 insertions(+), 79 deletions(-)xen/common/grant_table.c +0
-0@@ -4013,6 +4013,72 @@ static int gnttab_get_shared_frame_mfn(struct domain *d,return 0 ; }
Quoted 13 lines
xen/common/grant_table.c static int gnttab_get_shared_frame_mfn(struct domain *d, · line 4027
Quoted 12 lines
xen/common/grant_table.c +12
-0+int gnttab_acquire_resource ( +struct domain *d,unsigned int id,unsigned long frame, +unsigned int nr_frames,xen_pfn_t mfn_list[]) +{ +struct grant_table *gt = d->grant_table; +unsigned int i = nr_frames, tot_frames; +void **vaddrs; +int rc =0 ; + +/* Input sanity. */ +if ( !nr_frames ) +return -EINVAL ;This was not an error before. Does mapping 0 frames really need to be a failure?
Yes.
You have spotted the -1 which depends on nr_frames being nonzero to function correctly.
Quoted 7 lines
xen/common/grant_table.c static int gnttab_get_shared_frame_mfn(struct domain *d, · line 4033
Quoted 6 lines
xen/common/grant_table.c +6
-0+ +/* Overflow checks */ +if ( frame + nr_frames < frame ) +return -EINVAL ; + + tot_frames = frame + nr_frames;That name is confusing. '
last_frame' perhaps (and then make the -1 implicit)?
How is that naming any less confusing?
Quoted 9 lines
xen/common/grant_table.c static int gnttab_get_shared_frame_mfn(struct domain *d, · line 4077
Quoted 8 lines
xen/common/grant_table.c +8
-0+break ; + } + +for ( i =0 ; i < nr_frames; ++i ) + mfn_list[i] =virt_to_mfn (vaddrs[frame + i]); + + out: +grant_write_unlock (gt);Since you deliberately grew the table first, could you not drop the write lock and acquire it a read lock before looping over the frames?
I tried originally. That's not an operation supported by percpu read/write locks, and this isn't a fastpath.
~Andrew
Re: [PATCH 2/5] xen/gnttab: Rework resource acquisition
In reply to Andrew CooperQuoted 26 lines · Andrew Cooper
Quoted 16 lines · Jan Beulich
Quoted 15 lines · Andrew Cooper
xen/common/grant_table.c +0
-0@@ -4013,6 +4013,72 @@ static int gnttab_get_shared_frame_mfn(struct domain *d, return 0; }+int gnttab_acquire_resource(
+ struct domain *d, unsigned int id, unsigned long frame, + unsigned intnr_frames,xen_pfn_tmfn_list[])xen/common/grant_table.c +6
-0+{ + struct grant_table *gt = d->grant_table; + unsigned int i = nr_frames, tot_frames; + void **vaddrs; + int rc = 0; +Quoted 8 lines
Quoted 3 lines
+ /* Input sanity. */
+ if ( !nr_frames )
+ return-EINVAL;I can't seem to be able to find an equivalent of this in the old logic, and hence this looks like an unwarranted change in behavior to me. We have quite a few hypercall ops where some count being zero is simply a no-op, i.e. yielding success without doing anything.
Quoted 5 lines
There is no possible circumstance when getting here requesting 0 is legitimate.
Tolerating a zero input for a mapping request is a very expensive way of hiding caller bugs.
That's just one possible view. There are people thinking that some extra runtime overhead doesn't outweigh the "clutter" of extra conditionals in their code, and hence would rather avoid checking if some calculation of theirs yielded zero before making a hypercall. I think we should try to consistently regard counts of zero as "nothing to do" in all our hypercalls.
Quoted 2 lines
Most importantly however, the correctness of the logic does depends on
nr_framesbeing nonzero.
I didn't mean to ask to drop the conditional if it's needed; all I'm opposed to is raising an error in this case.
Jan
Re: [PATCH 2/5] xen/gnttab: Rework resource acquisition
Quoted 25 lines · Jan Beulich
Quoted 24 lines · Andrew Cooper
Quoted 23 lines · Jan Beulich
Quoted 9 lines · Andrew Cooper
xen/common/grant_table.c +0
-0@@ -4013,6 +4013,72 @@ static int gnttab_get_shared_frame_mfn(struct domain *d, return 0; }+int gnttab_acquire_resource(
+ struct domain *d, unsigned int id, unsigned long frame, + unsigned intnr_frames,xen_pfn_tmfn_list[])xen/common/grant_table.c static int gnttab_get_shared_frame_mfn(struct domain *d, · line 4027
Quoted 9 lines
xen/common/grant_table.c +9
-0+{ + struct grant_table *gt = d->grant_table; + unsigned int i = nr_frames, tot_frames; + void **vaddrs; + int rc = 0; + + /* Input sanity. */ + if ( !nr_frames ) + return -EINVAL;I can't seem to be able to find an equivalent of this in the old logic, and hence this looks like an unwarranted change in behavior to me. We have quite a few hypercall ops where some count being zero is simply a no-op, i.e. yielding success without doing anything.
Quoted 6 lines
Quoted 5 lines
There is no possible circumstance when getting here requesting 0 is legitimate.
Tolerating a zero input for a mapping request is a very expensive way of hiding caller bugs.
That's just one possible view.
... that is fully enforced in the ecosystem we work in.
You can't get a 0 to here in the first place.
However, when it comes to the XLAT and the chunking fixes, getting 0 here is a hard error, and I'm not interested in breaking the that logic for a non-existent corner case.
~Andrew
Re: [PATCH 2/5] xen/gnttab: Rework resource acquisition
Quoted 29 lines · Andrew Cooper
Quoted 28 lines · Jan Beulich
Quoted 27 lines · Andrew Cooper
Quoted 15 lines · Jan Beulich
Quoted 14 lines · Andrew Cooper
xen/common/grant_table.c +0
-0@@ -4013,6 +4013,72 @@ static int gnttab_get_shared_frame_mfn(struct domain *d, return 0; }+int gnttab_acquire_resource(
+ struct domain *d, unsigned int id, unsigned long frame, + unsigned intnr_frames,xen_pfn_tmfn_list[]) +{
+ structgrant_table*gt =d->grant_table;
+ unsigned int i =nr_frames,tot_frames;
+ void **vaddrs;
+ int rc = 0;Quoted 8 lines
Quoted 4 lines
+
+ /* Input sanity. */
+ if ( !nr_frames )
+ return-EINVAL;I can't seem to be able to find an equivalent of this in the old logic, and hence this looks like an unwarranted change in behavior to me. We have quite a few hypercall ops where some count being zero is simply a no-op, i.e. yielding success without doing anything.
There is no possible circumstance when getting here requesting 0 is legitimate.
Quoted 7 lines
Quoted 3 lines
Quoted 2 lines
Tolerating a zero input for a mapping request is a very expensive way of hiding caller bugs.
That's just one possible view.
... that is fully enforced in the ecosystem we work in.
You can't get a 0 to here in the first place.
Would you mind pointing me at why this is? All I can see is
if ( xmar.nr_frames > ARRAY_SIZE(mfn_list) )
return -E2BIG;
as far as the caller's argument checking goes here. And acquire_grant_table(), which is what you're replacing, also is dealing with zero quite fine afaics. As is arch_acquire_resource().
Quoted 3 lines
However, when it comes to the XLAT and the chunking fixes, getting 0 here is a hard error, and I'm not interested in breaking the that logic for a non-existent corner case.
I don't share this view. It's also interesting that you had to address similar questions by Paul, isn't it?
Jan