DOMCTL-based guest magic region allocation for 11 domUs
v3 cover DOMCTL-based guest magic region allocation for 11 domUs
An error message can seen from the init-dom0less application on direct-mapped 1:1 domains:
``` Allocating magic pages memory.c:238:d0v0 mfn 0x39000 doesn't belong to d1 Error on alloc magic pages ```
This is because populate_physmap() automatically assumes gfn == mfn for direct mapped domains. This cannot be true for the magic pages that are allocated later for 1:1 Dom0less DomUs from the init-dom0less helper application executed in Dom0. For domain using statically allocated memory but not 1:1 direct-mapped, similar error "failed to retrieve a reserved page" can be seen as the reserved memory list is empty at that time.
This series tries to fix this issue using a DOMCTL-based approach, because for 1:1 direct-mapped domUs, we need to avoid the RAM regions and inform the toolstack about the region found by hypervisor for mapping the magic pages. Patch 1 introduced a new DOMCTL to get the guest memory map, currently only used for the magic page regions. Patch 2 generalized the extended region finding logic so that it can be reused for other use cases such as finding 1:1 domU magic regions. Patch 3 uses the same approach as finding the extended regions to find the guest magic page regions for direct-mapped DomUs. Patch 4 avoids hardcoding all base addresses of guest magic region in the init-dom0less application by consuming the newly introduced DOMCTL.
Gitlab pipeline for this series:
https://gitlab.com/xen-project/people/henryw/xen/-/pipelines/1238005196
Henry Wang (4):
xen/domctl, tools: Introduce a new domctl to get guest memory map xen/arm: Generalize the extended region finding logic xen/arm: Find unallocated spaces for magic pages of direct-mapped domU xen/memory, tools: Avoid hardcoding GUEST_MAGIC_BASE in init-dom0less
tools/helpers/init-dom0less.c | 36 +++++-- tools/include/xenctrl.h | 4 + tools/libs/ctrl/xc_domain.c | 33 ++++++ xen/arch/arm/dom0less-build.c | 11 ++ xen/arch/arm/domain.c | 15 +++ xen/arch/arm/domain_build.c | 131 +++++++++++++++-------- xen/arch/arm/domctl.c | 32 +++++- xen/arch/arm/include/asm/domain.h | 8 ++ xen/arch/arm/include/asm/domain_build.h | 4 + xen/arch/arm/include/asm/setup.h | 5 + xen/arch/arm/include/asm/static-memory.h | 7 ++ xen/arch/arm/static-memory.c | 39 +++++++ xen/common/memory.c | 22 +++- xen/include/public/arch-arm.h | 11 ++ xen/include/public/domctl.h | 27 +++++ xen/include/public/memory.h | 5 + xen/include/xen/mm.h | 7 ++ 17 files changed, 339 insertions(+), 58 deletions(-)
Signature · 1 line
2.34.1
PATCH v3 1/4 xen/domctl, tools: Introduce a new domctl to get guest memory map
There are some use cases where the toolstack needs to know the guest memory map. For example, the toolstack helper application "init-dom0less" needs to know the guest magic page regions for 1:1 direct-mapped dom0less DomUs to allocate magic pages.
To address such needs, add XEN_DOMCTL_get_mem_map hypercall and related data structures to query the hypervisor for the guest memory map. The guest memory map is recorded in the domain structure and currently only guest magic page region is recorded in the guest memory map. The guest magic page region is initialized at the domain creation time as the layout in the public header, and it is updated for 1:1 dom0less DomUs (see the following commit) to avoid conflict with RAM.
Take the opportunity to drop an unnecessary empty line to keep the coding style consistent in the file.
---
RFC: I think the newly introduced "struct xen_domctl_mem_map" is quite duplicated with "struct xen_memory_map", any comment on reuse the "struct xen_memory_map" for simplicity?
v3:
- Init the return rc for XEN_DOMCTL_get_mem_map.
- Copy the nr_mem_regions back as it should be both IN & OUT.
- Check if mem_map->nr_mem_regions exceeds the XEN_MAX_MEM_REGIONS when adding a new entry.
- Allow XEN_MAX_MEM_REGIONS to be different between different archs.
- Add explicit padding and check to the domctl structures. v2:
- New patch
--- tools/include/xenctrl.h | 4 ++++ tools/libs/ctrl/xc_domain.c | 33 +++++++++++++++++++++++++++++++ xen/arch/arm/domain.c | 15 ++++++++++++++ xen/arch/arm/domctl.c | 32 +++++++++++++++++++++++++++++- xen/arch/arm/include/asm/domain.h | 8 ++++++++ xen/include/public/arch-arm.h | 11 +++++++++++ xen/include/public/domctl.h | 27 +++++++++++++++++++++++++ 7 files changed, 129 insertions(+), 1 deletion(-)
tools/include/xenctrl.h +4 -0
@@ -1195,6 +1195,10 @@ int xc_domain_setmaxmem(xc_interface *xch,uint32_t domid, uint64_t max_memkb); +int xc_get_domain_mem_map (xc_interface *xch,uint32_t domid, +struct xen_mem_region mem_regions[], +uint32_t *nr_regions); +int xc_domain_set_memmap_limit (xc_interface *xch,uint32_t domid,unsigned long map_limitkb);
tools/libs/ctrl/xc_domain.c +33 -0
@@ -697,6 +697,39 @@ int xc_domain_setmaxmem(xc_interface *xch,return do_domctl (xch, &domctl); } +int xc_get_domain_mem_map (xc_interface *xch,uint32_t domid, +struct xen_mem_region mem_regions[], +uint32_t *nr_regions) +{ +int rc; +struct xen_domctl domctl = { + .cmd = XEN_DOMCTL_get_mem_map, + .domain = domid, + .u.mem_map = { + .nr_mem_regions = *nr_regions, + .pad =0 , + }, + }; + +DECLARE_HYPERCALL_BOUNCE (mem_regions, +sizeof (xen_mem_region_t) * (*nr_regions), +XC_HYPERCALL_BUFFER_BOUNCE_OUT ); + +if ( !mem_regions ||xc_hypercall_bounce_pre (xch, mem_regions) || + (*nr_regions) <1 ) +return -1 ; + +set_xen_guest_handle (domctl.u.mem_map.buffer, mem_regions); + + rc =do_domctl (xch, &domctl); + +xc_hypercall_bounce_post (xch, mem_regions); + + *nr_regions = domctl.u.mem_map.nr_mem_regions; + +return rc; +} +#if defined(__i386__) || defined(__x86_64__) int xc_domain_set_memory_map(xc_interface *xch, uint32_t domid,
xen/arch/arm/domain.c +15 -0
@@ -696,6 +696,7 @@ int arch_domain_create(struct domain *d, {unsigned int count =0 ;int rc; +struct mem_map_domain *mem_map = &d->arch.mem_map;BUILD_BUG_ON (GUEST_MAX_VCPUS <MAX_VIRT_CPUS );
@@ -785,6 +786,20 @@ int arch_domain_create(struct domain *d, d->arch.sve_vl = config->arch.sve_vl;#endif +if ( mem_map->nr_mem_regions <XEN_MAX_MEM_REGIONS ) + { + mem_map->regions[mem_map->nr_mem_regions].start =GUEST_MAGIC_BASE ; + mem_map->regions[mem_map->nr_mem_regions].size =GUEST_MAGIC_SIZE ; + mem_map->regions[mem_map->nr_mem_regions].type =GUEST_MEM_REGION_MAGIC ; + mem_map->nr_mem_regions++; + } +else + { +printk ("Exceed max number of supported memory map regions\n" ); + rc = -ENOSPC ; + goto fail; + } +return 0 ; fail:
xen/arch/arm/domctl.c +31 -1
@@ -148,7 +148,6 @@ long arch_do_domctl(struct xen_domctl *domctl, struct domain *d,return 0 ; } -case XEN_DOMCTL_vuart_op: {int rc;
@@ -176,6 +175,37 @@ long arch_do_domctl(struct xen_domctl *domctl, struct domain *d,return rc; } +case XEN_DOMCTL_get_mem_map: + { +int rc =0 ; +uint32_t nr_regions, i; + +if ( domctl->u.mem_map.pad ) +return -EINVAL ; + +/* +* Cap the number of regions to the minimum value between toolstack and +* hypervisor to avoid overflowing the buffer. +*/ + nr_regions =min (d->arch.mem_map.nr_mem_regions, + domctl->u.mem_map.nr_mem_regions); + + domctl->u.mem_map.nr_mem_regions = nr_regions; + +for ( i =0 ; i < nr_regions; i++ ) + { +if ( d->arch.mem_map.regions[i].pad ) +return -EINVAL ; + } + +if (copy_to_guest (domctl->u.mem_map.buffer, + d->arch.mem_map.regions, + nr_regions) || +__copy_to_guest (u_domctl, domctl,1 ) ) + rc = -EFAULT ; + +return rc; + }default :return subarch_do_domctl (domctl, d, u_domctl); }
xen/arch/arm/include/asm/domain.h +8 -0
@@ -10,6 +10,7 @@#include <asm/gic.h> #include <asm/vgic.h> #include <asm/vpl011.h> +#include <public/domctl.h> #include <public/hvm/params.h> struct hvm_domain
@@ -59,6 +60,11 @@ struct paging_domain {unsigned long p2m_total_pages; }; +struct mem_map_domain { +unsigned int nr_mem_regions; +struct xen_mem_region regions[XEN_MAX_MEM_REGIONS ]; +}; +struct arch_domain {#ifdef CONFIG_ARM_64
@@ -77,6 +83,8 @@ struct arch_domainstruct paging_domain paging; +struct mem_map_domain mem_map; +struct vmmio vmmio;/* Continuable domain_relinquish_resources(). */
xen/include/public/arch-arm.h +11 -0
@@ -223,6 +223,13 @@ typedef uint64_t xen_pfn_t;*/ #define XEN_LEGACY_MAX_VCPUS 1 +/* +* Maximum number of memory map regions for guest memory layout. +* Used by XEN_DOMCTL_get_mem_map, currently there is only one region +* for the guest magic pages. +*/ +#define XEN_MAX_MEM_REGIONS 1 +typedef uint64_t xen_ulong_t ;#define PRI_xen_ulong PRIx64
@@ -420,6 +427,10 @@ typedef uint64_t xen_callback_t;* should instead use the FDT. */ +/* Guest memory region types */ +#define GUEST_MEM_REGION_DEFAULT 0 +#define GUEST_MEM_REGION_MAGIC 1 +/* Physical Address Space */ /* Virtio MMIO mappings */
xen/include/public/domctl.h +27 -0
@@ -946,6 +946,31 @@ struct xen_domctl_paging_mempool {uint64_aligned_t size;/* Size in bytes. */ }; +#ifndef XEN_MAX_MEM_REGIONS +#define XEN_MAX_MEM_REGIONS 1 +#endif + +struct xen_mem_region { +uint64_aligned_t start; +uint64_aligned_t size; +uint32_t type; +/* Must be zero */ +uint32_t pad; +}; +typedef struct xen_mem_region xen_mem_region_t ; +DEFINE_XEN_GUEST_HANDLE (xen_mem_region_t); + +struct xen_domctl_mem_map { +/* IN & OUT */ +uint32_t nr_mem_regions; +/* Must be zero */ +uint32_t pad; +/* OUT */ +XEN_GUEST_HANDLE_64 (xen_mem_region_t) buffer; +}; +typedef struct xen_domctl_mem_map xen_domctl_mem_map_t ; +DEFINE_XEN_GUEST_HANDLE (xen_domctl_mem_map_t); + #if defined(__i386__) || defined(__x86_64__) struct xen_domctl_vcpu_msr { uint32_t index;
@@ -1277,6 +1302,7 @@ struct xen_domctl {#define XEN_DOMCTL_vmtrace_op 84#define XEN_DOMCTL_get_paging_mempool_size 85#define XEN_DOMCTL_set_paging_mempool_size 86 +#define XEN_DOMCTL_get_mem_map 87#define XEN_DOMCTL_gdbsx_guestmemio 1000#define XEN_DOMCTL_gdbsx_pausevcpu 1001#define XEN_DOMCTL_gdbsx_unpausevcpu 1002
@@ -1339,6 +1365,7 @@ struct xen_domctl {struct xen_domctl_vuart_op vuart_op;struct xen_domctl_vmtrace_op vmtrace_op;struct xen_domctl_paging_mempool paging_mempool; +struct xen_domctl_mem_map mem_map;uint8_t pad[128 ]; } u; };
Signature · 1 line
2.34.1
Re: [PATCH v3 1/4] xen/domctl, tools: Introduce a new domctl to get guest memory map
Quoted 7 lines · Henry Wang
tools/libs/ctrl/xc_domain.c +1
-0@@ -697,6 +697,39 @@ int xc_domain_setmaxmem(xc_interface *xch,return do_domctl (xch, &domctl); } +int xc_get_domain_mem_map(xc_interface *xch, uint32_t domid,
tools/libs/ctrl/xc_domain.c int xc_domain_setmaxmem(xc_interface *xch, · line 710
Quoted 10 lines
tools/libs/ctrl/xc_domain.c +10
-0+struct xen_mem_region mem_regions[], + uint32_t *nr_regions) +{ + int rc; + struct xen_domctl domctl = { + .cmd = XEN_DOMCTL_get_mem_map, + .domain = domid, + .u.mem_map = { + .nr_mem_regions = *nr_regions, + .pad = 0,
This isn't needed: By there being an initializer for the struct, all unmentioned fields will be set to 0 anyway.
Quoted 14 lines
xen/arch/arm/domain.c +1
-0@@ -696,6 +696,7 @@ int arch_domain_create(struct domain *d, {unsigned int count =0 ;int rc; +struct mem_map_domain *mem_map = &d->arch.mem_map;BUILD_BUG_ON (GUEST_MAX_VCPUS <MAX_VIRT_CPUS );@@ -785,6 +786,20 @@ int arch_domain_create(struct domain *d, d->arch.sve_vl = config->arch.sve_vl;#endif
xen/arch/arm/domain.c int arch_domain_create(struct domain *d, · line 798
Quoted 10 lines
xen/arch/arm/domain.c +10
-0+if ( mem_map->nr_mem_regions <XEN_MAX_MEM_REGIONS ) + { + mem_map->regions[mem_map->nr_mem_regions].start =GUEST_MAGIC_BASE ; + mem_map->regions[mem_map->nr_mem_regions].size =GUEST_MAGIC_SIZE ; + mem_map->regions[mem_map->nr_mem_regions].type =GUEST_MEM_REGION_MAGIC ; + mem_map->nr_mem_regions++; + } +else + { +printk ("Exceed max number of supported memory map regions\n" );
Debugging leftover?
xen/arch/arm/domctl.c long arch_do_domctl(struct xen_domctl *domctl, struct domain *d, · line 153
Quoted 10 lines
xen/arch/arm/domctl.c +0
-1@@ -148,7 +148,6 @@ long arch_do_domctl(struct xen_domctl *domctl, struct domain *d,return 0 ; } -case XEN_DOMCTL_vuart_op: {int rc;
Why? Instead you want ...
xen/arch/arm/domctl.c long arch_do_domctl(struct xen_domctl *domctl, struct domain *d, · line 178
Quoted 5 lines
xen/arch/arm/domctl.c +1
-0@@ -176,6 +175,37 @@ long arch_do_domctl(struct xen_domctl *domctl, struct domain *d,return rc; } +case XEN_DOMCTL_get_mem_map:
... separating blank line above this line and ...
Quoted 21 lines
xen/arch/arm/domctl.c +21
-0+ { +int rc =0 ; +uint32_t nr_regions, i; + +if ( domctl->u.mem_map.pad ) +return -EINVAL ; + +/* +* Cap the number of regions to the minimum value between toolstack and +* hypervisor to avoid overflowing the buffer. +*/ + nr_regions =min (d->arch.mem_map.nr_mem_regions, + domctl->u.mem_map.nr_mem_regions); + + domctl->u.mem_map.nr_mem_regions = nr_regions; + +for ( i =0 ; i < nr_regions; i++ ) + { +if ( d->arch.mem_map.regions[i].pad ) +return -EINVAL ; + }
xen/arch/arm/domctl.c long arch_do_domctl(struct xen_domctl *domctl, struct domain *d, · line 209
Quoted 10 lines
xen/arch/arm/domctl.c +9
-0+ +if (copy_to_guest (domctl->u.mem_map.buffer, + d->arch.mem_map.regions, + nr_regions) || +__copy_to_guest (u_domctl, domctl,1 ) ) + rc = -EFAULT ; + +return rc; + }default :
... this one.
Further with the way you use min() above, how is the caller going to know whether it simply specified too small an array?
And then you check d->arch.mem_map.regions[i].pad. Why's that? And even if needed here for some reason, that's surely not EINVAL, but an internal error in Xen.
Finally instead of __copy_to_guest() can't you use __copy_field_to_guest(), for just nr_regions?
Quoted 4 lines
xen/include/public/domctl.h +0
-0@@ -946,6 +946,31 @@ struct xen_domctl_paging_mempool {uint64_aligned_t size;/* Size in bytes. */
xen/include/public/domctl.h struct xen_domctl_paging_mempool { · line 956
Quoted 10 lines
xen/include/public/domctl.h +8
-0}; +#ifndef XEN_MAX_MEM_REGIONS +#define XEN_MAX_MEM_REGIONS 1 +#endif + +struct xen_mem_region { +uint64_aligned_t start; +uint64_aligned_t size; +uint32_t type;
What is this field set to? I see no #define(s) in this header. If it's the GUEST_MEM_REGION_* in the Arm header, a connection needs to be made. Also note that GUEST_MEM_REGION_* violate name space requirements: New additions should have XEN_ / xen_ prefixes on their names.
xen/include/public/domctl.h struct xen_domctl_paging_mempool { · line 958
Quoted 2 lines
xen/include/public/domctl.h +2
-0+/* Must be zero */ +uint32_t pad;
This, being OUT only, should not be required to be set by the caller. As long as no use appears, Xen merely ought to guarantee that it'll be 0 upon return.
Jan
Re: [PATCH v3 1/4] xen/domctl, tools: Introduce a new domctl to get guest memory map
Hi Jan,
Quoted 7 lines · Jan Beulich
Quoted 6 lines · Henry Wang
tools/libs/ctrl/xc_domain.c +0
-0@@ -697,6 +697,39 @@ int xc_domain_setmaxmem(xc_interface *xch,return do_domctl (xch, &domctl); }
Quoted 13 lines
tools/libs/ctrl/xc_domain.c int xc_domain_setmaxmem(xc_interface *xch, · line 710
Quoted 11 lines
tools/libs/ctrl/xc_domain.c +11
-0+int xc_get_domain_mem_map (xc_interface *xch,uint32_t domid, +struct xen_mem_region mem_regions[], +uint32_t *nr_regions) +{ +int rc; +struct xen_domctl domctl = { + .cmd = XEN_DOMCTL_get_mem_map, + .domain = domid, + .u.mem_map = { + .nr_mem_regions = *nr_regions, + .pad =0 ,This isn't needed: By there being an initializer for the struct, all unmentioned fields will be set to 0 anyway.
Ok, I can drop the initialization of the .pad field.
Quoted 14 lines
Quoted 14 lines
xen/arch/arm/domain.c +1
-0@@ -696,6 +696,7 @@ int arch_domain_create(struct domain *d, {unsigned int count =0 ;int rc; +struct mem_map_domain *mem_map = &d->arch.mem_map;BUILD_BUG_ON (GUEST_MAX_VCPUS <MAX_VIRT_CPUS );@@ -785,6 +786,20 @@ int arch_domain_create(struct domain *d, d->arch.sve_vl = config->arch.sve_vl;#endif
Quoted 11 lines
xen/arch/arm/domain.c int arch_domain_create(struct domain *d, · line 798
Quoted 10 lines
xen/arch/arm/domain.c +10
-0+if ( mem_map->nr_mem_regions <XEN_MAX_MEM_REGIONS ) + { + mem_map->regions[mem_map->nr_mem_regions].start =GUEST_MAGIC_BASE ; + mem_map->regions[mem_map->nr_mem_regions].size =GUEST_MAGIC_SIZE ; + mem_map->regions[mem_map->nr_mem_regions].type =GUEST_MEM_REGION_MAGIC ; + mem_map->nr_mem_regions++; + } +else + { +printk ("Exceed max number of supported memory map regions\n" );Debugging leftover?
Well, not really, I did this on purpose to print some info before exit. But now I realize other error paths in arch_domain_create() do not do that. I will drop this printk in v4.
Quoted 11 lines
xen/arch/arm/domctl.c long arch_do_domctl(struct xen_domctl *domctl, struct domain *d, · line 153
Quoted 10 lines
xen/arch/arm/domctl.c +0
-1@@ -148,7 +148,6 @@ long arch_do_domctl(struct xen_domctl *domctl, struct domain *d,return 0 ; } -case XEN_DOMCTL_vuart_op: {int rc;Why? Instead you want ...
Because there are no empty line between the other sub-ops in the arm version of arch_do_domctl(). Since there is no explicit guideline in CODING_STYLE, I was trying to take the opportunity to keep the coding style consistent within the file. However since you are asking, I realized that the x86 arch_do_domctl() is using the other way, i.e adding an empty line between the sub-ops, so...
Quoted 31 lines
xen/arch/arm/domctl.c long arch_do_domctl(struct xen_domctl *domctl, struct domain *d, · line 178
Quoted 5 lines
xen/arch/arm/domctl.c +1
-0@@ -176,6 +175,37 @@ long arch_do_domctl(struct xen_domctl *domctl, struct domain *d,return rc; } +case XEN_DOMCTL_get_mem_map:... separating blank line above this line and ...
Quoted 24 lines
xen/arch/arm/domctl.c +24
-0+ { +int rc =0 ; +uint32_t nr_regions, i; + +if ( domctl->u.mem_map.pad ) +return -EINVAL ; + +/* +* Cap the number of regions to the minimum value between toolstack and +* hypervisor to avoid overflowing the buffer. +*/ + nr_regions =min (d->arch.mem_map.nr_mem_regions, + domctl->u.mem_map.nr_mem_regions); + + domctl->u.mem_map.nr_mem_regions = nr_regions; + +for ( i =0 ; i < nr_regions; i++ ) + { +if ( d->arch.mem_map.regions[i].pad ) +return -EINVAL ; + } + + if ( copy_to_guest(domctl->u.mem_map.buffer, + d->arch.mem_map.regions,
Quoted 8 lines
xen/arch/arm/domctl.c long arch_do_domctl(struct xen_domctl *domctl, struct domain *d, · line 209
Quoted 7 lines
xen/arch/arm/domctl.c +6
-0+ nr_regions) || + __copy_to_guest(u_domctl, domctl, 1) ) + rc = -EFAULT ; + +return rc; + }default :... this one.
...personally I don't have strong opinions on the style as long as we keep consistent. I can switch the Arm one following the x86 style or just leave it as is.
Quoted 2 lines
Further with the way you use
min()above, how is the caller going to know whether it simply specified too small an array?
I am a bit unsure if we need to forbid caller to specify a smaller value than the max number of regions supported by the hypervisor, technically it is legal, although I agree it will lead to some issues in the toolstack side. It looks like the similar hypercall of e820 also does not forbid this (see get_mem_mapping_layout() and related XENMEM_memory_map). Do you have any suggestions?
Quoted 3 lines
And then you check d->arch.mem_map.regions[i].pad. Why's that? And even if needed here for some reason, that's surely not EINVAL, but an internal error in Xen.
I did that under the impression that we need to check the value of padding field being 0. Also you mentioned in one of the comments below that Xen should guarantee that the padding field should be 0 before return. Apologize if I misunderstand your comment. The -EINVAL is taken from the same way of checking the padding field in XEN_DOMCTL_vuart_op above. Personally I would keep some consistency, but I am open to suggestions to make it better.
Quoted 2 lines
Finally instead of
__copy_to_guest()can't you use__copy_field_to_guest(), for justnr_regions?
You mean replacing __copy_to_guest(u_domctl, domctl, 1) with only the __copy_field_to_guest(u_domctl, domctl, u.mem_map.nr_mem_regions)? Ok I can do that in v4.
Quoted 6 lines
Quoted 6 lines
xen/include/public/domctl.h +0
-0@@ -946,6 +946,31 @@ struct xen_domctl_paging_mempool {uint64_aligned_t size;/* Size in bytes. */ };
Quoted 12 lines
xen/include/public/domctl.h struct xen_domctl_paging_mempool { · line 956
Quoted 8 lines
xen/include/public/domctl.h +8
-0+#ifndef XEN_MAX_MEM_REGIONS +#define XEN_MAX_MEM_REGIONS 1 +#endif + +struct xen_mem_region { +uint64_aligned_t start; +uint64_aligned_t size; +uint32_t type;What is this field set to? I see no #define(s) in this header. If it's the GUEST_MEM_REGION_* in the Arm header, a connection needs to be made. Also note that GUEST_MEM_REGION_* violate name space requirements: New additions should have
XEN_/xen_prefixes on their names.
Yeah it is the GUEST_MEM_REGION_* in the Arm header. The default value will be set to 0 when struct domain is created. I will switch to the XEN_* prefix in v4.
Quoted 5 lines
xen/include/public/domctl.h struct xen_domctl_paging_mempool { · line 958
Quoted 2 lines
xen/include/public/domctl.h +2
-0+/* Must be zero */ +uint32_t pad;This, being OUT only, should not be required to be set by the caller. As long as no use appears, Xen merely ought to guarantee that it'll be 0 upon return.
See above.
Kind regards,
Henry
Quoted 1 line
Jan
Re: [PATCH v3 1/4] xen/domctl, tools: Introduce a new domctl to get guest memory map
Quoted 79 lines · Henry Wang
Quoted 15 lines · Jan Beulich
Quoted 14 lines · Henry Wang
xen/arch/arm/domain.c +1
-0@@ -696,6 +696,7 @@ int arch_domain_create(struct domain *d, {unsigned int count =0 ;int rc; +struct mem_map_domain *mem_map = &d->arch.mem_map;BUILD_BUG_ON (GUEST_MAX_VCPUS <MAX_VIRT_CPUS );@@ -785,6 +786,20 @@ int arch_domain_create(struct domain *d, d->arch.sve_vl = config->arch.sve_vl;#endif Quoted 11 lines
xen/arch/arm/domain.c int arch_domain_create(struct domain *d, · line 798
Quoted 10 lines
xen/arch/arm/domain.c +10
-0+if ( mem_map->nr_mem_regions <XEN_MAX_MEM_REGIONS ) + { + mem_map->regions[mem_map->nr_mem_regions].start =GUEST_MAGIC_BASE ; + mem_map->regions[mem_map->nr_mem_regions].size =GUEST_MAGIC_SIZE ; + mem_map->regions[mem_map->nr_mem_regions].type =GUEST_MEM_REGION_MAGIC ; + mem_map->nr_mem_regions++; + } +else + { +printk ("Exceed max number of supported memory map regions\n" );Debugging leftover?
Well, not really, I did this on purpose to print some info before exit. But now I realize other error paths in
arch_domain_create()do not do that. I will drop this printk in v4.Quoted 31 lines
xen/arch/arm/domain.c long arch_do_domctl(struct xen_domctl *domctl, struct domain *d, · line 178
Quoted 5 lines
xen/arch/arm/domain.c +1
-0@@ -176,6 +175,37 @@ long arch_do_domctl(struct xen_domctl *domctl, struct domain *d,return rc; } +case XEN_DOMCTL_get_mem_map:... separating blank line above this line and ...
Quoted 24 lines
xen/arch/arm/domain.c +24
-0+ { +int rc =0 ; +uint32_t nr_regions, i; + +if ( domctl->u.mem_map.pad ) +return -EINVAL ; + +/* +* Cap the number of regions to the minimum value between toolstack and +* hypervisor to avoid overflowing the buffer. +*/ + nr_regions =min (d->arch.mem_map.nr_mem_regions, + domctl->u.mem_map.nr_mem_regions); + + domctl->u.mem_map.nr_mem_regions = nr_regions; + +for ( i =0 ; i < nr_regions; i++ ) + { +if ( d->arch.mem_map.regions[i].pad ) +return -EINVAL ; + } + + if ( copy_to_guest(domctl->u.mem_map.buffer, + d->arch.mem_map.regions,Quoted 8 lines
xen/arch/arm/domain.c long arch_do_domctl(struct xen_domctl *domctl, struct domain *d, · line 209
Quoted 7 lines
xen/arch/arm/domain.c +6
-0+ nr_regions) || + __copy_to_guest(u_domctl, domctl, 1) ) + rc = -EFAULT ; + +return rc; + }default :... this one.
...personally I don't have strong opinions on the style as long as we keep consistent. I can switch the Arm one following the x86 style or just leave it as is.
Quoted 2 lines
Further with the way you use
min()above, how is the caller going to know whether it simply specified too small an array?
Quoted 6 lines
I am a bit unsure if we need to forbid caller to specify a smaller value than the max number of regions supported by the hypervisor, technically it is legal, although I agree it will lead to some issues in the toolstack side. It looks like the similar hypercall of e820 also does not forbid this (see
get_mem_mapping_layout()and relatedXENMEM_memory_map). Do you have any suggestions?
Fill only as much of the array as there is space for, but return the full count to the caller. Another option (less desirable imo) would be to return -ENOBUFS. If to be written anew now, I'd likely code XENMEM_memory_map handling that way, too. But that's too late now.
Quoted 11 lines
Quoted 3 lines
And then you check d->arch.mem_map.regions[i].pad. Why's that? And even if needed here for some reason, that's surely not EINVAL, but an internal error in Xen.
I did that under the impression that we need to check the value of padding field being 0. Also you mentioned in one of the comments below that Xen should guarantee that the padding field should be 0 before return. Apologize if I misunderstand your comment. The
-EINVALis taken from the same way of checking the padding field inXEN_DOMCTL_vuart_opabove. Personally I would keep some consistency, but I am open to suggestions to make it better.
In XEN_DOMCTL_vuart_op it is caller input which is being checked (and needs checking). You're checking internal Xen state here instead. Considering the nature of the issue arising if the assumption was broken, ASSERT() would seem to be the construct to use for the internal state check.
Quoted 6 lines
Quoted 2 lines
Finally instead of
__copy_to_guest()can't you use__copy_field_to_guest(), for justnr_regions?You mean replacing
__copy_to_guest(u_domctl, domctl, 1)with only the__copy_field_to_guest(u_domctl, domctl, u.mem_map.nr_mem_regions)? Ok I can do that in v4.
Yes (unless there are technical reasons not to, of course).
Jan
Re: [PATCH v3 1/4] xen/domctl, tools: Introduce a new domctl to get guest memory map
Hi Jan,
Quoted 83 lines · Jan Beulich
Quoted 74 lines · Henry Wang
Quoted 15 lines · Jan Beulich
Quoted 14 lines · Henry Wang
xen/arch/arm/domain.c +1
-0@@ -696,6 +696,7 @@ int arch_domain_create(struct domain *d, {unsigned int count =0 ;int rc; +struct mem_map_domain *mem_map = &d->arch.mem_map;BUILD_BUG_ON (GUEST_MAX_VCPUS <MAX_VIRT_CPUS );@@ -785,6 +786,20 @@ int arch_domain_create(struct domain *d, d->arch.sve_vl = config->arch.sve_vl;#endif Quoted 11 lines
xen/arch/arm/domain.c int arch_domain_create(struct domain *d, · line 798
Quoted 10 lines
xen/arch/arm/domain.c +10
-0+if ( mem_map->nr_mem_regions <XEN_MAX_MEM_REGIONS ) + { + mem_map->regions[mem_map->nr_mem_regions].start =GUEST_MAGIC_BASE ; + mem_map->regions[mem_map->nr_mem_regions].size =GUEST_MAGIC_SIZE ; + mem_map->regions[mem_map->nr_mem_regions].type =GUEST_MEM_REGION_MAGIC ; + mem_map->nr_mem_regions++; + } +else + { +printk ("Exceed max number of supported memory map regions\n" );Debugging leftover?
Well, not really, I did this on purpose to print some info before exit. But now I realize other error paths in
arch_domain_create()do not do that. I will drop this printk in v4.Quoted 31 lines
xen/arch/arm/domain.c long arch_do_domctl(struct xen_domctl *domctl, struct domain *d, · line 178
Quoted 5 lines
xen/arch/arm/domain.c +1
-0@@ -176,6 +175,37 @@ long arch_do_domctl(struct xen_domctl *domctl, struct domain *d,return rc; } +case XEN_DOMCTL_get_mem_map:... separating blank line above this line and ...
Quoted 24 lines
xen/arch/arm/domain.c +24
-0+ { +int rc =0 ; +uint32_t nr_regions, i; + +if ( domctl->u.mem_map.pad ) +return -EINVAL ; + +/* +* Cap the number of regions to the minimum value between toolstack and +* hypervisor to avoid overflowing the buffer. +*/ + nr_regions =min (d->arch.mem_map.nr_mem_regions, + domctl->u.mem_map.nr_mem_regions); + + domctl->u.mem_map.nr_mem_regions = nr_regions; + +for ( i =0 ; i < nr_regions; i++ ) + { +if ( d->arch.mem_map.regions[i].pad ) +return -EINVAL ; + } + + if ( copy_to_guest(domctl->u.mem_map.buffer, + d->arch.mem_map.regions,Quoted 8 lines
xen/arch/arm/domain.c long arch_do_domctl(struct xen_domctl *domctl, struct domain *d, · line 209
Quoted 7 lines
xen/arch/arm/domain.c +6
-0+ nr_regions) || + __copy_to_guest(u_domctl, domctl, 1) ) + rc = -EFAULT ; + +return rc; + }default :... this one.
...personally I don't have strong opinions on the style as long as we keep consistent. I can switch the Arm one following the x86 style or just leave it as is.
Quoted 8 lines
Quoted 2 lines
Further with the way you use
min()above, how is the caller going to know whether it simply specified too small an array?I am a bit unsure if we need to forbid caller to specify a smaller value than the max number of regions supported by the hypervisor, technically it is legal, although I agree it will lead to some issues in the toolstack side. It looks like the similar hypercall of e820 also does not forbid this (see
get_mem_mapping_layout()and relatedXENMEM_memory_map). Do you have any suggestions?
Quoted 4 lines
Fill only as much of the array as there is space for, but return the full count to the caller. Another option (less desirable imo) would be to return
-ENOBUFS. If to be written anew now, I'd likely codeXENMEM_memory_maphandling that way, too. But that's too late now.
Thanks for the input! Sure I will follow the suggestion in v4.
Quoted 10 lines
Quoted 10 lines
Quoted 3 lines
And then you check d->arch.mem_map.regions[i].pad. Why's that? And even if needed here for some reason, that's surely not EINVAL, but an internal error in Xen.
I did that under the impression that we need to check the value of padding field being 0. Also you mentioned in one of the comments below that Xen should guarantee that the padding field should be 0 before return. Apologize if I misunderstand your comment. The
-EINVALis taken from the same way of checking the padding field inXEN_DOMCTL_vuart_opabove. Personally I would keep some consistency, but I am open to suggestions to make it better.
Quoted 5 lines
In
XEN_DOMCTL_vuart_opit is caller input which is being checked (and needs checking). You're checking internal Xen state here instead. Considering the nature of the issue arising if the assumption was broken,ASSERT()would seem to be the construct to use for the internal state check.
You are right, I will drop the Xen internal state check here.
Quoted 6 lines
Quoted 5 lines
Quoted 2 lines
Finally instead of
__copy_to_guest()can't you use__copy_field_to_guest(), for justnr_regions?You mean replacing
__copy_to_guest(u_domctl, domctl, 1)with only the__copy_field_to_guest(u_domctl, domctl, u.mem_map.nr_mem_regions)? Ok I can do that in v4.Yes (unless there are technical reasons not to, of course).
Thanks for confirming, I will do this way.
Kind regards,
Henry
Quoted 1 line
Jan
PATCH v3 2/4 xen/arm: Generalize the extended region finding logic
In reply to Henry WangRecently there are needs to find unallocated/used memory regions for different use cases at boot time, and the logic of finding extended regions can be reused for some of the new use cases.
This commit therefore generalize the extended region finding logic. Firstly, extract the extended region finding logic to a dedicated helper find_unused_regions(). Then add and pass down a min_region_size parameter so that the helpers can take a flexible region size instead of the hardcoded 64MB for extended region. Finally, adjust the variable and function names to make them general instead of for extended region only.
Take the opportunity to update the stale in-code comment of find_unallocated_memory() and find_memory_holes().
---
v3:
- New patch
--- xen/arch/arm/domain_build.c | 107 ++++++++++++++---------- xen/arch/arm/include/asm/domain_build.h | 2 + xen/arch/arm/include/asm/setup.h | 5 ++ 3 files changed, 70 insertions(+), 44 deletions(-)
xen/arch/arm/domain_build.c +63 -44
@@ -814,19 +814,21 @@ int __init make_memory_node(const struct domain *d,return res; } -static int __init add_ext_regions (unsigned long s_gfn,unsigned long e_gfn, -void *data) +static int __init add_regions (unsigned long s_gfn,unsigned long e_gfn, +void *data) { -struct meminfo *ext_regions = data; +struct mem_unused_info *unused = data; +struct meminfo *regions = unused->regions; +paddr_t min_region_size = unused->min_region_size;paddr_t start, size;paddr_t s =pfn_to_paddr (s_gfn);paddr_t e =pfn_to_paddr (e_gfn); -if ( ext_regions->nr_banks >=ARRAY_SIZE (ext_regions->bank) ) +if ( regions->nr_banks >=ARRAY_SIZE (regions->bank) )return 0 ;/* -* Both start and size of the extended region should be 2MB aligned to +* Both start and size of the region should be 2MB aligned to * potentially allow superpage mapping. */ start = (s +SZ_2M -1 ) & ~(SZ_2M -1 );
@@ -845,26 +847,27 @@ static int __init add_ext_regions(unsigned long s_gfn, unsigned long e_gfn,* not quite useful but increase bookkeeping and not too large * to skip a large proportion of unused address space. */ -if ( size <MB (64 ) ) +if ( size < min_region_size )return 0 ; - ext_regions->bank[ext_regions->nr_banks].start = start; - ext_regions->bank[ext_regions->nr_banks].size = size; - ext_regions->nr_banks++; + regions->bank[regions->nr_banks].start = start; + regions->bank[regions->nr_banks].size = size; + regions->nr_banks++;return 0 ; }/* -* Find unused regions of Host address space which can be exposed to Dom0 -* as extended regions for the special memory mappings. In order to calculate -* regions we exclude every region assigned to Dom0 from the Host RAM: +* Find unused regions of Host address space which can be exposed to +* direct-mapped domains as regions for the special memory mappings. +* In order to calculate regions we exclude every region assigned to +* direct-mapped domains from the Host RAM: * - domain RAM * - reserved-memory * - grant table space */ static int __init find_unallocated_memory (const struct kernel_info *kinfo, -struct meminfo *ext_regions) +struct mem_unused_info *unused) {const struct meminfo *assign_mem = &kinfo->mem;struct rangeset *unalloc_mem;
@@ -893,7 +896,7 @@ static int __init find_unallocated_memory(const struct kernel_info *kinfo, } } -/* Remove RAM assigned to Dom0 */ +/* Remove RAM assigned to domain */ for ( i =0 ; i < assign_mem->nr_banks; i++ ) { start = assign_mem->bank[i].start;
@@ -942,10 +945,10 @@ static int __init find_unallocated_memory(const struct kernel_info *kinfo, start =0 ; end = (1ULL << p2m_ipa_bits) -1 ; res =rangeset_report_ranges (unalloc_mem,PFN_DOWN (start),PFN_DOWN (end), - add_ext_regions, ext_regions); + add_regions, unused);if ( res ) - ext_regions->nr_banks =0 ; -else if ( !ext_regions->nr_banks ) + unused->regions->nr_banks =0 ; +else if ( !unused->regions->nr_banks ) res = -ENOENT ; out:
@@ -982,16 +985,16 @@ static int __init handle_pci_range(const struct dt_device_node *dev, }/* -* Find the holes in the Host DT which can be exposed to Dom0 as extended -* regions for the special memory mappings. In order to calculate regions -* we exclude every addressable memory region described by "reg" and "ranges" +* Find the holes in the Host DT which can be exposed to direct-mapped domains +* as regions for the special memory mappings. In order to calculate regions we +* exclude every addressable memory region described by "reg" and "ranges" * properties from the maximum possible addressable physical memory range: * - MMIO * - Host RAM * - PCI aperture */ static int __init find_memory_holes (const struct kernel_info *kinfo, -struct meminfo *ext_regions) +struct mem_unused_info *unused) {struct dt_device_node *np;struct rangeset *mem_holes;
@@ -1068,10 +1071,10 @@ static int __init find_memory_holes(const struct kernel_info *kinfo, start =0 ; end = (1ULL << p2m_ipa_bits) -1 ; res =rangeset_report_ranges (mem_holes,PFN_DOWN (start),PFN_DOWN (end), - add_ext_regions, ext_regions); + add_regions, unused);if ( res ) - ext_regions->nr_banks =0 ; -else if ( !ext_regions->nr_banks ) + unused->regions->nr_banks =0 ; +else if ( !unused->regions->nr_banks ) res = -ENOENT ; out:
@@ -1081,35 +1084,62 @@ out: }static int __init find_domU_holes (const struct kernel_info *kinfo, -struct meminfo *ext_regions) +struct mem_unused_info *unused) {unsigned int i;uint64_t bankend;const uint64_t bankbase[] =GUEST_RAM_BANK_BASES ;const uint64_t banksize[] =GUEST_RAM_BANK_SIZES ; +struct meminfo *regions = unused->regions; +paddr_t min_region_size = unused->min_region_size;int res = -ENOENT ;for ( i =0 ; i <GUEST_RAM_BANKS ; i++ ) { -struct membank *ext_bank = &(ext_regions->bank[ext_regions->nr_banks]); +struct membank *bank = &(regions->bank[regions->nr_banks]); - ext_bank->start =ROUNDUP (bankbase[i] + kinfo->mem.bank[i].size,SZ_2M ); + bank->start =ROUNDUP (bankbase[i] + kinfo->mem.bank[i].size,SZ_2M ); bankend = ~0ULL >> (64 - p2m_ipa_bits); bankend =min (bankend, bankbase[i] + banksize[i] -1 ); -if ( bankend > ext_bank->start ) - ext_bank->size = bankend - ext_bank->start +1 ; +if ( bankend > bank->start ) + bank->size = bankend - bank->start +1 ; -/* 64MB is the minimum size of an extended region */ -if ( ext_bank->size <MB (64 ) ) +if ( bank->size < min_region_size )continue ; - ext_regions->nr_banks++; + regions->nr_banks++; res =0 ; }return res; } +int __init find_unused_regions (struct domain *d, +const struct kernel_info *kinfo, +struct meminfo *regions, +paddr_t min_region_size) +{ +int res; +struct mem_unused_info unused; + + unused.regions = regions; + unused.min_region_size = min_region_size; + +if (is_domain_direct_mapped (d) ) + { +if ( !is_iommu_enabled (d) ) + res =find_unallocated_memory (kinfo, &unused); +else + res =find_memory_holes (kinfo, &unused); + } +else + { + res =find_domU_holes (kinfo, &unused); + } + +return res; +} +int __init make_hypervisor_node (struct domain *d,const struct kernel_info *kinfo,int addrcells,int sizecells)
@@ -1161,18 +1191,7 @@ int __init make_hypervisor_node(struct domain *d,if ( !ext_regions )return -ENOMEM ; -if (is_domain_direct_mapped (d) ) - { -if ( !is_iommu_enabled (d) ) - res =find_unallocated_memory (kinfo, ext_regions); -else - res =find_memory_holes (kinfo, ext_regions); - } -else - { - res =find_domU_holes (kinfo, ext_regions); - } - + res =find_unused_regions (d, kinfo, ext_regions,MB (64 ));if ( res )printk (XENLOG_WARNING "%pd: failed to allocate extended regions\n" , d);
xen/arch/arm/include/asm/domain_build.h +2 -0
@@ -10,6 +10,8 @@ bool allocate_bank_memory(struct domain *d, struct kernel_info *kinfo, gfn_t sgfn, paddr_t tot_size);int construct_domain (struct domain *d,struct kernel_info *kinfo);int domain_fdt_begin_node (void *fdt,const char *name,uint64_t unit); +int find_unused_regions (struct domain *d,const struct kernel_info *kinfo, +struct meminfo *regions,paddr_t min_region_size);int make_chosen_node (const struct kernel_info *kinfo);int make_cpus_node (const struct domain *d,void *fdt); int make_hypervisor_node(struct domain *d, const struct kernel_info *kinfo,
xen/arch/arm/include/asm/setup.h +5 -0
@@ -61,6 +61,11 @@ struct meminfo {struct membank bank[NR_MEM_BANKS ]; }; +struct mem_unused_info { +struct meminfo *regions; +paddr_t min_region_size; +}; +/* * The domU flag is set for kernels and ramdisks of "xen,domain" nodes. * The purpose of the domU flag is to avoid getting confused in
Signature · 1 line
2.34.1
PATCH v3 3/4 xen/arm: Find unallocated spaces for magic pages of direct-mapped domU
In reply to Henry WangFor 1:1 direct-mapped dom0less DomUs, the magic pages should not clash with any RAM region. To find a proper region for guest magic pages, we can reuse the logic of finding domain extended regions.
If the extended region is enabled, since the extended region banks are at least 64MB, carve out the first 16MB from the first extended region bank for magic pages of direct-mapped domU. If the extended region is disabled, call the newly introduced helper find_11_domU_magic_region() to find a GUEST_MAGIC_SIZE sized unused region.
---
v3:
- Extract the logic of finding unallocated spaces for magic pages of direct-mapped domU to a dedicated function in static-memory.c
- Properly handle error and free memory in find_11_domU_magic_region() v2:
- New patch
--- xen/arch/arm/dom0less-build.c | 11 +++++++ xen/arch/arm/domain_build.c | 24 ++++++++++++++- xen/arch/arm/include/asm/domain_build.h | 2 ++ xen/arch/arm/include/asm/static-memory.h | 7 +++++ xen/arch/arm/static-memory.c | 39 ++++++++++++++++++++++++ 5 files changed, 82 insertions(+), 1 deletion(-)
xen/arch/arm/dom0less-build.c +11 -0
@@ -682,6 +682,17 @@ static int __init prepare_dtb_domU(struct domain *d, struct kernel_info *kinfo)if ( kinfo->dom0less_feature &DOM0LESS_ENHANCED_NO_XS ) { +/* +* Find the guest magic region for 1:1 dom0less domU when the extended +* region is not enabled. +*/ +if ( !opt_ext_regions ||is_32bit_domain (d) ) + { + ret =find_11_domU_magic_region (d, kinfo); +if ( ret ) + goto err; + } + ret =make_hypervisor_node (d, kinfo, addrcells, sizecells);if ( ret ) goto err;
xen/arch/arm/domain_build.c +23 -1
@@ -46,7 +46,7 @@ integer_param("dom0_max_vcpus", opt_dom0_max_vcpus);* If true, the extended regions support is enabled for dom0 and * dom0less domUs. */ -static bool __initdata opt_ext_regions = true; +bool __initdata opt_ext_regions = true;boolean_param ("ext_regions" , opt_ext_regions);static u64 __initdata dom0_mem;
@@ -1196,6 +1196,28 @@ int __init make_hypervisor_node(struct domain *d,printk (XENLOG_WARNING "%pd: failed to allocate extended regions\n" , d); nr_ext_regions = ext_regions->nr_banks; + +/* +* If extended region is enabled, carve out the 16MB guest magic page +* regions from the first bank of extended region (at least 64MB) for +* the 1:1 dom0less DomUs +*/ +if (is_domain_direct_mapped (d) && !is_hardware_domain (d) ) + { +struct mem_map_domain *mem_map = &d->arch.mem_map; + +for ( i =0 ; i < mem_map->nr_mem_regions; i++ ) + { +if ( mem_map->regions[i].type ==GUEST_MEM_REGION_MAGIC ) + { + mem_map->regions[i].start = ext_regions->bank[0 ].start; + mem_map->regions[i].size =GUEST_MAGIC_SIZE ; + + ext_regions->bank[0 ].start +=GUEST_MAGIC_SIZE ; + ext_regions->bank[0 ].size -=GUEST_MAGIC_SIZE ; + } + } + } } reg =xzalloc_array (__be32, (nr_ext_regions +1 ) * (addrcells + sizecells));
xen/arch/arm/include/asm/domain_build.h +2 -0
@@ -4,6 +4,8 @@#include <xen/sched.h> #include <asm/kernel.h> +extern bool opt_ext_regions; +typedef __be32 gic_interrupt_t [3 ]; bool allocate_bank_memory(struct domain *d, struct kernel_info *kinfo,
xen/arch/arm/include/asm/static-memory.h +7 -0
@@ -12,6 +12,7 @@ void allocate_static_memory(struct domain *d, struct kernel_info *kinfo,void assign_static_memory_11 (struct domain *d,struct kernel_info *kinfo,const struct dt_device_node *node);void init_staticmem_pages (void ); +int find_11_domU_magic_region (struct domain *d,struct kernel_info *kinfo);#else /* !CONFIG_STATIC_MEMORY */
@@ -31,6 +32,12 @@ static inline void assign_static_memory_11(struct domain *d,static inline void init_staticmem_pages (void ) {}; +static inline int find_11_domU_magic_region (struct domain *d, +struct kernel_info *kinfo) +{ +return 0 ; +} +#endif /* CONFIG_STATIC_MEMORY */ #endif /* __ASM_STATIC_MEMORY_H_ */
xen/arch/arm/static-memory.c +39 -0
@@ -2,6 +2,7 @@#include <xen/sched.h> +#include <asm/domain_build.h> #include <asm/static-memory.h> static bool __init append_static_memory_to_bank(struct domain *d,
@@ -276,6 +277,44 @@ void __init init_staticmem_pages(void) } } +int __init find_11_domU_magic_region (struct domain *d, +struct kernel_info *kinfo) +{ +if (is_domain_direct_mapped (d) ) + { + struct meminfo *magic_region = xzalloc(struct meminfo); +struct mem_map_domain *mem_map = &d->arch.mem_map; +unsigned int i; +int ret =0 ; + +if ( !magic_region ) +return -ENOMEM ; + + ret =find_unused_regions (d, kinfo, magic_region,GUEST_MAGIC_SIZE ); +if ( ret ) + { +printk (XENLOG_WARNING +"%pd: failed to find a region for domain magic pages\n" , d); +xfree (magic_region); +return -ENOENT ; + } + +/* Update the domain memory map. */ +for ( i =0 ; i < mem_map->nr_mem_regions; i++ ) + { +if ( mem_map->regions[i].type ==GUEST_MEM_REGION_MAGIC ) + { + mem_map->regions[i].start = magic_region->bank[0 ].start; + mem_map->regions[i].size =GUEST_MAGIC_SIZE ; + } + } + +xfree (magic_region); + } + +return 0 ; +} +/* * Local variables: * mode: C
Signature · 1 line
2.34.1
PATCH v3 4/4 xen/memory, tools: Avoid hardcoding GUEST_MAGIC_BASE in init-dom0less
In reply to Henry WangCurrently the GUEST_MAGIC_BASE in the init-dom0less application is hardcoded, which will lead to failures for 1:1 direct-mapped Dom0less DomUs.
Instead of hardcoding the guest magic pages region, use the XEN_DOMCTL_get_mem_map domctl to get the start address of the guest magic pages region. Add the (XEN)MEMF_force_heap_alloc memory flags to force populate_physmap() to allocate page from domheap instead of using 1:1 or static allocated pages to map the magic pages.
---
v3:
- Don't ignore the error from xc_get_domain_mem_map().
- Re-purposing the _MEMF_no_refcount as _MEMF_force_heap_alloc to avoid introduction of a new, single-use flag.
- Reject other reservation sub-ops to use the newly added flag.
- Replace all the GUEST_MAGIC_BASE usages.
v2:
- New patch
--- tools/helpers/init-dom0less.c | 36 +++++++++++++++++++++++++---------- xen/common/memory.c | 22 +++++++++++++++++++-- xen/include/public/memory.h | 5 +++++ xen/include/xen/mm.h | 7 +++++++ 4 files changed, 58 insertions(+), 12 deletions(-)
tools/helpers/init-dom0less.c +26 -10
@@ -19,24 +19,43 @@#define XENSTORE_PFN_OFFSET 1#define STR_MAX_LENGTH 128 +static xen_pfn_t xs_page_base; +static xen_pfn_t xs_page_p2m; +static int alloc_xs_page (struct xc_interface_core *xch,libxl_dominfo *info,uint64_t *xenstore_pfn) { -int rc; -const xen_pfn_t base =GUEST_MAGIC_BASE >>XC_PAGE_SHIFT ; -xen_pfn_t p2m = (GUEST_MAGIC_BASE >>XC_PAGE_SHIFT ) +XENSTORE_PFN_OFFSET ; +int rc, i; +uint32_t nr_regions =XEN_MAX_MEM_REGIONS ; +struct xen_mem_region mem_regions[XEN_MAX_MEM_REGIONS ] = {0 }; + + rc =xc_get_domain_mem_map (xch, info->domid, mem_regions, &nr_regions); +if (rc <0 ) +return rc; + +for ( i =0 ; i < nr_regions; i++ ) + { +if ( mem_regions[i].type ==GUEST_MEM_REGION_MAGIC ) + { + xs_page_base = mem_regions[i].start >>XC_PAGE_SHIFT ; + xs_page_p2m = (mem_regions[i].start >>XC_PAGE_SHIFT ) + +XENSTORE_PFN_OFFSET ; + } + } rc =xc_domain_setmaxmem (xch, info->domid, info->max_memkb + (XC_PAGE_SIZE /1024 ));if (rc <0 )return rc; - rc =xc_domain_populate_physmap_exact (xch, info->domid,1 ,0 ,0 , &p2m); + rc =xc_domain_populate_physmap_exact (xch, info->domid,1 ,0 , + XENMEMF_force_heap_alloc, + &xs_page_p2m);if (rc <0 )return rc; - *xenstore_pfn = base +XENSTORE_PFN_OFFSET ; + *xenstore_pfn = xs_page_base +XENSTORE_PFN_OFFSET ; rc =xc_clear_domain_page (xch, info->domid, *xenstore_pfn);if (rc <0 )return rc;
@@ -145,8 +164,7 @@ static int create_xenstore(struct xs_handle *xsh, rc =snprintf (target_memkb_str,STR_MAX_LENGTH ,"%" PRIu64, info->current_memkb);if (rc <0 || rc >=STR_MAX_LENGTH )return rc; - rc =snprintf (ring_ref_str,STR_MAX_LENGTH ,"%lld" , - (GUEST_MAGIC_BASE >>XC_PAGE_SHIFT ) +XENSTORE_PFN_OFFSET ); + rc =snprintf (ring_ref_str,STR_MAX_LENGTH ,"%" PRIu_xen_pfn, xs_page_p2m);if (rc <0 || rc >=STR_MAX_LENGTH )return rc; rc =snprintf (xenstore_port_str,STR_MAX_LENGTH ,"%u" , xenstore_port);
@@ -282,9 +300,7 @@ static int init_domain(struct xs_handle *xsh,if (rc)err (1 ,"writing to xenstore" ); - rc =xs_introduce_domain (xsh, info->domid, - (GUEST_MAGIC_BASE >>XC_PAGE_SHIFT ) +XENSTORE_PFN_OFFSET , - xenstore_evtchn); + rc =xs_introduce_domain (xsh, info->domid, xs_page_p2m, xenstore_evtchn);if (!rc)err (1 ,"xs_introduce_domain" );return 0 ;
xen/common/memory.c +20 -2
@@ -219,7 +219,8 @@ static void populate_physmap(struct memop_args *a) }else { -if (is_domain_direct_mapped (d) ) +if (is_domain_direct_mapped (d) && + !(a->memflags & MEMF_force_heap_alloc) ) { mfn =_mfn (gpfn);
@@ -246,7 +247,8 @@ static void populate_physmap(struct memop_args *a) mfn =_mfn (gpfn); } -else if (is_domain_using_staticmem (d) ) +else if (is_domain_using_staticmem (d) && + !(a->memflags & MEMF_force_heap_alloc) ) {/* * No easy way to guarantee the retrieved pages are contiguous,
@@ -271,6 +273,14 @@ static void populate_physmap(struct memop_args *a) }else { +/* +* Avoid passing MEMF_force_heap_alloc down to +* alloc_domheap_pages() where the meaning would be the +* original MEMF_no_refcount. +*/ +if (unlikely (a->memflags & MEMF_force_heap_alloc) ) +clear_bit (_MEMF_force_heap_alloc, &a->memflags); + page =alloc_domheap_pages (d, a->extent_order, a->memflags);if (unlikely (!page) )
@@ -1408,6 +1418,10 @@ long do_memory_op(unsigned long cmd, XEN_GUEST_HANDLE_PARAM(void) arg)if (copy_from_guest (&reservation, arg,1 ) )return start_extent; +if ( op != XENMEM_populate_physmap + && (reservation.mem_flags & XENMEMF_force_heap_alloc) ) +return -EINVAL ; +/* Is size too large for us to encode a continuation? */ if ( reservation.nr_extents > (UINT_MAX >>MEMOP_EXTENT_SHIFT ) )return start_extent;
@@ -1433,6 +1447,10 @@ long do_memory_op(unsigned long cmd, XEN_GUEST_HANDLE_PARAM(void) arg) && (reservation.mem_flags & XENMEMF_populate_on_demand) ) args.memflags |= MEMF_populate_on_demand; +if ( op == XENMEM_populate_physmap + && (reservation.mem_flags & XENMEMF_force_heap_alloc) ) + args.memflags |= MEMF_force_heap_alloc; +if (xsm_memory_adjust_reservation (XSM_TARGET , curr_d, d) ) {rcu_unlock_domain (d);
xen/include/public/memory.h +5 -0
@@ -41,6 +41,11 @@#define XENMEMF_exact_node (n) (XENMEMF_node(n) | XENMEMF_exact_node_request)/* Flag to indicate the node specified is virtual node */ #define XENMEMF_vnode (1<<18) +/* +* Flag to force populate physmap to use pages from domheap instead of 1:1 +* or static allocation. +*/ +#define XENMEMF_force_heap_alloc (1<<19)#endif struct xen_memory_reservation {
xen/include/xen/mm.h +7 -0
@@ -192,6 +192,13 @@ struct npfec {/* memflags: */ #define _MEMF_no_refcount 0#define MEMF_no_refcount (1U<<_MEMF_no_refcount) +/* +* Alias of _MEMF_no_refcount to avoid introduction of a new, single-use flag. +* This flag should be used for populate_physmap() only as a re-purposing of +* _MEMF_no_refcount to force a non-1:1 allocation from domheap. +*/ +#define _MEMF_force_heap_alloc _MEMF_no_refcount +#define MEMF_force_heap_alloc (1U<<_MEMF_force_heap_alloc)#define _MEMF_populate_on_demand 1#define MEMF_populate_on_demand (1U<<_MEMF_populate_on_demand)#define _MEMF_no_dma 3
Signature · 1 line
2.34.1
Re: [PATCH v3 4/4] xen/memory, tools: Avoid hardcoding GUEST_MAGIC_BASE in init-dom0less
Quoted 23 lines · Henry Wang
xen/common/memory.c +4
-2@@ -219,7 +219,8 @@ static void populate_physmap(struct memop_args *a) }else { -if (is_domain_direct_mapped (d) ) +if (is_domain_direct_mapped (d) && + !(a->memflags & MEMF_force_heap_alloc) ) { mfn =_mfn (gpfn);@@ -246,7 +247,8 @@ static void populate_physmap(struct memop_args *a) mfn =_mfn (gpfn); } -else if (is_domain_using_staticmem (d) ) +else if (is_domain_using_staticmem (d) && + !(a->memflags & MEMF_force_heap_alloc) ) {/* * No easy way to guarantee the retrieved pages are contiguous,
xen/common/memory.c static void populate_physmap(struct memop_args *a) · line 282
Quoted 10 lines
xen/common/memory.c +7
-0@@ -271,6 +273,14 @@ static void populate_physmap(struct memop_args *a) }else { +/* +* Avoid passing MEMF_force_heap_alloc down to +* alloc_domheap_pages() where the meaning would be the +* original MEMF_no_refcount. +*/ +if (unlikely (a->memflags & MEMF_force_heap_alloc) ) +clear_bit (_MEMF_force_heap_alloc, &a->memflags);
Why an atomic operation? &= will to quite fine here. And you can also drop the if().
Quoted 12 lines
xen/common/memory.c +4
-0@@ -1408,6 +1418,10 @@ long do_memory_op(unsigned long cmd, XEN_GUEST_HANDLE_PARAM(void) arg)if (copy_from_guest (&reservation, arg,1 ) )return start_extent; +if ( op != XENMEM_populate_physmap + && (reservation.mem_flags & XENMEMF_force_heap_alloc) ) +return -EINVAL ; +/* Is size too large for us to encode a continuation? */ if ( reservation.nr_extents > (UINT_MAX >>MEMOP_EXTENT_SHIFT ) )return start_extent;
xen/common/memory.c long do_memory_op(unsigned long cmd, XEN_GUEST_HANDLE_PARAM(void) arg) · line 1452
Quoted 6 lines
xen/common/memory.c +3
-0@@ -1433,6 +1447,10 @@ long do_memory_op(unsigned long cmd, XEN_GUEST_HANDLE_PARAM(void) arg) && (reservation.mem_flags & XENMEMF_populate_on_demand) ) args.memflags |= MEMF_populate_on_demand; +if ( op == XENMEM_populate_physmap + && (reservation.mem_flags & XENMEMF_force_heap_alloc) ) + args.memflags |= MEMF_force_heap_alloc;
If in the end no new sub-op is used (see below), this and the earlier if() want combining.
You further may want to assert that the flag isn't already set (as coming back from construct_memop_from_reservation()).
xen/include/public/memory.h line 48
Quoted 11 lines
xen/include/public/memory.h +5
-0@@ -41,6 +41,11 @@#define XENMEMF_exact_node (n) (XENMEMF_node(n) | XENMEMF_exact_node_request)/* Flag to indicate the node specified is virtual node */ #define XENMEMF_vnode (1<<18) +/* +* Flag to force populate physmap to use pages from domheap instead of 1:1 +* or static allocation. +*/ +#define XENMEMF_force_heap_alloc (1<<19)
As before, a separate new sub-op would look to me as being the cleaner approach, avoiding the need to consume a bit position for something not even going to be used on all architectures.
xen/include/xen/mm.h struct npfec { · line 201
Quoted 13 lines
xen/include/xen/mm.h +7
-0@@ -192,6 +192,13 @@ struct npfec {/* memflags: */ #define _MEMF_no_refcount 0#define MEMF_no_refcount (1U<<_MEMF_no_refcount) +/* +* Alias of _MEMF_no_refcount to avoid introduction of a new, single-use flag. +* This flag should be used for populate_physmap() only as a re-purposing of +* _MEMF_no_refcount to force a non-1:1 allocation from domheap. +*/ +#define _MEMF_force_heap_alloc _MEMF_no_refcount +#define MEMF_force_heap_alloc (1U<<_MEMF_force_heap_alloc)
Given its purpose and scope, this alias wants to be local to common/memory.c.
Jan
Re: [PATCH v3 4/4] xen/memory, tools: Avoid hardcoding GUEST_MAGIC_BASE in init-dom0less
Hi Jan,
Quoted 28 lines · Jan Beulich
Quoted 27 lines · Henry Wang
xen/common/memory.c +5
-2@@ -219,7 +219,8 @@ static void populate_physmap(struct memop_args *a) }else { -if (is_domain_direct_mapped (d) ) +if (is_domain_direct_mapped (d) && + !(a->memflags & MEMF_force_heap_alloc) ) { mfn =_mfn (gpfn);@@ -246,7 +247,8 @@ static void populate_physmap(struct memop_args *a) mfn =_mfn (gpfn); } -else if (is_domain_using_staticmem (d) ) +else if (is_domain_using_staticmem (d) && + !(a->memflags & MEMF_force_heap_alloc) ) {/* * No easy way to guarantee the retrieved pages are contiguous, @@ -271,6 +273,14 @@ static void populate_physmap(struct memop_args *a) }else { +/*
Quoted 8 lines
xen/common/memory.c static void populate_physmap(struct memop_args *a) · line 282
Quoted 6 lines
xen/common/memory.c +6
-0+* Avoid passing MEMF_force_heap_alloc down to +* alloc_domheap_pages() where the meaning would be the +* original MEMF_no_refcount. +*/ +if (unlikely (a->memflags & MEMF_force_heap_alloc) ) +clear_bit (_MEMF_force_heap_alloc, &a->memflags);Why an atomic operation? &= will to quite fine here. And you can also drop the
if().
Ok, I will use &= and drop the if here.
Quoted 15 lines
Quoted 15 lines
xen/common/memory.c +4
-0@@ -1408,6 +1418,10 @@ long do_memory_op(unsigned long cmd, XEN_GUEST_HANDLE_PARAM(void) arg)if (copy_from_guest (&reservation, arg,1 ) )return start_extent; +if ( op != XENMEM_populate_physmap + && (reservation.mem_flags & XENMEMF_force_heap_alloc) ) +return -EINVAL ; +/* Is size too large for us to encode a continuation? */ if ( reservation.nr_extents > (UINT_MAX >>MEMOP_EXTENT_SHIFT ) )return start_extent;@@ -1433,6 +1447,10 @@ long do_memory_op(unsigned long cmd, XEN_GUEST_HANDLE_PARAM(void) arg) && (reservation.mem_flags & XENMEMF_populate_on_demand) ) args.memflags |= MEMF_populate_on_demand;
Quoted 8 lines
xen/common/memory.c long do_memory_op(unsigned long cmd, XEN_GUEST_HANDLE_PARAM(void) arg) · line 1452
Quoted 3 lines
xen/common/memory.c +3
-0+if ( op == XENMEM_populate_physmap + && (reservation.mem_flags & XENMEMF_force_heap_alloc) ) + args.memflags |= MEMF_force_heap_alloc;If in the end no new sub-op is used (see below), this and the earlier
if()want combining.You further may want to assert that the flag isn't already set (as coming back from
construct_memop_from_reservation()).
Ok I will check and do the combining as suggested. Thanks!
Quoted 6 lines
Quoted 6 lines
xen/include/public/memory.h +0
-0@@ -41,6 +41,11 @@#define XENMEMF_exact_node (n) (XENMEMF_node(n) | XENMEMF_exact_node_request)/* Flag to indicate the node specified is virtual node */ #define XENMEMF_vnode (1<<18)
Quoted 8 lines
xen/include/public/memory.h line 48
Quoted 5 lines
xen/include/public/memory.h +5
-0+/* +* Flag to force populate physmap to use pages from domheap instead of 1:1 +* or static allocation. +*/ +#define XENMEMF_force_heap_alloc (1<<19)As before, a separate new sub-op would look to me as being the cleaner approach, avoiding the need to consume a bit position for something not even going to be used on all architectures.
Like discussed in v2, I doubt that if introducing a new sub-op, the helpers added to duplicate mainly populate_physmap() and the toolstack helpers would be a good idea. Similarly as the way that we do for the MEMF_force_heap_alloc, if in the future we run out of the bit positions, can't we reuse this bit for different architectures as an alias? Or maybe we can even alias it now?
Quoted 6 lines
Quoted 6 lines
xen/include/xen/mm.h +0
-0@@ -192,6 +192,13 @@ struct npfec {/* memflags: */ #define _MEMF_no_refcount 0#define MEMF_no_refcount (1U<<_MEMF_no_refcount)
Quoted 8 lines
xen/include/xen/mm.h struct npfec { · line 201
Quoted 7 lines
xen/include/xen/mm.h +7
-0+/* +* Alias of _MEMF_no_refcount to avoid introduction of a new, single-use flag. +* This flag should be used for populate_physmap() only as a re-purposing of +* _MEMF_no_refcount to force a non-1:1 allocation from domheap. +*/ +#define _MEMF_force_heap_alloc _MEMF_no_refcount +#define MEMF_force_heap_alloc (1U<<_MEMF_force_heap_alloc)Given its purpose and scope, this alias wants to be local to
common/memory.c.
Sure, I will move it to common/memory.c
Kind regards,
Henry
Quoted 1 line
Jan
Re: [PATCH v3 4/4] xen/memory, tools: Avoid hardcoding GUEST_MAGIC_BASE in init-dom0less
Quoted 17 lines · Henry Wang
Quoted 15 lines · Jan Beulich
xen/include/public/memory.h line 48
Quoted 11 lines · Henry Wang
xen/include/public/memory.h +5
-0@@ -41,6 +41,11 @@#define XENMEMF_exact_node (n) (XENMEMF_node(n) | XENMEMF_exact_node_request)/* Flag to indicate the node specified is virtual node */ #define XENMEMF_vnode (1<<18) +/* +* Flag to force populate physmap to use pages from domheap instead of 1:1 +* or static allocation. +*/ +#define XENMEMF_force_heap_alloc (1<<19)As before, a separate new sub-op would look to me as being the cleaner approach, avoiding the need to consume a bit position for something not even going to be used on all architectures.
Quoted 3 lines
Like discussed in v2, I doubt that if introducing a new sub-op, the helpers added to duplicate mainly
populate_physmap()and the toolstack helpers would be a good idea.
I'm curious what amount of duplication you still see left. By suitably adding a new parameter, there should be very little left.
Quoted 4 lines
Similarly as the way that we do for the
MEMF_force_heap_alloc, if in the future we run out of the bit positions, can't we reuse this bit for different architectures as an alias? Or maybe we can even alias it now?
I view this as far less desirable in the public interface.
Jan
Re: [PATCH v3 4/4] xen/memory, tools: Avoid hardcoding GUEST_MAGIC_BASE in init-dom0less
Hi Jan,
Quoted 17 lines · Jan Beulich
Quoted 16 lines · Henry Wang
Quoted 15 lines · Jan Beulich
xen/include/public/memory.h line 48
Quoted 11 lines · Henry Wang
xen/include/public/memory.h +5
-0@@ -41,6 +41,11 @@#define XENMEMF_exact_node (n) (XENMEMF_node(n) | XENMEMF_exact_node_request)/* Flag to indicate the node specified is virtual node */ #define XENMEMF_vnode (1<<18) +/* +* Flag to force populate physmap to use pages from domheap instead of 1:1 +* or static allocation. +*/ +#define XENMEMF_force_heap_alloc (1<<19)As before, a separate new sub-op would look to me as being the cleaner approach, avoiding the need to consume a bit position for something not even going to be used on all architectures.
Quoted 5 lines
Quoted 3 lines
Like discussed in v2, I doubt that if introducing a new sub-op, the helpers added to duplicate mainly
populate_physmap()and the toolstack helpers would be a good idea.I'm curious what amount of duplication you still see left. By suitably adding a new parameter, there should be very little left.
The duplication I see so far is basically the exact xc_domain_populate_physmap(), say xc_domain_populate_physmap_heap_alloc(). In init-dom0less.c, We can replace the original call xc_domain_populate_physmap_exact() to call the newly added xc_domain_populate_physmap_heap_alloc() which evokes the new sub-op, then from the hypervisor side we set the alias MEMF flag and share the populate_physmap().
Adding a new parameter to xc_domain_populate_physmap() or maybe even xc_domain_populate_physmap_exact() is also a good idea (thanks). I was just worrying there are already too many use cases of these two functions in the existing code: there are 14 for xc_domain_populate_physmap_exact() and 8 for xc_domain_populate_physmap(). Adding a new parameter needs the update of all these and the function declaration. If you really insist this way, I can do this, sure.
Quoted 5 lines
Quoted 4 lines
Similarly as the way that we do for the
MEMF_force_heap_alloc, if in the future we run out of the bit positions, can't we reuse this bit for different architectures as an alias? Or maybe we can even alias it now?I view this as far less desirable in the public interface.
I agree.
Kind regards,
Henry
Quoted 1 line
Jan
Re: [PATCH v3 4/4] xen/memory, tools: Avoid hardcoding GUEST_MAGIC_BASE in init-dom0less
Quoted 34 lines · Henry Wang
Hi Jan,
Quoted 17 lines · Jan Beulich
Quoted 16 lines · Henry Wang
Quoted 15 lines · Jan Beulich
xen/include/public/memory.h line 48
Quoted 11 lines · Henry Wang
xen/include/public/memory.h +5
-0@@ -41,6 +41,11 @@#define XENMEMF_exact_node (n) (XENMEMF_node(n) | XENMEMF_exact_node_request)/* Flag to indicate the node specified is virtual node */ #define XENMEMF_vnode (1<<18) +/* +* Flag to force populate physmap to use pages from domheap instead of 1:1 +* or static allocation. +*/ +#define XENMEMF_force_heap_alloc (1<<19)As before, a separate new sub-op would look to me as being the cleaner approach, avoiding the need to consume a bit position for something not even going to be used on all architectures.
Quoted 5 lines
Quoted 3 lines
Like discussed in v2, I doubt that if introducing a new sub-op, the helpers added to duplicate mainly
populate_physmap()and the toolstack helpers would be a good idea.I'm curious what amount of duplication you still see left. By suitably adding a new parameter, there should be very little left.
The duplication I see so far is basically the exact
xc_domain_populate_physmap(), sayxc_domain_populate_physmap_heap_alloc(). Ininit-dom0less.c, We can replace the original callxc_domain_populate_physmap_exact()to call the newly added xc_domain_populate_physmap_heap_alloc() which evokes the new sub-op, then from the hypervisor side we set the alias MEMF flag and share thepopulate_physmap().
Quoted 8 lines
Adding a new parameter to
xc_domain_populate_physmap()or maybe evenxc_domain_populate_physmap_exact()is also a good idea (thanks). I was just worrying there are already too many use cases of these two functions in the existing code: there are 14 forxc_domain_populate_physmap_exact()and 8 forxc_domain_populate_physmap(). Adding a new parameter needs the update of all these and the function declaration. If you really insist this way, I can do this, sure.
You don't need to change all the callers. You can morph xc_domain_populate_physmap() into an internal helper, which a new trivial wrapper named xc_domain_populate_physmap() would then call, alongside with the new trivial wrapper you want to introduce.
Jan
Re: [PATCH v3 4/4] xen/memory, tools: Avoid hardcoding GUEST_MAGIC_BASE in init-dom0less
Hi Jan,
Quoted 42 lines · Jan Beulich
Quoted 33 lines · Henry Wang
Hi Jan,
Quoted 17 lines · Jan Beulich
Quoted 16 lines · Henry Wang
Quoted 15 lines · Jan Beulich
xen/include/public/memory.h line 48
Quoted 11 lines · Henry Wang
xen/include/public/memory.h +5
-0@@ -41,6 +41,11 @@#define XENMEMF_exact_node (n) (XENMEMF_node(n) | XENMEMF_exact_node_request)/* Flag to indicate the node specified is virtual node */ #define XENMEMF_vnode (1<<18) +/* +* Flag to force populate physmap to use pages from domheap instead of 1:1 +* or static allocation. +*/ +#define XENMEMF_force_heap_alloc (1<<19)As before, a separate new sub-op would look to me as being the cleaner approach, avoiding the need to consume a bit position for something not even going to be used on all architectures.
Quoted 5 lines
Quoted 3 lines
Like discussed in v2, I doubt that if introducing a new sub-op, the helpers added to duplicate mainly
populate_physmap()and the toolstack helpers would be a good idea.I'm curious what amount of duplication you still see left. By suitably adding a new parameter, there should be very little left.
The duplication I see so far is basically the exact
xc_domain_populate_physmap(), sayxc_domain_populate_physmap_heap_alloc(). Ininit-dom0less.c, We can replace the original callxc_domain_populate_physmap_exact()to call the newly added xc_domain_populate_physmap_heap_alloc() which evokes the new sub-op, then from the hypervisor side we set the alias MEMF flag and share thepopulate_physmap().Quoted 8 lines
Adding a new parameter to
xc_domain_populate_physmap()or maybe evenxc_domain_populate_physmap_exact()is also a good idea (thanks). I was just worrying there are already too many use cases of these two functions in the existing code: there are 14 forxc_domain_populate_physmap_exact()and 8 forxc_domain_populate_physmap(). Adding a new parameter needs the update of all these and the function declaration. If you really insist this way, I can do this, sure.
Quoted 4 lines
You don't need to change all the callers. You can morph
xc_domain_populate_physmap()into an internal helper, which a new trivial wrapper namedxc_domain_populate_physmap()would then call, alongside with the new trivial wrapper you want to introduce.
Thanks for the good suggestion. Would below key diff make sense to you (naming can be further discussed)? Also by checking the code, if we go this way, maybe we can even simplify the xc_domain_decrease_reservation() and xc_domain_increase_reservation()? (Although there are some hardcoded hypercall name in the error message and some small differences between the memflags)
tools/libs/ctrl/xc_domain.c +0 -0
@@ -1124,12 +1124,13 @@ int xc_domain_claim_pages(xc_interface *xch,
return err;
}
-int xc_domain_populate_physmap(xc_interface *xch,
- uint32_t domid,
- unsigned long nr_extents,
- unsigned int extent_order,
- unsigned int mem_flags,
- xen_pfn_t *extent_start)
+static int xc_populate_physmap_cmd(xc_interface *xch,
+ unsigned int cmd;
+ uint32_t domid,
+ unsigned long nr_extents,
+ unsigned int extent_order,
+ unsigned int mem_flags,
+ xen_pfn_t *extent_start)
{
int err;
DECLARE_HYPERCALL_BOUNCE(extent_start, nr_extents * sizeof(*extent_start), XC_HYPERCALL_BUFFER_BOUNCE_BOTH); @@ -1147,12 +1148,50 @@ int xc_domain_populate_physmap(xc_interface *xch,
}
set_xen_guest_handle(reservation.extent_start, extent_start);
- err = xc_memory_op(xch, XENMEM_populate_physmap, &reservation, sizeof(reservation)); + err = xc_memory_op(xch, cmd, &reservation, sizeof(reservation));
xc_hypercall_bounce_post(xch, extent_start);
return err;
}
+int xc_domain_populate_physmap(xc_interface *xch,
+ uint32_t domid,
+ unsigned long nr_extents,
+ unsigned int extent_order,
+ unsigned int mem_flags,
+ xen_pfn_t *extent_start)
+{
+ return xc_populate_physmap_cmd(xch, XENMEM_populate_physmap, domid, + nr_extents, extent_order, mem_flags, + extent_start);
+}
+
+int xc_domain_populate_physmap_heap_exact(xc_interface *xch,
+ uint32_t domid,
+ unsigned long nr_extents, + unsigned int extent_order, + unsigned int mem_flags, + xen_pfn_t *extent_start) +{
+ int err;
+
+ err = xc_populate_physmap_cmd(xch, XENMEM_populate_physmap_heapalloc, + domid, nr_extents, extent_order, mem_flags, + extent_start);
+ if ( err == nr_extents )
+ return 0;
+
+ if ( err >= 0 )
+ {
+ DPRINTF("Failed allocation for dom %d: %ld extents of order %d\n", + domid, nr_extents, extent_order);
+ errno = EBUSY;
+ err = -1;
+ }
+
+ return err;
+}
+
Kind regards,
Henry
Quoted 1 line
Jan
Re: [PATCH v3 4/4] xen/memory, tools: Avoid hardcoding GUEST_MAGIC_BASE in init-dom0less
Quoted 45 lines · Henry Wang
Hi Jan,
Quoted 42 lines · Jan Beulich
Quoted 41 lines · Henry Wang
Hi Jan,
Quoted 17 lines · Jan Beulich
Quoted 16 lines · Henry Wang
Quoted 15 lines · Jan Beulich
xen/include/public/memory.h line 48
Quoted 11 lines · Henry Wang
xen/include/public/memory.h +5
-0@@ -41,6 +41,11 @@#define XENMEMF_exact_node (n) (XENMEMF_node(n) | XENMEMF_exact_node_request)/* Flag to indicate the node specified is virtual node */ #define XENMEMF_vnode (1<<18) +/* +* Flag to force populate physmap to use pages from domheap instead of 1:1 +* or static allocation. +*/ +#define XENMEMF_force_heap_alloc (1<<19)As before, a separate new sub-op would look to me as being the cleaner approach, avoiding the need to consume a bit position for something not even going to be used on all architectures.
Quoted 5 lines
Quoted 3 lines
Like discussed in v2, I doubt that if introducing a new sub-op, the helpers added to duplicate mainly
populate_physmap()and the toolstack helpers would be a good idea.I'm curious what amount of duplication you still see left. By suitably adding a new parameter, there should be very little left.
The duplication I see so far is basically the exact
xc_domain_populate_physmap(), sayxc_domain_populate_physmap_heap_alloc(). Ininit-dom0less.c, We can replace the original callxc_domain_populate_physmap_exact()to call the newly added xc_domain_populate_physmap_heap_alloc() which evokes the new sub-op, then from the hypervisor side we set the alias MEMF flag and share thepopulate_physmap().Adding a new parameter to
xc_domain_populate_physmap()or maybe evenxc_domain_populate_physmap_exact()is also a good idea (thanks). I was just worrying there are already too many use cases of these two functions in the existing code: there are 14 forxc_domain_populate_physmap_exact()and 8 forxc_domain_populate_physmap(). Adding a new parameter needs the update of all these and the function declaration. If you really insist this way, I can do this, sure.
Quoted 6 lines
Quoted 4 lines
You don't need to change all the callers. You can morph
xc_domain_populate_physmap()into an internal helper, which a new trivial wrapper namedxc_domain_populate_physmap()would then call, alongside with the new trivial wrapper you want to introduce.Thanks for the good suggestion. Would below key diff make sense to you
Yes.
Quoted 1 line
(naming can be further discussed)?
Personally I wouldn't use xc_ on internal helpers. But for guidance on naming in the libraries the maintainer(s) would need consulting.
Quoted 5 lines
Also by checking the code, if we go
this way, maybe we can even simplify thexc_domain_decrease_reservation()andxc_domain_increase_reservation()? (Although there are some hardcoded hypercall name in the error message and some small differences between the memflags)
There may be room for improvement there, but as you say, some care would need applying.
Jan
Re: [PATCH v3 4/4] xen/memory, tools: Avoid hardcoding GUEST_MAGIC_BASE in init-dom0less
Hi Jan,
Quoted 46 lines · Jan Beulich
Quoted 45 lines · Henry Wang
Hi Jan,
Quoted 42 lines · Jan Beulich
Quoted 41 lines · Henry Wang
Hi Jan,
Quoted 17 lines · Jan Beulich
Quoted 16 lines · Henry Wang
Quoted 15 lines · Jan Beulich
xen/include/public/memory.h line 48
Quoted 11 lines · Henry Wang
xen/include/public/memory.h +5
-0@@ -41,6 +41,11 @@#define XENMEMF_exact_node (n) (XENMEMF_node(n) | XENMEMF_exact_node_request)/* Flag to indicate the node specified is virtual node */ #define XENMEMF_vnode (1<<18) +/* +* Flag to force populate physmap to use pages from domheap instead of 1:1 +* or static allocation. +*/ +#define XENMEMF_force_heap_alloc (1<<19)As before, a separate new sub-op would look to me as being the cleaner approach, avoiding the need to consume a bit position for something not even going to be used on all architectures.
Quoted 5 lines
Quoted 3 lines
Like discussed in v2, I doubt that if introducing a new sub-op, the helpers added to duplicate mainly
populate_physmap()and the toolstack helpers would be a good idea.I'm curious what amount of duplication you still see left. By suitably adding a new parameter, there should be very little left.
The duplication I see so far is basically the exact
xc_domain_populate_physmap(), sayxc_domain_populate_physmap_heap_alloc(). Ininit-dom0less.c, We can replace the original callxc_domain_populate_physmap_exact()to call the newly added xc_domain_populate_physmap_heap_alloc() which evokes the new sub-op, then from the hypervisor side we set the alias MEMF flag and share thepopulate_physmap().Adding a new parameter to
xc_domain_populate_physmap()or maybe evenxc_domain_populate_physmap_exact()is also a good idea (thanks). I was just worrying there are already too many use cases of these two functions in the existing code: there are 14 forxc_domain_populate_physmap_exact()and 8 forxc_domain_populate_physmap(). Adding a new parameter needs the update of all these and the function declaration. If you really insist this way, I can do this, sure.
Quoted 6 lines
Quoted 5 lines
Quoted 4 lines
You don't need to change all the callers. You can morph
xc_domain_populate_physmap()into an internal helper, which a new trivial wrapper namedxc_domain_populate_physmap()would then call, alongside with the new trivial wrapper you want to introduce.Thanks for the good suggestion. Would below key diff make sense to you
Yes.
Thanks for confirming!
Quoted 3 lines
Quoted 1 line
(naming can be further discussed)?
Personally I wouldn't use
xc_on internal helpers. But for guidance on naming in the libraries the maintainer(s) would need consulting.
Sure, thanks for sharing your idea. Well I added xc_ prefix because seeing static int xc_domain_pod_target() & int xc_domain_{set, get}_pod_target() etc. pairs in the existing code of the file. I agree that the opinion of libxc/toolstack maintainers matter too, and hence we can continue this discussion in the formal v4.
Quoted 7 lines
Quoted 5 lines
Also by checking the code, if we go
this way, maybe we can even simplify thexc_domain_decrease_reservation()andxc_domain_increase_reservation()? (Although there are some hardcoded hypercall name in the error message and some small differences between the memflags)There may be room for improvement there, but as you say, some care would need applying.
Yeah...not really sure if it is ok (TBH I think not) to do it all together in this patch, I will probably add another patch to clean-up all these duplications when sending v4 (depending on how far I can go).
Kind regards,
Henry
Quoted 1 line
Jan