mini-os: some memory map updates for PVH
Subsystems (2)
Juergen Gross
maintainerv3 cover mini-os: some memory map updates for PVH
Do some memory map related changes/fixes for PVH mode:
- Prefer the memory map delivered via start-info over the one obtained from the hypervisor. This is a prerequisite for Xenstore-stubdom live-update with rising the memory limit.
- Fix a bug related to ballooning in PVH mode: PVH Xenstore-stubdom can't read its target memory size from Xenstore, as this introduces a chicken-and-egg problem. The memory size read from the hypervisor OTOH includes additional "special" pages marked as reserved in the memory map. Those pages need to be subtracted from the read size.
- Fix a bug in ballooning code in PVH mode when using memory beyond a RAM hole in the memory map
Changes in V3:
- minor comment for patch 3 addressed
Changes in V2:
- added patch 4
- addressed comment regarding patch 3
Juergen Gross (4):
mini-os: take newest version of arch-x86/hvm/start_info.h
mini-os: prefer memory map via start_info for PVH
mini-os: fix number of pages for PVH
mini-os: fix bug in ballooning on PVH
arch/x86/mm.c | 23 ++++---- balloon.c | 18 ++---- e820.c | 83 ++++++++++++++++++++++++--- include/e820.h | 6 ++ include/x86/arch_mm.h | 2 + include/xen/arch-x86/hvm/start_info.h | 63 +++++++++++++++++++- 6 files changed, 163 insertions(+), 32 deletions(-)
Signature · 1 line
2.35.3
Juergen Gross
maintainerPATCH v3 1/4 mini-os: take newest version of arch-x86/hvm/start_info.h
Update include/xen/arch-x86/hvm/start_info.h to the newest version from the Xen tree.
--- include/xen/arch-x86/hvm/start_info.h | 63 ++++++++++++++++++++++++++- 1 file changed, 62 insertions(+), 1 deletion(-)
include/xen/arch-x86/hvm/start_info.h +62 -1
@@ -33,7 +33,7 @@* | magic | Contains the magic value XEN_HVM_START_MAGIC_VALUE * | | ("xEn3" with the 0x80 bit of the "E" set). * 4 +----------------+ -* | version | Version of this structure. Current version is 0. New +* | version | Version of this structure. Current version is 1. New * | | versions are guaranteed to be backwards-compatible. * 8 +----------------+ * | flags | SIF_xxx flags.
@@ -48,6 +48,15 @@* 32 +----------------+ * | rsdp_paddr | Physical address of the RSDP ACPI data structure. * 40 +----------------+ +* | memmap_paddr | Physical address of the (optional) memory map. Only +* | | present in version 1 and newer of the structure. +* 48 +----------------+ +* | memmap_entries | Number of entries in the memory map table. Zero +* | | if there is no memory map being provided. Only +* | | present in version 1 and newer of the structure. +* 52 +----------------+ +* | reserved | Version 1 and newer only. +* 56 +----------------+ * * The layout of each entry in the module structure is the following: *
@@ -62,13 +71,51 @@* | reserved | * 32 +----------------+ * +* The layout of each entry in the memory map table is as follows: +* +* 0 +----------------+ +* | addr | Base address +* 8 +----------------+ +* | size | Size of mapping in bytes +* 16 +----------------+ +* | type | Type of mapping as defined between the hypervisor +* | | and guest. See XEN_HVM_MEMMAP_TYPE_* values below. +* 20 +----------------| +* | reserved | +* 24 +----------------+ +* * The address and sizes are always a 64bit little endian unsigned integer. * * NB: Xen on x86 will always try to place all the data below the 4GiB * boundary. +* +* Version numbers of the hvm_start_info structure have evolved like this: +* +* Version 0: Initial implementation. +* +* Version 1: Added the memmap_paddr/memmap_entries fields (plus 4 bytes of +* padding) to the end of the hvm_start_info struct. These new +* fields can be used to pass a memory map to the guest. The +* memory map is optional and so guests that understand version 1 +* of the structure must check that memmap_entries is non-zero +* before trying to read the memory map. */ #define XEN_HVM_START_MAGIC_VALUE 0x336ec578 +/* +* The values used in the type field of the memory map table entries are +* defined below and match the Address Range Types as defined in the "System +* Address Map Interfaces" section of the ACPI Specification. Please refer to +* section 15 in version 6.2 of the ACPI spec: http://uefi.org/specifications +*/ +#define XEN_HVM_MEMMAP_TYPE_RAM 1 +#define XEN_HVM_MEMMAP_TYPE_RESERVED 2 +#define XEN_HVM_MEMMAP_TYPE_ACPI 3 +#define XEN_HVM_MEMMAP_TYPE_NVS 4 +#define XEN_HVM_MEMMAP_TYPE_UNUSABLE 5 +#define XEN_HVM_MEMMAP_TYPE_DISABLED 6 +#define XEN_HVM_MEMMAP_TYPE_PMEM 7 +/* * C representation of the x86/HVM start info layout. *
@@ -86,6 +133,13 @@ struct hvm_start_info {uint64_t cmdline_paddr;/* Physical address of the command line. */ uint64_t rsdp_paddr;/* Physical address of the RSDP ACPI data */ /* structure. */ +/* All following fields only present in version 1 and newer */ +uint64_t memmap_paddr;/* Physical address of an array of */ +/* hvm_memmap_table_entry. */ +uint32_t memmap_entries;/* Number of entries in the memmap table. */ +/* Value will be zero if there is no memory */ +/* map being provided. */ +uint32_t reserved;/* Must be zero. */ };struct hvm_modlist_entry {
@@ -95,4 +149,11 @@ struct hvm_modlist_entry { uint64_t reserved; }; +struct hvm_memmap_table_entry { +uint64_t addr;/* Base address of the memory region */ +uint64_t size;/* Size of the memory region in bytes */ +uint32_t type;/* Mapping type */ +uint32_t reserved;/* Must be zero for Version 1. */ +}; +#endif /* __XEN_PUBLIC_ARCH_X86_HVM_START_INFO_H__ */
Signature · 1 line
2.35.3
Juergen Gross
maintainerPATCH v3 2/4 mini-os: prefer memory map via start_info for PVH
In reply to Juergen GrossSince some time now a guest started in PVH mode will get the memory map from Xen via the start_info structure.
Modify the PVH initialization to prefer this memory map over the one obtained via hypercall, as this will allow to add information to the memory map for a new kernel when supporting kexec.
In case the start_info structure doesn't contain memory map information fall back to the hypercall.
--- arch/x86/mm.c | 6 ++++++ e820.c | 25 +++++++++++++++++++++++++ include/e820.h | 4 ++++ 3 files changed, 35 insertions(+)
arch/x86/mm.c +6 -0
@@ -45,6 +45,7 @@#include <mini-os/xmalloc.h> #include <mini-os/e820.h> #include <xen/memory.h> +#include <xen/arch-x86/hvm/start_info.h> #ifdef MM_DEBUG #define DEBUG(_f, _a...) \
@@ -108,6 +109,11 @@ void arch_mm_preinit(void *p) {long ret;domid_t domid =DOMID_SELF ; +struct hvm_start_info *hsi = p; + +if ( hsi->version >=1 && hsi->memmap_entries >0 ) +e820_init_memmap ((struct hvm_memmap_table_entry *)(unsigned long ) + hsi->memmap_paddr, hsi->memmap_entries); pt_base = page_table_base; first_free_pfn =PFN_UP (to_phys (&_end));
e820.c +25 -0
@@ -54,6 +54,7 @@ static char *e820_types[E820_TYPES] = { [E820_ACPI] = "ACPI", [E820_NVS] = "NVS", [E820_UNUSABLE] = "Unusable", + [E820_DISABLED] = "Disabled", [E820_PMEM] = "PMEM" };
@@ -259,6 +260,30 @@ static void e820_get_memmap(void)e820_sanitize (); } +void e820_init_memmap (struct hvm_memmap_table_entry *entry,unsigned int num) +{ +unsigned int i; + +BUILD_BUG_ON (XEN_HVM_MEMMAP_TYPE_RAM !=E820_RAM ); +BUILD_BUG_ON (XEN_HVM_MEMMAP_TYPE_RESERVED !=E820_RESERVED ); +BUILD_BUG_ON (XEN_HVM_MEMMAP_TYPE_ACPI !=E820_ACPI ); +BUILD_BUG_ON (XEN_HVM_MEMMAP_TYPE_NVS !=E820_NVS ); +BUILD_BUG_ON (XEN_HVM_MEMMAP_TYPE_UNUSABLE !=E820_UNUSABLE ); +BUILD_BUG_ON (XEN_HVM_MEMMAP_TYPE_DISABLED !=E820_DISABLED ); +BUILD_BUG_ON (XEN_HVM_MEMMAP_TYPE_PMEM !=E820_PMEM ); + +for ( i =0 ; i < num; i++ ) + { + e820_map[i].addr = entry[i].addr; + e820_map[i].size = entry[i].size; + e820_map[i].type = entry[i].type; + } + + e820_entries = num; + +e820_sanitize (); +} +void arch_print_memmap (void ) {int i;
include/e820.h +4 -0
@@ -26,6 +26,8 @@#if defined(__arm__) || defined(__aarch64__) || defined(CONFIG_PARAVIRT )#define CONFIG_E820_TRIVIAL +#else +#include <xen/arch-x86/hvm/start_info.h> #endif /* PC BIOS standard E820 types and structure. */
@@ -34,6 +36,7 @@#define E820_ACPI 3#define E820_NVS 4#define E820_UNUSABLE 5 +#define E820_DISABLED 6#define E820_PMEM 7#define E820_TYPES 8
@@ -54,6 +57,7 @@ unsigned long e820_get_max_contig_pages(unsigned long pfn, unsigned long pages);#ifndef CONFIG_E820_TRIVIAL unsigned long e820_get_reserved_pfns (int pages);void e820_put_reserved_pfns (unsigned long start_pfn,int pages); +void e820_init_memmap (struct hvm_memmap_table_entry *entry,unsigned int num);#endif #endif /*__E820_HEADER*/
Signature · 1 line
2.35.3
Juergen Gross
maintainerPATCH v3 3/4 mini-os: fix number of pages for PVH
In reply to Juergen GrossWhen getting the current allocation from Xen, this value includes the pages allocated in the MMIO area. Fix the highest available RAM page by subtracting the size of that area.
This requires to read the E820 map before needing this value. Add two functions returning the current and the maximum number of RAM pages taking this correction into account.
At the same time add the LAPIC page to the memory map in order to avoid reusing that PFN for internal purposes.
---
V2:
- make e820_initial_reserved_pfns static (Samuel Thibault)
- add e820_get_current_pages() and e820_get_max_pages()
V3:
- change return type of e820_get_current_pages() and e820_get_max_pages() to unsigned long (Samuel Thibault)
--- arch/x86/mm.c | 17 +++++-------- balloon.c | 16 +++--------- e820.c | 58 +++++++++++++++++++++++++++++++++++++------ include/e820.h | 2 ++ include/x86/arch_mm.h | 2 ++ 5 files changed, 65 insertions(+), 30 deletions(-)
arch/x86/mm.c +6 -11
@@ -107,25 +107,20 @@ desc_ptr idt_ptr =void arch_mm_preinit (void *p) { -long ret; -domid_t domid =DOMID_SELF ; +unsigned int pages;struct hvm_start_info *hsi = p;if ( hsi->version >=1 && hsi->memmap_entries >0 )e820_init_memmap ((struct hvm_memmap_table_entry *)(unsigned long ) hsi->memmap_paddr, hsi->memmap_entries); +else +e820_init_memmap (NULL ,0 ); pt_base = page_table_base; first_free_pfn =PFN_UP (to_phys (&_end)); - ret =HYPERVISOR_memory_op (XENMEM_current_reservation, &domid); -if ( ret <0 ) - { -xprintk ("could not get memory size\n" ); -do_exit (); - } - - last_free_pfn =e820_get_maxpfn (ret); -balloon_set_nr_pages (ret, last_free_pfn); + pages =e820_get_current_pages (); + last_free_pfn =e820_get_maxpfn (pages); +balloon_set_nr_pages (pages, last_free_pfn); }#endif
balloon.c +4 -12
@@ -44,20 +44,12 @@ void balloon_set_nr_pages(unsigned long pages, unsigned long pfn)void get_max_pages (void ) { -long ret; -domid_t domid =DOMID_SELF ; - - ret =HYPERVISOR_memory_op (XENMEM_maximum_reservation, &domid); -if ( ret <0 ) + nr_max_pages =e820_get_max_pages (); +if ( nr_max_pages ) { -printk ("Could not get maximum pfn\n" ); -return ; +printk ("Maximum memory size: %ld pages\n" , nr_max_pages); + nr_max_pfn =e820_get_maxpfn (nr_max_pages); } - - nr_max_pages = ret; -printk ("Maximum memory size: %ld pages\n" , nr_max_pages); - - nr_max_pfn =e820_get_maxpfn (nr_max_pages); }void mm_alloc_bitmap_remap (void )
e820.c +51 -7
96 lines not on this page · raw message
include/e820.h +2 -0
@@ -52,6 +52,8 @@ struct __packed e820entry {extern struct e820entry e820_map[];extern unsigned e820_entries; +unsigned long e820_get_current_pages (void ); +unsigned long e820_get_max_pages (void );unsigned long e820_get_maxpfn (unsigned long pages);unsigned long e820_get_max_contig_pages (unsigned long pfn,unsigned long pages);#ifndef CONFIG_E820_TRIVIAL
include/x86/arch_mm.h +2 -0
@@ -207,6 +207,8 @@ typedef unsigned long pgentry_t;/* to align the pointer to the (next) page boundary */ #define PAGE_ALIGN (addr) (((addr)+PAGE_SIZE-1)&PAGE_MASK) +#define LAPIC_ADDRESS CONST(0xfee00000) +#ifndef __ASSEMBLY__/* Definitions for machine and pseudophysical addresses. */ #ifdef __i386__
Signature · 1 line
2.35.3
Re: [PATCH v3 3/4] mini-os: fix number of pages for PVH
Quoted 9 lines · Juergen Gross
e820.c +1
-0@@ -29,6 +29,38 @@#include <mini-os/e820.h> #include <xen/memory.h> +static unsigned long e820_initial_reserved_pfns;
e820.c line 42
Quoted 10 lines
e820.c +10
-0+ +unsigned long e820_get_current_pages (void ) +{ +domid_t domid =DOMID_SELF ; +long ret; + + ret =HYPERVISOR_memory_op (XENMEM_current_reservation, &domid); +if ( ret <0 ) + { +xprintk ("could not get memory size\n" );
%ld ret
Also, xprintk() vs ...
Quoted 5 lines
e820.c +5
-0+do_exit (); + } + +return ret - e820_initial_reserved_pfns; +}
e820.c line 57
Quoted 10 lines
e820.c +10
-0+ +unsigned long e820_get_max_pages (void ) +{ +domid_t domid =DOMID_SELF ; +long ret; + + ret =HYPERVISOR_memory_op (XENMEM_maximum_reservation, &domid); +if ( ret <0 ) + { +printk ("Could not get maximum pfn\n" );
... printk()?
Shouldn't they both be printk()? Can fix both issues on commit.
~Andrew
Juergen Gross
maintainerRe: [PATCH v3 3/4] mini-os: fix number of pages for PVH
Quoted 43 lines · Andrew Cooper
Quoted 9 lines · Juergen Gross
e820.c +1
-0@@ -29,6 +29,38 @@#include <mini-os/e820.h> #include <xen/memory.h> +static unsigned long e820_initial_reserved_pfns;e820.c line 42
Quoted 10 lines
e820.c +10
-0+ +unsigned long e820_get_current_pages (void ) +{ +domid_t domid =DOMID_SELF ; +long ret; + + ret =HYPERVISOR_memory_op (XENMEM_current_reservation, &domid); +if ( ret <0 ) + { +xprintk ("could not get memory size\n" );%ld ret
Also,
xprintk()vs ...Quoted 5 lines
e820.c +5
-0+do_exit (); + } + +return ret - e820_initial_reserved_pfns; +}e820.c line 57
Quoted 10 lines
e820.c +10
-0+ +unsigned long e820_get_max_pages (void ) +{ +domid_t domid =DOMID_SELF ; +long ret; + + ret =HYPERVISOR_memory_op (XENMEM_maximum_reservation, &domid); +if ( ret <0 ) + { +printk ("Could not get maximum pfn\n" );...
printk()?
Quoted 1 line
Shouldn't they both be printk()? Can fix both issues on commit.
e820_get_current_pages() is being called before console initialization, so it should really use xprintk().
Adding the returned error might be interesting, though (even if an error is very unlikely).
Note that the error message printing has just been moved from the previous use case into the new function.
Juergen
Juergen Gross
maintainerPATCH v3 4/4 mini-os: fix bug in ballooning on PVH
In reply to Juergen GrossThere is a subtle bug in ballooning code for PVH: in case ballooning extends above a non-RAM area of the memory map, wrong pages will be used.
---
V2:
- new patch
--- balloon.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
balloon.c +1 -1
@@ -124,7 +124,7 @@ int balloon_up(unsigned long n_pages)for ( pfn =0 ; pfn < rc; pfn++ ) {arch_pfn_add (start_pfn + pfn, balloon_frames[pfn]); -free_page (pfn_to_virt (nr_mem_pages + pfn)); +free_page (pfn_to_virt (start_pfn + pfn)); } nr_mem_pages += rc;
Signature · 1 line
2.35.3
Juergen Gross
maintainerQuoted 28 lines · Juergen Gross
Do some memory map related changes/fixes for PVH mode:
- Prefer the memory map delivered via start-info over the one obtained from the hypervisor. This is a prerequisite for Xenstore-stubdom live-update with rising the memory limit.
- Fix a bug related to ballooning in PVH mode: PVH Xenstore-stubdom can't read its target memory size from Xenstore, as this introduces a chicken-and-egg problem. The memory size read from the hypervisor OTOH includes additional "special" pages marked as reserved in the memory map. Those pages need to be subtracted from the read size.
- Fix a bug in ballooning code in PVH mode when using memory beyond a RAM hole in the memory map
Changes in V3:
- minor comment for patch 3 addressedChanges in V2:
- added patch 4
- addressed comment regarding patch 3Juergen Gross (4):
mini-os: take newest version ofarch-x86/hvm/start_info.h
mini-os: prefer memory map viastart_infofor PVH
mini-os: fix number of pages for PVH
mini-os: fix bug in ballooning on PVH
Quoted 8 lines
arch/x86/mm.c | 23 ++++---- balloon.c | 18 ++---- e820.c | 83 ++++++++++++++++++++++++--- include/e820.h | 6 ++ include/x86/arch_mm.h | 2 + include/xen/arch-x86/hvm/start_info.h | 63 +++++++++++++++++++- 6 files changed, 163 insertions(+), 32 deletions(-)
Could someone please commit this series? Samuel gave his R-b nearly 2 weeks ago.
Juergen
Hi Juergen,
Quoted 38 lines · Juergen Gross
Quoted 28 lines · Juergen Gross
Do some memory map related changes/fixes for PVH mode:
- Prefer the memory map delivered via start-info over the one obtained from the hypervisor. This is a prerequisite for Xenstore-stubdom live-update with rising the memory limit.
- Fix a bug related to ballooning in PVH mode: PVH Xenstore-stubdom can't read its target memory size from Xenstore, as this introduces a chicken-and-egg problem. The memory size read from the hypervisor OTOH includes additional "special" pages marked as reserved in the memory map. Those pages need to be subtracted from the read size.
- Fix a bug in ballooning code in PVH mode when using memory beyond a RAM hole in the memory map
Changes in V3:
- minor comment for patch 3 addressedChanges in V2:
- added patch 4
- addressed comment regarding patch 3Juergen Gross (4):
mini-os: take newest version ofarch-x86/hvm/start_info.h
mini-os: prefer memory map viastart_infofor PVH
mini-os: fix number of pages for PVH
mini-os: fix bug in ballooning on PVHQuoted 8 lines
arch/x86/mm.c | 23 ++++---- balloon.c | 18 ++----
e820.c | 83 ++++++++++++++++++++++++--- include/e820.h | 6 ++ include/x86/arch_mm.h | 2 +include/xen/arch-x86/hvm/start_info.h| 63 +++++++++++++++++++-6 files changed, 163 insertions(+), 32 deletions(-)
Quoted 1 line
Could someone please commit this series?
Done.
Quoted 2 lines
Samuel gave his R-b nearly 2 weeks
ago.
xen-devel is quite high volume and I don't read all the e-mails. If you need someone to commit then please CC committers@ and this will land in my inbox.
Cheers,
Signature · 1 line
Julien Grall
Juergen Gross
maintainerQuoted 41 lines · Julien Grall
Hi Juergen,
Quoted 38 lines · Juergen Gross
Quoted 36 lines · Juergen Gross
Do some memory map related changes/fixes for PVH mode:
- Prefer the memory map delivered via start-info over the one obtained from the hypervisor. This is a prerequisite for Xenstore-stubdom live-update with rising the memory limit.
- Fix a bug related to ballooning in PVH mode: PVH Xenstore-stubdom can't read its target memory size from Xenstore, as this introduces a chicken-and-egg problem. The memory size read from the hypervisor OTOH includes additional "special" pages marked as reserved in the memory map. Those pages need to be subtracted from the read size.
- Fix a bug in ballooning code in PVH mode when using memory beyond a RAM hole in the memory map
Changes in V3:
- minor comment for patch 3 addressedChanges in V2:
- added patch 4
- addressed comment regarding patch 3Juergen Gross (4):
mini-os: take newest version ofarch-x86/hvm/start_info.h
mini-os: prefer memory map viastart_infofor PVH
mini-os: fix number of pages for PVH
mini-os: fix bug in ballooning on PVHarch/x86/mm.c | 23 ++++---- balloon.c | 18 ++----
e820.c | 83 ++++++++++++++++++++++++--- include/e820.h | 6 ++ include/x86/arch_mm.h | 2 +include/xen/arch-x86/hvm/start_info.h| 63 +++++++++++++++++++-6 files changed, 163 insertions(+), 32 deletions(-)
Quoted 8 lines
Quoted 1 line
Could someone please commit this series?
Done.
Quoted 2 lines
Samuel gave his R-b nearly 2 weeks
ago.xen-devel is quite high volume and I don't read all the e-mails. If you need someone to commit then please CC committers@ and this will land in my inbox.
This is the reason why Wei wanted to be CC-ed for Mini-OS patches in the past. He doesn't seem to be interested any more, unfortunately...
Juergen
Quoted 46 lines · Juergen Gross
Quoted 45 lines · Julien Grall
Hi Juergen,
Quoted 38 lines · Juergen Gross
Quoted 28 lines · Juergen Gross
Do some memory map related changes/fixes for PVH mode:
- Prefer the memory map delivered via start-info over the one obtained from the hypervisor. This is a prerequisite for Xenstore-stubdom live-update with rising the memory limit.
- Fix a bug related to ballooning in PVH mode: PVH Xenstore-stubdom can't read its target memory size from Xenstore, as this introduces a chicken-and-egg problem. The memory size read from the hypervisor OTOH includes additional "special" pages marked as reserved in the memory map. Those pages need to be subtracted from the read size.
- Fix a bug in ballooning code in PVH mode when using memory beyond a RAM hole in the memory map
Changes in V3:
- minor comment for patch 3 addressedChanges in V2:
- added patch 4
- addressed comment regarding patch 3Juergen Gross (4):
mini-os: take newest version ofarch-x86/hvm/start_info.h
mini-os: prefer memory map viastart_infofor PVH
mini-os: fix number of pages for PVH
mini-os: fix bug in ballooning on PVHQuoted 8 lines
arch/x86/mm.c | 23 ++++---- balloon.c | 18 ++----
e820.c | 83 ++++++++++++++++++++++++--- include/e820.h | 6 ++ include/x86/arch_mm.h | 2 +include/xen/arch-x86/hvm/start_info.h| 63 +++++++++++++++++++-6 files changed, 163 insertions(+), 32 deletions(-)Quoted 1 line
Could someone please commit this series?
Done.
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
Samuel gave his R-b nearly 2 weeks
ago.xen-devel is quite high volume and I don't read all the e-mails. If you need someone to commit then please CC committers@ and this will land in my inbox.
This is the reason why Wei wanted to be CC-ed for Mini-OS patches in the past. He doesn't seem to be interested any more, unfortunately...
And really, with no other indication so far, I've been skipping mini-os patches on the assumption that Wei would be looking after them. Wei, if you can't do so anymore, could you please explicitly indicate so?
Jan