Skip to contentSkip to conversation

libxl: fix memory map reported to PVH guests

PATCH In review Roger Pau Monne 3 participants 3 messages Apr 20, 2018 – Apr 23, 2018

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 libxl: fix memory map reported to PVH guests

PVH guests with 4GB of RAM or more get a memory map like the following:

0x00000000000000 - 0x000000fee00000 RAM
0x000000fee00000 - 0x00000100000000 RESERVED
0x000000fc009000 - 0x000000fc009040 ACPI
0x000000fc000000 - 0x000000fc001000 ACPI
0x000000fc001000 - 0x000000fc009000 ACPI
0x00000100000000 - 0x000001fb200400 RAM

This is wrong because ACPI regions overlap with RAM regions. The cause of this issue is not setting a big enough MMIO hole and marking the whole MMIO hole as reserved, when it actually contains several pieces:

- local APIC page.
- ACPI tables.
- HVM special pages.

Of those items only HVM special pages need to be marked as reserved in order to advise the guest against using them for example for memory hotplug.

After the fix the layout reported for the same guest is:

0x00000000000000 - 0x000000fc000000 RAM
0x000000feff8000 - 0x000000ff000000 RESERVED
0x000000fc009000 - 0x000000fc009040 ACPI
0x000000fc000000 - 0x000000fc001000 ACPI
0x000000fc001000 - 0x000000fc009000 ACPI
0x00000100000000 - 0x000001fe000400 RAM

---
Cc: Ian Jackson <[email protected]>
Cc: Wei Liu <[email protected]>
Cc: Juergen Gross <[email protected]>
---
Changes since v1:
- Only mark HVM special pages as reserved, in order to avoid overlap of the memory map regions.

---
 tools/libxl/libxl_arch.h     |  1 +
 tools/libxl/libxl_dom.c      | 20 +++++++++++---------
 tools/libxl/libxl_x86.c      | 13 +++++++------
 tools/libxl/libxl_x86_acpi.c |  1 -
 4 files changed, 19 insertions(+), 16 deletions(-)
tools/libxl/libxl_arch.h +1 -0
index 318c111bb4..74a5af3cf3 100644
@@ -76,6 +76,7 @@ int libxl__arch_extra_memory(libxl__gc *gc,
 #if defined(__i386__) || defined(__x86_64__)
 
 #define LAPIC_BASE_ADDRESS  0xfee00000
+#define ACPI_INFO_PHYSICAL_ADDRESS 0xfc000000
 
 int libxl__dom_load_acpi(libxl__gc *gc,
                          const libxl_domain_build_info *b_info,
tools/libxl/libxl_dom.c +11 -9
index 8c3607b0f5..f0fd5fd3a3 100644
@@ -1229,15 +1229,17 @@ int libxl__build_hvm(libxl__gc *gc, uint32_t domid,
         dom->mmio_size = HVM_BELOW_4G_MMIO_LENGTH;
     else if (dom->mmio_size == 0 && !device_model) {
 #if defined(__i386__) || defined(__x86_64__)
-        if (libxl_defbool_val(info->apic)) {
-            /* Make sure LAPIC_BASE_ADDRESS is below special pages */
-            assert(((((X86_HVM_END_SPECIAL_REGION - X86_HVM_NR_SPECIAL_PAGES)
-                      << XC_PAGE_SHIFT) - LAPIC_BASE_ADDRESS)) >= XC_PAGE_SIZE);
-            dom->mmio_size = GB(4) - LAPIC_BASE_ADDRESS;
-        } else
-            dom->mmio_size = GB(4) -
-                ((X86_HVM_END_SPECIAL_REGION - X86_HVM_NR_SPECIAL_PAGES)
-                 << XC_PAGE_SHIFT);
+        /*
+         * Make sure the local APIC page, the ACPI tables and the special pages
+         * are inside the MMIO hole.
+         */
+        xen_paddr_t start =
+            (X86_HVM_END_SPECIAL_REGION - X86_HVM_NR_SPECIAL_PAGES) <<
+            XC_PAGE_SHIFT;
+
+        start = min_t(xen_paddr_t, start, LAPIC_BASE_ADDRESS);
+        start = min_t(xen_paddr_t, start, ACPI_INFO_PHYSICAL_ADDRESS);
+        dom->mmio_size = GB(4) - start;
 #else
         assert(1);
 #endif
tools/libxl/libxl_x86.c +7 -6
index 4573d6764d..ab88562619 100644
@@ -512,8 +512,8 @@ static int domain_construct_memmap(libxl__gc *gc,
         if (d_config->rdms[i].policy != LIBXL_RDM_RESERVE_POLICY_INVALID)
             e820_entries++;
 
-    /* Add mmio entry for PVH. */
-    if (dom->mmio_size && d_config->b_info.type == LIBXL_DOMAIN_TYPE_PVH)
+    /* Add the HVM special pages to PVH memmap as RESERVED. */
+    if (d_config->b_info.type == LIBXL_DOMAIN_TYPE_PVH)
         e820_entries++;
 
     /* If we should have a highmem range. */
@@ -549,10 +549,11 @@ static int domain_construct_memmap(libxl__gc *gc,
         nr++;
     }
 
-    /* mmio area */
-    if (dom->mmio_size && d_config->b_info.type == LIBXL_DOMAIN_TYPE_PVH) {
-        e820[nr].addr = dom->mmio_start;
-        e820[nr].size = dom->mmio_size;
+    /* HVM special pages */
+    if (d_config->b_info.type == LIBXL_DOMAIN_TYPE_PVH) {
+        e820[nr].addr = (X86_HVM_END_SPECIAL_REGION - X86_HVM_NR_SPECIAL_PAGES)
+                        << XC_PAGE_SHIFT;
+        e820[nr].size = X86_HVM_NR_SPECIAL_PAGES << XC_PAGE_SHIFT;
         e820[nr].type = E820_RESERVED;
         nr++;
     }
tools/libxl/libxl_x86_acpi.c +0 -1
index 143ce66644..ed6610c84e 100644
@@ -22,7 +22,6 @@
 
  /* Number of pages holding ACPI tables */
 #define NUM_ACPI_PAGES 16
-#define ACPI_INFO_PHYSICAL_ADDRESS 0xfc000000
 
 struct libxl_acpi_ctxt {
     struct acpi_ctxt c;
Signature · 7 lines
2.17.0


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#1 lore raw
Quoted 24 lines · Roger Pau Monne

PVH guests with 4GB of RAM or more get a memory map like the following:

0x00000000000000 - 0x000000fee00000 RAM
0x000000fee00000 - 0x00000100000000 RESERVED
0x000000fc009000 - 0x000000fc009040 ACPI
0x000000fc000000 - 0x000000fc001000 ACPI
0x000000fc001000 - 0x000000fc009000 ACPI
0x00000100000000 - 0x000001fb200400 RAM

This is wrong because ACPI regions overlap with RAM regions. The cause of this issue is not setting a big enough MMIO hole and marking the whole MMIO hole as reserved, when it actually contains several pieces:

- local APIC page.
- ACPI tables.
- HVM special pages.

Of those items only HVM special pages need to be marked as reserved in order to advise the guest against using them for example for memory hotplug.

After the fix the layout reported for the same guest is:

Quoted 8 lines

0x00000000000000 - 0x000000fc000000 RAM
0x000000feff8000 - 0x000000ff000000 RESERVED
0x000000fc009000 - 0x000000fc009040 ACPI
0x000000fc000000 - 0x000000fc001000 ACPI
0x000000fc001000 - 0x000000fc009000 ACPI
0x00000100000000 - 0x000001fe000400 RAM

You could have kept my Rab.

Juergen

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

Acked-by on v2 1/1
#2 lore raw
In reply to Roger Pau Monne
Quoted 24 lines · Roger Pau Monne

PVH guests with 4GB of RAM or more get a memory map like the following:

0x00000000000000 - 0x000000fee00000 RAM
0x000000fee00000 - 0x00000100000000 RESERVED
0x000000fc009000 - 0x000000fc009040 ACPI
0x000000fc000000 - 0x000000fc001000 ACPI
0x000000fc001000 - 0x000000fc009000 ACPI
0x00000100000000 - 0x000001fb200400 RAM

This is wrong because ACPI regions overlap with RAM regions. The cause of this issue is not setting a big enough MMIO hole and marking the whole MMIO hole as reserved, when it actually contains several pieces:

- local APIC page.
- ACPI tables.
- HVM special pages.

Of those items only HVM special pages need to be marked as reserved in order to advise the guest against using them for example for memory hotplug.

After the fix the layout reported for the same guest is:

Quoted 8 lines

0x00000000000000 - 0x000000fc000000 RAM
0x000000feff8000 - 0x000000ff000000 RESERVED
0x000000fc009000 - 0x000000fc009040 ACPI
0x000000fc000000 - 0x000000fc001000 ACPI
0x000000fc001000 - 0x000000fc009000 ACPI
0x00000100000000 - 0x000001fe000400 RAM

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel