Skip to contentSkip to conversation

xen/arm: validate hwdom first bank boot placement

PATCH Superseded Mykola Kvach 3 participants 11 messages Jun 5 1 message not in the archive

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

v2 cover xen/arm: validate hwdom first bank boot placement

From: Mykola Kvach <[email protected]>

With LLC coloring enabled, the hardware domain memory is allocated by allocate_hwdom_memory() instead of relying on the fixed direct-map layout. Since de99f3263555 ("device-tree: Improve hwdom memory allocation for DMA"), the allocator prefers lower host regions, but the first-bank filter still only enforces the old 128MB heuristic.

A low candidate bank can satisfy that heuristic while still being unsuitable for the Arm kernel/DTB/initrd placement rules. Split the Arm placement logic into reusable helpers first, then validate the candidate hardware-domain bank 0 against those helpers before allocating it.

Changes in v2:
- Split the previous single patch into a behavior-preserving Arm placement refactoring and a separate hardware-domain first-bank fix.
- Rename find_module_placement() and place_modules() to use DTB/initrd terminology.
- Pass ramend to find_dtb_initrd_placement().
- Update the stale tools/libs/guest/xg_dom_arm.c reference.

Link to v1:
https://patchew.org/Xen/4f862bb2dc323914b8120b0f16af7516140cf42b.1780065103.git.mykola._5Fkvach@epam.com/

Mykola Kvach (2):
xen/arm: split DTB/initrd placement helpers
device-tree: device-tree: validate hwdom bank 0 boot placement

 xen/arch/arm/acpi/domain_build.c        |   2 -
 xen/arch/arm/domain_build.c             |   8 +
 xen/arch/arm/include/asm/domain_build.h |   4 +
 xen/arch/arm/include/asm/kernel.h       |   9 ++
 xen/arch/arm/kernel.c                   | 190 ++++++++++++++++++------
 xen/common/device-tree/domain-build.c   |  30 ++--
 xen/include/xen/fdt-kernel.h            |   9 ++
 7 files changed, 192 insertions(+), 60 deletions(-)
Signature · 1 line
2.43.0
#1 lore raw

PATCH v2 1/2 xen/arm: split DTB/initrd placement helpers

From: Mykola Kvach <[email protected]>

The Arm zImage loader currently computes the kernel load address and places the DTB/initrd in one local flow. The hardware-domain memory allocator needs to reuse those placement rules before it chooses bank 0, but open-coding the same calculations there would make the fix harder to audit.

Split the existing logic into small helpers:
- kernel_zimage_place_in_bank() computes the zImage load address for a given bank.
- first_bank_can_fit_modules() checks the aggregate first-bank footprint.
- find_dtb_initrd_placement() chooses the DTB/initrd location within a known bank and kernel range.

Rename place_modules() to place_dtb_initrd() so the code distinguishes the kernel image from the DTB/initrd placement area. Also update the stale xg_dom_arm.c path in the placement comment.

The caller still panics in the same cases as before, so this is intended to be behavior preserving.

---
Changes in v2:
- New patch split out from the hardware-domain first-bank fix.
- Rename the DTB/initrd placement helpers to avoid treating the kernel and DTB/initrd as the same kind of module.
- Pass the RAM end address to find_dtb_initrd_placement() instead of recomputing it from the RAM size.
- Update the stale xg_dom_arm.c reference in the placement comment.

---
 xen/arch/arm/kernel.c                 | 147 ++++++++++++++++----------
 xen/common/device-tree/domain-build.c |   6 +-
 2 files changed, 97 insertions(+), 56 deletions(-)
xen/arch/arm/kernel.c +94 -53
index b72585b7fe..d1be4d8074 100644
@@ -40,27 +40,59 @@ struct minimal_dtb_header {
     /* There are other fields but we don't use them yet. */
 };
 
-static void __init place_modules(struct kernel_info *info,
-                                 paddr_t kernbase, paddr_t kernend)
+static paddr_t __init
+kernel_zimage_place_in_bank(const struct kernel_info *info,
+                            paddr_t bank_start, paddr_t bank_size)
 {
-    /* Align DTB and initrd size to 2Mb. Linux only requires 4 byte alignment */
-    const struct boot_module *mod = info->bd.initrd;
-    const struct membanks *mem = kernel_info_get_mem(info);
-    const paddr_t initrd_len = ROUNDUP(mod ? mod->size : 0, MB(2));
-    const paddr_t dtb_len = ROUNDUP(fdt_totalsize(info->fdt), MB(2));
-    const paddr_t modsize = initrd_len + dtb_len;
+    paddr_t load_addr;
 
-    /* Convenient */
-    const paddr_t rambase = mem->bank[0].start;
-    const paddr_t ramsize = mem->bank[0].size;
-    const paddr_t ramend = rambase + ramsize;
+#ifdef CONFIG_HAS_DOMAIN_TYPE
+    if ( (info->type == DOMAIN_64BIT) && (info->image.start == 0) )
+        return bank_start + info->image.text_offset;
+#endif
+
+    /*
+     * If start is zero, the zImage is position independent, in this
+     * case Documentation/arm/Booting recommends loading below 128MiB
+     * and above 32MiB. Load it as high as possible within these
+     * constraints, while also avoiding the DTB.
+     */
+    if ( info->image.start == 0 )
+    {
+        paddr_t load_end;
+
+        load_end = bank_start + bank_size;
+        load_end = MIN(bank_start + MB(128), load_end);
+
+        load_addr = load_end - info->image.len;
+        /* Align to 2MB */
+        load_addr &= ~(MB(2) - 1);
+    }
+    else
+        load_addr = info->image.start;
+
+    return load_addr;
+}
+
+static bool __init first_bank_can_fit_modules(paddr_t ramsize,
+                                              paddr_t kernbase, paddr_t kernend,
+                                              paddr_t dtb_initrd_size)
+{
     const paddr_t kernsize = ROUNDUP(kernend, MB(2)) - kernbase;
-    const paddr_t ram128mb = rambase + MB(128);
 
-    paddr_t modbase;
+    /*
+     * Check only the aggregate kernel + DTB/initrd footprint. The actual
+     * DTB/initrd location is selected by find_dtb_initrd_placement().
+     */
+    return dtb_initrd_size + kernsize <= ramsize;
+}
 
-    if ( modsize + kernsize > ramsize )
-        panic("Not enough memory in the first bank for the kernel+dtb+initrd\n");
+static bool __init find_dtb_initrd_placement(paddr_t rambase, paddr_t ramend,
+                                             paddr_t kernbase, paddr_t kernend,
+                                             paddr_t dtb_initrd_size,
+                                             paddr_t *dtb_base)
+{
+    const paddr_t ram128mb = rambase + MB(128);
 
     /*
      * DTB must be loaded such that it does not conflict with the
@@ -77,55 +109,64 @@ static void __init place_modules(struct kernel_info *info,
      * just before the kernel.
      *
      * If changing this then consider
-     * tools/libxc/xc_dom_arm.c:arch_setup_meminit as well.
+     * tools/libs/guest/xg_dom_arm.c:meminit as well.
      */
-    if ( ramend >= ram128mb + modsize && kernend < ram128mb )
-        modbase = ram128mb;
-    else if ( ramend - modsize > ROUNDUP(kernend, MB(2)) )
-        modbase = ramend - modsize;
-    else if ( kernbase - rambase > modsize )
-        modbase = kernbase - modsize;
-    else
+    if ( ramend >= ram128mb + dtb_initrd_size && kernend < ram128mb )
     {
-        panic("Unable to find suitable location for dtb+initrd\n");
-        return;
+        *dtb_base = ram128mb;
+        return true;
     }
 
-    info->dtb_paddr = modbase;
-    info->initrd_paddr = info->dtb_paddr + dtb_len;
+    if ( ramend - dtb_initrd_size > ROUNDUP(kernend, MB(2)) )
+    {
+        *dtb_base = ramend - dtb_initrd_size;
+        return true;
+    }
+
+    if ( kernbase - rambase > dtb_initrd_size )
+    {
+        *dtb_base = kernbase - dtb_initrd_size;
+        return true;
+    }
+
+    return false;
 }
 
-static paddr_t __init kernel_zimage_place(struct kernel_info *info)
+static void __init place_dtb_initrd(struct kernel_info *info,
+                                    paddr_t kernbase, paddr_t kernend)
 {
+    /* Align DTB and initrd size to 2Mb. Linux only requires 4 byte alignment */
+    const struct boot_module *initrd = info->bd.initrd;
     const struct membanks *mem = kernel_info_get_mem(info);
-    paddr_t load_addr;
+    const paddr_t initrd_len = ROUNDUP(initrd ? initrd->size : 0, MB(2));
+    const paddr_t dtb_len = ROUNDUP(fdt_totalsize(info->fdt), MB(2));
+    const paddr_t dtb_initrd_size = initrd_len + dtb_len;
 
-#ifdef CONFIG_HAS_DOMAIN_TYPE
-    if ( (info->type == DOMAIN_64BIT) && (info->image.start == 0) )
-        return mem->bank[0].start + info->image.text_offset;
-#endif
+    /* Convenient */
+    const paddr_t rambase = mem->bank[0].start;
+    const paddr_t ramsize = mem->bank[0].size;
+    const paddr_t ramend = rambase + ramsize;
 
-    /*
-     * If start is zero, the zImage is position independent, in this
-     * case Documentation/arm/Booting recommends loading below 128MiB
-     * and above 32MiB. Load it as high as possible within these
-     * constraints, while also avoiding the DTB.
-     */
-    if ( info->image.start == 0 )
-    {
-        paddr_t load_end;
+    paddr_t dtb_base;
 
-        load_end = mem->bank[0].start + mem->bank[0].size;
-        load_end = MIN(mem->bank[0].start + MB(128), load_end);
+    if ( !first_bank_can_fit_modules(ramsize, kernbase, kernend,
+                                     dtb_initrd_size) )
+        panic("Not enough memory in the first bank for the kernel+dtb+initrd\n");
 
-        load_addr = load_end - info->image.len;
-        /* Align to 2MB */
-        load_addr &= ~((2 << 20) - 1);
-    }
-    else
-        load_addr = info->image.start;
+    if ( !find_dtb_initrd_placement(rambase, ramend, kernbase, kernend,
+                                    dtb_initrd_size, &dtb_base) )
+        panic("Unable to find suitable location for dtb+initrd\n");
 
-    return load_addr;
+    info->dtb_paddr = dtb_base;
+    info->initrd_paddr = info->dtb_paddr + dtb_len;
+}
+
+static paddr_t __init kernel_zimage_place(struct kernel_info *info)
+{
+    const struct membanks *mem = kernel_info_get_mem(info);
+
+    return kernel_zimage_place_in_bank(info, mem->bank[0].start,
+                                       mem->bank[0].size);
 }
 
 static void __init kernel_zimage_load(struct kernel_info *info)
@@ -143,7 +184,7 @@ static void __init kernel_zimage_load(struct kernel_info *info)
     if ( info->entry == 0 )
         info->entry = load_addr;
 
-    place_modules(info, load_addr, load_addr + len);
+    place_dtb_initrd(info, load_addr, load_addr + len);
 
     printk("Loading zImage from %"PRIpaddr" to %"PRIpaddr"-%"PRIpaddr"\n",
            paddr, load_addr, load_addr + len);
xen/common/device-tree/domain-build.c +3 -3
index 2a760b007b..f3ba496f1e 100644
@@ -245,8 +245,8 @@ out:
  * hardware domain to have memory reachable by devices with limited DMA address
  * capabilities (e.g. 32-bit DMA).
  *
- * The first bank allocated must be large enough for place_modules() to fit
- * the kernel, DTB and initrd.
+ * The first bank allocated must be large enough for place_dtb_initrd() to
+ * fit the kernel, DTB and initrd.
  */
 static bool __init allocate_hwdom_memory(struct kernel_info *kinfo)
 {
@@ -302,7 +302,7 @@ static bool __init allocate_hwdom_memory(struct kernel_info *kinfo)
         paddr_t bank_size;
 
         /*
-         * The first bank must be large enough for place_modules() to
+         * The first bank must be large enough for place_dtb_initrd() to
          * fit the kernel, DTB and initrd.  Skip small regions to avoid
          * ending up with a tiny first bank.
          */
Signature · 1 line
2.43.0
Reviewed-by on v2 1/2
#2 lore raw

Re: [PATCH for-4.22 v2 1/2] xen/arm: split DTB/initrd placement helpers

We are in the RC phase, so please CC Oleksii (release manager) as we need his tags for fixes. Adding him now.

Quoted 74 lines · Mykola Kvach

From: Mykola Kvach <[email protected]>

The Arm zImage loader currently computes the kernel load address and places the DTB/initrd in one local flow. The hardware-domain memory allocator needs to reuse those placement rules before it chooses bank 0, but open-coding the same calculations there would make the fix harder to audit.

Split the existing logic into small helpers:
- kernel_zimage_place_in_bank() computes the zImage load address for a given bank.
- first_bank_can_fit_modules() checks the aggregate first-bank footprint.
- find_dtb_initrd_placement() chooses the DTB/initrd location within a known bank and kernel range.

Rename place_modules() to place_dtb_initrd() so the code distinguishes the kernel image from the DTB/initrd placement area. Also update the stale xg_dom_arm.c path in the placement comment.

The caller still panics in the same cases as before, so this is intended to be behavior preserving.

---
Changes in v2:
- New patch split out from the hardware-domain first-bank fix.
- Rename the DTB/initrd placement helpers to avoid treating the kernel and DTB/initrd as the same kind of module.
- Pass the RAM end address to find_dtb_initrd_placement() instead of recomputing it from the RAM size.
- Update the stale xg_dom_arm.c reference in the placement comment.

---
 xen/arch/arm/kernel.c                 | 147 ++++++++++++++++----------
 xen/common/device-tree/domain-build.c |   6 +-
 2 files changed, 97 insertions(+), 56 deletions(-)
xen/arch/arm/kernel.c +15 -12
index b72585b7fe..d1be4d8074 100644
@@ -40,27 +40,59 @@ struct minimal_dtb_header {
     /* There are other fields but we don't use them yet. */
 };
 
-static void __init place_modules(struct kernel_info *info,
-                                 paddr_t kernbase, paddr_t kernend)
+static paddr_t __init
+kernel_zimage_place_in_bank(const struct kernel_info *info,
+                            paddr_t bank_start, paddr_t bank_size)
 {
-    /* Align DTB and initrd size to 2Mb. Linux only requires 4 byte alignment */
-    const struct boot_module *mod = info->bd.initrd;
-    const struct membanks *mem = kernel_info_get_mem(info);
-    const paddr_t initrd_len = ROUNDUP(mod ? mod->size : 0, MB(2));
-    const paddr_t dtb_len = ROUNDUP(fdt_totalsize(info->fdt), MB(2));
-    const paddr_t modsize = initrd_len + dtb_len;
+    paddr_t load_addr;
 
-    /* Convenient */
-    const paddr_t rambase = mem->bank[0].start;
-    const paddr_t ramsize = mem->bank[0].size;
-    const paddr_t ramend = rambase + ramsize;
+#ifdef CONFIG_HAS_DOMAIN_TYPE
+    if ( (info->type == DOMAIN_64BIT) && (info->image.start == 0) )
+        return bank_start + info->image.text_offset;
+#endif
+
+    /*
+     * If start is zero, the zImage is position independent, in this
+     * case Documentation/arm/Booting recommends loading below 128MiB
+     * and above 32MiB. Load it as high as possible within these
+     * constraints, while also avoiding the DTB.
+     */

xen/arch/arm/kernel.c struct minimal_dtb_header { · line 69

Quoted 10 lines
xen/arch/arm/kernel.c +10 -0
+    if ( info->image.start == 0 )
+    {
+        paddr_t load_end;
+
+        load_end = bank_start + bank_size;
+        load_end = MIN(bank_start + MB(128), load_end);
+
+        load_addr = load_end - info->image.len;
+        /* Align to 2MB */
+        load_addr &= ~(MB(2) - 1);

For the future mechanical changes, you should not be making even such tiny changes like s/(2 << 20)/MB(2) without mentioning them in commit msg. For today, it's ok.

Quoted 34 lines
xen/arch/arm/kernel.c +23 -4
+    }
+    else
+        load_addr = info->image.start;
+
+    return load_addr;
+}
+
+static bool __init first_bank_can_fit_modules(paddr_t ramsize,
+                                              paddr_t kernbase, paddr_t kernend,
+                                              paddr_t dtb_initrd_size)
+{
     const paddr_t kernsize = ROUNDUP(kernend, MB(2)) - kernbase;
-    const paddr_t ram128mb = rambase + MB(128);
 
-    paddr_t modbase;
+    /*
+     * Check only the aggregate kernel + DTB/initrd footprint. The actual
+     * DTB/initrd location is selected by find_dtb_initrd_placement().
+     */
+    return dtb_initrd_size + kernsize <= ramsize;
+}
 
-    if ( modsize + kernsize > ramsize )
-        panic("Not enough memory in the first bank for the kernel+dtb+initrd\n");
+static bool __init find_dtb_initrd_placement(paddr_t rambase, paddr_t ramend,
+                                             paddr_t kernbase, paddr_t kernend,
+                                             paddr_t dtb_initrd_size,
+                                             paddr_t *dtb_base)
+{
+    const paddr_t ram128mb = rambase + MB(128);
 
     /*
      * DTB must be loaded such that it does not conflict with the

xen/arch/arm/kernel.c static void __init place_modules(struct kernel_info *info, · line 112

Quoted 5 lines
xen/arch/arm/kernel.c +1 -1
@@ -77,55 +109,64 @@ static void __init place_modules(struct kernel_info *info,
      * just before the kernel.
      *
      * If changing this then consider
-     * tools/libxc/xc_dom_arm.c:arch_setup_meminit as well.
+     * tools/libs/guest/xg_dom_arm.c:meminit as well.

This fixes the kernel.c -> tools pointer, but the reverse comment in xg_dom_arm.c:meminit still points at place_modules. This needs to be fixed.

I think this can be done on commit, therefore:

~Michal

#3 lore raw

Re: [PATCH for-4.22 v2 1/2] xen/arm: split DTB/initrd placement helpers

Hi Michal,

Thanks for the review.

Quoted 3 lines · Michal

We are in the RC phase, so please CC Oleksii (release manager) as we need his tags for fixes. Adding him now.

Ack, I will keep Oleksii Cc'ed for this series.

Quoted 81 lines · Mykola Kvach
Quoted 79 lines

From: Mykola Kvach <[email protected]>

The Arm zImage loader currently computes the kernel load address and places the DTB/initrd in one local flow. The hardware-domain memory allocator needs to reuse those placement rules before it chooses bank 0, but open-coding the same calculations there would make the fix harder to audit.

Split the existing logic into small helpers:
- kernel_zimage_place_in_bank() computes the zImage load address for a given bank.
- first_bank_can_fit_modules() checks the aggregate first-bank footprint.
- find_dtb_initrd_placement() chooses the DTB/initrd location within a known bank and kernel range.

Rename place_modules() to place_dtb_initrd() so the code distinguishes the kernel image from the DTB/initrd placement area. Also update the stale xg_dom_arm.c path in the placement comment.

The caller still panics in the same cases as before, so this is intended to be behavior preserving.

---
Changes in v2:
- New patch split out from the hardware-domain first-bank fix.
- Rename the DTB/initrd placement helpers to avoid treating the kernel and DTB/initrd as the same kind of module.
- Pass the RAM end address to find_dtb_initrd_placement() instead of recomputing it from the RAM size.
- Update the stale xg_dom_arm.c reference in the placement comment.

---
 xen/arch/arm/kernel.c                 | 147 ++++++++++++++++----------
 xen/common/device-tree/domain-build.c |   6 +-
 2 files changed, 97 insertions(+), 56 deletions(-)
xen/arch/arm/kernel.c +20 -12
index b72585b7fe..d1be4d8074 100644
@@ -40,27 +40,59 @@ struct minimal_dtb_header {
     /* There are other fields but we don't use them yet. */
 };
 
-static void __init place_modules(struct kernel_info *info,
-                                 paddr_t kernbase, paddr_t kernend)
+static paddr_t __init
+kernel_zimage_place_in_bank(const struct kernel_info *info,
+                            paddr_t bank_start, paddr_t bank_size)
 {
-    /* Align DTB and initrd size to 2Mb. Linux only requires 4 byte alignment */
-    const struct boot_module *mod = info->bd.initrd;
-    const struct membanks *mem = kernel_info_get_mem(info);
-    const paddr_t initrd_len = ROUNDUP(mod ? mod->size : 0, MB(2));
-    const paddr_t dtb_len = ROUNDUP(fdt_totalsize(info->fdt), MB(2));
-    const paddr_t modsize = initrd_len + dtb_len;
+    paddr_t load_addr;
 
-    /* Convenient */
-    const paddr_t rambase = mem->bank[0].start;
-    const paddr_t ramsize = mem->bank[0].size;
-    const paddr_t ramend = rambase + ramsize;
+#ifdef CONFIG_HAS_DOMAIN_TYPE
+    if ( (info->type == DOMAIN_64BIT) && (info->image.start == 0) )
+        return bank_start + info->image.text_offset;
+#endif
+
+    /*
+     * If start is zero, the zImage is position independent, in this
+     * case Documentation/arm/Booting recommends loading below 128MiB
+     * and above 32MiB. Load it as high as possible within these
+     * constraints, while also avoiding the DTB.
+     */
+    if ( info->image.start == 0 )
+    {
+        paddr_t load_end;
+
+        load_end = bank_start + bank_size;
Quoted 8 lines

xen/arch/arm/kernel.c struct minimal_dtb_header { · line 69

Quoted 5 lines
xen/arch/arm/kernel.c +5 -0
+        load_end = MIN(bank_start + MB(128), load_end);
+
+        load_addr = load_end - info->image.len;
+        /* Align to 2MB */
+        load_addr &= ~(MB(2) - 1);

For the future mechanical changes, you should not be making even such tiny changes like s/(2 << 20)/MB(2) without mentioning them in commit msg. For today, it's ok.

Got it.

Quoted 43 lines
Quoted 35 lines
xen/arch/arm/kernel.c +23 -4
+    }
+    else
+        load_addr = info->image.start;
+
+    return load_addr;
+}
+
+static bool __init first_bank_can_fit_modules(paddr_t ramsize,
+                                              paddr_t kernbase, paddr_t kernend,
+                                              paddr_t dtb_initrd_size)
+{
     const paddr_t kernsize = ROUNDUP(kernend, MB(2)) - kernbase;
-    const paddr_t ram128mb = rambase + MB(128);
 
-    paddr_t modbase;
+    /*
+     * Check only the aggregate kernel + DTB/initrd footprint. The actual
+     * DTB/initrd location is selected by find_dtb_initrd_placement().
+     */
+    return dtb_initrd_size + kernsize <= ramsize;
+}
 
-    if ( modsize + kernsize > ramsize )
-        panic("Not enough memory in the first bank for the kernel+dtb+initrd\n");
+static bool __init find_dtb_initrd_placement(paddr_t rambase, paddr_t ramend,
+                                             paddr_t kernbase, paddr_t kernend,
+                                             paddr_t dtb_initrd_size,
+                                             paddr_t *dtb_base)
+{
+    const paddr_t ram128mb = rambase + MB(128);
 
     /*
      * DTB must be loaded such that it does not conflict with the

xen/arch/arm/kernel.c static void __init place_modules(struct kernel_info *info, · line 112

Quoted 5 lines
xen/arch/arm/kernel.c +1 -1
@@ -77,55 +109,64 @@ static void __init place_modules(struct kernel_info *info,
      * just before the kernel.
      *
      * If changing this then consider
-     * tools/libxc/xc_dom_arm.c:arch_setup_meminit as well.
+     * tools/libs/guest/xg_dom_arm.c:meminit as well.

This fixes the kernel.c -> tools pointer, but the reverse comment in xg_dom_arm.c:meminit still points at place_modules. This needs to be fixed.

Quoted 2 lines

I think this can be done on commit, therefore:

Ack on the reverse pointer in xg_dom_arm.c. It should point at place_dtb_initrd() now. I am fine with this being fixed on commit, but since I will send a new revision for the comments on patch 2, I will fold this update into patch 1 as well.

Best regards,
Mykola

#4 lore raw

PATCH v2 2/2 device-tree: validate hwdom bank 0 boot placement

In reply to Mykola Kvach

From: Mykola Kvach <[email protected]>

With LLC coloring enabled, the hardware domain memory is allocated by allocate_hwdom_memory() rather than by using the fixed direct-map layout.

Commit de99f3263555 ("device-tree: Improve hwdom memory allocation for DMA") made that allocator prefer lower host regions. The first-bank filter, however, still only checked the old 128MB heuristic. A low region can satisfy that heuristic but still be too small, or otherwise unsuitable, for the hardware-domain kernel and the DTB/initrd area to fit in bank 0 according to the Arm placement rules.

Keep the existing first-bank size policy and add an architecture-specific candidate check. On Arm, compute the kernel load address for the candidate bank using the same logic as kernel_zimage_place(), verify that the kernel range is covered by that bank, and then reuse the same DTB/initrd placement helper as place_dtb_initrd(). The FDT is generated later, so use the hardware-domain FDT allocation size as a conservative upper bound for the final DTB size.

Check the candidate after capping the host region by the remaining unassigned hardware-domain memory, so the validation is performed against the size that would actually become bank 0.

This keeps the DMA-oriented allocation policy from de99f3263555 while preventing a too-small bank 0 from reaching place_dtb_initrd().

Make kernel_zimage_place_in_bank() return INVALID_PADDR when a position-independent zImage cannot be placed in the supplied bank; the real load path turns this into a panic, while the hwdom candidate check uses it to reject the bank.

Fixes de99f3263555 ("device-tree: Improve hwdom memory allocation for DMA")

---
Changes in v2:
- Split the behavior-preserving placement refactoring into the previous patch.
- Reuse the refactored Arm kernel and DTB/initrd placement helpers for the first-bank candidate check.

Link to v1:
https://patchew.org/Xen/4f862bb2dc323914b8120b0f16af7516140cf42b.1780065103.git.mykola._5Fkvach@epam.com/

Changes since RFC:
- Do not keep the RFC scalar minimum-size check. It can both reject valid layouts and accept layouts which still fail later. Instead, validate the candidate bank using the same kernel and DTB/initrd placement rules as the load path.
- Replace the scalar minimum-size check with arch_hwdom_first_bank_ok().
- Validate fixed-address and AArch32 start == 0 kernel placement against the candidate bank.
- Check the candidate after capping the host region by the remaining unassigned hardware-domain memory.
- Treat the hardware-domain FDT allocation size as a conservative upper bound because the final FDT is generated later.

Link to RFC: https://patchew.org/Xen/9ae4f7dd49f5b1f761193adae573c2675c92e883.1779051035.git.mykola._5Fkvach@epam.com/

Why the RFC scalar approach was not kept:

A simple minimum-size check is not sufficient here because the validity of the first bank depends on the actual Arm placement rules, not only on the aggregate size of the kernel, DTB and initrd. The DTB/initrd area may fit before a 64-bit Image loaded with a text offset, while an AArch32 position-independent kernel may leave no valid module location even when the aggregate size appears to fit. Fixed-address kernels also need the candidate bank start to be considered.

---
 xen/arch/arm/acpi/domain_build.c        |  2 -
 xen/arch/arm/domain_build.c             |  8 ++++
 xen/arch/arm/include/asm/domain_build.h |  4 ++
 xen/arch/arm/include/asm/kernel.h       |  9 ++++
 xen/arch/arm/kernel.c                   | 57 ++++++++++++++++++++++++-
 xen/common/device-tree/domain-build.c   | 24 ++++++++---
 xen/include/xen/fdt-kernel.h            |  9 ++++
 7 files changed, 102 insertions(+), 11 deletions(-)
xen/arch/arm/acpi/domain_build.c +0 -2
index 249d899c33..db16f7fa94 100644
@@ -26,8 +26,6 @@
 #undef virt_to_mfn
 #define virt_to_mfn(va) _mfn(__virt_to_mfn(va))
 
-#define ACPI_DOM0_FDT_MIN_SIZE 4096
-
 static int __init acpi_iomem_deny_access(struct domain *d)
 {
     acpi_status status;
xen/arch/arm/domain_build.c +8 -0
index 1efddc60ef..550617f152 100644
@@ -115,6 +115,14 @@ int __init parse_arch_dom0_param(const char *s, const char *e)
                              (IS_ENABLED(CONFIG_STATIC_SHM) ?         \
                               (NR_SHMEM_BANKS * (160 + 16)) : 0))
 
+paddr_t __init hwdom_get_fdt_alloc_size(void)
+{
+    if ( acpi_disabled )
+        return fdt_totalsize(device_tree_flattened) + DOM0_FDT_EXTRA_SIZE;
+
+    return ACPI_DOM0_FDT_MIN_SIZE;
+}
+
 unsigned int __init dom0_max_vcpus(void)
 {
     if ( opt_dom0_max_vcpus == 0 )
xen/arch/arm/include/asm/domain_build.h +4 -0
index df8b361b3d..85cf46a958 100644
@@ -19,6 +19,10 @@ int prepare_acpi(struct domain *d, struct kernel_info *kinfo);
 
 int add_ext_regions(unsigned long s_gfn, unsigned long e_gfn, void *data);
 
+#define ACPI_DOM0_FDT_MIN_SIZE 4096
+
+paddr_t hwdom_get_fdt_alloc_size(void);
+
 #if defined(CONFIG_MPU) && defined(CONFIG_ARM_64)
 /* Utility function to determine if an Armv8-R processor supports VMSA. */
 bool has_v8r_vmsa_support(void);
xen/arch/arm/include/asm/kernel.h +9 -0
index 21f4273fa1..bf14fb208a 100644
@@ -8,12 +8,21 @@
 
 #include <asm/domain.h>
 
+#include <xen/types.h>
+
+struct kernel_info;
+
 struct arch_kernel_info
 {
     /* Enable pl011 emulation */
     bool vpl011;
 };
 
+#define arch_hwdom_first_bank_ok arch_hwdom_first_bank_ok
+bool arch_hwdom_first_bank_ok(const struct kernel_info *info,
+                              paddr_t bank_start,
+                              paddr_t bank_size);
+
 #endif /* #ifdef __ARCH_ARM_KERNEL_H__ */
 
 /*
xen/arch/arm/kernel.c +55 -2

73 lines not on this page · raw message

xen/common/device-tree/domain-build.c +17 -7
index f3ba496f1e..2e806c1b09 100644
@@ -299,20 +299,30 @@ static bool __init allocate_hwdom_memory(struct kernel_info *kinfo)
 
     for ( i = 0; (kinfo->unassigned_mem > 0) && (i < nr_banks); i++ )
     {
-        paddr_t bank_size;
+        const paddr_t bank_start = hwdom_free_mem->bank[i].start;
+        paddr_t bank_size = hwdom_free_mem->bank[i].size;
+
+        /*
+         * Check the size that would actually be assigned, not just the size
+         * of the host region.
+         */
+        bank_size = min(bank_size, kinfo->unassigned_mem);
 
         /*
          * The first bank must be large enough for place_dtb_initrd() to
          * fit the kernel, DTB and initrd.  Skip small regions to avoid
          * ending up with a tiny first bank.
          */
-        if ( !mem->nr_banks && (hwdom_free_mem->bank[i].size < min_bank_size) )
-            continue;
+        if ( !mem->nr_banks )
+        {
+            if ( bank_size < min_bank_size )
+                continue;
+
+            if ( !arch_hwdom_first_bank_ok(kinfo, bank_start, bank_size) )
+                continue;
+        }
 
-        bank_size = MIN(hwdom_free_mem->bank[i].size, kinfo->unassigned_mem);
-        if ( !allocate_bank_memory(kinfo,
-                                   gaddr_to_gfn(hwdom_free_mem->bank[i].start),
-                                   bank_size) )
+        if ( !allocate_bank_memory(kinfo, gaddr_to_gfn(bank_start), bank_size) )
         {
             xfree(hwdom_free_mem);
             return false;
xen/include/xen/fdt-kernel.h +9 -0
index 00c37be101..71e2344b97 100644
@@ -93,6 +93,15 @@ kernel_info_get_mem_const(const struct kernel_info *kinfo)
     return container_of(&kinfo->mem.common, const struct membanks, common);
 }
 
+#ifndef arch_hwdom_first_bank_ok
+static inline bool arch_hwdom_first_bank_ok(const struct kernel_info *info,
+                                            paddr_t bank_start,
+                                            paddr_t bank_size)
+{
+    return true;
+}
+#endif
+
 #ifndef KERNEL_INFO_SHM_MEM_INIT
 
 #ifdef CONFIG_STATIC_SHM
Signature · 1 line
2.43.0
#5 lore raw

Re: [PATCH for-4.22 v2 2/2] device-tree: validate hwdom bank 0 boot placement

Quoted 202 lines · Mykola Kvach

From: Mykola Kvach <[email protected]>

With LLC coloring enabled, the hardware domain memory is allocated by allocate_hwdom_memory() rather than by using the fixed direct-map layout.

Commit de99f3263555 ("device-tree: Improve hwdom memory allocation for DMA") made that allocator prefer lower host regions. The first-bank filter, however, still only checked the old 128MB heuristic. A low region can satisfy that heuristic but still be too small, or otherwise unsuitable, for the hardware-domain kernel and the DTB/initrd area to fit in bank 0 according to the Arm placement rules.

Keep the existing first-bank size policy and add an architecture-specific candidate check. On Arm, compute the kernel load address for the candidate bank using the same logic as kernel_zimage_place(), verify that the kernel range is covered by that bank, and then reuse the same DTB/initrd placement helper as place_dtb_initrd(). The FDT is generated later, so use the hardware-domain FDT allocation size as a conservative upper bound for the final DTB size.

Check the candidate after capping the host region by the remaining unassigned hardware-domain memory, so the validation is performed against the size that would actually become bank 0.

This keeps the DMA-oriented allocation policy from de99f3263555 while preventing a too-small bank 0 from reaching place_dtb_initrd().

Make kernel_zimage_place_in_bank() return INVALID_PADDR when a position-independent zImage cannot be placed in the supplied bank; the real load path turns this into a panic, while the hwdom candidate check uses it to reject the bank.

Fixes de99f3263555 ("device-tree: Improve hwdom memory allocation for DMA")

---
Changes in v2:
- Split the behavior-preserving placement refactoring into the previous patch.
- Reuse the refactored Arm kernel and DTB/initrd placement helpers for the first-bank candidate check.

Link to v1:
https://patchew.org/Xen/4f862bb2dc323914b8120b0f16af7516140cf42b.1780065103.git.mykola._5Fkvach@epam.com/

Changes since RFC:
- Do not keep the RFC scalar minimum-size check. It can both reject valid layouts and accept layouts which still fail later. Instead, validate the candidate bank using the same kernel and DTB/initrd placement rules as the load path.
- Replace the scalar minimum-size check with arch_hwdom_first_bank_ok().
- Validate fixed-address and AArch32 start == 0 kernel placement against the candidate bank.
- Check the candidate after capping the host region by the remaining unassigned hardware-domain memory.
- Treat the hardware-domain FDT allocation size as a conservative upper bound because the final FDT is generated later.

Link to RFC: https://patchew.org/Xen/9ae4f7dd49f5b1f761193adae573c2675c92e883.1779051035.git.mykola._5Fkvach@epam.com/

Why the RFC scalar approach was not kept:

A simple minimum-size check is not sufficient here because the validity of the first bank depends on the actual Arm placement rules, not only on the aggregate size of the kernel, DTB and initrd. The DTB/initrd area may fit before a 64-bit Image loaded with a text offset, while an AArch32 position-independent kernel may leave no valid module location even when the aggregate size appears to fit. Fixed-address kernels also need the candidate bank start to be considered.

---
 xen/arch/arm/acpi/domain_build.c        |  2 -
 xen/arch/arm/domain_build.c             |  8 ++++
 xen/arch/arm/include/asm/domain_build.h |  4 ++
 xen/arch/arm/include/asm/kernel.h       |  9 ++++
 xen/arch/arm/kernel.c                   | 57 ++++++++++++++++++++++++-
 xen/common/device-tree/domain-build.c   | 24 ++++++++---
 xen/include/xen/fdt-kernel.h            |  9 ++++
 7 files changed, 102 insertions(+), 11 deletions(-)
xen/arch/arm/acpi/domain_build.c +0 -2
index 249d899c33..db16f7fa94 100644
@@ -26,8 +26,6 @@
 #undef virt_to_mfn
 #define virt_to_mfn(va) _mfn(__virt_to_mfn(va))
 
-#define ACPI_DOM0_FDT_MIN_SIZE 4096
-
 static int __init acpi_iomem_deny_access(struct domain *d)
 {
     acpi_status status;
xen/arch/arm/domain_build.c +8 -0
index 1efddc60ef..550617f152 100644
@@ -115,6 +115,14 @@ int __init parse_arch_dom0_param(const char *s, const char *e)
                              (IS_ENABLED(CONFIG_STATIC_SHM) ?         \
                               (NR_SHMEM_BANKS * (160 + 16)) : 0))
 
+paddr_t __init hwdom_get_fdt_alloc_size(void)
+{
+    if ( acpi_disabled )
+        return fdt_totalsize(device_tree_flattened) + DOM0_FDT_EXTRA_SIZE;
+
+    return ACPI_DOM0_FDT_MIN_SIZE;
+}
+
 unsigned int __init dom0_max_vcpus(void)
 {
     if ( opt_dom0_max_vcpus == 0 )
xen/arch/arm/include/asm/domain_build.h +4 -0
index df8b361b3d..85cf46a958 100644
@@ -19,6 +19,10 @@ int prepare_acpi(struct domain *d, struct kernel_info *kinfo);
 
 int add_ext_regions(unsigned long s_gfn, unsigned long e_gfn, void *data);
 
+#define ACPI_DOM0_FDT_MIN_SIZE 4096
+
+paddr_t hwdom_get_fdt_alloc_size(void);
+
 #if defined(CONFIG_MPU) && defined(CONFIG_ARM_64)
 /* Utility function to determine if an Armv8-R processor supports VMSA. */
 bool has_v8r_vmsa_support(void);
xen/arch/arm/include/asm/kernel.h +9 -0
index 21f4273fa1..bf14fb208a 100644
@@ -8,12 +8,21 @@
 
 #include <asm/domain.h>
 
+#include <xen/types.h>
+
+struct kernel_info;
+
 struct arch_kernel_info
 {
     /* Enable pl011 emulation */
     bool vpl011;
 };
 
+#define arch_hwdom_first_bank_ok arch_hwdom_first_bank_ok
+bool arch_hwdom_first_bank_ok(const struct kernel_info *info,
+                              paddr_t bank_start,
+                              paddr_t bank_size);
+
 #endif /* #ifdef __ARCH_ARM_KERNEL_H__ */
 
 /*
xen/arch/arm/kernel.c +33 -0
index d1be4d8074..ecea2822a1 100644
@@ -64,9 +64,15 @@ kernel_zimage_place_in_bank(const struct kernel_info *info,
         load_end = bank_start + bank_size;
         load_end = MIN(bank_start + MB(128), load_end);
 
+        if ( load_end - bank_start < info->image.len )
+            return INVALID_PADDR;
+
         load_addr = load_end - info->image.len;
         /* Align to 2MB */
         load_addr &= ~(MB(2) - 1);
+
+        if ( load_addr < bank_start )
+            return INVALID_PADDR;
     }
     else
         load_addr = info->image.start;
@@ -164,9 +170,56 @@ static void __init place_dtb_initrd(struct kernel_info *info,
 static paddr_t __init kernel_zimage_place(struct kernel_info *info)
 {
     const struct membanks *mem = kernel_info_get_mem(info);
+    paddr_t load_addr;
+
+    load_addr = kernel_zimage_place_in_bank(info, mem->bank[0].start,
+                                            mem->bank[0].size);
+    if ( load_addr == INVALID_PADDR )
+        panic("Unable to find suitable location for the kernel\n");
+
+    return load_addr;
+}
+
+bool __init arch_hwdom_first_bank_ok(const struct kernel_info *info,
+                                     paddr_t bank_start,
+                                     paddr_t bank_size)
+{
+    const struct boot_module *initrd = info->bd.initrd;
+    /*
+     * place_dtb_initrd() rounds the DTB and initrd placement to 2MB boundaries;
+     * use the same granularity when checking whether the first bank can hold
+     * them.
+     */
+    const paddr_t initrd_len = ROUNDUP(initrd ? initrd->size : 0, MB(2));
+    /*
+     * The hardware domain FDT has not been generated yet. Use the allocation
+     * size as a conservative upper bound for the final DTB size.
+     */
+    const paddr_t dtb_len = ROUNDUP(hwdom_get_fdt_alloc_size(), MB(2));
+    const paddr_t rambase = bank_start;

xen/arch/arm/kernel.c static void __init place_dtb_initrd(struct kernel_info *info, · line 209

Quoted 10 lines
xen/arch/arm/kernel.c +10 -0
+    const paddr_t ramsize = bank_size;
+    const paddr_t dtb_initrd_size = initrd_len + dtb_len;
+    const paddr_t ramend = rambase + ramsize;
+    paddr_t kernbase;
+    paddr_t kernend;
+    paddr_t dtb_base;
+
+    kernbase = kernel_zimage_place_in_bank(info, bank_start, bank_size);
+    if ( kernbase == INVALID_PADDR ||
+         info->image.len > INVALID_PADDR - kernbase )

Max IPA is 48bit, far from 64bit, where the arch max is 52bit and image.len is a kernel image size, so kernbase + image.len cannot wrap a 64-bit paddr_t, so this check is dead. Drop it.

xen/arch/arm/kernel.c static void __init place_dtb_initrd(struct kernel_info *info, · line 214

Quoted 5 lines
xen/arch/arm/kernel.c +5 -0
+        return false;
+
+    kernend = kernbase + info->image.len;
+
+    if ( kernbase < rambase || kernend > ramend )

Please add braces around individual expressions.

Quoted 60 lines
xen/arch/arm/kernel.c +7 -2
+        return false;
+
+    if ( !first_bank_can_fit_modules(ramsize, kernbase, kernend,
+                                     dtb_initrd_size) )
+        return false;
 
-    return kernel_zimage_place_in_bank(info, mem->bank[0].start,
-                                       mem->bank[0].size);
+    return find_dtb_initrd_placement(rambase, ramend, kernbase, kernend,
+                                     dtb_initrd_size, &dtb_base);
 }
 
 static void __init kernel_zimage_load(struct kernel_info *info)
xen/common/device-tree/domain-build.c +17 -7
index f3ba496f1e..2e806c1b09 100644
@@ -299,20 +299,30 @@ static bool __init allocate_hwdom_memory(struct kernel_info *kinfo)
 
     for ( i = 0; (kinfo->unassigned_mem > 0) && (i < nr_banks); i++ )
     {
-        paddr_t bank_size;
+        const paddr_t bank_start = hwdom_free_mem->bank[i].start;
+        paddr_t bank_size = hwdom_free_mem->bank[i].size;
+
+        /*
+         * Check the size that would actually be assigned, not just the size
+         * of the host region.
+         */
+        bank_size = min(bank_size, kinfo->unassigned_mem);
 
         /*
          * The first bank must be large enough for place_dtb_initrd() to
          * fit the kernel, DTB and initrd.  Skip small regions to avoid
          * ending up with a tiny first bank.
          */
-        if ( !mem->nr_banks && (hwdom_free_mem->bank[i].size < min_bank_size) )
-            continue;
+        if ( !mem->nr_banks )
+        {
+            if ( bank_size < min_bank_size )
+                continue;
+
+            if ( !arch_hwdom_first_bank_ok(kinfo, bank_start, bank_size) )
+                continue;
+        }
 
-        bank_size = MIN(hwdom_free_mem->bank[i].size, kinfo->unassigned_mem);
-        if ( !allocate_bank_memory(kinfo,
-                                   gaddr_to_gfn(hwdom_free_mem->bank[i].start),
-                                   bank_size) )
+        if ( !allocate_bank_memory(kinfo, gaddr_to_gfn(bank_start), bank_size) )
         {
             xfree(hwdom_free_mem);
             return false;
xen/include/xen/fdt-kernel.h +0 -0
index 00c37be101..71e2344b97 100644

xen/include/xen/fdt-kernel.h kernel_info_get_mem_const(const struct kernel_info *kinfo) · line 97

Quoted 5 lines
xen/include/xen/fdt-kernel.h +2 -0
@@ -93,6 +93,15 @@ kernel_info_get_mem_const(const struct kernel_info *kinfo)
     return container_of(&kinfo->mem.common, const struct membanks, common);
 }
 
+#ifndef arch_hwdom_first_bank_ok
+static inline bool arch_hwdom_first_bank_ok(const struct kernel_info *info,

This should deserve a comment describing its contract. Also, the name is not very descriptive: how about arch_hwdom_first_bank_can_fit_modules() similar to generic Arm's first_bank_can_fit_modules()?

Rest is ok.

~Michal

#6 lore raw

Re: [PATCH for-4.22 v2 2/2] device-tree: validate hwdom bank 0 boot placement

Quoted 163 lines · Michal
Quoted 159 lines · Mykola Kvach

From: Mykola Kvach <[email protected]>

With LLC coloring enabled, the hardware domain memory is allocated by allocate_hwdom_memory() rather than by using the fixed direct-map layout.

Commit de99f3263555 ("device-tree: Improve hwdom memory allocation for DMA") made that allocator prefer lower host regions. The first-bank filter, however, still only checked the old 128MB heuristic. A low region can satisfy that heuristic but still be too small, or otherwise unsuitable, for the hardware-domain kernel and the DTB/initrd area to fit in bank 0 according to the Arm placement rules.

Keep the existing first-bank size policy and add an architecture-specific candidate check. On Arm, compute the kernel load address for the candidate bank using the same logic as kernel_zimage_place(), verify that the kernel range is covered by that bank, and then reuse the same DTB/initrd placement helper as place_dtb_initrd(). The FDT is generated later, so use the hardware-domain FDT allocation size as a conservative upper bound for the final DTB size.

Check the candidate after capping the host region by the remaining unassigned hardware-domain memory, so the validation is performed against the size that would actually become bank 0.

This keeps the DMA-oriented allocation policy from de99f3263555 while preventing a too-small bank 0 from reaching place_dtb_initrd().

Make kernel_zimage_place_in_bank() return INVALID_PADDR when a position-independent zImage cannot be placed in the supplied bank; the real load path turns this into a panic, while the hwdom candidate check uses it to reject the bank.

Fixes de99f3263555 ("device-tree: Improve hwdom memory allocation for DMA")

---
Changes in v2:
- Split the behavior-preserving placement refactoring into the previous patch.
- Reuse the refactored Arm kernel and DTB/initrd placement helpers for the first-bank candidate check.

Link to v1:
https://patchew.org/Xen/4f862bb2dc323914b8120b0f16af7516140cf42b.1780065103.git.mykola._5Fkvach@epam.com/

Changes since RFC:
- Do not keep the RFC scalar minimum-size check. It can both reject valid layouts and accept layouts which still fail later. Instead, validate the candidate bank using the same kernel and DTB/initrd placement rules as the load path.
- Replace the scalar minimum-size check with arch_hwdom_first_bank_ok().
- Validate fixed-address and AArch32 start == 0 kernel placement against the candidate bank.
- Check the candidate after capping the host region by the remaining unassigned hardware-domain memory.
- Treat the hardware-domain FDT allocation size as a conservative upper bound because the final FDT is generated later.

Link to RFC: https://patchew.org/Xen/9ae4f7dd49f5b1f761193adae573c2675c92e883.1779051035.git.mykola._5Fkvach@epam.com/

Why the RFC scalar approach was not kept:

A simple minimum-size check is not sufficient here because the validity of the first bank depends on the actual Arm placement rules, not only on the aggregate size of the kernel, DTB and initrd. The DTB/initrd area may fit before a 64-bit Image loaded with a text offset, while an AArch32 position-independent kernel may leave no valid module location even when the aggregate size appears to fit. Fixed-address kernels also need the candidate bank start to be considered.

---
 xen/arch/arm/acpi/domain_build.c        |  2 -
 xen/arch/arm/domain_build.c             |  8 ++++
 xen/arch/arm/include/asm/domain_build.h |  4 ++
 xen/arch/arm/include/asm/kernel.h       |  9 ++++
 xen/arch/arm/kernel.c                   | 57 ++++++++++++++++++++++++-
 xen/common/device-tree/domain-build.c   | 24 ++++++++---
 xen/include/xen/fdt-kernel.h            |  9 ++++
 7 files changed, 102 insertions(+), 11 deletions(-)
xen/arch/arm/acpi/domain_build.c +0 -2
index 249d899c33..db16f7fa94 100644
@@ -26,8 +26,6 @@
 #undef virt_to_mfn
 #define virt_to_mfn(va) _mfn(__virt_to_mfn(va))
 
-#define ACPI_DOM0_FDT_MIN_SIZE 4096
-
 static int __init acpi_iomem_deny_access(struct domain *d)
 {
     acpi_status status;
xen/arch/arm/domain_build.c +8 -0
index 1efddc60ef..550617f152 100644
@@ -115,6 +115,14 @@ int __init parse_arch_dom0_param(const char *s, const char *e)
                              (IS_ENABLED(CONFIG_STATIC_SHM) ?         \
                               (NR_SHMEM_BANKS * (160 + 16)) : 0))
 
+paddr_t __init hwdom_get_fdt_alloc_size(void)
+{
+    if ( acpi_disabled )
+        return fdt_totalsize(device_tree_flattened) + DOM0_FDT_EXTRA_SIZE;
+
+    return ACPI_DOM0_FDT_MIN_SIZE;
+}
+
 unsigned int __init dom0_max_vcpus(void)
 {
     if ( opt_dom0_max_vcpus == 0 )
xen/arch/arm/include/asm/domain_build.h +4 -0
index df8b361b3d..85cf46a958 100644
@@ -19,6 +19,10 @@ int prepare_acpi(struct domain *d, struct kernel_info *kinfo);
 
 int add_ext_regions(unsigned long s_gfn, unsigned long e_gfn, void *data);
 
+#define ACPI_DOM0_FDT_MIN_SIZE 4096
+
+paddr_t hwdom_get_fdt_alloc_size(void);
+
 #if defined(CONFIG_MPU) && defined(CONFIG_ARM_64)
 /* Utility function to determine if an Armv8-R processor supports VMSA. */
 bool has_v8r_vmsa_support(void);
xen/arch/arm/include/asm/kernel.h +9 -0
index 21f4273fa1..bf14fb208a 100644
@@ -8,12 +8,21 @@
 
 #include <asm/domain.h>
 
+#include <xen/types.h>
+
+struct kernel_info;
+
 struct arch_kernel_info
 {
     /* Enable pl011 emulation */
     bool vpl011;
 };
 
+#define arch_hwdom_first_bank_ok arch_hwdom_first_bank_ok
+bool arch_hwdom_first_bank_ok(const struct kernel_info *info,
+                              paddr_t bank_start,
+                              paddr_t bank_size);
+
 #endif /* #ifdef __ARCH_ARM_KERNEL_H__ */
 
 /*
xen/arch/arm/kernel.c +0 -0
index d1be4d8074..ecea2822a1 100644
@@ -64,9 +64,15 @@ kernel_zimage_place_in_bank(const struct kernel_info *info,
         load_end = bank_start + bank_size;
         load_end = MIN(bank_start + MB(128), load_end);
 
Quoted 9 lines

xen/arch/arm/kernel.c kernel_zimage_place_in_bank(const struct kernel_info *info, · line 75

Quoted 9 lines
xen/arch/arm/kernel.c +6 -0
+        if ( load_end - bank_start < info->image.len )
+            return INVALID_PADDR;
+
         load_addr = load_end - info->image.len;
         /* Align to 2MB */
         load_addr &= ~(MB(2) - 1);
+
+        if ( load_addr < bank_start )
+            return INVALID_PADDR;

After re-checking the hwdom allocation path, this post-alignment check looks redundant. Candidate bank starts come from the hwdom free-region list, where regions are already 2MB-aligned. The check before subtracting image.len already ensures that the computed load address does not go below bank_start, so aligning it down to 2MB cannot move it below bank_start for this path.

I will drop this check in the next revision.

Quoted 39 lines
Quoted 39 lines
xen/arch/arm/kernel.c +32 -0
     }
     else
         load_addr = info->image.start;
@@ -164,9 +170,56 @@ static void __init place_dtb_initrd(struct kernel_info *info,
 static paddr_t __init kernel_zimage_place(struct kernel_info *info)
 {
     const struct membanks *mem = kernel_info_get_mem(info);
+    paddr_t load_addr;
+
+    load_addr = kernel_zimage_place_in_bank(info, mem->bank[0].start,
+                                            mem->bank[0].size);
+    if ( load_addr == INVALID_PADDR )
+        panic("Unable to find suitable location for the kernel\n");
+
+    return load_addr;
+}
+
+bool __init arch_hwdom_first_bank_ok(const struct kernel_info *info,
+                                     paddr_t bank_start,
+                                     paddr_t bank_size)
+{
+    const struct boot_module *initrd = info->bd.initrd;
+    /*
+     * place_dtb_initrd() rounds the DTB and initrd placement to 2MB boundaries;
+     * use the same granularity when checking whether the first bank can hold
+     * them.
+     */
+    const paddr_t initrd_len = ROUNDUP(initrd ? initrd->size : 0, MB(2));
+    /*
+     * The hardware domain FDT has not been generated yet. Use the allocation
+     * size as a conservative upper bound for the final DTB size.
+     */
+    const paddr_t dtb_len = ROUNDUP(hwdom_get_fdt_alloc_size(), MB(2));
+    const paddr_t rambase = bank_start;
+    const paddr_t ramsize = bank_size;
+    const paddr_t dtb_initrd_size = initrd_len + dtb_len;
+    const paddr_t ramend = rambase + ramsize;
+    paddr_t kernbase;
+    paddr_t kernend;
Quoted 8 lines

xen/arch/arm/kernel.c static void __init place_dtb_initrd(struct kernel_info *info, · line 209

Quoted 5 lines
xen/arch/arm/kernel.c +5 -0
+    paddr_t dtb_base;
+
+    kernbase = kernel_zimage_place_in_bank(info, bank_start, bank_size);
+    if ( kernbase == INVALID_PADDR ||
+         info->image.len > INVALID_PADDR - kernbase )

Max IPA is 48bit, far from 64bit, where the arch max is 52bit and image.len is a kernel image size, so kernbase + image.len cannot wrap a 64-bit paddr_t, so this check is dead. Drop it.

Ack, I will drop this check.

Quoted 7 lines

xen/arch/arm/kernel.c static void __init place_dtb_initrd(struct kernel_info *info, · line 214

Quoted 6 lines
xen/arch/arm/kernel.c +5 -0
+        return false;
+
+    kernend = kernbase + info->image.len;
+
+    if ( kernbase < rambase || kernend > ramend )

Please add braces around individual expressions.

Ack, I will make the condition explicit.

Quoted 64 lines
Quoted 64 lines
xen/arch/arm/kernel.c +7 -2
+        return false;
+
+    if ( !first_bank_can_fit_modules(ramsize, kernbase, kernend,
+                                     dtb_initrd_size) )
+        return false;
 
-    return kernel_zimage_place_in_bank(info, mem->bank[0].start,
-                                       mem->bank[0].size);
+    return find_dtb_initrd_placement(rambase, ramend, kernbase, kernend,
+                                     dtb_initrd_size, &dtb_base);
 }
 
 static void __init kernel_zimage_load(struct kernel_info *info)
xen/common/device-tree/domain-build.c +17 -7
index f3ba496f1e..2e806c1b09 100644
@@ -299,20 +299,30 @@ static bool __init allocate_hwdom_memory(struct kernel_info *kinfo)
 
     for ( i = 0; (kinfo->unassigned_mem > 0) && (i < nr_banks); i++ )
     {
-        paddr_t bank_size;
+        const paddr_t bank_start = hwdom_free_mem->bank[i].start;
+        paddr_t bank_size = hwdom_free_mem->bank[i].size;
+
+        /*
+         * Check the size that would actually be assigned, not just the size
+         * of the host region.
+         */
+        bank_size = min(bank_size, kinfo->unassigned_mem);
 
         /*
          * The first bank must be large enough for place_dtb_initrd() to
          * fit the kernel, DTB and initrd.  Skip small regions to avoid
          * ending up with a tiny first bank.
          */
-        if ( !mem->nr_banks && (hwdom_free_mem->bank[i].size < min_bank_size) )
-            continue;
+        if ( !mem->nr_banks )
+        {
+            if ( bank_size < min_bank_size )
+                continue;
+
+            if ( !arch_hwdom_first_bank_ok(kinfo, bank_start, bank_size) )
+                continue;
+        }
 
-        bank_size = MIN(hwdom_free_mem->bank[i].size, kinfo->unassigned_mem);
-        if ( !allocate_bank_memory(kinfo,
-                                   gaddr_to_gfn(hwdom_free_mem->bank[i].start),
-                                   bank_size) )
+        if ( !allocate_bank_memory(kinfo, gaddr_to_gfn(bank_start), bank_size) )
         {
             xfree(hwdom_free_mem);
             return false;
xen/include/xen/fdt-kernel.h +0 -0
index 00c37be101..71e2344b97 100644
@@ -93,6 +93,15 @@ kernel_info_get_mem_const(const struct kernel_info *kinfo)
     return container_of(&kinfo->mem.common, const struct membanks, common);
 }
 
Quoted 6 lines

xen/include/xen/fdt-kernel.h kernel_info_get_mem_const(const struct kernel_info *kinfo) · line 97

Quoted 2 lines
xen/include/xen/fdt-kernel.h +2 -0
+#ifndef arch_hwdom_first_bank_ok
+static inline bool arch_hwdom_first_bank_ok(const struct kernel_info *info,

This should deserve a comment describing its contract. Also, the name is not very descriptive: how about arch_hwdom_first_bank_can_fit_modules() similar to generic Arm's first_bank_can_fit_modules()?

Ack, I will add a contract comment and rename the hook to arch_hwdom_first_bank_can_fit_modules().

Best regards,
Mykola

#7 lore raw
In reply to Mykola Kvach

Hello Mykola,

Quoted 1 line · Mykola Kvach

From: Mykola Kvach <[email protected]>

Just out of curiosity, is it okay that the email address listed here differs from the one you're using to send patches? (By the way, nice email address ;))

Quoted 20 lines

With LLC coloring enabled, the hardware domain memory is allocated by allocate_hwdom_memory() instead of relying on the fixed direct-map layout. Since de99f3263555 ("device-tree: Improve hwdom memory allocation for DMA"), the allocator prefers lower host regions, but the first-bank filter still only enforces the old 128MB heuristic.

A low candidate bank can satisfy that heuristic while still being unsuitable for the Arm kernel/DTB/initrd placement rules. Split the Arm placement logic into reusable helpers first, then validate the candidate hardware-domain bank 0 against those helpers before allocating it.

Changes in v2:
- Split the previous single patch into a behavior-preserving Arm placement refactoring and a separate hardware-domain first-bank fix.
- Rename find_module_placement() and place_modules() to use DTB/initrd terminology.
- Pass ramend to find_dtb_initrd_placement().
- Update the stale tools/libs/guest/xg_dom_arm.c reference.

Quoted 3 lines

Link to v1:

   https://patchew.org/Xen/4f862bb2dc323914b8120b0f16af7516140cf42b.1780065103.git.mykola._5Fkvach@epam.com/

I would like to clarify whether my understanding is correct.

With LLC coloring enabled on ARM, the allocator could assign the hardware domain a bank 0 that passes the size check but is unsuitable for kernel/DTB/initrd placement. As a result, the hardware domain may fail to boot or could silently corrupt memory if the images are placed in an incorrect location.

Is my understanding correct?

Thanks in advance.

~ Oleksii

#8 lore raw
Quoted 35 lines · Oleksii Kurochko

Hello Mykola,

Quoted 1 line · Mykola Kvach

From: Mykola Kvach <[email protected]>

Just out of curiosity, is it okay that the email address listed here differs from the one you're using to send patches? (By the way, nice email address ;))

Quoted 20 lines

With LLC coloring enabled, the hardware domain memory is allocated by allocate_hwdom_memory() instead of relying on the fixed direct-map layout. Since de99f3263555 ("device-tree: Improve hwdom memory allocation for DMA"), the allocator prefers lower host regions, but the first-bank filter still only enforces the old 128MB heuristic.

A low candidate bank can satisfy that heuristic while still being unsuitable for the Arm kernel/DTB/initrd placement rules. Split the Arm placement logic into reusable helpers first, then validate the candidate hardware-domain bank 0 against those helpers before allocating it.

Changes in v2:
- Split the previous single patch into a behavior-preserving Arm placement refactoring and a separate hardware-domain first-bank fix.
- Rename find_module_placement() and place_modules() to use DTB/initrd terminology.
- Pass ramend to find_dtb_initrd_placement().
- Update the stale tools/libs/guest/xg_dom_arm.c reference.

Quoted 3 lines

Link to v1:

   https://patchew.org/Xen/4f862bb2dc323914b8120b0f16af7516140cf42b.1780065103.git.mykola._5Fkvach@epam.com/

I would like to clarify whether my understanding is correct.

Quoted 7 lines

With LLC coloring enabled on ARM, the allocator could assign the hardware domain a bank 0 that passes the size check but is unsuitable for kernel/DTB/initrd placement. As a result, the hardware domain may fail to boot or could silently corrupt memory if the images are placed in an incorrect location.

Is my understanding correct?

Yes, Xen would fail to boot such domain. On Arm we carry a lot of placement logic we need to deal with...

~Michal

#9 lore raw
Quoted 44 lines · Michal
Quoted 41 lines · Oleksii Kurochko

Hello Mykola,

Quoted 1 line · Mykola Kvach

From: Mykola Kvach <[email protected]>

Just out of curiosity, is it okay that the email address listed here differs from the one you're using to send patches? (By the way, nice email address ;))

Quoted 20 lines

With LLC coloring enabled, the hardware domain memory is allocated by allocate_hwdom_memory() instead of relying on the fixed direct-map layout. Since de99f3263555 ("device-tree: Improve hwdom memory allocation for DMA"), the allocator prefers lower host regions, but the first-bank filter still only enforces the old 128MB heuristic.

A low candidate bank can satisfy that heuristic while still being unsuitable for the Arm kernel/DTB/initrd placement rules. Split the Arm placement logic into reusable helpers first, then validate the candidate hardware-domain bank 0 against those helpers before allocating it.

Changes in v2:
- Split the previous single patch into a behavior-preserving Arm placement refactoring and a separate hardware-domain first-bank fix.
- Rename find_module_placement() and place_modules() to use DTB/initrd terminology.
- Pass ramend to find_dtb_initrd_placement().
- Update the stale tools/libs/guest/xg_dom_arm.c reference.

Quoted 3 lines

Link to v1:

    https://patchew.org/Xen/4f862bb2dc323914b8120b0f16af7516140cf42b.1780065103.git.mykola._5Fkvach@epam.com/

I would like to clarify whether my understanding is correct.

With LLC coloring enabled on ARM, the allocator could assign the hardware domain a bank 0 that passes the size check but is unsuitable for kernel/DTB/initrd placement. As a result, the hardware domain may fail to boot or could silently corrupt memory if the images are placed in an incorrect location.

Quoted 3 lines
Quoted 1 line

Is my understanding correct?

Yes, Xen would fail to boot such domain. On Arm we carry a lot of placement logic we need to deal with...

Thanks for confirming. Then for this patch series:

~ Oleksii

A message that is not in the archive (sent off-list or not yet received): CAGeoDV_RRjMm7dHa4VJ4AwUypUg4kUx7kPYjM6Sg5_NafHQX1A@mail.gmail.com

#11 lore raw
Quoted 14 lines · Mykola Kvach

Hi Oleksii,

Quoted 9 lines · Oleksii Kurochko

Hello Mykola,

Quoted 1 line · Mykola Kvach

From: Mykola Kvach <[email protected]>

Just out of curiosity, is it okay that the email address listed here differs from the one you're using to send patches? (By the way, nice email address ;))

Quoted 5 lines

Yes, this is intentional. The patch author and Signed-off-by use my work address, while I send patches through another account because that is the mailing-list setup I currently have configured.

Please let me know if this is a problem for Xen submissions.

No, I don’t think it’s a problem. I was just curious whether it was intentional.

~ Oleksii