Skip to contentSkip to conversation

Hyperlaunch device tree for dom0

PATCH In review Alejandro Vallejo 4 participants 33 messages Apr 29, 2025 – Jun 12, 2025

lore
Share & tools
lore
#0 lore raw

v6 cover Hyperlaunch device tree for dom0

Hi,

Not very many changes here. Just:

v5->v6:
* Denis' suggestion to rename a few helpers to fdt_*
* Change to last patch to only pass CDF_iommu to domains with DOMAIN_CAPS_HARDWARE.

I _think_ this addresses all feedback I got so far and I don't expect anything major remaining before commit. If there's something I was asked and I haven't delivered yet, please bring it up again.

v5: https://lore.kernel.org/xen-devel/[email protected]/
v4: https://lore.kernel.org/xen-devel/[email protected]/
v3: https://lore.kernel.org/xen-devel/[email protected]/
v2: https://lore.kernel.org/xen-devel/[email protected]/
v1: https://lore.kernel.org/xen-devel/[email protected]/

========= Original cover letter:

The Hyperlaunch device tree for dom0 series is the second split out for the introduction of the Hyperlaunch domain builder logic. These changes focus on introducing the ability to express a domain configuration that is then used to populate the struct boot_domain structure for dom0. This ability to express a domain configuration provides the next step towards a general domain builder.

The splitting of Hyperlaunch into a set of series are twofold, to reduce the effort in reviewing a much larger series, and to reduce the effort in handling the knock-on effects to the construction logic from requested review changes.

Alejandro Vallejo (1):
x86/hyperlaunch: Add helpers to locate multiboot modules

Daniel P. Smith (11):
kconfig: introduce CONFIG_DOMAIN_BUILDER common/hyperlaunch: introduce the domain builder x86/hyperlaunch: initial support for hyperlaunch device tree x86/hyperlaunch: locate dom0 kernel with hyperlaunch x86/hyperlaunch: obtain cmdline from device tree x86/hyperlaunch: locate dom0 initrd with hyperlaunch x86/hyperlaunch: add domain id parsing to domain config x86/hyperlaunch: specify dom0 mode with device tree x86/hyperlaunch: add memory parsing to domain config x86/hyperlaunch: add max vcpu parsing of hyperlaunch device tree x86/hyperlaunch: add capabilities to boot domain

 xen/arch/x86/dom0_build.c              |  11 +
 xen/arch/x86/include/asm/boot-domain.h |  14 +
 xen/arch/x86/include/asm/bootinfo.h    |  10 +-
 xen/arch/x86/setup.c                   |  66 +++-
 xen/common/Kconfig                     |   2 +
 xen/common/Makefile                    |   1 +
 xen/common/domain-builder/Kconfig      |  15 +
 xen/common/domain-builder/Makefile     |   2 +
 xen/common/domain-builder/core.c       |  86 +++++
 xen/common/domain-builder/fdt.c        | 512 +++++++++++++++++++++++++
 xen/common/domain-builder/fdt.h        |  40 ++
 xen/include/xen/domain-builder.h       |  37 ++
 xen/include/xen/libfdt/libfdt-xen.h    |  23 ++
 13 files changed, 801 insertions(+), 18 deletions(-)
 create mode 100644 xen/common/domain-builder/Kconfig
 create mode 100644 xen/common/domain-builder/Makefile
 create mode 100644 xen/common/domain-builder/core.c
 create mode 100644 xen/common/domain-builder/fdt.c
 create mode 100644 xen/common/domain-builder/fdt.h
 create mode 100644 xen/include/xen/domain-builder.h
Signature · 1 line
2.43.0
#1 lore raw

PATCH v6 1/12 kconfig: introduce CONFIG_DOMAIN_BUILDER

From: "Daniel P. Smith" <[email protected]>

Hyperlaunch domain builder will be the consolidated boot time domain building logic framework. Introduce the config option to enable this domain builder to eventually turn on the ability to load the domain configuration via a flattened device tree.

---
 xen/common/Kconfig                |  2 ++
 xen/common/domain-builder/Kconfig | 15 +++++++++++++++
 2 files changed, 17 insertions(+)
 create mode 100644 xen/common/domain-builder/Kconfig
xen/common/Kconfig +2 -0
index be28060716..e025fbe257 100644
@@ -144,6 +144,8 @@ config STATIC_MEMORY
 
 	  If unsure, say N.
 
+source "common/domain-builder/Kconfig"
+
 menu "Speculative hardening"
 
 config INDIRECT_THUNK
xen/common/domain-builder/Kconfig new file +15 -0
new file mode 100644
index 0000000000..44b8351af8
@@ -0,0 +1,15 @@
+menu "Domain Builder Features"
+
+config DOMAIN_BUILDER
+	bool "Domain builder (UNSUPPORTED)" if UNSUPPORTED && X86
+	select LIBFDT
+	help
+	  Support for constructing predefined domains described by a flattened
+	  device tree. This allows constructing multiple domains at boot time
+	  instead of being limited to a single dom0.
+
+	  This feature is currently experimental.
+
+	  If unsure, say N.
+
+endmenu
Signature · 1 line
2.43.0
#2 lore raw

PATCH v6 2/12 common/hyperlaunch: introduce the domain builder

In reply to Alejandro Vallejo

From: "Daniel P. Smith" <[email protected]>

Introduce the domain builder which is capable of consuming a device tree as the first boot module. If it finds a device tree as the first boot module, it will set its type to BOOTMOD_FDT. This change only detects the boot module and continues to boot with slight change to the boot convention that the dom0 kernel is no longer first boot module but is the second.

No functional change intended.

---
 xen/arch/x86/include/asm/bootinfo.h |  3 ++
 xen/arch/x86/setup.c                | 19 ++++++++----
 xen/common/Makefile                 |  1 +
 xen/common/domain-builder/Makefile  |  2 ++
 xen/common/domain-builder/core.c    | 45 +++++++++++++++++++++++++++++
 xen/common/domain-builder/fdt.c     | 39 +++++++++++++++++++++++++
 xen/common/domain-builder/fdt.h     | 14 +++++++++
 xen/include/xen/domain-builder.h    | 29 +++++++++++++++++++
 8 files changed, 146 insertions(+), 6 deletions(-)
 create mode 100644 xen/common/domain-builder/Makefile
 create mode 100644 xen/common/domain-builder/core.c
 create mode 100644 xen/common/domain-builder/fdt.c
 create mode 100644 xen/common/domain-builder/fdt.h
 create mode 100644 xen/include/xen/domain-builder.h
xen/arch/x86/include/asm/bootinfo.h +3 -0
index 3afc214c17..82c2650fcf 100644
@@ -27,6 +27,7 @@ enum bootmod_type {
     BOOTMOD_RAMDISK,
     BOOTMOD_MICROCODE,
     BOOTMOD_XSM_POLICY,
+    BOOTMOD_FDT,
 };
 
 struct boot_module {
@@ -80,6 +81,8 @@ struct boot_info {
     paddr_t memmap_addr;
     size_t memmap_length;
 
+    bool hyperlaunch_enabled;
+
     unsigned int nr_modules;
     struct boot_module mods[MAX_NR_BOOTMODS + 1];
     struct boot_domain domains[MAX_NR_BOOTDOMS];
xen/arch/x86/setup.c +13 -6
index 2518954124..f3b5c83a3c 100644
@@ -6,6 +6,7 @@
 #include <xen/cpuidle.h>
 #include <xen/dmi.h>
 #include <xen/domain.h>
+#include <xen/domain-builder.h>
 #include <xen/domain_page.h>
 #include <xen/efi.h>
 #include <xen/err.h>
@@ -1284,9 +1285,14 @@ void asmlinkage __init noreturn __start_xen(void)
                bi->nr_modules);
     }
 
-    /* Dom0 kernel is always first */
-    bi->mods[0].type = BOOTMOD_KERNEL;
-    bi->domains[0].kernel = &bi->mods[0];
+    if ( builder_init(bi) == FDT_KIND_NONE )
+    {
+        /* Find first unknown boot module to use as dom0 kernel */
+        i = first_boot_module_index(bi, BOOTMOD_UNKNOWN);
+        bi->mods[i].type = BOOTMOD_KERNEL;
+        bi->domains[0].kernel = &bi->mods[i];
+        bi->hyperlaunch_enabled = false;
+    }
 
     if ( pvh_boot )
     {
@@ -1469,8 +1475,9 @@ void asmlinkage __init noreturn __start_xen(void)
         xen->size  = __2M_rwdata_end - _stext;
     }
 
-    bi->mods[0].headroom =
-        bzimage_headroom(bootstrap_map_bm(&bi->mods[0]), bi->mods[0].size);
+    bi->domains[0].kernel->headroom =
+        bzimage_headroom(bootstrap_map_bm(bi->domains[0].kernel),
+                         bi->domains[0].kernel->size);
     bootstrap_unmap();
 
 #ifndef highmem_start
@@ -1594,7 +1601,7 @@ void asmlinkage __init noreturn __start_xen(void)
 #endif
     }
 
-    if ( bi->mods[0].headroom && !bi->mods[0].relocated )
+    if ( bi->domains[0].kernel->headroom && !bi->domains[0].kernel->relocated )
         panic("Not enough memory to relocate the dom0 kernel image\n");
     for ( i = 0; i < bi->nr_modules; ++i )
     {
xen/common/Makefile +1 -0
index 98f0873056..e42af71e3f 100644
@@ -11,6 +11,7 @@ obj-$(filter-out $(CONFIG_X86),$(CONFIG_ACPI)) += device.o
 obj-$(CONFIG_HAS_DEVICE_TREE) += device-tree/
 obj-$(CONFIG_IOREQ_SERVER) += dm.o
 obj-y += domain.o
+obj-$(CONFIG_DOMAIN_BUILDER) += domain-builder/
 obj-y += event_2l.o
 obj-y += event_channel.o
 obj-$(CONFIG_EVTCHN_FIFO) += event_fifo.o
xen/common/domain-builder/Makefile new file +2 -0
new file mode 100644
index 0000000000..bfd2f6267e
@@ -0,0 +1,2 @@
+obj-y += fdt.init.o
+obj-y += core.init.o
xen/common/domain-builder/core.c new file +45 -0

45 lines not on this page · raw message

xen/common/domain-builder/fdt.c new file +39 -0
new file mode 100644
index 0000000000..4b07bd22c8
@@ -0,0 +1,39 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2024, Apertus Solutions, LLC
+ */
+#include <xen/err.h>
+#include <xen/init.h>
+#include <xen/lib.h>
+#include <xen/libfdt/libfdt.h>
+
+#include <asm/bootinfo.h>
+#include <asm/page.h>
+#include <asm/setup.h>
+
+#include "fdt.h"
+
+enum fdt_kind __init fdt_detect_kind(const struct boot_info *bi)
+{
+    enum fdt_kind kind;
+    const void *fdt = bootstrap_map_bm(&bi->mods[HYPERLAUNCH_MODULE_IDX]);
+
+    if ( !fdt || fdt_check_header(fdt) < 0 )
+        kind = FDT_KIND_NONE;
+    else
+        kind = FDT_KIND_UNKNOWN;
+
+    bootstrap_unmap();
+
+    return kind;
+}
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
xen/common/domain-builder/fdt.h new file +14 -0
new file mode 100644
index 0000000000..ef897fc412
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef __XEN_DOMAIN_BUILDER_FDT_H__
+#define __XEN_DOMAIN_BUILDER_FDT_H__
+
+#include <xen/domain-builder.h>
+
+struct boot_info;
+
+/* hyperlaunch fdt is required to be module 0 */
+#define HYPERLAUNCH_MODULE_IDX 0
+
+enum fdt_kind fdt_detect_kind(const struct boot_info *bi);
+
+#endif /* __XEN_DOMAIN_BUILDER_FDT_H__ */
xen/include/xen/domain-builder.h new file +29 -0
new file mode 100644
index 0000000000..b9702db735
@@ -0,0 +1,29 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef __XEN_DOMAIN_BUILDER_H__
+#define __XEN_DOMAIN_BUILDER_H__
+
+struct boot_info;
+
+/* Return status of builder_init(). Shows which boot mechanism was detected */
+enum fdt_kind
+{
+    /* FDT not found. Skipped builder. */
+    FDT_KIND_NONE,
+    /* Found an FDT that wasn't hyperlaunch. */
+    FDT_KIND_UNKNOWN,
+};
+
+/*
+ * Initialises `bi` if it detects a compatible FDT. Otherwise returns
+ * FDT_KIND_NONE and leaves initialisation up to the caller.
+ */
+#if IS_ENABLED(CONFIG_DOMAIN_BUILDER)
+enum fdt_kind builder_init(struct boot_info *bi);
+#else
+static inline enum fdt_kind builder_init(struct boot_info *bi)
+{
+    return FDT_KIND_NONE;
+}
+#endif /* !IS_ENABLED(CONFIG_DOMAIN_BUILDER) */
+
+#endif /* __XEN_DOMAIN_BUILDER_H__ */
Signature · 1 line
2.43.0
#3 lore raw

Re: [PATCH v6 02/12] common/hyperlaunch: introduce the domain builder

Quoted 107 lines · Alejandro Vallejo

From: "Daniel P. Smith" <[email protected]>

Introduce the domain builder which is capable of consuming a device tree as the first boot module. If it finds a device tree as the first boot module, it will set its type to BOOTMOD_FDT. This change only detects the boot module and continues to boot with slight change to the boot convention that the dom0 kernel is no longer first boot module but is the second.

No functional change intended.

---
  xen/arch/x86/include/asm/bootinfo.h |  3 ++
  xen/arch/x86/setup.c                | 19 ++++++++----
  xen/common/Makefile                 |  1 +
  xen/common/domain-builder/Makefile  |  2 ++
  xen/common/domain-builder/core.c    | 45 +++++++++++++++++++++++++++++
  xen/common/domain-builder/fdt.c     | 39 +++++++++++++++++++++++++
  xen/common/domain-builder/fdt.h     | 14 +++++++++
  xen/include/xen/domain-builder.h    | 29 +++++++++++++++++++
  8 files changed, 146 insertions(+), 6 deletions(-)

create mode 100644 xen/common/domain-builder/Makefile create mode 100644 xen/common/domain-builder/core.c create mode 100644 xen/common/domain-builder/fdt.c create mode 100644 xen/common/domain-builder/fdt.h create mode 100644 xen/include/xen/domain-builder.h

xen/arch/x86/include/asm/bootinfo.h +3 -0
index 3afc214c17..82c2650fcf 100644
@@ -27,6 +27,7 @@ enum bootmod_type {
      BOOTMOD_RAMDISK,
      BOOTMOD_MICROCODE,
      BOOTMOD_XSM_POLICY,
+    BOOTMOD_FDT,
  };
 
  struct boot_module {
@@ -80,6 +81,8 @@ struct boot_info {
      paddr_t memmap_addr;
      size_t memmap_length;
 
+    bool hyperlaunch_enabled;
+
      unsigned int nr_modules;
      struct boot_module mods[MAX_NR_BOOTMODS + 1];
      struct boot_domain domains[MAX_NR_BOOTDOMS];
xen/arch/x86/setup.c +13 -6
index 2518954124..f3b5c83a3c 100644
@@ -6,6 +6,7 @@
  #include <xen/cpuidle.h>
  #include <xen/dmi.h>
  #include <xen/domain.h>
+#include <xen/domain-builder.h>
  #include <xen/domain_page.h>
  #include <xen/efi.h>
  #include <xen/err.h>
@@ -1284,9 +1285,14 @@ void asmlinkage __init noreturn __start_xen(void)
                 bi->nr_modules);
      }
 
-    /* Dom0 kernel is always first */
-    bi->mods[0].type = BOOTMOD_KERNEL;
-    bi->domains[0].kernel = &bi->mods[0];
+    if ( builder_init(bi) == FDT_KIND_NONE )
+    {
+        /* Find first unknown boot module to use as dom0 kernel */
+        i = first_boot_module_index(bi, BOOTMOD_UNKNOWN);
+        bi->mods[i].type = BOOTMOD_KERNEL;
+        bi->domains[0].kernel = &bi->mods[i];
+        bi->hyperlaunch_enabled = false;
+    }
 
      if ( pvh_boot )
      {
@@ -1469,8 +1475,9 @@ void asmlinkage __init noreturn __start_xen(void)
          xen->size  = __2M_rwdata_end - _stext;
      }
 
-    bi->mods[0].headroom =
-        bzimage_headroom(bootstrap_map_bm(&bi->mods[0]), bi->mods[0].size);
+    bi->domains[0].kernel->headroom =
+        bzimage_headroom(bootstrap_map_bm(bi->domains[0].kernel),
+                         bi->domains[0].kernel->size);
      bootstrap_unmap();
 
  #ifndef highmem_start
@@ -1594,7 +1601,7 @@ void asmlinkage __init noreturn __start_xen(void)
  #endif
      }
 
-    if ( bi->mods[0].headroom && !bi->mods[0].relocated )
+    if ( bi->domains[0].kernel->headroom && !bi->domains[0].kernel->relocated )
          panic("Not enough memory to relocate the dom0 kernel image\n");
      for ( i = 0; i < bi->nr_modules; ++i )
      {
xen/common/Makefile +0 -0
index 98f0873056..e42af71e3f 100644

xen/common/Makefile obj-$(filter-out $(CONFIG_X86),$(CONFIG_ACPI)) += device.o · line 14

Quoted 4 lines
xen/common/Makefile +1 -0
@@ -11,6 +11,7 @@ obj-$(filter-out $(CONFIG_X86),$(CONFIG_ACPI)) += device.o
  obj-$(CONFIG_HAS_DEVICE_TREE) += device-tree/
  obj-$(CONFIG_IOREQ_SERVER) += dm.o
  obj-y += domain.o
+obj-$(CONFIG_DOMAIN_BUILDER) += domain-builder/

Please don't do this, use IF_ENABLED in core.c and then hide the unnecessary units in domain-builder/Makefile as I originally had it. This allows for a much easier time incrementally converting the dom0 construction path into a generalized domain construction path.

Quoted 162 lines
xen/common/Makefile +0 -0
  obj-y += event_2l.o
  obj-y += event_channel.o
  obj-$(CONFIG_EVTCHN_FIFO) += event_fifo.o
xen/common/domain-builder/Makefile new file +2 -0
new file mode 100644
index 0000000000..bfd2f6267e
@@ -0,0 +1,2 @@
+obj-y += fdt.init.o
+obj-y += core.init.o
xen/common/domain-builder/core.c new file +45 -0
new file mode 100644
index 0000000000..97c92db571
@@ -0,0 +1,45 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2024, Apertus Solutions, LLC
+ */
+#include <xen/bug.h>
+#include <xen/err.h>
+#include <xen/init.h>
+#include <xen/kconfig.h>
+#include <xen/domain-builder.h>
+#include <xen/lib.h>
+
+#include <asm/bootinfo.h>
+
+#include "fdt.h"
+
+enum fdt_kind __init builder_init(struct boot_info *bi)
+{
+    enum fdt_kind kind;
+
+    bi->hyperlaunch_enabled = false;
+    switch ( (kind = fdt_detect_kind(bi)) )
+    {
+    case FDT_KIND_NONE:
+        /* No DT found */
+        return kind;
+
+    case FDT_KIND_UNKNOWN:
+        printk(XENLOG_DEBUG "DT found: non-hyperlaunch\n");
+        bi->mods[0].type = BOOTMOD_FDT;
+        return kind;
+
+    default:
+        BUG();
+    }
+}
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
xen/common/domain-builder/fdt.c new file +39 -0
new file mode 100644
index 0000000000..4b07bd22c8
@@ -0,0 +1,39 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2024, Apertus Solutions, LLC
+ */
+#include <xen/err.h>
+#include <xen/init.h>
+#include <xen/lib.h>
+#include <xen/libfdt/libfdt.h>
+
+#include <asm/bootinfo.h>
+#include <asm/page.h>
+#include <asm/setup.h>
+
+#include "fdt.h"
+
+enum fdt_kind __init fdt_detect_kind(const struct boot_info *bi)
+{
+    enum fdt_kind kind;
+    const void *fdt = bootstrap_map_bm(&bi->mods[HYPERLAUNCH_MODULE_IDX]);
+
+    if ( !fdt || fdt_check_header(fdt) < 0 )
+        kind = FDT_KIND_NONE;
+    else
+        kind = FDT_KIND_UNKNOWN;
+
+    bootstrap_unmap();
+
+    return kind;
+}
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
xen/common/domain-builder/fdt.h new file +14 -0
new file mode 100644
index 0000000000..ef897fc412
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef __XEN_DOMAIN_BUILDER_FDT_H__
+#define __XEN_DOMAIN_BUILDER_FDT_H__
+
+#include <xen/domain-builder.h>
+
+struct boot_info;
+
+/* hyperlaunch fdt is required to be module 0 */
+#define HYPERLAUNCH_MODULE_IDX 0
+
+enum fdt_kind fdt_detect_kind(const struct boot_info *bi);
+
+#endif /* __XEN_DOMAIN_BUILDER_FDT_H__ */
xen/include/xen/domain-builder.h new file +29 -0
new file mode 100644
index 0000000000..b9702db735
@@ -0,0 +1,29 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef __XEN_DOMAIN_BUILDER_H__
+#define __XEN_DOMAIN_BUILDER_H__
+
+struct boot_info;
+
+/* Return status of builder_init(). Shows which boot mechanism was detected */
+enum fdt_kind
+{
+    /* FDT not found. Skipped builder. */
+    FDT_KIND_NONE,
+    /* Found an FDT that wasn't hyperlaunch. */
+    FDT_KIND_UNKNOWN,
+};
+
+/*
+ * Initialises `bi` if it detects a compatible FDT. Otherwise returns
+ * FDT_KIND_NONE and leaves initialisation up to the caller.
+ */
+#if IS_ENABLED(CONFIG_DOMAIN_BUILDER)
+enum fdt_kind builder_init(struct boot_info *bi);
+#else
+static inline enum fdt_kind builder_init(struct boot_info *bi)
+{
+    return FDT_KIND_NONE;
+}
+#endif /* !IS_ENABLED(CONFIG_DOMAIN_BUILDER) */
+
+#endif /* __XEN_DOMAIN_BUILDER_H__ */
#4 lore raw

Re: [PATCH v6 02/12] common/hyperlaunch: introduce the domain builder

Quoted 13 lines · Daniel P. Smith

xen/common/Makefile obj-$(filter-out $(CONFIG_X86),$(CONFIG_ACPI)) += device.o · line 14

Quoted 7 lines · Alejandro Vallejo
xen/common/Makefile +1 -0
@@ -11,6 +11,7 @@ obj-$(filter-out $(CONFIG_X86),$(CONFIG_ACPI)) += device.o
  obj-$(CONFIG_HAS_DEVICE_TREE) += device-tree/
  obj-$(CONFIG_IOREQ_SERVER) += dm.o
  obj-y += domain.o
+obj-$(CONFIG_DOMAIN_BUILDER) += domain-builder/

Please don't do this, use IF_ENABLED in core.c and then hide the unnecessary units in domain-builder/Makefile as I originally had it. This allows for a much easier time incrementally converting the dom0 construction path into a generalized domain construction path.

That is, are you viewing this as a transitional thing only? If the end goal is to have it as Alejandro has it above, that may be acceptable (even if not nice).

Jan

#5 lore raw

Re: [PATCH v6 02/12] common/hyperlaunch: introduce the domain builder

Quoted 15 lines · Jan Beulich
Quoted 13 lines · Daniel P. Smith

xen/common/Makefile obj-$(filter-out $(CONFIG_X86),$(CONFIG_ACPI)) += device.o · line 14

Quoted 7 lines · Alejandro Vallejo
xen/common/Makefile +1 -0
@@ -11,6 +11,7 @@ obj-$(filter-out $(CONFIG_X86),$(CONFIG_ACPI)) += device.o
   obj-$(CONFIG_HAS_DEVICE_TREE) += device-tree/
   obj-$(CONFIG_IOREQ_SERVER) += dm.o
   obj-y += domain.o
+obj-$(CONFIG_DOMAIN_BUILDER) += domain-builder/

Please don't do this, use IF_ENABLED in core.c and then hide the unnecessary units in domain-builder/Makefile as I originally had it. This allows for a much easier time incrementally converting the dom0 construction path into a generalized domain construction path.

Quoted 3 lines

That is, are you viewing this as a transitional thing only? If the end goal is to have it as Alejandro has it above, that may be acceptable (even if not nice).

There is/will be shared domain construction code between dom0-only and multidomain construction, with it will all living under domain builder. So no, the end goal is not what Alejandro just did. The end result will be the way I had it, with a different kconfig option to enable/disable the multidomain construction path.

v/r,
dps

#6 lore raw

Re: [PATCH v6 02/12] common/hyperlaunch: introduce the domain builder

Quoted 20 lines · Daniel P. Smith
Quoted 15 lines · Jan Beulich
Quoted 13 lines · Daniel P. Smith

xen/common/Makefile obj-$(filter-out $(CONFIG_X86),$(CONFIG_ACPI)) += device.o · line 14

Quoted 7 lines · Alejandro Vallejo
xen/common/Makefile +1 -0
@@ -11,6 +11,7 @@ obj-$(filter-out $(CONFIG_X86),$(CONFIG_ACPI)) += device.o
   obj-$(CONFIG_HAS_DEVICE_TREE) += device-tree/
   obj-$(CONFIG_IOREQ_SERVER) += dm.o
   obj-y += domain.o
+obj-$(CONFIG_DOMAIN_BUILDER) += domain-builder/

Please don't do this, use IF_ENABLED in core.c and then hide the unnecessary units in domain-builder/Makefile as I originally had it. This allows for a much easier time incrementally converting the dom0 construction path into a generalized domain construction path.

Quoted 3 lines

That is, are you viewing this as a transitional thing only? If the end goal is to have it as Alejandro has it above, that may be acceptable (even if not nice).

Quoted 5 lines

There is/will be shared domain construction code between dom0-only and multidomain construction, with it will all living under domain builder. So no, the end goal is not what Alejandro just did. The end result will be the way I had it, with a different kconfig option to enable/disable the multidomain construction path.

Isn't CONFIG_DOMAIN_BUILDER a misnomer then?

Jan

#7 lore raw

Re: [PATCH v6 02/12] common/hyperlaunch: introduce the domain builder

Quoted 21 lines · Jan Beulich
Quoted 20 lines · Daniel P. Smith
Quoted 18 lines · Jan Beulich
Quoted 13 lines · Daniel P. Smith

xen/common/Makefile obj-$(filter-out $(CONFIG_X86),$(CONFIG_ACPI)) += device.o · line 14

Quoted 7 lines · Alejandro Vallejo
xen/common/Makefile +1 -0
@@ -11,6 +11,7 @@ obj-$(filter-out $(CONFIG_X86),$(CONFIG_ACPI)) += device.o
    obj-$(CONFIG_HAS_DEVICE_TREE) += device-tree/
    obj-$(CONFIG_IOREQ_SERVER) += dm.o
    obj-y += domain.o
+obj-$(CONFIG_DOMAIN_BUILDER) += domain-builder/

Please don't do this, use IF_ENABLED in core.c and then hide the unnecessary units in domain-builder/Makefile as I originally had it. This allows for a much easier time incrementally converting the dom0 construction path into a generalized domain construction path.

That is, are you viewing this as a transitional thing only? If the end goal is to have it as Alejandro has it above, that may be acceptable (even if not nice).

Quoted 7 lines
Quoted 5 lines

There is/will be shared domain construction code between dom0-only and multidomain construction, with it will all living under domain builder. So no, the end goal is not what Alejandro just did. The end result will be the way I had it, with a different kconfig option to enable/disable the multidomain construction path.

Isn't CONFIG_DOMAIN_BUILDER a misnomer then?

Which is why I originally had two kconfig flags, one to enable dtb parsing for domain configuration, which allowed dom0 construction from dtb. Then there was the second flag that was to enable multi-domain construction. I have reworked a portion of the approach in the RFC based on feedback, which is based off of this series. In it, I tried to minimize how much went into common, but you will see how I still had to rework the kconfig flags.

v/r,
dps

#8 lore raw

Re: [PATCH v6 02/12] common/hyperlaunch: introduce the domain builder

Quoted 38 lines · Daniel P. Smith
Quoted 21 lines · Jan Beulich
Quoted 20 lines · Daniel P. Smith
Quoted 18 lines · Jan Beulich
Quoted 13 lines · Daniel P. Smith

xen/common/Makefile obj-$(filter-out $(CONFIG_X86),$(CONFIG_ACPI)) += device.o · line 14

Quoted 7 lines · Alejandro Vallejo
xen/common/Makefile +1 -0
@@ -11,6 +11,7 @@ obj-$(filter-out $(CONFIG_X86),$(CONFIG_ACPI)) += device.o
    obj-$(CONFIG_HAS_DEVICE_TREE) += device-tree/
    obj-$(CONFIG_IOREQ_SERVER) += dm.o
    obj-y += domain.o
+obj-$(CONFIG_DOMAIN_BUILDER) += domain-builder/

Please don't do this, use IF_ENABLED in core.c and then hide the unnecessary units in domain-builder/Makefile as I originally had it. This allows for a much easier time incrementally converting the dom0 construction path into a generalized domain construction path.

That is, are you viewing this as a transitional thing only? If the end goal is to have it as Alejandro has it above, that may be acceptable (even if not nice).

Quoted 7 lines
Quoted 5 lines

There is/will be shared domain construction code between dom0-only and multidomain construction, with it will all living under domain builder. So no, the end goal is not what Alejandro just did. The end result will be the way I had it, with a different kconfig option to enable/disable the multidomain construction path.

Isn't CONFIG_DOMAIN_BUILDER a misnomer then?

Which is why I originally had two kconfig flags, one to enable dtb parsing for domain configuration, which allowed dom0 construction from dtb. Then there was the second flag that was to enable multi-domain construction. I have reworked a portion of the approach in the RFC based on feedback, which is based off of this series. In it, I tried to minimize how much went into common, but you will see how I still had to rework the kconfig flags.

Quoted 2 lines

v/r,
dps

Does it really make sense to have a flag to restrict multidomain while allowing parsing the DTB? There's virtually nothing compiled out in that case.

If you did it that way because it doesn't initially build several domains, that's just transitional and to be expected on any feature tagged as unsupported with (EXPERIMENTAL) in the name.

What if I collapse everything under a single CONFIG_MULTIDOMAIN_BUILDER that compiles-in support for parsing DTBs while introducing an unconditional builder as I go? From what I'm seeing, there are no breaking changes in the series and the end goal is to have the builder be unconditionally used, after all.

In fact, with the bindings code in common, I can also collapse everything in core.c (and later domain.c) into a single arch/x86/domain-builder.c that is unconditionally compiled in. The DTB parsing logic is already in separate files and that can be compiled out with CONFIG_MULTIDOMAIN_BUILDER flag.

In retrospect, after looking at dom0less long enough there's bootfdt.c and bootinfo.c with similar intent, but far more ad-hoc cohesion. While the builder wants to be in common, no other arch is in a position to take it. It needs merging with the stuff done in bootfdt.c/bootinfo.c

So, in short. I'm planning to:

1. Collapse domain-builder/{core,domain}.c into domain-builder.c under arch/x86. There's little reason to have them separate.
2. Remove CONFIG_DOMAIN_BUILDER, and replace it with something that reflects the intent of using a DTB. Either CONFIG_MULTIDOMAIN_BUILDER or CONFIG_DTB_BUILDER. Or maybe even CONFIG_DTB_BOOT.
* I specifically want to avoid CONFIG_BOOTFDT, because that'd create confusion with the already existing bootfdt.c in common.

and, from the discussion in the other thread about code sharing in the spirit of getting somewhere soon:

3. Do minimal parsing at builder_init() (module identification, basically), and do the full parsing by the create_dom0() mark, immediately before constructing all domains. With the unflattened tree.

Moving forward I for sure want to merge the boot paths in the x86 builder with those of arm/riscv. Having a unified boot frontend can only bring niceness.

Cheers,
Alejandro

#9 lore raw

Re: [PATCH v6 02/12] common/hyperlaunch: introduce the domain builder

In reply to Jan Beulich
Quoted 15 lines · Jan Beulich
Quoted 13 lines · Daniel P. Smith

xen/common/Makefile obj-$(filter-out $(CONFIG_X86),$(CONFIG_ACPI)) += device.o · line 14

Quoted 7 lines · Alejandro Vallejo
xen/common/Makefile +1 -0
@@ -11,6 +11,7 @@ obj-$(filter-out $(CONFIG_X86),$(CONFIG_ACPI)) += device.o
   obj-$(CONFIG_HAS_DEVICE_TREE) += device-tree/
   obj-$(CONFIG_IOREQ_SERVER) += dm.o
   obj-y += domain.o
+obj-$(CONFIG_DOMAIN_BUILDER) += domain-builder/

Please don't do this, use IF_ENABLED in core.c and then hide the unnecessary units in domain-builder/Makefile as I originally had it. This allows for a much easier time incrementally converting the dom0 construction path into a generalized domain construction path.

Quoted 3 lines

That is, are you viewing this as a transitional thing only? If the end goal is to have it as Alejandro has it above, that may be acceptable (even if not nice).

In the end, it became too much of a mess to keep the core builder functions with the fdt code in common. So I left this as is, and kept all the builder logic in x86/domain-builer/core.c.

v/r,
dps

#10 lore raw

Re: [PATCH v6 02/12] common/hyperlaunch: introduce the domain builder

In reply to Alejandro Vallejo

xen/arch/x86/setup.c void asmlinkage __init noreturn __start_xen(void) · line 1288

Quoted 8 lines · Alejandro Vallejo
xen/arch/x86/setup.c +1 -3
@@ -1284,9 +1285,14 @@ void asmlinkage __init noreturn __start_xen(void)
                bi->nr_modules);
     }
 
-    /* Dom0 kernel is always first */
-    bi->mods[0].type = BOOTMOD_KERNEL;
-    bi->domains[0].kernel = &bi->mods[0];
+    if ( builder_init(bi) == FDT_KIND_NONE )

With this, can ...

xen/arch/x86/setup.c void asmlinkage __init noreturn __start_xen(void) · line 1291

Quoted 3 lines
xen/arch/x86/setup.c +3 -0
+    {
+        /* Find first unknown boot module to use as dom0 kernel */
+        i = first_boot_module_index(bi, BOOTMOD_UNKNOWN);

... i ever be anything else than 0? If not, perhaps keeping the call here is still fine (kind of for doc purposes), but an assertion may then want adding.

xen/arch/x86/setup.c void asmlinkage __init noreturn __start_xen(void) · line 1294

Quoted 3 lines
xen/arch/x86/setup.c +3 -0
+        bi->mods[i].type = BOOTMOD_KERNEL;
+        bi->domains[0].kernel = &bi->mods[i];
+        bi->hyperlaunch_enabled = false;

Is this necessary, when the field is supposed to be starting out clear?

xen/common/domain-builder/Makefile line 2

Quoted 5 lines
xen/common/domain-builder/Makefile new file +2 -0
@@ -0,0 +1,2 @@
+obj-y += fdt.init.o
+obj-y += core.init.o

Any reason for these not both adding to obj-bin-y, like we do elsewhere for *.init.o?

Also please sort object lists alphabetically.

Quoted 13 lines
xen/include/xen/domain-builder.h new file +10 -0
@@ -0,0 +1,29 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef __XEN_DOMAIN_BUILDER_H__
+#define __XEN_DOMAIN_BUILDER_H__
+
+struct boot_info;
+
+/* Return status of builder_init(). Shows which boot mechanism was detected */
+enum fdt_kind
+{
+    /* FDT not found. Skipped builder. */

xen/include/xen/domain-builder.h line 20

Quoted 10 lines
xen/include/xen/domain-builder.h new file +10 -0
+    FDT_KIND_NONE,
+    /* Found an FDT that wasn't hyperlaunch. */
+    FDT_KIND_UNKNOWN,
+};
+
+/*
+ * Initialises `bi` if it detects a compatible FDT. Otherwise returns
+ * FDT_KIND_NONE and leaves initialisation up to the caller.
+ */
+#if IS_ENABLED(CONFIG_DOMAIN_BUILDER)

For the pre-processor it wants to be the simpler #ifdef.

Jan

#11 lore raw

Re: [PATCH v6 02/12] common/hyperlaunch: introduce the domain builder

Quoted 12 lines · Jan Beulich

xen/arch/x86/setup.c void asmlinkage __init noreturn __start_xen(void) · line 1288

Quoted 8 lines · Alejandro Vallejo
xen/arch/x86/setup.c +1 -3
@@ -1284,9 +1285,14 @@ void asmlinkage __init noreturn __start_xen(void)
                bi->nr_modules);
     }
 
-    /* Dom0 kernel is always first */
-    bi->mods[0].type = BOOTMOD_KERNEL;
-    bi->domains[0].kernel = &bi->mods[0];
+    if ( builder_init(bi) == FDT_KIND_NONE )

With this, can ...

Quoted 6 lines

xen/arch/x86/setup.c void asmlinkage __init noreturn __start_xen(void) · line 1291

Quoted 3 lines
xen/arch/x86/setup.c +3 -0
+    {
+        /* Find first unknown boot module to use as dom0 kernel */
+        i = first_boot_module_index(bi, BOOTMOD_UNKNOWN);

... i ever be anything else than 0? If not, perhaps keeping the call here is still fine (kind of for doc purposes), but an assertion may then want adding.

I don't think so, no. It's there for symmetry with the way the initrd is discovered later on, as that might be on module1, or 2, or 3 depending on whether there's microcode, or an XSM policy or anything else. in other modules.

Quoted 6 lines

xen/arch/x86/setup.c void asmlinkage __init noreturn __start_xen(void) · line 1294

Quoted 4 lines
xen/arch/x86/setup.c +3 -0
+        bi->mods[i].type = BOOTMOD_KERNEL;
+        bi->domains[0].kernel = &bi->mods[i];
+        bi->hyperlaunch_enabled = false;

Is this necessary, when the field is supposed to be starting out clear?

It isn't necessary, but I think it gave a sense of intent. That said I'm pondering removing that boolean though in favour of something like

bi->mods[0].type == BOOTMOD_FDT

At which point that assignment disappears.

Quoted 29 lines

xen/common/domain-builder/Makefile line 2

Quoted 6 lines
xen/common/domain-builder/Makefile new file +2 -0
@@ -0,0 +1,2 @@
+obj-y += fdt.init.o
+obj-y += core.init.o

Any reason for these not both adding to obj-bin-y, like we do elsewhere for *.init.o?

Also please sort object lists alphabetically.

Quoted 17 lines
xen/include/xen/domain-builder.h new file +14 -0
@@ -0,0 +1,29 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef __XEN_DOMAIN_BUILDER_H__
+#define __XEN_DOMAIN_BUILDER_H__
+
+struct boot_info;
+
+/* Return status of builder_init(). Shows which boot mechanism was detected */
+enum fdt_kind
+{
+    /* FDT not found. Skipped builder. */
+    FDT_KIND_NONE,
+    /* Found an FDT that wasn't hyperlaunch. */
+    FDT_KIND_UNKNOWN,
+};
Quoted 8 lines

xen/include/xen/domain-builder.h line 20

Quoted 6 lines
xen/include/xen/domain-builder.h new file +6 -0
+
+/*
+ * Initialises `bi` if it detects a compatible FDT. Otherwise returns
+ * FDT_KIND_NONE and leaves initialisation up to the caller.
+ */
+#if IS_ENABLED(CONFIG_DOMAIN_BUILDER)

For the pre-processor it wants to be the simpler #ifdef.

True. Don't know what I was thinking.

Quoted 2 lines

Jan

Cheers,
Alejandro

#12 lore raw

PATCH v6 3/12 x86/hyperlaunch: initial support for hyperlaunch device tree

In reply to Alejandro Vallejo

From: "Daniel P. Smith" <[email protected]>

Add the ability to detect both a formal hyperlaunch device tree or a dom0less device tree. If the hyperlaunch device tree is found, then count the number of domain entries, reporting an error if more than one is found.

---
 xen/arch/x86/include/asm/bootinfo.h |  1 +
 xen/common/domain-builder/core.c    | 15 +++++++
 xen/common/domain-builder/fdt.c     | 62 +++++++++++++++++++++++++++++
 xen/common/domain-builder/fdt.h     |  1 +
 xen/include/xen/domain-builder.h    |  2 +
 5 files changed, 81 insertions(+)
xen/arch/x86/include/asm/bootinfo.h +1 -0
index 82c2650fcf..1e3d582e45 100644
@@ -84,6 +84,7 @@ struct boot_info {
     bool hyperlaunch_enabled;
 
     unsigned int nr_modules;
+    unsigned int nr_domains;
     struct boot_module mods[MAX_NR_BOOTMODS + 1];
     struct boot_domain domains[MAX_NR_BOOTDOMS];
 };
xen/common/domain-builder/core.c +15 -0
index 97c92db571..c6917532be 100644
@@ -16,10 +16,17 @@
 enum fdt_kind __init builder_init(struct boot_info *bi)
 {
     enum fdt_kind kind;
+    int ret;
 
     bi->hyperlaunch_enabled = false;
     switch ( (kind = fdt_detect_kind(bi)) )
     {
+    case FDT_KIND_HYPERLAUNCH:
+        printk(XENLOG_DEBUG "DT found: hyperlaunch\n");
+        bi->hyperlaunch_enabled = true;
+        bi->mods[0].type = BOOTMOD_FDT;
+        break;
+
     case FDT_KIND_NONE:
         /* No DT found */
         return kind;
@@ -32,6 +39,14 @@ enum fdt_kind __init builder_init(struct boot_info *bi)
     default:
         BUG();
     }
+
+    printk(XENLOG_INFO "Hyperlaunch configuration:\n");
+    if ( (ret = fdt_walk_hyperlaunch(bi)) < 0 )
+        panic("Walk of device tree failed (%d)\n", ret);
+
+    printk(XENLOG_INFO "  number of domains: %u\n", bi->nr_domains);
+
+    return FDT_KIND_HYPERLAUNCH;
 }
 
 /*
xen/common/domain-builder/fdt.c +62 -0

80 lines not on this page · raw message

xen/common/domain-builder/fdt.h +1 -0
index ef897fc412..d1bcc23fa2 100644
@@ -10,5 +10,6 @@ struct boot_info;
 #define HYPERLAUNCH_MODULE_IDX 0
 
 enum fdt_kind fdt_detect_kind(const struct boot_info *bi);
+int fdt_walk_hyperlaunch(struct boot_info *bi);
 
 #endif /* __XEN_DOMAIN_BUILDER_FDT_H__ */
xen/include/xen/domain-builder.h +2 -0
index b9702db735..cbb3cbea7c 100644
@@ -9,6 +9,8 @@ enum fdt_kind
 {
     /* FDT not found. Skipped builder. */
     FDT_KIND_NONE,
+    /* Found Hyperlaunch FDT */
+    FDT_KIND_HYPERLAUNCH,
     /* Found an FDT that wasn't hyperlaunch. */
     FDT_KIND_UNKNOWN,
 };
Signature · 1 line
2.43.0
#13 lore raw

Re: [PATCH v6 03/12] x86/hyperlaunch: initial support for hyperlaunch device tree

Quoted 10 lines · Alejandro Vallejo

From: "Daniel P. Smith" <[email protected]>

Add the ability to detect both a formal hyperlaunch device tree or a dom0less device tree. If the hyperlaunch device tree is found, then count the number of domain entries, reporting an error if more than one is found.

First: With your code re-use proposal sent earlier today I wonder how meaningful it is to further review this series. Much of it would change if that proposal was followed, I expect?

Then: When you say "hyperlaunch or dom0less" - is it entirely benign which of the two is found, as to further parsing? I ask because I can't spot anywhere that you would record which of the two (if any) was found.

Quoted 9 lines
xen/common/domain-builder/fdt.c +3 -0
@@ -13,6 +13,36 @@
 
 #include "fdt.h"
 
+static int __init find_hyperlaunch_node(const void *fdt)
+{
+    int hv_node = fdt_path_offset(fdt, "/chosen/hypervisor");

xen/common/domain-builder/fdt.c line 28

Quoted 10 lines
xen/common/domain-builder/fdt.c +10 -0
+
+    if ( hv_node >= 0 )
+    {
+        /* Anything other than zero indicates no match */
+        if ( fdt_node_check_compatible(fdt, hv_node, "hypervisor,xen") )
+            return -ENODATA;
+
+        return hv_node;
+    }
+    else

Please can such unnecessary (and potentially misleading) "else" be omitted? As ...

xen/common/domain-builder/fdt.c line 31

Quoted 3 lines
xen/common/domain-builder/fdt.c +3 -0
+    {
+        /* Look for dom0less config */
+        int node, chosen_node = fdt_path_offset(fdt, "/chosen");

... these will need to move to function scope then, one of the two may want folding with "hv_node" above.

Jan

#14 lore raw

Re: [PATCH v6 03/12] x86/hyperlaunch: initial support for hyperlaunch device tree

Quoted 7 lines · Jan Beulich
Quoted 6 lines · Alejandro Vallejo

From: "Daniel P. Smith" <[email protected]>

Add the ability to detect both a formal hyperlaunch device tree or a dom0less device tree. If the hyperlaunch device tree is found, then count the number of domain entries, reporting an error if more than one is found.

Quoted 8 lines
Quoted 4 lines

First: With your code re-use proposal sent earlier today I wonder how meaningful it is to further review this series. Much of it would change if that proposal was followed, I expect?

Should I follow through with that proposal, that would indeed have large knock-on effects here. Sorry, I took longer than I thought I would evaluating.

I'll go over your reviews and answer them in case they stay relevant afterwards. Thanks for that.

Quoted 4 lines

Then: When you say "hyperlaunch or dom0less" - is it entirely benign which of the two is found, as to further parsing? I ask because I can't spot anywhere that you would record which of the two (if any) was found.

Under dom0less everything is /chosen, mixed with other nodes. Hyperlaunch mandates the initial system configuration to reside in /chosen/hypervisor, which is meant to be "xen,hypervisor"-compatible.

The function is effectively finding a suitable root for the tree that contains the initial system config.

Quoted 7 lines
Quoted 7 lines
xen/common/domain-builder/fdt.c +0 -0
@@ -13,6 +13,36 @@
 
 #include "fdt.h"
 
Quoted 15 lines

xen/common/domain-builder/fdt.c line 28

Quoted 13 lines
xen/common/domain-builder/fdt.c +13 -0
+static int __init find_hyperlaunch_node(const void *fdt)
+{
+    int hv_node = fdt_path_offset(fdt, "/chosen/hypervisor");
+
+    if ( hv_node >= 0 )
+    {
+        /* Anything other than zero indicates no match */
+        if ( fdt_node_check_compatible(fdt, hv_node, "hypervisor,xen") )
+            return -ENODATA;
+
+        return hv_node;
+    }
+    else

Please can such unnecessary (and potentially misleading) "else" be omitted?

Not sure how it could be misleading, but...

Quoted 8 lines

As ...

xen/common/domain-builder/fdt.c line 31

Quoted 3 lines
xen/common/domain-builder/fdt.c +3 -0
+    {
+        /* Look for dom0less config */
+        int node, chosen_node = fdt_path_offset(fdt, "/chosen");

... these will need to move to function scope then, one of the two may want folding with "hv_node" above.

... there is indeed a more compact form the function could take. Noted.

Cheers,
Alejandro

#15 lore raw

Re: [PATCH v6 03/12] x86/hyperlaunch: initial support for hyperlaunch device tree

Quoted 24 lines · Alejandro Vallejo
Quoted 7 lines · Jan Beulich
Quoted 6 lines · Alejandro Vallejo
xen/common/domain-builder/fdt.c +0 -0
@@ -13,6 +13,36 @@
 
 #include "fdt.h"
 
Quoted 15 lines

xen/common/domain-builder/fdt.c line 28

Quoted 13 lines
xen/common/domain-builder/fdt.c +13 -0
+static int __init find_hyperlaunch_node(const void *fdt)
+{
+    int hv_node = fdt_path_offset(fdt, "/chosen/hypervisor");
+
+    if ( hv_node >= 0 )
+    {
+        /* Anything other than zero indicates no match */
+        if ( fdt_node_check_compatible(fdt, hv_node, "hypervisor,xen") )
+            return -ENODATA;
+
+        return hv_node;
+    }
+    else

Please can such unnecessary (and potentially misleading) "else" be omitted?

Quoted 1 line

Not sure how it could be misleading,

For context, just briefly looking at such a construct may leave one with the (wrong) impression that both branches of the conditional can fall through to subsequent code. This may be less of an issue as long as both sides are reasonably short, but then it is imo better to avoid the pattern altogether.

Jan

Quoted 15 lines

but...

Quoted 8 lines

As ...

xen/common/domain-builder/fdt.c line 31

Quoted 3 lines
xen/common/domain-builder/fdt.c +3 -0
+    {
+        /* Look for dom0less config */
+        int node, chosen_node = fdt_path_offset(fdt, "/chosen");

... these will need to move to function scope then, one of the two may want folding with "hv_node" above.

... there is indeed a more compact form the function could take. Noted.

Cheers,
Alejandro

#16 lore raw

PATCH v6 4/12 x86/hyperlaunch: Add helpers to locate multiboot modules

In reply to Alejandro Vallejo

Hyperlaunch mandates either a reg or module-index DT prop on nodes that contain `multiboot,module" under their "compatible" prop. This patch introduces a helper to generically find such index, appending the module to the list of modules if it wasn't already (i.e: because it's given via the "reg" prop).

---
 xen/common/domain-builder/fdt.c  | 172 +++++++++++++++++++++++++++++++
 xen/common/domain-builder/fdt.h  |   1 +
 xen/include/xen/domain-builder.h |   4 +
 3 files changed, 177 insertions(+)
xen/common/domain-builder/fdt.c +172 -0
index 94ccff61e2..05ac3647cf 100644
@@ -13,6 +13,178 @@
 
 #include "fdt.h"
 
+static int __init fdt_prop_as_u32(const struct fdt_property *prop,
+                                  uint32_t *val)
+{
+    if ( !prop || fdt32_to_cpu(prop->len) < sizeof(u32) )
+        return -EINVAL;
+
+    *val = fdt32_to_cpu(*(const fdt32_t *)prop->data);
+    return 0;
+}
+
+/*
+ * Unpacks a "reg" property into its address and size constituents.
+ *
+ * @param prop          Pointer to an FDT "reg" property.
+ * @param address_cells Number of 4-octet cells that make up an "address".
+ * @param size_cells    Number of 4-octet cells that make up a "size".
+ * @param p_addr[out]   Address encoded in the property.
+ * @param p_size[out]   Size encoded in the property.
+ * @returns             -EINVAL on malformed property, 0 otherwise.
+ */
+static int __init fdt_prop_as_reg(const struct fdt_property *prop,
+                                  int address_cells, int size_cells,
+                                  uint64_t *p_addr, uint64_t *p_size)
+{
+    const fdt32_t *cell = (const fdt32_t *)prop->data;
+    uint64_t addr, size;
+
+    if ( fdt32_to_cpu(prop->len) !=
+         (address_cells + size_cells) * sizeof(*cell) )
+    {
+        printk(XENLOG_ERR "  cannot read reg %lu+%lu from prop len %u\n",
+            address_cells * sizeof(*cell), size_cells * sizeof(*cell),
+            fdt32_to_cpu(prop->len));
+        return -EINVAL;
+    }
+
+    switch ( address_cells )
+    {
+    case 1:
+        addr = fdt32_to_cpu(*cell);
+        break;
+    case 2:
+        addr = fdt64_to_cpu(*(const fdt64_t *)cell);
+        break;
+    default:
+        printk(XENLOG_ERR "  unsupported address_cells=%d\n", address_cells);
+        return -EINVAL;
+    }
+
+    cell += address_cells;
+    switch ( size_cells )
+    {
+    case 1:
+        size = fdt32_to_cpu(*cell);
+        break;
+    case 2:
+        size = fdt64_to_cpu(*(const fdt64_t *)cell);
+        break;
+    default:
+        printk(XENLOG_ERR "  unsupported size_cells=%d\n", size_cells);
+        return -EINVAL;
+    }
+
+    *p_addr = addr;
+    *p_size = size;
+
+    return 0;
+}
+
+/*
+ * Locate a multiboot module given its node offset in the FDT.
+ *
+ * The module location may be given via either FDT property:
+ *     * reg = <address, size>
+ *         * Mutates `bi` to append the module.
+ *     * module-index = <idx>
+ *         * Leaves `bi` unchanged.
+ *
+ * @param fdt           Pointer to the full FDT.
+ * @param node          Offset for the module node.
+ * @param address_cells Number of 4-octet cells that make up an "address".
+ * @param size_cells    Number of 4-octet cells that make up a "size".
+ * @param bi[inout]     Xen's representation of the boot parameters.
+ * @return              -EINVAL on malformed nodes, otherwise
+ *                      index inside `bi->mods`
+ */
+int __init fdt_read_multiboot_module(const void *fdt, int node,
+                                     int address_cells, int size_cells,
+                                     struct boot_info *bi)
+{
+    const struct fdt_property *prop;
+    uint64_t addr, size;
+    int ret;
+    uint32_t idx;
+
+    if ( fdt_node_check_compatible(fdt, node, "multiboot,module") )
+    {
+        printk(XENLOG_ERR "  bad module. multiboot,module not found");
+        return -ENODATA;
+    }
+
+    /* Location given as a `module-index` property. */
+    if ( (prop = fdt_get_property(fdt, node, "module-index", NULL)) )
+    {
+        if ( fdt_get_property(fdt, node, "reg", NULL) )
+        {
+            printk(XENLOG_ERR "  found both reg and module-index for module\n");
+            return -EINVAL;
+        }
+        if ( (ret = fdt_prop_as_u32(prop, &idx)) )
+        {
+            printk(XENLOG_ERR "  bad module-index prop\n");
+            return ret;
+        }
+        if ( idx >= MAX_NR_BOOTMODS )
+        {
+            printk(XENLOG_ERR "  module-index overflow. %s=%u\n",
+                   STR(MAX_NR_BOOTMODS), MAX_NR_BOOTMODS);
+            return -EINVAL;
+        }
+
+        return idx;
+    }
+
+    /* Otherwise location given as a `reg` property. */
+    if ( !(prop = fdt_get_property(fdt, node, "reg", NULL)) )
+    {
+        printk(XENLOG_ERR "  no location for multiboot,module\n");
+        return -EINVAL;
+    }
+    if ( fdt_get_property(fdt, node, "module-index", NULL) )
+    {
+        printk(XENLOG_ERR "  found both reg and module-index for module\n");
+        return -EINVAL;
+    }
+
+    ret = fdt_prop_as_reg(prop, address_cells, size_cells, &addr, &size);
+    if ( ret < 0 )
+    {
+        printk(XENLOG_ERR "  failed reading reg for multiboot,module\n");
+        return -EINVAL;
+    }
+
+    idx = bi->nr_modules;
+    if ( idx > MAX_NR_BOOTMODS )
+    {
+        /*
+         * MAX_NR_BOOTMODS must fit in 31 bits so it's representable in the
+         * positive side of an int; for the return value.
+         */
+        BUILD_BUG_ON(MAX_NR_BOOTMODS > (uint64_t)INT_MAX);
+        printk(XENLOG_ERR "  idx=%u exceeds len=%u\n", idx, MAX_NR_BOOTMODS);
+        return -EINVAL;
+    }
+
+    /*
+     * Append new module to the existing list
+     *
+     * Note that bi->nr_modules points to Xen itself, so we must shift it first
+     */
+    bi->nr_modules++;
+    bi->mods[bi->nr_modules] = bi->mods[idx];
+    bi->mods[idx] = (struct boot_module){
+        .start = addr,
+        .size = size,
+    };
+
+    printk(XENLOG_INFO "  module[%u]: addr %lx size %lx\n", idx, addr, size);
+
+    return idx;
+}
+
 static int __init find_hyperlaunch_node(const void *fdt)
 {
     int hv_node = fdt_path_offset(fdt, "/chosen/hypervisor");
xen/common/domain-builder/fdt.h +1 -0
index d1bcc23fa2..5570fb7a9c 100644
@@ -3,6 +3,7 @@
 #define __XEN_DOMAIN_BUILDER_FDT_H__
 
 #include <xen/domain-builder.h>
+#include <xen/libfdt/libfdt-xen.h>
 
 struct boot_info;
 
xen/include/xen/domain-builder.h +4 -0
index cbb3cbea7c..3ac3a0ab84 100644
@@ -28,4 +28,8 @@ static inline enum fdt_kind builder_init(struct boot_info *bi)
 }
 #endif /* !IS_ENABLED(CONFIG_DOMAIN_BUILDER) */
 
+int fdt_read_multiboot_module(const void *fdt, int node,
+                              int address_cells, int size_cells,
+                              struct boot_info *bi)
+
 #endif /* __XEN_DOMAIN_BUILDER_H__ */
Signature · 1 line
2.43.0
#17 lore raw

PATCH v6 5/12 x86/hyperlaunch: locate dom0 kernel with hyperlaunch

In reply to Alejandro Vallejo

From: "Daniel P. Smith" <[email protected]>

Look for a subnode of type multiboot,kernel within a domain node. If found, locate it using the multiboot module helper to generically ensure it lives in the module list. If the bootargs property is present and there was not an MB1 string, then use the command line from the device tree definition.

---
 xen/arch/x86/setup.c            |  1 +
 xen/common/domain-builder/fdt.c | 64 +++++++++++++++++++++++++++++++--
 2 files changed, 62 insertions(+), 3 deletions(-)
xen/arch/x86/setup.c +1 -0
index f3b5c83a3c..f8193f2c1c 100644
@@ -1291,6 +1291,7 @@ void asmlinkage __init noreturn __start_xen(void)
         i = first_boot_module_index(bi, BOOTMOD_UNKNOWN);
         bi->mods[i].type = BOOTMOD_KERNEL;
         bi->domains[0].kernel = &bi->mods[i];
+        bi->nr_domains = 1;
         bi->hyperlaunch_enabled = false;
     }
 
xen/common/domain-builder/fdt.c +61 -3

84 lines not on this page · raw message

Signature · 1 line
2.43.0
#18 lore raw

PATCH v6 6/12 x86/hyperlaunch: obtain cmdline from device tree

In reply to Alejandro Vallejo

From: "Daniel P. Smith" <[email protected]>

Add support to read the command line from the hyperlaunch device tree. The device tree command line is located in the "bootargs" property of the "multiboot,kernel" node.

A boot loader command line, e.g. a grub module string field, takes precendence over the device tree one since it is easier to modify.

---
 xen/arch/x86/include/asm/bootinfo.h |  6 ++++--
 xen/arch/x86/setup.c                | 22 ++++++++++++++++++++--
 xen/common/domain-builder/core.c    | 26 ++++++++++++++++++++++++++
 xen/common/domain-builder/fdt.c     |  6 ++++++
 xen/common/domain-builder/fdt.h     | 24 ++++++++++++++++++++++++
 xen/include/xen/domain-builder.h    |  8 +++++---
 xen/include/xen/libfdt/libfdt-xen.h | 23 +++++++++++++++++++++++
 7 files changed, 108 insertions(+), 7 deletions(-)
xen/arch/x86/include/asm/bootinfo.h +4 -2
index 1e3d582e45..5b2c93b1ef 100644
@@ -35,11 +35,13 @@ struct boot_module {
 
     /*
      * Module State Flags:
-     *   relocated: indicates module has been relocated in memory.
-     *   released:  indicates module's pages have been freed.
+     *   relocated:   indicates module has been relocated in memory.
+     *   released:    indicates module's pages have been freed.
+     *   fdt_cmdline: indicates module's cmdline is in the FDT.
      */
     bool relocated:1;
     bool released:1;
+    bool fdt_cmdline:1;
 
     /*
      * A boot module may need decompressing by Xen.  Headroom is an estimate of
xen/arch/x86/setup.c +20 -2
index f8193f2c1c..765b690c41 100644
@@ -985,7 +985,19 @@ static size_t __init domain_cmdline_size(const struct boot_info *bi,
 {
     size_t s = bi->kextra ? strlen(bi->kextra) : 0;
 
-    s += bd->kernel->cmdline_pa ? strlen(__va(bd->kernel->cmdline_pa)) : 0;
+
+    /*
+     * Bootloader cmdline takes precedence and replaces the DT provided one.
+     *
+     * In that case, fdt_cmdline is not be populated at all.
+     */
+    if ( bd->kernel->fdt_cmdline )
+    {
+        BUG_ON(!IS_ENABLED(CONFIG_DOMAIN_BUILDER));
+        s += builder_get_cmdline_size(bi, bd->kernel->cmdline_pa);
+    }
+    else if ( bd->kernel->cmdline_pa )
+        s += strlen(__va(bd->kernel->cmdline_pa));
 
     if ( s == 0 )
         return s;
@@ -1049,7 +1061,13 @@ static struct domain *__init create_dom0(struct boot_info *bi)
         if ( !(cmdline = xzalloc_array(char, cmdline_size)) )
             panic("Error allocating cmdline buffer for %pd\n", d);
 
-        if ( bd->kernel->cmdline_pa )
+        if ( bd->kernel->fdt_cmdline )
+        {
+            BUG_ON(!IS_ENABLED(CONFIG_DOMAIN_BUILDER));
+            builder_get_cmdline(
+                bi, bd->kernel->cmdline_pa, cmdline, cmdline_size);
+        }
+        else if ( bd->kernel->cmdline_pa )
             strlcpy(cmdline,
                     cmdline_cook(__va(bd->kernel->cmdline_pa), bi->loader),
                     cmdline_size);
xen/common/domain-builder/core.c +26 -0
index c6917532be..b2f49bb17b 100644
@@ -10,9 +10,35 @@
 #include <xen/lib.h>
 
 #include <asm/bootinfo.h>
+#include <asm/setup.h>
 
 #include "fdt.h"
 
+size_t __init builder_get_cmdline_size(const struct boot_info *bi, int offset)
+{
+    const void *fdt;
+    int size;
+
+    fdt = bootstrap_map_bm(&bi->mods[HYPERLAUNCH_MODULE_IDX]);
+    size = fdt_cmdline_prop_size(fdt, offset);
+    bootstrap_unmap();
+
+    return max(0, size);
+}
+
+int __init builder_get_cmdline(const struct boot_info *bi,
+                               int offset, char *cmdline, size_t size)
+{
+    const void *fdt;
+    int ret;
+
+    fdt = bootstrap_map_bm(&bi->mods[HYPERLAUNCH_MODULE_IDX]);
+    ret = fdt_cmdline_prop_copy(fdt, offset, cmdline, size);
+    bootstrap_unmap();
+
+    return ret;
+}
+
 enum fdt_kind __init builder_init(struct boot_info *bi)
 {
     enum fdt_kind kind;
xen/common/domain-builder/fdt.c +6 -0
index cbb0ed30a2..dabe201b04 100644
@@ -219,6 +219,12 @@ static int __init fdt_process_domain_node(
             printk(XENLOG_INFO "  kernel: multiboot-index=%d\n", idx);
             bi->mods[idx].type = BOOTMOD_KERNEL;
             bd->kernel = &bi->mods[idx];
+
+            /* If bootloader didn't set cmdline, see if FDT provides one. */
+            if ( bd->kernel->cmdline_pa &&
+                 !((char *)__va(bd->kernel->cmdline_pa))[0] )
+                bd->kernel->fdt_cmdline = fdt_get_prop_offset(
+                    fdt, node, "bootargs", &bd->kernel->cmdline_pa);
         }
     }
 
xen/common/domain-builder/fdt.h +24 -0
index 5570fb7a9c..68cbc42674 100644
@@ -10,6 +10,30 @@ struct boot_info;
 /* hyperlaunch fdt is required to be module 0 */
 #define HYPERLAUNCH_MODULE_IDX 0
 
+static inline int __init fdt_cmdline_prop_size(const void *fdt, int offset)
+{
+    int ret;
+
+    fdt_get_property_by_offset(fdt, offset, &ret);
+
+    return ret;
+}
+
+static inline int __init fdt_cmdline_prop_copy(
+    const void *fdt, int offset, char *cmdline, size_t size)
+{
+    int ret;
+    const struct fdt_property *prop =
+        fdt_get_property_by_offset(fdt, offset, &ret);
+
+    if ( ret < 0 )
+        return ret;
+
+    ASSERT(size > ret);
+
+    return strlcpy(cmdline, prop->data, ret);
+}
+
 enum fdt_kind fdt_detect_kind(const struct boot_info *bi);
 int fdt_walk_hyperlaunch(struct boot_info *bi);
 
xen/include/xen/domain-builder.h +5 -3
index 3ac3a0ab84..350e2eb2af 100644
@@ -2,6 +2,8 @@
 #ifndef __XEN_DOMAIN_BUILDER_H__
 #define __XEN_DOMAIN_BUILDER_H__
 
+#include <xen/types.h>
+
 struct boot_info;
 
 /* Return status of builder_init(). Shows which boot mechanism was detected */
@@ -28,8 +30,8 @@ static inline enum fdt_kind builder_init(struct boot_info *bi)
 }
 #endif /* !IS_ENABLED(CONFIG_DOMAIN_BUILDER) */
 
-int fdt_read_multiboot_module(const void *fdt, int node,
-                              int address_cells, int size_cells,
-                              struct boot_info *bi)
+size_t builder_get_cmdline_size(const struct boot_info *bi, int offset);
+int builder_get_cmdline(const struct boot_info *bi, int offset,
+                        char *cmdline, size_t size);
 
 #endif /* __XEN_DOMAIN_BUILDER_H__ */
xen/include/xen/libfdt/libfdt-xen.h +23 -0
index a5340bc9f4..9c20a26a35 100644
@@ -13,6 +13,29 @@
 
 #include <xen/libfdt/libfdt.h>
 
+static inline bool __init fdt_get_prop_offset(
+    const void *fdt, int node, const char *pname, unsigned long *poffset)
+{
+    int ret, offset;
+    const char *name;
+
+    fdt_for_each_property_offset(offset, fdt, node)
+    {
+        fdt_getprop_by_offset(fdt, offset, &name, &ret);
+
+        if ( ret < 0 )
+            continue;
+
+        if ( !strcmp(name, pname) )
+        {
+            *poffset = offset;
+            return true;
+        }
+    }
+
+    return false;
+}
+
 static inline int fdt_get_mem_rsv_paddr(const void *fdt, int n,
                                         paddr_t *address,
                                         paddr_t *size)
Signature · 1 line
2.43.0
#19 lore raw

Re: [PATCH v6 06/12] x86/hyperlaunch: obtain cmdline from device tree

Quoted 20 lines · Alejandro Vallejo

From: "Daniel P. Smith" <[email protected]>

Add support to read the command line from the hyperlaunch device tree. The device tree command line is located in the "bootargs" property of the "multiboot,kernel" node.

A boot loader command line, e.g. a grub module string field, takes precendence over the device tree one since it is easier to modify.

---

xen/common/domain-builder/fdt.c +0 -0
index cbb0ed30a2..dabe201b04 100644

xen/common/domain-builder/fdt.c static int __init fdt_process_domain_node( · line 225

Quoted 7 lines
xen/common/domain-builder/fdt.c +4 -0
@@ -219,6 +219,12 @@ static int __init fdt_process_domain_node(
              printk(XENLOG_INFO "  kernel: multiboot-index=%d\n", idx);
              bi->mods[idx].type = BOOTMOD_KERNEL;
              bd->kernel = &bi->mods[idx];
+
+            /* If bootloader didn't set cmdline, see if FDT provides one. */
+            if ( bd->kernel->cmdline_pa &&
+                 !((char *)__va(bd->kernel->cmdline_pa))[0] )

The logic is incorrect - it should be:

            if ( !bd->kernel->cmdline_pa ||
                 !((char *)__va(bd->kernel->cmdline_pa))[0] )

If there is no cmdline_pa (which happens with the "reg" property) or the if there is a 0-length string, then check the DT for bootargs.

This fixes the "reg" case to read from bootargs.

xen/common/domain-builder/fdt.c static int __init fdt_process_domain_node( · line 230

Quoted 5 lines
xen/common/domain-builder/fdt.c +2 -0
+                bd->kernel->fdt_cmdline = fdt_get_prop_offset(
+                    fdt, node, "bootargs", &bd->kernel->cmdline_pa);
          }
      }
 

Just giving a heads up in case anyone is using it.

Regards,
Jason

#20 lore raw

Re: [PATCH v6 06/12] x86/hyperlaunch: obtain cmdline from device tree

Quoted 31 lines · Jason Andryuk
Quoted 20 lines · Alejandro Vallejo

From: "Daniel P. Smith" <[email protected]>

Add support to read the command line from the hyperlaunch device tree. The device tree command line is located in the "bootargs" property of the "multiboot,kernel" node.

A boot loader command line, e.g. a grub module string field, takes precendence over the device tree one since it is easier to modify.

---

xen/common/domain-builder/fdt.c +0 -0
index cbb0ed30a2..dabe201b04 100644

xen/common/domain-builder/fdt.c static int __init fdt_process_domain_node( · line 225

Quoted 7 lines
xen/common/domain-builder/fdt.c +4 -0
@@ -219,6 +219,12 @@ static int __init fdt_process_domain_node(
               printk(XENLOG_INFO "  kernel: multiboot-index=%d\n", idx);
               bi->mods[idx].type = BOOTMOD_KERNEL;
               bd->kernel = &bi->mods[idx];
+
+            /* If bootloader didn't set cmdline, see if FDT provides one. */
+            if ( bd->kernel->cmdline_pa &&
+                 !((char *)__va(bd->kernel->cmdline_pa))[0] )

The logic is incorrect - it should be:

Quoted 4 lines

            if ( !bd->kernel->cmdline_pa ||
                 !((char *)__va(bd->kernel->cmdline_pa))[0] )

If there is no cmdline_pa (which happens with the "reg" property) or the if there is a 0-length string, then check the DT for bootargs.

Even that sounds bogus to me: There's a difference between "no command line" and "empty command line".

Jan

#21 lore raw

Re: [PATCH v6 06/12] x86/hyperlaunch: obtain cmdline from device tree

Quoted 32 lines · Jan Beulich
Quoted 31 lines · Jason Andryuk
Quoted 20 lines · Alejandro Vallejo

From: "Daniel P. Smith" <[email protected]>

Add support to read the command line from the hyperlaunch device tree. The device tree command line is located in the "bootargs" property of the "multiboot,kernel" node.

A boot loader command line, e.g. a grub module string field, takes precendence over the device tree one since it is easier to modify.

---

xen/common/domain-builder/fdt.c +0 -0
index cbb0ed30a2..dabe201b04 100644

xen/common/domain-builder/fdt.c static int __init fdt_process_domain_node( · line 225

Quoted 7 lines
xen/common/domain-builder/fdt.c +4 -0
@@ -219,6 +219,12 @@ static int __init fdt_process_domain_node(
               printk(XENLOG_INFO "  kernel: multiboot-index=%d\n", idx);
               bi->mods[idx].type = BOOTMOD_KERNEL;
               bd->kernel = &bi->mods[idx];
+
+            /* If bootloader didn't set cmdline, see if FDT provides one. */
+            if ( bd->kernel->cmdline_pa &&
+                 !((char *)__va(bd->kernel->cmdline_pa))[0] )

The logic is incorrect - it should be:

Quoted 7 lines
Quoted 4 lines

            if ( !bd->kernel->cmdline_pa ||
                 !((char *)__va(bd->kernel->cmdline_pa))[0] )

If there is no cmdline_pa (which happens with the "reg" property) or the if there is a 0-length string, then check the DT for bootargs.

Even that sounds bogus to me: There's a difference between "no command line" and "empty command line".

Yes, you have a point. The difficulty is grub always provides a NUL terminated string, so Xen can't differentiate the two.

Regards,
Jason

#22 lore raw

Re: [PATCH v6 06/12] x86/hyperlaunch: obtain cmdline from device tree

Quoted 38 lines · Jason Andryuk
Quoted 35 lines · Jan Beulich
Quoted 34 lines · Jason Andryuk
Quoted 20 lines · Alejandro Vallejo

From: "Daniel P. Smith" <[email protected]>

Add support to read the command line from the hyperlaunch device tree. The device tree command line is located in the "bootargs" property of the "multiboot,kernel" node.

A boot loader command line, e.g. a grub module string field, takes precendence over the device tree one since it is easier to modify.

---

xen/common/domain-builder/fdt.c +0 -0
index cbb0ed30a2..dabe201b04 100644

xen/common/domain-builder/fdt.c static int __init fdt_process_domain_node( · line 225

Quoted 7 lines
xen/common/domain-builder/fdt.c +4 -0
@@ -219,6 +219,12 @@ static int __init fdt_process_domain_node(
                printk(XENLOG_INFO "  kernel: multiboot-index=%d\n", idx);
                bi->mods[idx].type = BOOTMOD_KERNEL;
                bd->kernel = &bi->mods[idx];
+
+            /* If bootloader didn't set cmdline, see if FDT provides one. */
+            if ( bd->kernel->cmdline_pa &&
+                 !((char *)__va(bd->kernel->cmdline_pa))[0] )

The logic is incorrect - it should be:

             if ( !bd->kernel->cmdline_pa ||
                  !((char *)__va(bd->kernel->cmdline_pa))[0] )

Quoted 6 lines
Quoted 4 lines
Quoted 1 line

If there is no cmdline_pa (which happens with the "reg" property) or the if there is a 0-length string, then check the DT for bootargs.

Even that sounds bogus to me: There's a difference between "no command line" and "empty command line".

Yes, you have a point.  The difficulty is grub always provides a NUL terminated string, so Xen can't differentiate the two.

Which may call for either special-casing GrUB, or at least calling out that behavior in the comment. (Ideally we'd still have a way to distinguish between both cases, but likely we'll need to resort to documenting that some dummy option will need adding to tell "none" from [intended to be] empty.)

Jan

#23 lore raw

Re: [PATCH v6 06/12] x86/hyperlaunch: obtain cmdline from device tree

Quoted 44 lines · Jan Beulich
Quoted 43 lines · Jason Andryuk
Quoted 39 lines · Jan Beulich
Quoted 31 lines · Jason Andryuk
Quoted 20 lines · Alejandro Vallejo

From: "Daniel P. Smith" <[email protected]>

Add support to read the command line from the hyperlaunch device tree. The device tree command line is located in the "bootargs" property of the "multiboot,kernel" node.

A boot loader command line, e.g. a grub module string field, takes precendence over the device tree one since it is easier to modify.

---

xen/common/domain-builder/fdt.c +0 -0
index cbb0ed30a2..dabe201b04 100644

xen/common/domain-builder/fdt.c static int __init fdt_process_domain_node( · line 225

Quoted 7 lines
xen/common/domain-builder/fdt.c +4 -0
@@ -219,6 +219,12 @@ static int __init fdt_process_domain_node(
                printk(XENLOG_INFO "  kernel: multiboot-index=%d\n", idx);
                bi->mods[idx].type = BOOTMOD_KERNEL;
                bd->kernel = &bi->mods[idx];
+
+            /* If bootloader didn't set cmdline, see if FDT provides one. */
+            if ( bd->kernel->cmdline_pa &&
+                 !((char *)__va(bd->kernel->cmdline_pa))[0] )

The logic is incorrect - it should be:

Quoted 4 lines

             if ( !bd->kernel->cmdline_pa ||
                  !((char *)__va(bd->kernel->cmdline_pa))[0] )

If there is no cmdline_pa (which happens with the "reg" property) or the if there is a 0-length string, then check the DT for bootargs.

Even that sounds bogus to me: There's a difference between "no command line" and "empty command line".

Quoted 8 lines
Quoted 1 line

Yes, you have a point.  The difficulty is grub always provides a NUL terminated string, so Xen can't differentiate the two.

Which may call for either special-casing GrUB, or at least calling out that behavior in the comment. (Ideally we'd still have a way to distinguish between both cases, but likely we'll need to resort to documenting that some dummy option will need adding to tell "none" from [intended to be] empty.)

Jan

We can add suitable comments where required, sure.

About the dummy option, note that even if we have an option for Xen, that does nothing for the kernel cmdlines. If there's such dummy option there I don't know of it.

Cheers,
Alejandro

#24 lore raw

Re: [PATCH v6 06/12] x86/hyperlaunch: obtain cmdline from device tree

Quoted 52 lines · Alejandro Vallejo
Quoted 44 lines · Jan Beulich
Quoted 43 lines · Jason Andryuk
Quoted 39 lines · Jan Beulich
Quoted 31 lines · Jason Andryuk
Quoted 20 lines · Alejandro Vallejo

From: "Daniel P. Smith" <[email protected]>

Add support to read the command line from the hyperlaunch device tree. The device tree command line is located in the "bootargs" property of the "multiboot,kernel" node.

A boot loader command line, e.g. a grub module string field, takes precendence over the device tree one since it is easier to modify.

---

xen/common/domain-builder/fdt.c +0 -0
index cbb0ed30a2..dabe201b04 100644

xen/common/domain-builder/fdt.c static int __init fdt_process_domain_node( · line 225

Quoted 7 lines
xen/common/domain-builder/fdt.c +4 -0
@@ -219,6 +219,12 @@ static int __init fdt_process_domain_node(
                printk(XENLOG_INFO "  kernel: multiboot-index=%d\n", idx);
                bi->mods[idx].type = BOOTMOD_KERNEL;
                bd->kernel = &bi->mods[idx];
+
+            /* If bootloader didn't set cmdline, see if FDT provides one. */
+            if ( bd->kernel->cmdline_pa &&
+                 !((char *)__va(bd->kernel->cmdline_pa))[0] )

The logic is incorrect - it should be:

Quoted 4 lines

             if ( !bd->kernel->cmdline_pa ||
                  !((char *)__va(bd->kernel->cmdline_pa))[0] )

If there is no cmdline_pa (which happens with the "reg" property) or the if there is a 0-length string, then check the DT for bootargs.

Even that sounds bogus to me: There's a difference between "no command line" and "empty command line".

Quoted 6 lines
Quoted 1 line

Yes, you have a point.  The difficulty is grub always provides a NUL terminated string, so Xen can't differentiate the two.

Which may call for either special-casing GrUB, or at least calling out that behavior in the comment. (Ideally we'd still have a way to distinguish between both cases, but likely we'll need to resort to documenting that some dummy option will need adding to tell "none" from [intended to be] empty.)

Quoted 5 lines

We can add suitable comments where required, sure.

About the dummy option, note that even if we have an option for Xen, that does nothing for the kernel cmdlines. If there's such dummy option there I don't know of it.

Indeed I meant we may need to introduce something.

Jan

#25 lore raw

PATCH v6 7/12 x86/hyperlaunch: locate dom0 initrd with hyperlaunch

In reply to Alejandro Vallejo

From: "Daniel P. Smith" <[email protected]>

Look for a subnode of type multiboot,ramdisk within a domain node and parse via the fdt_read_multiboot_module() helper. After a successful helper call, the module index is returned and the module is guaranteed to be in the module list.

Fix unused typo in adjacent comment.

---
 xen/arch/x86/setup.c            |  4 ++--
 xen/common/domain-builder/fdt.c | 24 ++++++++++++++++++++++++
 2 files changed, 26 insertions(+), 2 deletions(-)
xen/arch/x86/setup.c +2 -2
index 765b690c41..c8de028439 100644
@@ -2172,11 +2172,11 @@ void asmlinkage __init noreturn __start_xen(void)
      * At this point all capabilities that consume boot modules should have
      * claimed their boot modules. Find the first unclaimed boot module and
      * claim it as the initrd ramdisk. Do a second search to see if there are
-     * any remaining unclaimed boot modules, and report them as unusued initrd
+     * any remaining unclaimed boot modules, and report them as unused initrd
      * candidates.
      */
     initrdidx = first_boot_module_index(bi, BOOTMOD_UNKNOWN);
-    if ( initrdidx < MAX_NR_BOOTMODS )
+    if ( !bi->hyperlaunch_enabled && initrdidx < MAX_NR_BOOTMODS )
     {
         bi->mods[initrdidx].type = BOOTMOD_RAMDISK;
         bi->domains[0].module = &bi->mods[initrdidx];
xen/common/domain-builder/fdt.c +24 -0
index dabe201b04..507f383f8e 100644
@@ -226,6 +226,30 @@ static int __init fdt_process_domain_node(
                 bd->kernel->fdt_cmdline = fdt_get_prop_offset(
                     fdt, node, "bootargs", &bd->kernel->cmdline_pa);
         }
+        else if ( !fdt_node_check_compatible(fdt, node, "multiboot,ramdisk") )
+        {
+            int idx;
+
+            if ( bd->module )
+            {
+                printk(XENLOG_WARNING
+                       "Duplicate module for domain %s\n", name);
+                continue;
+            }
+
+            idx = fdt_read_multiboot_module(fdt, node, address_cells,
+                                            size_cells, bi);
+            if ( idx < 0 )
+            {
+                printk(XENLOG_ERR "  failed processing module for domain %s\n",
+                       name);
+                return idx;
+            }
+
+            printk(XENLOG_INFO "  module: multiboot-index=%d\n", idx);
+            bi->mods[idx].type = BOOTMOD_RAMDISK;
+            bd->module = &bi->mods[idx];
+        }
     }
 
     if ( !bd->kernel )
Signature · 1 line
2.43.0
#26 lore raw

PATCH v6 8/12 x86/hyperlaunch: add domain id parsing to domain config

In reply to Alejandro Vallejo

From: "Daniel P. Smith" <[email protected]>

Introduce the ability to specify the desired domain id for the domain definition. The domain id will be populated in the domid property of the domain node in the device tree configuration.

---
 xen/arch/x86/setup.c            |  5 ++--
 xen/common/domain-builder/fdt.c | 52 ++++++++++++++++++++++++++++++++-
 2 files changed, 54 insertions(+), 3 deletions(-)
xen/arch/x86/setup.c +3 -2
index c8de028439..10ff67ac37 100644
@@ -1043,8 +1043,9 @@ static struct domain *__init create_dom0(struct boot_info *bi)
     if ( iommu_enabled )
         dom0_cfg.flags |= XEN_DOMCTL_CDF_iommu;
 
-    /* Create initial domain.  Not d0 for pvshim. */
-    bd->domid = get_initial_domain_id();
+    if ( bd->domid == DOMID_INVALID )
+        /* Create initial domain.  Not d0 for pvshim. */
+        bd->domid = get_initial_domain_id();
     d = domain_create(bd->domid, &dom0_cfg,
                       pv_shim ? 0 : CDF_privileged | CDF_hardware);
     if ( IS_ERR(d) )
xen/common/domain-builder/fdt.c +51 -1

75 lines not on this page · raw message

Signature · 1 line
2.43.0
#27 lore raw

PATCH v6 9/12 x86/hyperlaunch: specify dom0 mode with device tree

In reply to Alejandro Vallejo

From: "Daniel P. Smith" <[email protected]>

Enable selecting the mode in which the domain will be built and ran. This includes:

- whether it will be either a 32/64 bit domain
- if it will be run as a PV or HVM domain
- and if it will require a device model (not applicable for dom0)

In the device tree, this will be represented as a bit map that will be carried through into struct boot_domain.

---
 xen/arch/x86/include/asm/boot-domain.h |  5 +++++
 xen/arch/x86/setup.c                   |  3 ++-
 xen/common/domain-builder/fdt.c        | 21 +++++++++++++++++++++
 3 files changed, 28 insertions(+), 1 deletion(-)
xen/arch/x86/include/asm/boot-domain.h +5 -0
index d7c6042e25..e316d4bcde 100644
@@ -13,6 +13,11 @@
 struct boot_domain {
     domid_t domid;
 
+                                          /* On     | Off    */
+#define BUILD_MODE_PARAVIRT      (1 << 0) /* PV     | PVH/HVM */
+#define BUILD_MODE_ENABLE_DM     (1 << 1) /* HVM    | PVH     */
+    uint32_t mode;
+
     struct boot_module *kernel;
     struct boot_module *module;
     const char *cmdline;
xen/arch/x86/setup.c +2 -1
index 10ff67ac37..343f87ee9d 100644
@@ -1030,7 +1030,8 @@ static struct domain *__init create_dom0(struct boot_info *bi)
     struct boot_domain *bd = &bi->domains[0];
     struct domain *d;
 
-    if ( opt_dom0_pvh )
+    if ( opt_dom0_pvh ||
+         (bi->hyperlaunch_enabled && !(bd->mode & BUILD_MODE_PARAVIRT)) )
     {
         dom0_cfg.flags |= (XEN_DOMCTL_CDF_hvm |
                            ((hvm_hap_supported() && !opt_dom0_shadow) ?
xen/common/domain-builder/fdt.c +21 -0
index 2c05b0a22d..aadca11dfa 100644
@@ -235,6 +235,27 @@ static int __init fdt_process_domain_node(
             bd->domid = val;
             printk(XENLOG_INFO "  domid: %d\n", bd->domid);
         }
+        else if ( !strncmp(prop_name, "mode", name_len) )
+        {
+            if ( (rc = fdt_prop_as_u32(prop, &bd->mode)) )
+            {
+                printk(XENLOG_ERR
+                       "  failed processing mode for domain %s\n", name);
+                return rc;
+            }
+
+            if ( (bd->mode & BUILD_MODE_PARAVIRT)  &&
+                 (bd->mode & BUILD_MODE_ENABLE_DM) )
+            {
+                printk(XENLOG_ERR "  mode: invalid (pv+hvm)\n");
+                return -EINVAL;
+            }
+
+            printk(XENLOG_INFO "  mode: %s\n",
+                   (bd->mode & BUILD_MODE_PARAVIRT)  ? "pv"  :
+                   (bd->mode & BUILD_MODE_ENABLE_DM) ? "hvm" :
+                                                       "pvh");
+        }
     }
 
     fdt_for_each_subnode(node, fdt, dom_node)
Signature · 1 line
2.43.0
#28 lore raw

PATCH v6 10/12 x86/hyperlaunch: add memory parsing to domain config

In reply to Alejandro Vallejo

From: "Daniel P. Smith" <[email protected]>

Add three properties, memory, mem-min, and mem-max, to the domain node device tree parsing to define the memory allocation for a domain. All three fields are expressed in kb and written as a u64 in the device tree entries.

---
 xen/arch/x86/dom0_build.c              |  8 +++++
 xen/arch/x86/include/asm/boot-domain.h |  4 +++
 xen/arch/x86/setup.c                   |  5 ++-
 xen/common/domain-builder/fdt.c        | 46 ++++++++++++++++++++++++++
 4 files changed, 62 insertions(+), 1 deletion(-)
xen/arch/x86/dom0_build.c +8 -0
index 0b467fd4a4..8db24dbc0a 100644
@@ -627,6 +627,14 @@ int __init construct_dom0(const struct boot_domain *bd)
 
     process_pending_softirqs();
 
+    /* If param dom0_size was not set and HL config provided memory size */
+    if ( !get_memsize(&dom0_size, ULONG_MAX) && bd->mem_pages )
+        dom0_size.nr_pages = bd->mem_pages;
+    if ( !get_memsize(&dom0_min_size, ULONG_MAX) && bd->min_pages )
+        dom0_min_size.nr_pages = bd->min_pages;
+    if ( !get_memsize(&dom0_max_size, ULONG_MAX) && bd->max_pages )
+        dom0_max_size.nr_pages = bd->max_pages;
+
     if ( is_hvm_domain(d) )
         rc = dom0_construct_pvh(bd);
     else if ( is_pv_domain(d) )
xen/arch/x86/include/asm/boot-domain.h +4 -0
index e316d4bcde..fa8ea1cc66 100644
@@ -18,6 +18,10 @@ struct boot_domain {
 #define BUILD_MODE_ENABLE_DM     (1 << 1) /* HVM    | PVH     */
     uint32_t mode;
 
+    unsigned long mem_pages;
+    unsigned long min_pages;
+    unsigned long max_pages;
+
     struct boot_module *kernel;
     struct boot_module *module;
     const char *cmdline;
xen/arch/x86/setup.c +4 -1
index 343f87ee9d..4a3c41ad71 100644
@@ -297,7 +297,10 @@ static const char *cmdline_cook(const char *p, const char *loader_name);
 struct boot_info __initdata xen_boot_info = {
     .loader = "unknown",
     .cmdline = "",
-    .domains = { [0 ... MAX_NR_BOOTDOMS - 1] = { .domid = DOMID_INVALID } },
+    .domains = { [0 ... MAX_NR_BOOTDOMS - 1] = {
+        .domid = DOMID_INVALID,
+        .max_pages = ULONG_MAX,
+    }},
 };
 
 static struct boot_info *__init multiboot_fill_boot_info(
xen/common/domain-builder/fdt.c +46 -0

64 lines not on this page · raw message

Signature · 1 line
2.43.0
#29 lore raw

PATCH v6 11/12 x86/hyperlaunch: add max vcpu parsing of hyperlaunch device tree

In reply to Alejandro Vallejo

From: "Daniel P. Smith" <[email protected]>

Introduce the cpus property, named as such for dom0less compatibility, that represents the maximum number of vcpus to allocate for a domain. In the device tree, it will be encoded as a u32 value.

---
 xen/arch/x86/dom0_build.c              |  3 +++
 xen/arch/x86/include/asm/boot-domain.h |  2 ++
 xen/common/domain-builder/fdt.c        | 11 +++++++++++
 3 files changed, 16 insertions(+)
xen/arch/x86/dom0_build.c +3 -0
index 8db24dbc0a..f734104c74 100644
@@ -635,6 +635,9 @@ int __init construct_dom0(const struct boot_domain *bd)
     if ( !get_memsize(&dom0_max_size, ULONG_MAX) && bd->max_pages )
         dom0_max_size.nr_pages = bd->max_pages;
 
+    if ( opt_dom0_max_vcpus_max == UINT_MAX && bd->max_vcpus )
+        opt_dom0_max_vcpus_max = bd->max_vcpus;
+
     if ( is_hvm_domain(d) )
         rc = dom0_construct_pvh(bd);
     else if ( is_pv_domain(d) )
xen/arch/x86/include/asm/boot-domain.h +2 -0
index fa8ea1cc66..969c02a6ea 100644
@@ -22,6 +22,8 @@ struct boot_domain {
     unsigned long min_pages;
     unsigned long max_pages;
 
+    unsigned int max_vcpus;
+
     struct boot_module *kernel;
     struct boot_module *module;
     const char *cmdline;
xen/common/domain-builder/fdt.c +11 -0
index d9babe9d56..02cfe8085d 100644
@@ -302,6 +302,17 @@ static int __init fdt_process_domain_node(
             bd->max_pages = PFN_DOWN(kb * SZ_1K);
             printk(XENLOG_INFO "  max memory: %lu KiB\n", kb);
         }
+        else if ( !strncmp(prop_name, "cpus", name_len) )
+        {
+            uint32_t val = UINT32_MAX;
+            if ( (rc = fdt_prop_as_u32(prop, &val)) )
+            {
+                printk(XENLOG_ERR "  bad \"cpus\" prop on domain %s\n", name);
+                return rc;
+            }
+            bd->max_vcpus = val;
+            printk(XENLOG_INFO "  cpus: %d\n", bd->max_vcpus);
+        }
     }
 
     fdt_for_each_subnode(node, fdt, dom_node)
Signature · 1 line
2.43.0
#30 lore raw

PATCH v6 12/12 x86/hyperlaunch: add capabilities to boot domain

In reply to Alejandro Vallejo

From: "Daniel P. Smith" <[email protected]>

Introduce the ability to assign capabilities to a domain via its definition in device tree. The capability property is a bitfield in both the device tree and struct boot_domain.

---
v6:
* Reworded the commit message to reflect the fact that the DOMAIN_CAPS_HARDWARE is also done here.
* Conditionalise setting CDF_iommu to DOMAIN_CAPS_HARDWARE being set.

---
 xen/arch/x86/include/asm/boot-domain.h |  3 +++
 xen/arch/x86/setup.c                   |  7 +++++--
 xen/common/domain-builder/fdt.c        | 23 +++++++++++++++++++++++
 3 files changed, 31 insertions(+), 2 deletions(-)
xen/arch/x86/include/asm/boot-domain.h +3 -0
index 969c02a6ea..cb5351241a 100644
@@ -13,6 +13,9 @@
 struct boot_domain {
     domid_t domid;
 
+    /* Bitmap. See DOMAIN_CAPS_MASK for a list */
+    uint32_t capabilities;
+
                                           /* On     | Off    */
 #define BUILD_MODE_PARAVIRT      (1 << 0) /* PV     | PVH/HVM */
 #define BUILD_MODE_ENABLE_DM     (1 << 1) /* HVM    | PVH     */
xen/arch/x86/setup.c +5 -2
index 4a3c41ad71..6a1e874b2e 100644
@@ -62,6 +62,7 @@
 
 #include <xsm/xsm.h>
 
+#include <public/bootfdt.h>
 #include <public/version.h>
 #ifdef CONFIG_COMPAT
 #include <compat/platform.h>
@@ -1044,14 +1045,15 @@ static struct domain *__init create_dom0(struct boot_info *bi)
             XEN_X86_EMU_LAPIC | XEN_X86_EMU_IOAPIC | XEN_X86_EMU_VPCI;
     }
 
-    if ( iommu_enabled )
+    if ( iommu_enabled && (bd->capabilities & DOMAIN_CAPS_HARDWARE) )
         dom0_cfg.flags |= XEN_DOMCTL_CDF_iommu;
 
     if ( bd->domid == DOMID_INVALID )
         /* Create initial domain.  Not d0 for pvshim. */
         bd->domid = get_initial_domain_id();
     d = domain_create(bd->domid, &dom0_cfg,
-                      pv_shim ? 0 : CDF_privileged | CDF_hardware);
+            ((bd->capabilities & DOMAIN_CAPS_CONTROL)  ? CDF_privileged : 0) |
+            ((bd->capabilities & DOMAIN_CAPS_HARDWARE) ? CDF_hardware   : 0));
     if ( IS_ERR(d) )
         panic("Error creating d%u: %ld\n", bd->domid, PTR_ERR(d));
 
@@ -1314,6 +1316,7 @@ void asmlinkage __init noreturn __start_xen(void)
         i = first_boot_module_index(bi, BOOTMOD_UNKNOWN);
         bi->mods[i].type = BOOTMOD_KERNEL;
         bi->domains[0].kernel = &bi->mods[i];
+        bi->domains[0].capabilities = pv_shim ? 0 : DOMAIN_CAPS_MASK;
         bi->nr_domains = 1;
         bi->hyperlaunch_enabled = false;
     }
xen/common/domain-builder/fdt.c +23 -0
index 02cfe8085d..232621ae46 100644
@@ -7,6 +7,7 @@
 #include <xen/init.h>
 #include <xen/lib.h>
 #include <xen/libfdt/libfdt.h>
+#include <public/bootfdt.h>
 #include <xen/sizes.h>
 
 #include <asm/bootinfo.h>
@@ -313,6 +314,28 @@ static int __init fdt_process_domain_node(
             bd->max_vcpus = val;
             printk(XENLOG_INFO "  cpus: %d\n", bd->max_vcpus);
         }
+        else if ( !strncmp(prop_name, "capabilities", name_len) )
+        {
+            if ( (rc = fdt_prop_as_u32(prop, &bd->capabilities)) )
+            {
+                printk(XENLOG_ERR
+                       "  bad \"capabilities\" on domain %s\n", name);
+                return rc;
+            }
+
+            if ( bd->capabilities & ~DOMAIN_CAPS_MASK )
+            {
+                printk(XENLOG_WARNING "  unknown capabilities: %#x\n",
+                       bd->capabilities & ~DOMAIN_CAPS_MASK);
+
+                bd->capabilities &= DOMAIN_CAPS_MASK;
+            }
+
+            printk(XENLOG_INFO "  caps: %s%s%s\n",
+                   (bd->capabilities & DOMAIN_CAPS_CONTROL)  ? "c" : "",
+                   (bd->capabilities & DOMAIN_CAPS_HARDWARE) ? "h" : "",
+                   (bd->capabilities & DOMAIN_CAPS_XENSTORE) ? "x" : "");
+        }
     }
 
     fdt_for_each_subnode(node, fdt, dom_node)
Signature · 1 line
2.43.0
#31 lore raw
In reply to Alejandro Vallejo
Quoted 9 lines · Alejandro Vallejo

Not very many changes here. Just:

v5->v6:
* Denis' suggestion to rename a few helpers to fdt_*
* Change to last patch to only pass CDF_iommu to domains with DOMAIN_CAPS_HARDWARE.

I _think_ this addresses all feedback I got so far and I don't expect anything major remaining before commit.

That's optimistic. I for one didn't even get around looking at v5.

Jan

#32 lore raw
Quoted 8 lines · Jan Beulich
Quoted 7 lines · Alejandro Vallejo

Not very many changes here. Just:

v5->v6:
* Denis' suggestion to rename a few helpers to fdt_*
* Change to last patch to only pass CDF_iommu to domains with DOMAIN_CAPS_HARDWARE.

Quoted 4 lines
Quoted 2 lines

I _think_ this addresses all feedback I got so far and I don't expect anything major remaining before commit.

That's optimistic. I for one didn't even get around looking at v5.

What can I say? I refill my glass often enough for it never to be half empty :)

Jokes aside, I did cover most of your large-scope concerns by v5 and v6 isn't a dramatica change. I'm hopeful the bigger thorns ought to be gone by now.

Cheers,
Alejandro