Hyperlaunch
cover Hyperlaunch
This work being submitted in this series was made possible with a great thanks to Star Lab Cop ration and their generous funding of this project.
The patch series is based on the existing xsm series for starting the idle domain privileged. After that, the first four commits were previously submitted as an RFC and expanded by an additional commit to refactor xen commandline handling. The remaining preliminary patches are FDT refactoring and a doc update. From there is where the series begins to morph x86 arch to support building multiple domains at boot.
This series has been fairly well tested using qemu with a multiboot1 bootoader and under EFI + GRUB multiboot2 boot. While there are likely some rough spots remaining in the series, it is at a point now where the series should be reviewed, exercised, and tested for consideration into tree.
Information, including docuemntation, meeting minutes, presentations, and past series postings can be found on the Xen wiki:
https://wiki.xenproject.org/wiki/Hyperlaunch
Daniel P. Smith (18):
kconfig: allow configuration of maximum modules introduction of generalized boot info
x86: adopt new boot info structures
x86: refactor entrypoints to new boot info
x86: refactor xen cmdline into general framework
fdt: make fdt handling reusable across arch
docs: update hyperlaunch device tree documentation
kconfig: introduce domain builder config option
x86: introduce abstractions for domain builder
x86: introduce the domain builder
x86: initial conversion to domain builder
x86: convert dom0 creation to domain builder
x86: generalize physmap logic
x86: generalize vcpu for domain building
x86: rework domain page allocation
x86: add pv multidomain construction
builder: introduce domain builder hypfs tree
tools: introduce example late pv helper
.gitignore | 1 +
.../designs/launch/hyperlaunch-devicetree.rst | 497 +++++++++++-------
tools/helpers/Makefile | 11 +
tools/helpers/builder-hypfs.c | 253 +++++++++
tools/helpers/hypfs-helpers.h | 9 +
tools/helpers/late-init-pv.c | 287 ++++++++++
tools/helpers/late-init-pv.h | 29 +
tools/helpers/xs-helpers.c | 117 +++++
tools/helpers/xs-helpers.h | 27 +
xen/arch/Kconfig | 12 +
xen/arch/arm/bootfdt.c | 115 +---
xen/arch/arm/include/asm/setup.h | 5 +-
xen/arch/x86/Makefile | 1 +
xen/arch/x86/boot/boot_info32.h | 97 ++++
xen/arch/x86/boot/defs.h | 17 +-
xen/arch/x86/boot/reloc.c | 187 +++++--
xen/arch/x86/bzimage.c | 18 +-
xen/arch/x86/cpu/microcode/core.c | 133 +++--
xen/arch/x86/dom0_build.c | 129 +----
xen/arch/x86/domain_builder.c | 284 ++++++++++
xen/arch/x86/efi/efi-boot.h | 96 ++--
xen/arch/x86/guest/xen/pvh-boot.c | 64 ++-
xen/arch/x86/hvm/dom0_build.c | 62 +--
xen/arch/x86/include/asm/bootdomain.h | 30 ++
xen/arch/x86/include/asm/bootinfo.h | 99 ++++
xen/arch/x86/include/asm/bzimage.h | 5 +-
xen/arch/x86/include/asm/dom0_build.h | 27 +-
xen/arch/x86/include/asm/guest/pvh-boot.h | 6 +-
xen/arch/x86/include/asm/setup.h | 18 +-
xen/arch/x86/pv/Makefile | 2 +-
.../x86/pv/{dom0_build.c => domain_builder.c} | 141 ++---
xen/arch/x86/pv/shim.c | 4 +-
xen/arch/x86/setup.c | 392 ++++++--------
xen/common/Kconfig | 5 +
xen/common/Makefile | 4 +-
xen/common/domain-builder/Kconfig | 36 ++
xen/common/domain-builder/Makefile | 3 +
xen/common/domain-builder/core.c | 207 ++++++++
xen/common/domain-builder/fdt.c | 295 +++++++++++
xen/common/domain-builder/fdt.h | 7 +
xen/common/domain-builder/hypfs.c | 193 +++++++
xen/common/efi/boot.c | 4 +-
xen/common/fdt.c | 131 +++++
xen/common/sched/core.c | 25 +-
xen/include/xen/bootdomain.h | 58 ++
xen/include/xen/bootinfo.h | 132 +++++
xen/include/xen/device_tree.h | 50 +-
xen/include/xen/domain_builder.h | 88 ++++
xen/include/xen/fdt.h | 79 +++
xen/include/xen/sched.h | 3 +-
xen/include/xsm/xsm.h | 26 +-
xen/xsm/xsm_core.c | 43 +-
xen/xsm/xsm_policy.c | 56 +-
53 files changed, 3544 insertions(+), 1076 deletions(-)
create mode 100644 tools/helpers/builder-hypfs.c
create mode 100644 tools/helpers/hypfs-helpers.h
create mode 100644 tools/helpers/late-init-pv.c
create mode 100644 tools/helpers/late-init-pv.h
create mode 100644 tools/helpers/xs-helpers.c
create mode 100644 tools/helpers/xs-helpers.h
create mode 100644 xen/arch/x86/boot/boot_info32.h
create mode 100644 xen/arch/x86/domain_builder.c
create mode 100644 xen/arch/x86/include/asm/bootdomain.h
create mode 100644 xen/arch/x86/include/asm/bootinfo.h
rename xen/arch/x86/pv/{dom0_build.c => domain_builder.c} (88%)
create mode 100644 xen/common/domain-builder/Kconfig
create mode 100644 xen/common/domain-builder/Makefile
create mode 100644 xen/common/domain-builder/core.c
create mode 100644 xen/common/domain-builder/fdt.c
create mode 100644 xen/common/domain-builder/fdt.h
create mode 100644 xen/common/domain-builder/hypfs.c
create mode 100644 xen/common/fdt.c
create mode 100644 xen/include/xen/bootdomain.h
create mode 100644 xen/include/xen/bootinfo.h
create mode 100644 xen/include/xen/domain_builder.h
create mode 100644 xen/include/xen/fdt.hSignature · 1 line
2.20.1
PATCH 1/18 kconfig: allow configuration of maximum modules
For x86 the number of allowable multiboot modules varies between the different entry points, non-efi boot, pvh boot, and efi boot. In the case of both Arm and x86 this value is fixed to values based on generalized assumptions. With hyperlaunch for x86 and dom0less on Arm, use of static sizes results in large allocations compiled into the hypervisor that will go unused by many use cases.
This commit introduces a Kconfig variable that is set with sane defaults based on configuration selection. This variable is in turned used as the array size for the cases where a static allocated array of boot modules is declared.
--- xen/arch/Kconfig | 12 ++++++++++++ xen/arch/arm/include/asm/setup.h | 5 +++-- xen/arch/x86/efi/efi-boot.h | 2 +- xen/arch/x86/guest/xen/pvh-boot.c | 2 +- xen/arch/x86/setup.c | 4 ++-- 5 files changed, 19 insertions(+), 6 deletions(-)
xen/arch/Kconfig +12 -0
@@ -17,3 +17,15 @@ config NR_CPUS For CPU cores which support Simultaneous Multi-Threading or similar technologies, this the number of logical threads which Xen will support. + +config NR_BOOTMODS + int "Maximum number of boot modules that a loader can pass" + range 1 32768 + default "8" if X86 + default "32" if ARM + help + Controls the build-time size of various arrays allocated for + parsing the boot modules passed by a loader when starting Xen. + + This is of particular interest when using Xen's hypervisor domain + capabilities such as dom0less.
xen/arch/arm/include/asm/setup.h +3 -2
@@ -10,7 +10,8 @@#define NR_MEM_BANKS 256 -#define MAX_MODULES 32/* Current maximum useful modules */ +/* Current maximum useful modules */ +#define MAX_MODULES CONFIG_NR_BOOTMODS typedef enum { BOOTMOD_XEN,
@@ -38,7 +39,7 @@ struct meminfo {* The domU flag is set for kernels and ramdisks of "xen,domain" nodes. * The purpose of the domU flag is to avoid getting confused in * kernel_probe, where we try to guess which is the dom0 kernel and -* initrd to be compatible with all versions of the multiboot spec. +* initrd to be compatible with all versions of the multiboot spec. */ #define BOOTMOD_MAX_CMDLINE 1024struct bootmodule {
xen/arch/x86/efi/efi-boot.h +1 -1
@@ -18,7 +18,7 @@ static multiboot_info_t __initdata mbi = {* The array size needs to be one larger than the number of modules we * support - see __start_xen(). */ -static module_t __initdata mb_modules[5 ]; +static module_t __initdata mb_modules[CONFIG_NR_BOOTMODS +1 ];static void __init edd_put_string (u8 *dst,size_t n,const char *src) {
xen/arch/x86/guest/xen/pvh-boot.c +1 -1
@@ -32,7 +32,7 @@ bool __initdata pvh_boot;uint32_t __initdata pvh_start_info_pa;static multiboot_info_t __initdata pvh_mbi; -static module_t __initdata pvh_mbi_mods[8 ]; +static module_t __initdata pvh_mbi_mods[CONFIG_NR_BOOTMOD +1 ];static const char *__initdata pvh_loader ="PVH Directboot" ; static void __init convert_pvh_info(multiboot_info_t **mbi,
xen/arch/x86/setup.c +2 -2
@@ -1020,9 +1020,9 @@ void __init noreturn __start_xen(unsigned long mbi_p)panic ("dom0 kernel not specified. Check bootloader configuration\n" );/* Check that we don't have a silly number of modules. */ -if ( mbi->mods_count >sizeof (module_map) *8 ) +if ( mbi->mods_count >CONFIG_NR_BOOTMODS ) { - mbi->mods_count =sizeof (module_map) *8 ; + mbi->mods_count =CONFIG_NR_BOOTMODS ;printk ("Excessive multiboot modules - using the first %u only\n" , mbi->mods_count); }
Signature · 1 line
2.20.1
RE: [PATCH v1 01/18] kconfig: allow configuration of maximum modules
Hi Daniel,
Quoted 12 lines
-----Original Message-----
Subject: [PATCH v1 01/18] kconfig: allow configuration of maximum modulesFor x86 the number of allowable multiboot modules varies between the different
entry points, non-efi boot, pvh boot, and efi boot. In the case of both Arm and
x86 this value is fixed to values based on generalized assumptions. With hyperlaunch for x86 and dom0less on Arm, use of static sizes results in large allocations compiled into the hypervisor that will go unused by many use cases.
Quoted 6 lines
This commit introduces a
Kconfigvariable that is set with sane defaults based on configuration selection. This variable is in turned used as the array size for the cases where a static allocated array of boot modules is declared.Signed-off-by Daniel P. Smith <[email protected]>Reviewed-by Christopher Clark <[email protected]>
Quoted 45 lines
--- xen/arch/Kconfig | 12 ++++++++++++ xen/arch/arm/include/asm/setup.h | 5 +++-- xen/arch/x86/efi/efi-boot.h | 2 +- xen/arch/x86/guest/xen/pvh-boot.c | 2 +- xen/arch/x86/setup.c | 4 ++-- 5 files changed, 19 insertions(+), 6 deletions(-)xen/arch/Kconfig +0
-0@@ -17,3 +17,15 @@ config NR_CPUS For CPU cores which support Simultaneous Multi-Threading orsimilar
xen/arch/Kconfig +12
-0technologies, this the number of logical threads which Xen will support. + +config NR_BOOTMODS + int "Maximum number of boot modules that a loader can pass" + range 1 32768 + default "8" if X86 + default "32" if ARM + help + Controls the build-time size of various arrays allocated for + parsing the boot modules passed by a loader when starting Xen. + + This is of particular interest when using Xen's hypervisor domain + capabilities such as dom0less.xen/arch/arm/include/asm/setup.h +2
-1@@ -10,7 +10,8 @@#define NR_MEM_BANKS 256 -#define MAX_MODULES 32/* Current maximum useful modules */ +/* Current maximum useful modules */ +#define MAX_MODULES CONFIG_NR_BOOTMODS typedef enum { BOOTMOD_XEN,
xen/arch/arm/include/asm/setup.h struct meminfo { · line 42
Quoted 5 lines
xen/arch/arm/include/asm/setup.h +1
-1@@ -38,7 +39,7 @@ struct meminfo {* The domU flag is set for kernels and ramdisks of "xen,domain" nodes. * The purpose of the domU flag is to avoid getting confused in * kernel_probe, where we try to guess which is the dom0 kernel and -* initrd to be compatible with all versions of the multiboot spec. +* initrd to be compatible with all versions of the multiboot spec.
Thanks for taking the chance to remove the space in the end of the sentence.
Kind regards,
Henry
Re: [PATCH v1 01/18] kconfig: allow configuration of maximum modules
In reply to Daniel P. SmithHi Daniel,
Quoted 6 lines · Daniel P. Smith
For x86 the number of allowable multiboot modules varies between the different entry points, non-efi boot, pvh boot, and efi boot. In the case of both Arm and x86 this value is fixed to values based on generalized assumptions. With hyperlaunch for x86 and dom0less on Arm, use of static sizes results in large allocations compiled into the hypervisor that will go unused by many use cases.
Quoted 6 lines
This commit introduces a
Kconfigvariable that is set with sane defaults based on configuration selection. This variable is in turned used as the array size for the cases where a static allocated array of boot modules is declared.Signed-off-by Daniel P. Smith <[email protected]>Reviewed-by Christopher Clark <[email protected]>
I am not entirely sure where this reviewed-by is coming from. Is this from internal review?
If yes, my recommendation would be to provide the reviewed-by on the mailing list. Ideally, the review should also be done in the open, but I understand some company wish to do a fully internal review first.
At least from a committer perspective, this helps me to know whether the reviewed-by still apply. An example would be if you send a v2, I would not be able to know whether Christoffer still agreed on the change.
Quoted 43 lines
--- xen/arch/Kconfig | 12 ++++++++++++ xen/arch/arm/include/asm/setup.h | 5 +++-- xen/arch/x86/efi/efi-boot.h | 2 +- xen/arch/x86/guest/xen/pvh-boot.c | 2 +- xen/arch/x86/setup.c | 4 ++-- 5 files changed, 19 insertions(+), 6 deletions(-)xen/arch/Kconfig +12
-0@@ -17,3 +17,15 @@ config NR_CPUS For CPU cores which support Simultaneous Multi-Threading or similar technologies, this the number of logical threads which Xen will support. + +config NR_BOOTMODS + int "Maximum number of boot modules that a loader can pass" + range 1 32768 + default "8" if X86 + default "32" if ARM + help + Controls the build-time size of various arrays allocated for + parsing the boot modules passed by a loader when starting Xen. + + This is of particular interest when using Xen's hypervisor domain + capabilities such as dom0less.xen/arch/arm/include/asm/setup.h +2
-1@@ -10,7 +10,8 @@#define NR_MEM_BANKS 256 -#define MAX_MODULES 32/* Current maximum useful modules */ +/* Current maximum useful modules */ +#define MAX_MODULES CONFIG_NR_BOOTMODS typedef enum { BOOTMOD_XEN,
xen/arch/arm/include/asm/setup.h struct meminfo { · line 42
Quoted 5 lines
xen/arch/arm/include/asm/setup.h +1
-1@@ -38,7 +39,7 @@ struct meminfo {* The domU flag is set for kernels and ramdisks of "xen,domain" nodes. * The purpose of the domU flag is to avoid getting confused in * kernel_probe, where we try to guess which is the dom0 kernel and -* initrd to be compatible with all versions of the multiboot spec. +* initrd to be compatible with all versions of the multiboot spec.
In general, I much prefer if coding style changes are done separately because it helps the review (I don't have to stare at the line to figure out what changed).
I am not going to force this here. However, the strict minimum is to mention the change in the commit message.
xen/arch/x86/efi/efi-boot.h static multiboot_info_t __initdata mbi = { · line 21
Quoted 13 lines
xen/arch/arm/include/asm/setup.h +0
-0*/ #define BOOTMOD_MAX_CMDLINE 1024struct bootmodule {xen/arch/x86/efi/efi-boot.h +1
-1@@ -18,7 +18,7 @@ static multiboot_info_t __initdata mbi = {* The array size needs to be one larger than the number of modules we * support - see __start_xen(). */ -static module_t __initdata mb_modules[5 ]; +static module_t __initdata mb_modules[CONFIG_NR_BOOTMODS +1 ];
Please explain in the commit message why the number of modules was bumped from 5 to 9.
xen/arch/x86/guest/xen/pvh-boot.c bool __initdata pvh_boot; · line 35
Quoted 13 lines
xen/arch/x86/efi/efi-boot.h +0
-0static void __init edd_put_string (u8 *dst,size_t n,const char *src) {xen/arch/x86/guest/xen/pvh-boot.c +1
-1@@ -32,7 +32,7 @@ bool __initdata pvh_boot;uint32_t __initdata pvh_start_info_pa;static multiboot_info_t __initdata pvh_mbi; -static module_t __initdata pvh_mbi_mods[8 ]; +static module_t __initdata pvh_mbi_mods[CONFIG_NR_BOOTMOD +1 ];
What's the +1 for?
Quoted 9 lines
xen/arch/x86/guest/xen/pvh-boot.c +0
-0static const char *__initdata pvh_loader ="PVH Directboot" ; static void __init convert_pvh_info(multiboot_info_t **mbi,xen/arch/x86/setup.c +0
-0@@ -1020,9 +1020,9 @@ void __init noreturn __start_xen(unsigned long mbi_p)panic ("dom0 kernel not specified. Check bootloader configuration\n" );
xen/arch/x86/setup.c void __init noreturn __start_xen(unsigned long mbi_p) · line 1028
Quoted 10 lines
xen/arch/x86/setup.c +2
-2/* Check that we don't have a silly number of modules. */ -if ( mbi->mods_count >sizeof (module_map) *8 ) +if ( mbi->mods_count >CONFIG_NR_BOOTMODS ) { - mbi->mods_count =sizeof (module_map) *8 ; + mbi->mods_count =CONFIG_NR_BOOTMODS ;printk ("Excessive multiboot modules - using the first %u only\n" , mbi->mods_count); }
AFAIU, this check is to make sure that we will not overrun module_map in the next line:
bitmap_fill(module_map, mbi->mods_count);
The current definition of module_map will allow 64 modules. But you are allowing 32768. So I think you either want to keep the check or define module_map as:
DECLARE_BITMAP(module_map, CONFIG_NR_BOOTMODS);
Cheers,
Signature · 1 line
Julien Grall
Re: [PATCH v1 01/18] kconfig: allow configuration of maximum modules
Quoted 19 lines · Julien Grall
Hi Daniel,
Quoted 16 lines · Daniel P. Smith
For x86 the number of allowable multiboot modules varies between the different
entry points, non-efi boot, pvh boot, and efi boot. In the case of both Arm and
x86 this value is fixed to values based on generalized assumptions. With hyperlaunch for x86 and dom0less on Arm, use of static sizes results in large
allocations compiled into the hypervisor that will go unused by many use cases.This commit introduces a
Kconfigvariable that is set with sane defaults based
on configuration selection. This variable is in turned used as the array size
for the cases where a static allocated array of boot modules is declared.
Quoted 5 lines
Quoted 2 lines
Signed-off-by Daniel P. Smith <[email protected]>Reviewed-by Christopher Clark <[email protected]>I am not entirely sure where this reviewed-by is coming from. Is this from internal review?
Yes.
Quoted 3 lines
If yes, my recommendation would be to provide the reviewed-by on the mailing list. Ideally, the review should also be done in the open, but I understand some company wish to do a fully internal review first.
Since this capability is being jointly developed by Christopher and I, with myself being the author of code, Christopher reviewed the code as the co-developer. He did so as a second pair of eyes for any obvious mistakes and to concur that the implementation was in line with the approach the two of us architected. Perhaps a SoB line might be more appropriate than an R-b line.
Quoted 3 lines
At least from a committer perspective, this helps me to know whether the reviewed-by still apply. An example would be if you send a v2, I would not be able to know whether Christoffer still agreed on the change.
If an SoB line is more appropriate, then on the next version I can switch it
Quoted 43 lines
Quoted 43 lines
---
xen/arch/Kconfig | 12 ++++++++++++ xen/arch/arm/include/asm/setup.h | 5 +++-- xen/arch/x86/efi/efi-boot.h | 2 +-xen/arch/x86/guest/xen/pvh-boot.c| 2 +- xen/arch/x86/setup.c | 4 ++--5 files changed, 19 insertions(+), 6 deletions(-)xen/arch/Kconfig +0
-0@@ -17,3 +17,15 @@ config NR_CPUSFor CPU cores which support Simultaneous Multi-Threading or similar
technologies, this the number of logical threads which Xen will support.xen/arch/Kconfig +12
-0+ +config NR_BOOTMODS + int "Maximum number of boot modules that a loader can pass" + range 1 32768 + default "8" if X86 + default "32" if ARM + help + Controls the build-time size of various arrays allocated for + parsing the boot modules passed by a loader when starting Xen. + + This is of particular interest when using Xen's hypervisor domain + capabilities such as dom0less.xen/arch/arm/include/asm/setup.h +0
-0@@ -10,7 +10,8 @@ #define NR_MEM_BANKS 256-#define
MAX_MODULES32 /* Current maximum useful modules */xen/arch/arm/include/asm/setup.h +2
-0+/* Current maximum useful modules */ +#define MAX_MODULES CONFIG_NR_BOOTMODStypedef enum {
BOOTMOD_XEN,xen/arch/arm/include/asm/setup.h +0
-0@@ -38,7 +39,7 @@ struct meminfo { * The domU flag is set for kernels and ramdisks of "xen,domain" nodes.
Quoted 8 lines
xen/arch/arm/include/asm/setup.h struct meminfo { · line 42
Quoted 4 lines
xen/arch/arm/include/asm/setup.h +1
-1* The purpose of the domU flag is to avoid getting confused in * kernel_probe, where we try to guess which is the dom0 kernel and - * initrd to be compatible with all versions of the multiboot spec. + * initrd to be compatible with all versions of the multiboot spec.In general, I much prefer if coding style changes are done separately because it helps the review (I don't have to stare at the line to figure out what changed).
Actually, on a past review of another series I got dinged on this, and I did try to get most of them out of this series. This is just a straggler that I missed. I will clean up on next revision.
Quoted 19 lines
I am not going to force this here. However, the strict minimum is to mention the change in the commit message.
xen/arch/x86/efi/efi-boot.h static multiboot_info_t __initdata mbi = { · line 21
Quoted 13 lines
xen/arch/arm/include/asm/setup.h +0
-0*/ #define BOOTMOD_MAX_CMDLINE 1024 struct bootmodule {xen/arch/x86/efi/efi-boot.h +1
-1@@ -18,7 +18,7 @@ static multiboot_info_t __initdata mbi = {* The array size needs to be one larger than the number of modules we * support - see __start_xen(). */ -static module_t __initdata mb_modules[5 ]; +static module_t __initdata mb_modules[CONFIG_NR_BOOTMODS +1 ];Please explain in the commit message why the number of modules was bumped from 5 to 9.
The number of modules were inconsistent between the different entry points into __start_xen(). By switching to a Kconfig variable, whose default was set to the largest value used across the entry points, results in change for the locations using another value.
See below for +1 explanation.
Quoted 6 lines
Quoted 6 lines
xen/arch/x86/efi/efi-boot.h +0
-0static void __init edd_put_string(u8 *dst, size_t n, const char *src) {xen/arch/x86/guest/xen/pvh-boot.c +0
-1--- a/xen/arch/x86/guest/xen/pvh-boot.c
Quoted 8 lines
xen/arch/x86/guest/xen/pvh-boot.c bool __initdata pvh_boot; · line 34
Quoted 6 lines
+++
b/xen/arch/x86/guest/xen/pvh-boot.cxen/arch/x86/guest/xen/pvh-boot.c +1
-1@@ -32,7 +32,7 @@ bool __initdata pvh_boot; uint32_t __initdata pvh_start_info_pa; static multiboot_info_t __initdata pvh_mbi; -static module_t __initdata pvh_mbi_mods[8 ]; +static module_t __initdata pvh_mbi_mods[CONFIG_NR_BOOTMOD +1 ];What's the +1 for?
I should clarify in the commit message, but the value set in CONFIG_NR_BOOTMOD is the max modules that Xen would accept from a bootloader. Xen startup code expects to be able to append Xen itself as the array. The +1 allocates an additional entry to store Xen in the array should a bootloader actually pass CONFIG_NR_BOOTMOD modules to Xen. There is an existing comment floating in one of these locations that explained it.
Quoted 24 lines
Quoted 12 lines
xen/arch/x86/guest/xen/pvh-boot.c +0
-0static const char *__initdata pvh_loader = "PVH Directboot"; static void __init convert_pvh_info(multiboot_info_t **mbi,xen/arch/x86/setup.c +0
-0@@ -1020,9 +1020,9 @@ void __init noreturn __start_xen(unsigned long mbi_p)panic("dom0 kernel not specified. Check bootloader configuration\n");
/* Check that we don't have a silly number of modules. */ - if (mbi->mods_count>sizeof(module_map)* 8 )Quoted 8 lines
+ if (
mbi->mods_count>CONFIG_NR_BOOTMODS)
{
-mbi->mods_count=sizeof(module_map)* 8;
+mbi->mods_count=CONFIG_NR_BOOTMODS; printk("Excessive multiboot modules - using the first %u only\n",
mbi->mods_count);
}AFAIU, this check is to make sure that we will not overrun
module_mapin the next line:
Quoted 7 lines
bitmap_fill(module_map, mbi->mods_count);The current definition of
module_mapwill allow 64 modules. But you are allowing 32768. So I think you either want to keep the check or definemodule_mapas:
DECLARE_BITMAP(module_map, CONFIG_NR_BOOTMODS);
Yes, in the RFC I had it capped to 64 and lost track of this related changed when it was bumped to 32768 per the review discussion. Later in the series, module_map goes away. To ensure stability at this point I would be inclined to restore the 64 module clamp down check. Thoughts?
v/r,
dps
Re: [PATCH v1 01/18] kconfig: allow configuration of maximum modules
Hi Daniel,
Quoted 40 lines · Daniel P. Smith
Quoted 19 lines · Julien Grall
Hi Daniel,
Quoted 16 lines · Daniel P. Smith
For x86 the number of allowable multiboot modules varies between the different
entry points, non-efi boot, pvh boot, and efi boot. In the case of both Arm and
x86 this value is fixed to values based on generalized assumptions. With hyperlaunch for x86 and dom0less on Arm, use of static sizes results in large
allocations compiled into the hypervisor that will go unused by many use cases.This commit introduces a
Kconfigvariable that is set with sane defaults based
on configuration selection. This variable is in turned used as the array size
for the cases where a static allocated array of boot modules is declared.Quoted 5 lines
Quoted 2 lines
Signed-off-by Daniel P. Smith <[email protected]>Reviewed-by Christopher Clark <[email protected]>I am not entirely sure where this reviewed-by is coming from. Is this from internal review?
Yes.
Quoted 3 lines
If yes, my recommendation would be to provide the reviewed-by on the mailing list. Ideally, the review should also be done in the open, but I understand some company wish to do a fully internal review first.
Since this capability is being jointly developed by Christopher and I, with myself being the author of code, Christopher reviewed the code as the co-developer. He did so as a second pair of eyes for any obvious mistakes and to concur that the implementation was in line with the approach the two of us architected. Perhaps a SoB line might be more appropriate than an R-b line.
Quoted 6 lines
Quoted 3 lines
At least from a committer perspective, this helps me to know whether the reviewed-by still apply. An example would be if you send a v2, I would not be able to know whether Christoffer still agreed on the change.
If an SoB line is more appropriate, then on the next version I can switch it
Thanks for the explanation. To me "signed-off-by" means the person wrote some code (or sent the patches) code. So from above, it sounds more like Christoffer did a review.
So I think it is more suitable for him to provide a reviewed-by. For follow-up, my preference would be Christoffer to provide the reviewed-by on the ML.
If it is too much overhead, I would suggest to log the latest version Christoffer reviewed-by in the changelog. I usually do:
Changes in vX:
- Add Christoffer's reviewed-by
Or if he will reviewing every version, just mention it in the cover letter.
Quoted 8 lines
Quoted 3 lines
Please explain in the commit message why the number of modules was bumped from 5 to 9.
The number of modules were inconsistent between the different entry points into
__start_xen(). By switching to aKconfigvariable, whose default was set to the largest value used across the entry points, results in change for the locations using another value.
Ok. Can you add something like: "For x86, the number of modules is not consistent across the code base. Use the maximum"?
Quoted 18 lines
See below for +1 explanation.
Quoted 6 lines
Quoted 6 lines
static void
__initedd_put_string(u8 *dst, size_t n, const char *src){xen/arch/x86/guest/xen/pvh-boot.c +0
-1--- a/xen/arch/x86/guest/xen/pvh-boot.cQuoted 8 lines
xen/arch/x86/guest/xen/pvh-boot.c bool __initdata pvh_boot; · line 34
Quoted 6 lines
+++
b/xen/arch/x86/guest/xen/pvh-boot.cxen/arch/x86/guest/xen/pvh-boot.c +1
-1@@ -32,7 +32,7 @@ bool __initdata pvh_boot; uint32_t __initdata pvh_start_info_pa; static multiboot_info_t __initdata pvh_mbi; -static module_t __initdata pvh_mbi_mods[8 ]; +static module_t __initdata pvh_mbi_mods[CONFIG_NR_BOOTMOD +1 ];What's the +1 for?
Quoted 7 lines
I should clarify in the commit message, but the value set in
CONFIG_NR_BOOTMODis the max modules that Xen would accept from a bootloader. Xen startup code expects to be able to append Xen itself as the array. The +1 allocates an additional entry to store Xen in the array should a bootloader actually passCONFIG_NR_BOOTMODmodules to Xen. There is an existing comment floating in one of these locations that explained it.
This makes sense. So every use of CONFIG_NR_BOOTMOD would end up to require +1. Is that correct?
If yes, then I think it would be better to require CONFIG_NR_BOOTMOD to be at minimum 1. This would reduce the risk to have different array size again. That said, this is x86 code, so the call is for the x86 maintainers.
Quoted 31 lines
Quoted 31 lines
Quoted 19 lines
xen/arch/x86/guest/xen/pvh-boot.c +0
-0static const char *__initdata pvh_loader = "PVH Directboot"; static void __init convert_pvh_info(multiboot_info_t **mbi,xen/arch/x86/setup.c +2
-2@@ -1020,9 +1020,9 @@ void __init noreturn __start_xen(unsigned long mbi_p) panic("dom0 kernel not specified. Check bootloader configuration\n"); /* Check that we don't have a silly number of modules. */ - if ( mbi->mods_count > sizeof(module_map) * 8 ) + if ( mbi->mods_count > CONFIG_NR_BOOTMODS ) { - mbi->mods_count = sizeof(module_map) * 8; + mbi->mods_count = CONFIG_NR_BOOTMODS; printk("Excessive multiboot modules - using the first %u only\n",Quoted 2 lines
mbi->mods_count);
}AFAIU, this check is to make sure that we will not overrun
module_mapin the next line:
bitmap_fill(module_map, mbi->mods_count);The current definition of
module_mapwill allow 64 modules. But you are allowing 32768. So I think you either want to keep the check or definemodule_mapas:
Quoted 6 lines
Quoted 1 line
DECLARE_BITMAP(module_map, CONFIG_NR_BOOTMODS);Yes, in the RFC I had it capped to 64 and lost track of this related changed when it was bumped to 32768 per the review discussion. Later in the series,
module_mapgoes away. To ensure stability at this point I would be inclined to restore the 64 module clamp down check. Thoughts?
I don't know what would a sensible value for x86. I will leave this question to the x86 maintainers.
Cheers,
Signature · 1 line
Julien Grall
Re: [PATCH v1 01/18] kconfig: allow configuration of maximum modules
Quoted 22 lines · Julien Grall
Quoted 13 lines · Daniel P. Smith
Quoted 11 lines · Julien Grall
xen/arch/x86/guest/xen/pvh-boot.c bool __initdata pvh_boot; · line 34
Quoted 8 lines · Daniel P. Smith
index
498625eae0..834b1ad16b100644xen/arch/x86/guest/xen/pvh-boot.c +1
-1@@ -32,7 +32,7 @@ bool __initdata pvh_boot; uint32_t __initdata pvh_start_info_pa; static multiboot_info_t __initdata pvh_mbi; -static module_t __initdata pvh_mbi_mods[8 ]; +static module_t __initdata pvh_mbi_mods[CONFIG_NR_BOOTMOD +1 ];What's the +1 for?
Quoted 7 lines
I should clarify in the commit message, but the value set in
CONFIG_NR_BOOTMODis the max modules that Xen would accept from a bootloader. Xen startup code expects to be able to append Xen itself as the array. The +1 allocates an additional entry to store Xen in the array should a bootloader actually passCONFIG_NR_BOOTMODmodules to Xen. There is an existing comment floating in one of these locations that explained it.
Quoted 6 lines
This makes sense. So every use of
CONFIG_NR_BOOTMODwould end up to require +1. Is that correct?If yes, then I think it would be better to require
CONFIG_NR_BOOTMODto be at minimum 1. This would reduce the risk to have different array size again. That said, this is x86 code, so the call is for the x86 maintainers.
I think the Kconfig setting should stand for "true" modules. Anywhere that x86 code internally uses one extra slot this should be expressed by an explicit "+ 1" imo.
Quoted 32 lines
Quoted 32 lines
Quoted 31 lines
Quoted 18 lines
xen/arch/x86/guest/xen/pvh-boot.c +0
-0static const char *__initdata pvh_loader = "PVH Directboot"; static void __init convert_pvh_info(multiboot_info_t **mbi,xen/arch/x86/setup.c +2
-2@@ -1020,9 +1020,9 @@ void __init noreturn __start_xen(unsigned long mbi_p) panic("dom0 kernel not specified. Check bootloader configuration\n"); /* Check that we don't have a silly number of modules. */ - if ( mbi->mods_count > sizeof(module_map) * 8 ) + if ( mbi->mods_count > CONFIG_NR_BOOTMODS ) { - mbi->mods_count = sizeof(module_map) * 8; + mbi->mods_count = CONFIG_NR_BOOTMODS; printk("Excessive multiboot modules - using the first %u only\n",Quoted 2 lines
mbi->mods_count);
}AFAIU, this check is to make sure that we will not overrun
module_mapin the next line:
bitmap_fill(module_map, mbi->mods_count);The current definition of
module_mapwill allow 64 modules. But you are allowing 32768. So I think you either want to keep the check or definemodule_mapas:
DECLARE_BITMAP(module_map, CONFIG_NR_BOOTMODS);
Quoted 7 lines
Quoted 4 lines
Yes, in the RFC I had it capped to 64 and lost track of this related changed when it was bumped to 32768 per the review discussion. Later in the series,
module_mapgoes away. To ensure stability at this point I would be inclined to restore the 64 module clamp down check. Thoughts?I don't know what would a sensible value for x86. I will leave this question to the x86 maintainers.
I guess I'd be fine either way, as long as the code is correct.
Jan
Re: [PATCH v1 01/18] kconfig: allow configuration of maximum modules
In reply to Daniel P. Smithxen/arch/Kconfig config NR_CPUS · line 25
Quoted 12 lines · Daniel P. Smith
xen/arch/Kconfig +6
-0@@ -17,3 +17,15 @@ config NR_CPUS For CPU cores which support Simultaneous Multi-Threading or similar technologies, this the number of logical threads which Xen will support. + +config NR_BOOTMODS + int "Maximum number of boot modules that a loader can pass" + range 1 32768 + default "8" if X86 + default "32" if ARM
Any reason for the larger default on Arm, irrespective of dom0less actually being in use? (I'm actually surprised I can't spot a Kconfig option controlling inclusion of dom0less. The default here imo isn't supposed to depend on the architecture, but on whether dom0less is supported. That way if another arch gained dom0less support, the higher default would apply to it without needing further adjustment.)
xen/arch/x86/efi/efi-boot.h static multiboot_info_t __initdata mbi = { · line 21
Quoted 8 lines
xen/arch/x86/efi/efi-boot.h +1
-1@@ -18,7 +18,7 @@ static multiboot_info_t __initdata mbi = {* The array size needs to be one larger than the number of modules we * support - see __start_xen(). */ -static module_t __initdata mb_modules[5 ]; +static module_t __initdata mb_modules[CONFIG_NR_BOOTMODS +1 ];
If the build admin selected 1, I'm pretty sure about nothing would work. I think you want max(5, CONFIG_NR_BOOTMODS) ormax(4, CONFIG_NR_BOOTMODS) + 1 here and ...
xen/arch/x86/guest/xen/pvh-boot.c bool __initdata pvh_boot; · line 35
Quoted 8 lines
xen/arch/x86/guest/xen/pvh-boot.c +1
-1@@ -32,7 +32,7 @@ bool __initdata pvh_boot;uint32_t __initdata pvh_start_info_pa;static multiboot_info_t __initdata pvh_mbi; -static module_t __initdata pvh_mbi_mods[8 ]; +static module_t __initdata pvh_mbi_mods[CONFIG_NR_BOOTMOD +1 ];
... max(8, CONFIG_NR_BOOTMODS) here (albeit the 8 may have room for lowering - I don't recall why 8 was chosen rather than going with the minimum possible value covering all module kinds known at that time).
Jan
Re: [PATCH v1 01/18] kconfig: allow configuration of maximum modules
Quoted 12 lines · Jan Beulich
Quoted 11 lines · Daniel P. Smith
xen/arch/Kconfig +5
-0@@ -17,3 +17,15 @@ config NR_CPUS For CPU cores which support Simultaneous Multi-Threading or similar technologies, this the number of logical threads which Xen will support. + +config NR_BOOTMODS + int "Maximum number of boot modules that a loader can pass" + range 1 32768 + default "8" if X86
Quoted 8 lines
xen/arch/Kconfig config NR_CPUS · line 25
Quoted 1 line
xen/arch/Kconfig +1
-0+ default "32" if ARMAny reason for the larger default on Arm, irrespective of dom0less actually being in use? (I'm actually surprised I can't spot a
Kconfigoption controlling inclusion of dom0less. The default here imo isn't supposed to depend on the architecture, but on whether dom0less is supported. That way if another arch gained dom0less support, the higher default would apply to it without needing further adjustment.)
Yes, multidomain construction is always on for Arm and the only configurable is a commandline parameter to enforce that dom0 is not created. As for the default, it was selected based on the largest value used in the locations replaced by the Kconfig variable. Since there was a significant difference between Arm and x86, I did not feel it was appropriate to reduce/increase either, since it drives multiple static array allocations for x86.
I have no attachments to any specific value, so I will freely adjust to whatever conscience the community might come to.
Quoted 4 lines
Quoted 4 lines
xen/arch/x86/efi/efi-boot.h +0
-0@@ -18,7 +18,7 @@ static multiboot_info_t __initdata mbi = {* The array size needs to be one larger than the number of modules we
Quoted 8 lines
xen/arch/x86/efi/efi-boot.h static multiboot_info_t __initdata mbi = { · line 21
Quoted 4 lines
xen/arch/x86/efi/efi-boot.h +1
-1* support - see __start_xen(). */ -static module_t __initdata mb_modules[5 ]; +static module_t __initdata mb_modules[CONFIG_NR_BOOTMODS +1 ];If the build admin selected 1, I'm pretty sure about nothing would work. I think you want
max(5, CONFIG_NR_BOOTMODS)ormax(4, CONFIG_NR_BOOTMODS)+ 1 here and ...
Actually, I reasoned this out and 1 is in fact a valid value. It would mean Xen + Dom0 Linux kernel with embedded initramfs with no externally loaded XSM policy and no boot time microcode patching. This is a working configuration, but open to debate if it is a desirable configuration. The question is whether it is desired to block someone from building such a configuration, or any number between 1 and 4. If the answer is yes, then why not just set the lower bound of the range in the Kconfig file instead of having to maintain a hard-coded lower bound in a max marco across multiple locations?
Quoted 5 lines
Quoted 5 lines
xen/arch/x86/guest/xen/pvh-boot.c +0
-0@@ -32,7 +32,7 @@ bool __initdata pvh_boot;uint32_t __initdata pvh_start_info_pa;
Quoted 8 lines
xen/arch/x86/guest/xen/pvh-boot.c bool __initdata pvh_boot; · line 35
Quoted 3 lines
xen/arch/x86/guest/xen/pvh-boot.c +1
-1static multiboot_info_t __initdata pvh_mbi; -static module_t __initdata pvh_mbi_mods[8 ]; +static module_t __initdata pvh_mbi_mods[CONFIG_NR_BOOTMOD +1 ];...
max(8, CONFIG_NR_BOOTMODS)here (albeit the 8 may have room for lowering - I don't recall why 8 was chosen rather than going with the minimum possible value covering all module kinds known at that time).
This is what drove the default for x86 in Kconfig to be 8. I thought it was excessive but assumed there was some reason for the value. And see my comment above whether it should be max({n},CONFIG_NR_BOOTMOD) vs range {n}..32768.
Re: [PATCH v1 01/18] kconfig: allow configuration of maximum modules
Quoted 15 lines · Daniel P. Smith
Quoted 13 lines · Jan Beulich
xen/arch/x86/efi/efi-boot.h static multiboot_info_t __initdata mbi = { · line 21
Quoted 8 lines · Daniel P. Smith
xen/arch/x86/efi/efi-boot.h +1
-1@@ -18,7 +18,7 @@ static multiboot_info_t __initdata mbi = {* The array size needs to be one larger than the number of modules we * support - see __start_xen(). */ -static module_t __initdata mb_modules[5 ]; +static module_t __initdata mb_modules[CONFIG_NR_BOOTMODS +1 ];If the build admin selected 1, I'm pretty sure about nothing would work. I think you want
max(5, CONFIG_NR_BOOTMODS)ormax(4, CONFIG_NR_BOOTMODS)+ 1 here and ...
Quoted 9 lines
Actually, I reasoned this out and 1 is in fact a valid value. It would mean Xen + Dom0 Linux kernel with embedded initramfs with no externally loaded XSM policy and no boot time microcode patching. This is a working configuration, but open to debate if it is a desirable configuration. The question is whether it is desired to block someone from building such a configuration, or any number between 1 and 4. If the answer is yes, then why not just set the lower bound of the range in the
Kconfigfile instead of having to maintain a hard-coded lower bound in a max marco across multiple locations?
While I'd be fine with the lower bounds being raised, I wouldn't be very happy with seeing those lower bounds becoming arch-specific.
Jan
Re: [PATCH v1 01/18] kconfig: allow configuration of maximum modules
Quoted 26 lines · Jan Beulich
Quoted 15 lines · Daniel P. Smith
Quoted 13 lines · Jan Beulich
xen/arch/x86/efi/efi-boot.h static multiboot_info_t __initdata mbi = { · line 21
Quoted 8 lines · Daniel P. Smith
xen/arch/x86/efi/efi-boot.h +1
-1@@ -18,7 +18,7 @@ static multiboot_info_t __initdata mbi = {* The array size needs to be one larger than the number of modules we * support - see __start_xen(). */ -static module_t __initdata mb_modules[5 ]; +static module_t __initdata mb_modules[CONFIG_NR_BOOTMODS +1 ];If the build admin selected 1, I'm pretty sure about nothing would work. I think you want
max(5, CONFIG_NR_BOOTMODS)ormax(4, CONFIG_NR_BOOTMODS)+ 1 here and ...Quoted 9 lines
Actually, I reasoned this out and 1 is in fact a valid value. It would mean Xen + Dom0 Linux kernel with embedded initramfs with no externally loaded XSM policy and no boot time microcode patching. This is a working configuration, but open to debate if it is a desirable configuration. The question is whether it is desired to block someone from building such a configuration, or any number between 1 and 4. If the answer is yes, then why not just set the lower bound of the range in the
Kconfigfile instead of having to maintain a hard-coded lower bound in a max marco across multiple locations?
Quoted 2 lines
While I'd be fine with the lower bounds being raised, I wouldn't be very happy with seeing those lower bounds becoming arch-specific.
Okay, and I am not sure how changing the range in Kconfig would make it arch-specific. I was not proposing making the existing range conditioned and having arch specific instances. There is one range, it will have a lower bound of 4 and the upper bound of 31768.
v/r,
dps
PATCH 2/18 introduction of generalized boot info
In reply to Daniel P. SmithThe x86 and Arm architectures represent in memory the general boot information and boot modules differently despite having commonality. The x86 representations are bound to the multiboot v1 structures while the Arm representations are a slightly generalized meta-data container for the boot material. The multiboot structure does not lend itself well to being expanded to accommodate additional metadata, both general and boot module specific. The Arm structures are not bound to an external specification and thus are able to be expanded for solutions such as dom0less.
This commit introduces a set of structures patterned off the Arm structures to represent the boot information in a manner that captures common data. The structures provide an arch field to allow arch specific expansions to the structures. The intended goal of these new common structures is to enable commonality between the different architectures. Specifically to enable dom0less and hyperlaunch to have a common representation of boot-time constructed domains.
--- xen/arch/x86/include/asm/bootinfo.h | 48 +++++++++++++++++++++++++ xen/include/xen/bootinfo.h | 54 +++++++++++++++++++++++++++++ 2 files changed, 102 insertions(+) create mode 100644 xen/arch/x86/include/asm/bootinfo.h create mode 100644 xen/include/xen/bootinfo.h
xen/arch/x86/include/asm/bootinfo.h new file +48 -0
@@ -0,0 +1,48 @@ +#ifndef __ARCH_X86_BOOTINFO_H__ +#define __ARCH_X86_BOOTINFO_H__ + +/* unused for x86 */ +struct arch_bootstring { }; + +struct __packed arch_bootmodule { +#define BOOTMOD_FLAG_X86_RELOCATED 1U << 0 +uint32_t flags; +uint32_t headroom; +}; + +struct __packed arch_boot_info { +uint32_t flags; +#define BOOTINFO_FLAG_X86_MEMLIMITS 1U << 0 +#define BOOTINFO_FLAG_X86_BOOTDEV 1U << 1 +#define BOOTINFO_FLAG_X86_CMDLINE 1U << 2 +#define BOOTINFO_FLAG_X86_MODULES 1U << 3 +#define BOOTINFO_FLAG_X86_AOUT_SYMS 1U << 4 +#define BOOTINFO_FLAG_X86_ELF_SYMS 1U << 5 +#define BOOTINFO_FLAG_X86_MEMMAP 1U << 6 +#define BOOTINFO_FLAG_X86_DRIVES 1U << 7 +#define BOOTINFO_FLAG_X86_BIOSCONFIG 1U << 8 +#define BOOTINFO_FLAG_X86_LOADERNAME 1U << 9 +#define BOOTINFO_FLAG_X86_APM 1U << 10 + +bool xen_guest; + +char *boot_loader_name; +char *kextra; + +uint32_t mem_lower; +uint32_t mem_upper; + +uint32_t mmap_length; +paddr_t mmap_addr; +}; + +struct __packed mb_memmap { +uint32_t size; +uint32_t base_addr_low; +uint32_t base_addr_high; +uint32_t length_low; +uint32_t length_high; +uint32_t type; +}; + +#endif
xen/include/xen/bootinfo.h new file +54 -0
54 lines not on this page · raw message
Signature · 1 line
2.20.1
Re: [PATCH v1 02/18] introduction of generalized boot info
Hi Daniel,
Quoted 70 lines · Daniel P. Smith
The x86 and Arm architectures represent in memory the general boot information and boot modules differently despite having commonality. The x86 representations are bound to the multiboot v1 structures while the Arm representations are a slightly generalized meta-data container for the boot material. The multiboot structure does not lend itself well to being expanded to accommodate additional metadata, both general and boot module specific. The Arm structures are not bound to an external specification and thus are able to be expanded for solutions such as dom0less.
This commit introduces a set of structures patterned off the Arm structures to represent the boot information in a manner that captures common data. The structures provide an arch field to allow arch specific expansions to the structures. The intended goal of these new common structures is to enable commonality between the different architectures. Specifically to enable dom0less and hyperlaunch to have a common representation of boot-time constructed domains.
Signed-off-by Daniel P. Smith <[email protected]>Reviewed-by Christopher Clark <[email protected]>--- xen/arch/x86/include/asm/bootinfo.h | 48 +++++++++++++++++++++++++ xen/include/xen/bootinfo.h | 54 +++++++++++++++++++++++++++++ 2 files changed, 102 insertions(+)create mode 100644
xen/arch/x86/include/asm/bootinfo.hcreate mode 100644xen/include/xen/bootinfo.hxen/arch/x86/include/asm/bootinfo.h new file +38
-0@@ -0,0 +1,48 @@ +#ifndef __ARCH_X86_BOOTINFO_H__ +#define __ARCH_X86_BOOTINFO_H__ + +/* unused for x86 */ +struct arch_bootstring { }; + +struct __packed arch_bootmodule { +#define BOOTMOD_FLAG_X86_RELOCATED 1U << 0 +uint32_t flags; +uint32_t headroom; +}; + +struct __packed arch_boot_info { +uint32_t flags; +#define BOOTINFO_FLAG_X86_MEMLIMITS 1U << 0 +#define BOOTINFO_FLAG_X86_BOOTDEV 1U << 1 +#define BOOTINFO_FLAG_X86_CMDLINE 1U << 2 +#define BOOTINFO_FLAG_X86_MODULES 1U << 3 +#define BOOTINFO_FLAG_X86_AOUT_SYMS 1U << 4 +#define BOOTINFO_FLAG_X86_ELF_SYMS 1U << 5 +#define BOOTINFO_FLAG_X86_MEMMAP 1U << 6 +#define BOOTINFO_FLAG_X86_DRIVES 1U << 7 +#define BOOTINFO_FLAG_X86_BIOSCONFIG 1U << 8 +#define BOOTINFO_FLAG_X86_LOADERNAME 1U << 9 +#define BOOTINFO_FLAG_X86_APM 1U << 10 + +bool xen_guest; + +char *boot_loader_name; +char *kextra; + +uint32_t mem_lower; +uint32_t mem_upper; + +uint32_t mmap_length; +paddr_t mmap_addr; +}; +
xen/arch/x86/include/asm/bootinfo.h line 48
Quoted 10 lines
xen/arch/x86/include/asm/bootinfo.h new file +10
-0+struct __packed mb_memmap { +uint32_t size; +uint32_t base_addr_low; +uint32_t base_addr_high; +uint32_t length_low; +uint32_t length_high; +uint32_t type; +}; + +#endif
NIT: Missing emacs magics.
Quoted 23 lines
xen/include/xen/bootinfo.h new file +17
-0@@ -0,0 +1,54 @@ +#ifndef __XEN_BOOTINFO_H__ +#define __XEN_BOOTINFO_H__ + +#include <xen/mm.h> +#include <xen/types.h> + +#include <asm/bootinfo.h> + +typedef enum { + BOOTMOD_UNKNOWN, + BOOTMOD_XEN, + BOOTMOD_FDT, + BOOTMOD_KERNEL, + BOOTMOD_RAMDISK, + BOOTMOD_XSM, + BOOTMOD_UCODE, + BOOTMOD_GUEST_DTB,
xen/include/xen/bootinfo.h line 27
Quoted 10 lines
xen/include/xen/bootinfo.h new file +10
-0+} bootmodule_kind; + +typedef enum { +BOOTSTR_EMPTY , +BOOTSTR_STRING , +BOOTSTR_CMDLINE , +}bootstring_kind ; + +#define BOOTMOD_MAX_STRING 1024 +struct __packed boot_string {
As you use __packed, the fields...
xen/include/xen/bootinfo.h line 29
Quoted 2 lines
xen/include/xen/bootinfo.h new file +2
-0+bootstring_kind kind; +struct arch_bootstring *arch;
... may not be naturally aligned anymore. Here it will depend on the size of bootstring_kind (this is an enum and it don't think C guarantees the size). This...
Quoted 10 lines
xen/include/xen/bootinfo.h new file +10
-0+ +char bytes[BOOTMOD_MAX_STRING ]; +size_t len; +}; + +struct __packed boot_module { +bootmodule_kind kind; +paddr_t start; +mfn_t mfn; +size_t size;
xen/include/xen/bootinfo.h line 49
Quoted 10 lines
xen/include/xen/bootinfo.h new file +10
-0+ +struct arch_bootmodule *arch; +struct boot_string string; +}; + +struct __packed boot_info { +char *cmdline; + +uint32_t nr_mods; +struct boot_module *mods;
... more obvious on this one because on 64-bit arch, there will be no 32-bit padding. So 'mods' will be 32-bit aligned even if the value 64-bit.
This is going to be a problem on any architecture that forbid unaligned access (or let the software decide).
In this case, I don't think any structures you defined warrant to be __packed.
xen/include/xen/bootinfo.h line 54
Quoted 5 lines
xen/include/xen/bootinfo.h new file +5
-0+ + struct arch_boot_info *arch; +}; + +#endif
NIT: Missing emacs magics.
Signature · 1 line
Julien Grall
Re: [PATCH v1 02/18] introduction of generalized boot info
Quoted 84 lines · Julien Grall
Hi Daniel,
Quoted 81 lines · Daniel P. Smith
The x86 and Arm architectures represent in memory the general boot information
and boot modules differently despite having commonality. The x86 representations are bound to the multiboot v1 structures while the Arm representations are a slightly generalized meta-data container for the boot
material. The multiboot structure does not lend itself well to being expanded
to accommodate additional metadata, both general and boot module specific. The
Arm structures are not bound to an external specification and thus are able to
be expanded for solutions such as dom0less.This commit introduces a set of structures patterned off the Arm structures to
represent the boot information in a manner that captures common data. The structures provide an arch field to allow arch specific expansions to the structures. The intended goal of these new common structures is to enable commonality between the different architectures. Specifically to enable dom0less and hyperlaunch to have a common representation of boot-time constructed domains.Signed-off-by Daniel P. Smith <[email protected]>Reviewed-by Christopher Clark <[email protected]>---
xen/arch/x86/include/asm/bootinfo.h| 48 +++++++++++++++++++++++++ xen/include/xen/bootinfo.h | 54 +++++++++++++++++++++++++++++2 files changed, 102 insertions(+)create mode 100644
xen/arch/x86/include/asm/bootinfo.hcreate mode 100644xen/include/xen/bootinfo.hxen/arch/x86/include/asm/bootinfo.h new file +42
-0@@ -0,0 +1,48 @@ +#ifndef __ARCH_X86_BOOTINFO_H__ +#define __ARCH_X86_BOOTINFO_H__ + +/* unused for x86 */ +struct arch_bootstring { }; + +struct __packed arch_bootmodule { +#define BOOTMOD_FLAG_X86_RELOCATED 1U << 0 + uint32_t flags; + uint32_t headroom; +}; + +struct __packed arch_boot_info { + uint32_t flags; +#define BOOTINFO_FLAG_X86_MEMLIMITS 1U << 0 +#define BOOTINFO_FLAG_X86_BOOTDEV 1U << 1 +#define BOOTINFO_FLAG_X86_CMDLINE 1U << 2 +#define BOOTINFO_FLAG_X86_MODULES 1U << 3 +#define BOOTINFO_FLAG_X86_AOUT_SYMS 1U << 4 +#define BOOTINFO_FLAG_X86_ELF_SYMS 1U << 5 +#define BOOTINFO_FLAG_X86_MEMMAP 1U << 6 +#define BOOTINFO_FLAG_X86_DRIVES 1U << 7 +#define BOOTINFO_FLAG_X86_BIOSCONFIG 1U << 8 +#define BOOTINFO_FLAG_X86_LOADERNAME 1U << 9 +#define BOOTINFO_FLAG_X86_APM 1U << 10 + + bool xen_guest; + + char *boot_loader_name; + char *kextra; + + uint32_t mem_lower; + uint32_t mem_upper; + + uint32_t mmap_length; + paddr_t mmap_addr; +}; + +struct __packed mb_memmap { + uint32_t size; + uint32_t base_addr_low; + uint32_t base_addr_high;
Quoted 8 lines
xen/arch/x86/include/asm/bootinfo.h line 48
Quoted 6 lines
xen/arch/x86/include/asm/bootinfo.h new file +6
-0+ uint32_t length_low; + uint32_t length_high; + uint32_t type; +}; + +#endif NIT: Missing emacs magics.
As a devoted vim user, begrudged ack. ( ^_^)
Quoted 36 lines
Quoted 23 lines
xen/include/xen/bootinfo.h new file +17
-0@@ -0,0 +1,54 @@ +#ifndef __XEN_BOOTINFO_H__ +#define __XEN_BOOTINFO_H__ + +#include <xen/mm.h> +#include <xen/types.h> + +#include <asm/bootinfo.h> + +typedef enum { + BOOTMOD_UNKNOWN, + BOOTMOD_XEN, + BOOTMOD_FDT, + BOOTMOD_KERNEL, + BOOTMOD_RAMDISK, + BOOTMOD_XSM, + BOOTMOD_UCODE, + BOOTMOD_GUEST_DTB,xen/include/xen/bootinfo.h line 27
Quoted 10 lines
xen/include/xen/bootinfo.h new file +10
-0+} bootmodule_kind; + +typedef enum { + BOOTSTR_EMPTY, + BOOTSTR_STRING, + BOOTSTR_CMDLINE, +}bootstring_kind ; + +#define BOOTMOD_MAX_STRING 1024 +struct __packed boot_string {As you use
__packed, the fields...
Quoted 6 lines
xen/include/xen/bootinfo.h line 29
Quoted 2 lines
xen/include/xen/bootinfo.h new file +2
-0+ bootstring_kind kind; + struct arch_bootstring *arch;... may not be naturally aligned anymore. Here it will depend on the size of
bootstring_kind(this is an enum and it don't think C guarantees the size). This...
Ack.
Quoted 24 lines
Quoted 10 lines
xen/include/xen/bootinfo.h new file +10
-0+ + char bytes[BOOTMOD_MAX_STRING]; + size_t len; +}; + +struct __packed boot_module { + bootmodule_kind kind; + paddr_t start; + mfn_t mfn; + size_t size;xen/include/xen/bootinfo.h line 49
Quoted 10 lines
xen/include/xen/bootinfo.h new file +10
-0+ + struct arch_bootmodule *arch; + struct boot_string string; +}; + +struct __packed boot_info { + char *cmdline; + + uint32_t nr_mods; + struct boot_module *mods;... more obvious on this one because on 64-bit arch, there will be no 32-bit padding. So 'mods' will be 32-bit aligned even if the value 64-bit.
Quoted 5 lines
This is going to be a problem on any architecture that forbid unaligned access (or let the software decide).
In this case, I don't think any structures you defined warrant to be
__packed.
Ack, I was too focused on 32bit alignment for x86 bootstrap entry point when I was laying out the structure, that was short-sighted on my part. I will go back and rework to be 64bit aligned.
Quoted 8 lines
xen/include/xen/bootinfo.h line 54
Quoted 5 lines
xen/include/xen/bootinfo.h new file +5
-0+ + struct arch_boot_info *arch; +}; + +#endif NIT: Missing emacs magics.
Ack.
Re: [PATCH v1 02/18] introduction of generalized boot info
In reply to Daniel P. Smithxen/arch/x86/include/asm/bootinfo.h line 8
Quoted 11 lines · Daniel P. Smith
xen/arch/x86/include/asm/bootinfo.h new file +8
-0@@ -0,0 +1,48 @@ +#ifndef __ARCH_X86_BOOTINFO_H__ +#define __ARCH_X86_BOOTINFO_H__ + +/* unused for x86 */ +struct arch_bootstring { }; + +struct __packed arch_bootmodule { +#define BOOTMOD_FLAG_X86_RELOCATED 1U << 0
Such macro expansions need parenthesizing.
xen/arch/x86/include/asm/bootinfo.h line 11
Quoted 3 lines
xen/arch/x86/include/asm/bootinfo.h new file +3
-0+uint32_t flags; +uint32_t headroom; +};
Since you're not following any external spec, on top of what Julien said about the __packed attribute I'd also like to point out that in many cases here there's no need to use fixed-width types.
Quoted 5 lines
xen/arch/x86/include/asm/bootinfo.h new file +5
-0+struct __packed arch_boot_info { +uint32_t flags; +#define BOOTINFO_FLAG_X86_MEMLIMITS 1U << 0 +#define BOOTINFO_FLAG_X86_BOOTDEV 1U << 1 +#define BOOTINFO_FLAG_X86_CMDLINE 1U << 2
xen/arch/x86/include/asm/bootinfo.h line 27
Quoted 10 lines
xen/arch/x86/include/asm/bootinfo.h new file +10
-0+#define BOOTINFO_FLAG_X86_MODULES 1U << 3 +#define BOOTINFO_FLAG_X86_AOUT_SYMS 1U << 4 +#define BOOTINFO_FLAG_X86_ELF_SYMS 1U << 5 +#define BOOTINFO_FLAG_X86_MEMMAP 1U << 6 +#define BOOTINFO_FLAG_X86_DRIVES 1U << 7 +#define BOOTINFO_FLAG_X86_BIOSCONFIG 1U << 8 +#define BOOTINFO_FLAG_X86_LOADERNAME 1U << 9 +#define BOOTINFO_FLAG_X86_APM 1U << 10 + +bool xen_guest;
As the example of this, with just the header files being introduced here it is not really possible to figure what these fields are to be used for and hence whether they're legitimately represented here.
xen/arch/x86/include/asm/bootinfo.h line 30
Quoted 2 lines
xen/arch/x86/include/asm/bootinfo.h new file +2
-0+char *boot_loader_name; +char *kextra;
const?
Jan
Re: [PATCH v1 02/18] introduction of generalized boot info
Quoted 14 lines · Jan Beulich
xen/arch/x86/include/asm/bootinfo.h line 8
Quoted 11 lines · Daniel P. Smith
xen/arch/x86/include/asm/bootinfo.h new file +8
-0@@ -0,0 +1,48 @@ +#ifndef __ARCH_X86_BOOTINFO_H__ +#define __ARCH_X86_BOOTINFO_H__ + +/* unused for x86 */ +struct arch_bootstring { }; + +struct __packed arch_bootmodule { +#define BOOTMOD_FLAG_X86_RELOCATED 1U << 0Such macro expansions need parenthesizing.
Ack.
Quoted 7 lines
xen/arch/x86/include/asm/bootinfo.h line 11
Quoted 3 lines
xen/arch/x86/include/asm/bootinfo.h new file +3
-0+uint32_t flags; +uint32_t headroom; +};Since you're not following any external spec, on top of what Julien said about the
__packedattribute I'd also like to point out that in many cases here there's no need to use fixed-width types.
Oh, I forgot to mention that in the reply to Julien. Yes, the __packed is needed to correctly cross the 32bit to 64bit bridge from the x86 bootstrap in patch 4.
Quoted 11 lines
Quoted 11 lines
xen/arch/x86/include/asm/bootinfo.h new file +11
-0+struct __packed arch_boot_info { +uint32_t flags; +#define BOOTINFO_FLAG_X86_MEMLIMITS 1U << 0 +#define BOOTINFO_FLAG_X86_BOOTDEV 1U << 1 +#define BOOTINFO_FLAG_X86_CMDLINE 1U << 2 +#define BOOTINFO_FLAG_X86_MODULES 1U << 3 +#define BOOTINFO_FLAG_X86_AOUT_SYMS 1U << 4 +#define BOOTINFO_FLAG_X86_ELF_SYMS 1U << 5 +#define BOOTINFO_FLAG_X86_MEMMAP 1U << 6 +#define BOOTINFO_FLAG_X86_DRIVES 1U << 7 +#define BOOTINFO_FLAG_X86_BIOSCONFIG 1U << 8
Quoted 8 lines
xen/arch/x86/include/asm/bootinfo.h line 27
Quoted 4 lines
xen/arch/x86/include/asm/bootinfo.h new file +4
-0+#define BOOTINFO_FLAG_X86_LOADERNAME 1U << 9 +#define BOOTINFO_FLAG_X86_APM 1U << 10 + +bool xen_guest;As the example of this, with just the header files being introduced here it is not really possible to figure what these fields are to be used for and hence whether they're legitimately represented here.
I can add a comment to clarify these are a mirror of the multiboot flags. These were mirrored to allow the multiboot flags to be direct copied and eased the replacement locations where an mb flag is checked.
Quoted 4 lines
xen/arch/x86/include/asm/bootinfo.h line 30
Quoted 2 lines
xen/arch/x86/include/asm/bootinfo.h new file +2
-0+char *boot_loader_name; +char *kextra;const?
I want to say const will not work based on usage, but I will double-check.
Re: [PATCH v1 02/18] introduction of generalized boot info
Quoted 22 lines · Daniel P. Smith
Quoted 14 lines · Jan Beulich
xen/arch/x86/include/asm/bootinfo.h line 8
Quoted 11 lines · Daniel P. Smith
xen/arch/x86/include/asm/bootinfo.h new file +8
-0@@ -0,0 +1,48 @@ +#ifndef __ARCH_X86_BOOTINFO_H__ +#define __ARCH_X86_BOOTINFO_H__ + +/* unused for x86 */ +struct arch_bootstring { }; + +struct __packed arch_bootmodule { +#define BOOTMOD_FLAG_X86_RELOCATED 1U << 0Such macro expansions need parenthesizing.
Ack.
Quoted 4 lines
Quoted 3 lines
xen/arch/x86/include/asm/bootinfo.h new file +3
-0+uint32_t flags; +uint32_t headroom; +};
Quoted 7 lines
Quoted 3 lines
Since you're not following any external spec, on top of what Julien said about the
__packedattribute I'd also like to point out that in many cases here there's no need to use fixed-width types.Oh, I forgot to mention that in the reply to Julien. Yes, the
__packedis needed to correctly cross the 32bit to 64bit bridge from the x86 bootstrap in patch 4.
I'm afraid I don't follow you here. I did briefly look at patch 4 (but that really also falls in the "wants to be split" category), but I can't see why a purely internally used struct may need packing. I'd appreciate if you could expand on that.
Quoted 20 lines
Quoted 11 lines
Quoted 11 lines
xen/arch/x86/include/asm/bootinfo.h new file +11
-0+struct __packed arch_boot_info { +uint32_t flags; +#define BOOTINFO_FLAG_X86_MEMLIMITS 1U << 0 +#define BOOTINFO_FLAG_X86_BOOTDEV 1U << 1 +#define BOOTINFO_FLAG_X86_CMDLINE 1U << 2 +#define BOOTINFO_FLAG_X86_MODULES 1U << 3 +#define BOOTINFO_FLAG_X86_AOUT_SYMS 1U << 4 +#define BOOTINFO_FLAG_X86_ELF_SYMS 1U << 5 +#define BOOTINFO_FLAG_X86_MEMMAP 1U << 6 +#define BOOTINFO_FLAG_X86_DRIVES 1U << 7 +#define BOOTINFO_FLAG_X86_BIOSCONFIG 1U << 8Quoted 8 lines
xen/arch/x86/include/asm/bootinfo.h line 27
Quoted 4 lines
xen/arch/x86/include/asm/bootinfo.h new file +4
-0+#define BOOTINFO_FLAG_X86_LOADERNAME 1U << 9 +#define BOOTINFO_FLAG_X86_APM 1U << 10 + +bool xen_guest;As the example of this, with just the header files being introduced here it is not really possible to figure what these fields are to be used for and hence whether they're legitimately represented here.
Quoted 3 lines
I can add a comment to clarify these are a mirror of the multiboot flags. These were mirrored to allow the multiboot flags to be direct copied and eased the replacement locations where an mb flag is checked.
Multiboot flags? The context here is the "xen_guest" field.
Jan
Re: [PATCH v1 02/18] introduction of generalized boot info
Quoted 27 lines · Jan Beulich
Quoted 26 lines · Daniel P. Smith
Quoted 14 lines · Jan Beulich
xen/arch/x86/include/asm/bootinfo.h line 8
Quoted 11 lines · Daniel P. Smith
xen/arch/x86/include/asm/bootinfo.h new file +8
-0@@ -0,0 +1,48 @@ +#ifndef __ARCH_X86_BOOTINFO_H__ +#define __ARCH_X86_BOOTINFO_H__ + +/* unused for x86 */ +struct arch_bootstring { }; + +struct __packed arch_bootmodule { +#define BOOTMOD_FLAG_X86_RELOCATED 1U << 0Such macro expansions need parenthesizing.
Ack.
Quoted 7 lines
xen/arch/x86/include/asm/bootinfo.h line 11
Quoted 3 lines
xen/arch/x86/include/asm/bootinfo.h new file +3
-0+uint32_t flags; +uint32_t headroom; +};Since you're not following any external spec, on top of what Julien said about the
__packedattribute I'd also like to point out that in many cases here there's no need to use fixed-width types.
Quoted 8 lines
Quoted 3 lines
Oh, I forgot to mention that in the reply to Julien. Yes, the
__packedis needed to correctly cross the 32bit to 64bit bridge from the x86 bootstrap in patch 4.I'm afraid I don't follow you here. I did briefly look at patch 4 (but that really also falls in the "wants to be split" category), but I can't see why a purely internally used struct may need packing. I'd appreciate if you could expand on that.
Originally, patch 3 and patch 4 were a single patch, and obviously was way too large. To split them, I realized I could introduce a temporary conversion function that would allow the patch to be split into a post start_xen() patch (patch 3) and a pre start_xen() patch, (patch 4). For x86, pre start_xen() consists of 3 different entry points. These being the classic/traditional/old multiboot1/2 entry, EFI entry, and PVH entry (aka Xen Guest). The latter two are all internal, 64bit, but the former is located in arch/x86/boot and is compiled as 32bit. I tried different approaches to support using a single header between these two environments. Ultimately, IMHO, the cleanest approach is what is introduced in patch 4 as it enabled the use of Xen types in the structures and maintain a single structure that need to be passed around. To do this, a 32bit specific version of the structures were defined in arch/x86/boot/boot_info32.h that is populated under 32bit mode, then they can be fixed up after getting into start_xen() and in 64bit code. To ensure no unexpected insertion of padding, I focused on ensuring everything was 32bit aligned and packed. As Julien pointed out, I messed up with the use of enum as its size is not guaranteed as the enum list grows and I forgot to consider keeping pointers 64bit aligned.
Does that help?
Quoted 20 lines
Quoted 20 lines
Quoted 19 lines
Quoted 5 lines
xen/arch/x86/include/asm/bootinfo.h new file +5
-0+struct __packed arch_boot_info { +uint32_t flags; +#define BOOTINFO_FLAG_X86_MEMLIMITS 1U << 0 +#define BOOTINFO_FLAG_X86_BOOTDEV 1U << 1 +#define BOOTINFO_FLAG_X86_CMDLINE 1U << 2xen/arch/x86/include/asm/bootinfo.h line 27
Quoted 10 lines
xen/arch/x86/include/asm/bootinfo.h new file +10
-0+#define BOOTINFO_FLAG_X86_MODULES 1U << 3 +#define BOOTINFO_FLAG_X86_AOUT_SYMS 1U << 4 +#define BOOTINFO_FLAG_X86_ELF_SYMS 1U << 5 +#define BOOTINFO_FLAG_X86_MEMMAP 1U << 6 +#define BOOTINFO_FLAG_X86_DRIVES 1U << 7 +#define BOOTINFO_FLAG_X86_BIOSCONFIG 1U << 8 +#define BOOTINFO_FLAG_X86_LOADERNAME 1U << 9 +#define BOOTINFO_FLAG_X86_APM 1U << 10 + +bool xen_guest;As the example of this, with just the header files being introduced here it is not really possible to figure what these fields are to be used for and hence whether they're legitimately represented here.
Quoted 5 lines
Quoted 3 lines
I can add a comment to clarify these are a mirror of the multiboot flags. These were mirrored to allow the multiboot flags to be direct copied and eased the replacement locations where an mb flag is checked.
Multiboot flags? The context here is the "
xen_guest" field.
Apologies, I thought you were referring to all the fields and I forgot to explain xen_guest. So to clarify, flags is to carry the MB flags passed up from the MB entry point and xen_guest is meant to carry the xen_guest bool passed up from the PVH/Xen Guest entry point.
v/r,
dps
Re: [PATCH v1 02/18] introduction of generalized boot info
Quoted 57 lines · Daniel P. Smith
Quoted 27 lines · Jan Beulich
Quoted 26 lines · Daniel P. Smith
Quoted 14 lines · Jan Beulich
xen/arch/x86/include/asm/bootinfo.h line 8
Quoted 11 lines · Daniel P. Smith
xen/arch/x86/include/asm/bootinfo.h new file +8
-0@@ -0,0 +1,48 @@ +#ifndef __ARCH_X86_BOOTINFO_H__ +#define __ARCH_X86_BOOTINFO_H__ + +/* unused for x86 */ +struct arch_bootstring { }; + +struct __packed arch_bootmodule { +#define BOOTMOD_FLAG_X86_RELOCATED 1U << 0Such macro expansions need parenthesizing.
Ack.
Quoted 7 lines
xen/arch/x86/include/asm/bootinfo.h line 11
Quoted 3 lines
xen/arch/x86/include/asm/bootinfo.h new file +3
-0+uint32_t flags; +uint32_t headroom; +};Since you're not following any external spec, on top of what Julien said about the
__packedattribute I'd also like to point out that in many cases here there's no need to use fixed-width types.Quoted 8 lines
Quoted 3 lines
Oh, I forgot to mention that in the reply to Julien. Yes, the
__packedis needed to correctly cross the 32bit to 64bit bridge from the x86 bootstrap in patch 4.I'm afraid I don't follow you here. I did briefly look at patch 4 (but that really also falls in the "wants to be split" category), but I can't see why a purely internally used struct may need packing. I'd appreciate if you could expand on that.
Originally, patch 3 and patch 4 were a single patch, and obviously was way too large. To split them, I realized I could introduce a temporary conversion function that would allow the patch to be split into a post
start_xen()patch (patch 3) and a prestart_xen()patch, (patch 4). For x86, prestart_xen()consists of 3 different entry points. These being the classic/traditional/old multiboot1/2 entry, EFI entry, and PVH entry (aka Xen Guest). The latter two are all internal, 64bit, but the former is located inarch/x86/bootand is compiled as 32bit. I tried different approaches to support using a single header between these two environments. Ultimately, IMHO, the cleanest approach is what is introduced in patch 4 as it enabled the use of Xen types in the structures and maintain a single structure that need to be passed around. To do this, a 32bit specific version of the structures were defined inarch/x86/boot/boot_info32.hthat is populated under 32bit mode, then they can be fixed up after getting intostart_xen()and in 64bit code. To ensure no unexpected insertion of padding, I focused on ensuring everything was 32bit aligned and packed. As Julien pointed out, I messed up with the use of enum as its size is not guaranteed as the enum list grows and I forgot to consider keeping pointers 64bit aligned.
Quoted 1 line
Does that help?
It helps as background info, yes, but I continue to be unhappy with the new uses of the __packed attribute.
Quoted 24 lines
Quoted 24 lines
Quoted 23 lines
Quoted 11 lines
Quoted 11 lines
xen/arch/x86/include/asm/bootinfo.h new file +11
-0+struct __packed arch_boot_info { +uint32_t flags; +#define BOOTINFO_FLAG_X86_MEMLIMITS 1U << 0 +#define BOOTINFO_FLAG_X86_BOOTDEV 1U << 1 +#define BOOTINFO_FLAG_X86_CMDLINE 1U << 2 +#define BOOTINFO_FLAG_X86_MODULES 1U << 3 +#define BOOTINFO_FLAG_X86_AOUT_SYMS 1U << 4 +#define BOOTINFO_FLAG_X86_ELF_SYMS 1U << 5 +#define BOOTINFO_FLAG_X86_MEMMAP 1U << 6 +#define BOOTINFO_FLAG_X86_DRIVES 1U << 7 +#define BOOTINFO_FLAG_X86_BIOSCONFIG 1U << 8Quoted 8 lines
xen/arch/x86/include/asm/bootinfo.h line 27
Quoted 4 lines
xen/arch/x86/include/asm/bootinfo.h new file +4
-0+#define BOOTINFO_FLAG_X86_LOADERNAME 1U << 9 +#define BOOTINFO_FLAG_X86_APM 1U << 10 + +bool xen_guest;As the example of this, with just the header files being introduced here it is not really possible to figure what these fields are to be used for and hence whether they're legitimately represented here.
I can add a comment to clarify these are a mirror of the multiboot flags. These were mirrored to allow the multiboot flags to be direct copied and eased the replacement locations where an mb flag is checked.
Quoted 6 lines
Quoted 1 line
Multiboot flags? The context here is the "
xen_guest" field.Apologies, I thought you were referring to all the fields and I forgot to explain
xen_guest. So to clarify, flags is to carry the MB flags passed up from the MB entry point andxen_guestis meant to carry thexen_guestbool passed up from the PVH/Xen Guest entry point.
That was my guess, but then my request stands: The fields should be added to the struct at the time they're being made use of.
Jan
Re: [PATCH v1 02/18] introduction of generalized boot info
In reply to Daniel P. SmithQuoted 22 lines · Daniel P. Smith
Quoted 14 lines · Jan Beulich
xen/arch/x86/include/asm/bootinfo.h line 8
Quoted 11 lines · Daniel P. Smith
xen/arch/x86/include/asm/bootinfo.h new file +8
-0@@ -0,0 +1,48 @@ +#ifndef __ARCH_X86_BOOTINFO_H__ +#define __ARCH_X86_BOOTINFO_H__ + +/* unused for x86 */ +struct arch_bootstring { }; + +struct __packed arch_bootmodule { +#define BOOTMOD_FLAG_X86_RELOCATED 1U << 0Such macro expansions need parenthesizing.
Ack.
Quoted 4 lines
Quoted 3 lines
xen/arch/x86/include/asm/bootinfo.h new file +3
-0+uint32_t flags; +uint32_t headroom; +};
Quoted 7 lines
Quoted 3 lines
Since you're not following any external spec, on top of what Julien said about the
__packedattribute I'd also like to point out that in many cases here there's no need to use fixed-width types.Oh, I forgot to mention that in the reply to Julien. Yes, the
__packedis needed to correctly cross the 32bit to 64bit bridge from the x86 bootstrap in patch 4.
I'm afraid I don't follow you here. I did briefly look at patch 4 (but that really also falls in the "wants to be split" category), but I can't see why a purely internally used struct may need packing. I'd appreciate if you could expand on that.
Quoted 20 lines
Quoted 11 lines
Quoted 11 lines
xen/arch/x86/include/asm/bootinfo.h new file +11
-0+struct __packed arch_boot_info { +uint32_t flags; +#define BOOTINFO_FLAG_X86_MEMLIMITS 1U << 0 +#define BOOTINFO_FLAG_X86_BOOTDEV 1U << 1 +#define BOOTINFO_FLAG_X86_CMDLINE 1U << 2 +#define BOOTINFO_FLAG_X86_MODULES 1U << 3 +#define BOOTINFO_FLAG_X86_AOUT_SYMS 1U << 4 +#define BOOTINFO_FLAG_X86_ELF_SYMS 1U << 5 +#define BOOTINFO_FLAG_X86_MEMMAP 1U << 6 +#define BOOTINFO_FLAG_X86_DRIVES 1U << 7 +#define BOOTINFO_FLAG_X86_BIOSCONFIG 1U << 8Quoted 8 lines
xen/arch/x86/include/asm/bootinfo.h line 27
Quoted 4 lines
xen/arch/x86/include/asm/bootinfo.h new file +4
-0+#define BOOTINFO_FLAG_X86_LOADERNAME 1U << 9 +#define BOOTINFO_FLAG_X86_APM 1U << 10 + +bool xen_guest;As the example of this, with just the header files being introduced here it is not really possible to figure what these fields are to be used for and hence whether they're legitimately represented here.
Quoted 3 lines
I can add a comment to clarify these are a mirror of the multiboot flags. These were mirrored to allow the multiboot flags to be direct copied and eased the replacement locations where an mb flag is checked.
Multiboot flags? The context here is the "xen_guest" field.
Jan
PATCH 3/18 x86: adopt new boot info structures
In reply to Daniel P. SmithThis commit replaces the use of the multiboot v1 structures starting at __start_xen(). The majority of this commit is converting the fields being accessed for the startup calculations. While adapting the ucode boot module location logic, this code was refactored to reduce some of the unnecessary complexity.
--- xen/arch/x86/bzimage.c | 18 +- xen/arch/x86/cpu/microcode/core.c | 133 ++++++++------ xen/arch/x86/dom0_build.c | 11 +- xen/arch/x86/hvm/dom0_build.c | 42 ++--- xen/arch/x86/include/asm/bootinfo.h | 2 +- xen/arch/x86/include/asm/bzimage.h | 5 +- xen/arch/x86/include/asm/dom0_build.h | 15 +- xen/arch/x86/include/asm/setup.h | 14 +- xen/arch/x86/pv/dom0_build.c | 34 ++-- xen/arch/x86/setup.c | 245 +++++++++++++++----------- xen/include/xen/bootinfo.h | 47 +++++ xen/include/xsm/xsm.h | 26 ++- xen/xsm/xsm_core.c | 43 +++-- xen/xsm/xsm_policy.c | 56 +++--- 14 files changed, 413 insertions(+), 278 deletions(-)
xen/arch/x86/bzimage.c +9 -9
@@ -69,10 +69,8 @@ static __init int bzimage_check(struct setup_header *hdr, unsigned long len)return 1 ; } -static unsigned long __initdata orig_image_len; - -unsigned long __init bzimage_headroom (void *image_start, -unsigned long image_length) +unsigned long __init bzimage_headroom ( +void *image_start,unsigned long image_length) {struct setup_header *hdr = (struct setup_header *)image_start;int err;
@@ -91,7 +89,6 @@ unsigned long __init bzimage_headroom(void *image_start,if (elf_is_elfbinary (image_start, image_length) )return 0 ; - orig_image_len = image_length; headroom =output_length (image_start, image_length);if (gzip_check (image_start, image_length)) {
@@ -104,12 +101,15 @@ unsigned long __init bzimage_headroom(void *image_start,return headroom; } -int __init bzimage_parse (void *image_base,void **image_start, -unsigned long *image_len) +int __init bzimage_parse ( +void *image_base,void **image_start,unsigned int headroom, +unsigned long *image_len) {struct setup_header *hdr = (struct setup_header *)(*image_start);int err =bzimage_check (hdr, *image_len); -unsigned long output_len; +unsigned long output_len, orig_image_len; + + orig_image_len = *image_len - headroom;if ( err <0 )return err;
@@ -125,7 +125,7 @@ int __init bzimage_parse(void *image_base, void **image_start,BUG_ON (!(image_base < *image_start)); - output_len =output_length (*image_start, orig_image_len); + output_len =output_length (*image_start, *image_len);if ( (err =perform_gunzip (image_base, *image_start, orig_image_len)) >0 ) err =decompress (*image_start, orig_image_len, image_base);
xen/arch/x86/cpu/microcode/core.c +81 -52
197 lines not on this page · raw message
xen/arch/x86/dom0_build.c +6 -5
@@ -4,6 +4,7 @@* Copyright (c) 2002-2005, K A Fraser */ +#include <xen/bootinfo.h> #include <xen/init.h> #include <xen/iocap.h> #include <xen/libelf.h>
@@ -574,9 +575,9 @@ int __init dom0_setup_permissions(struct domain *d)return rc; } -int __init construct_dom0 (struct domain *d,const module_t *image, -unsigned long image_headroom,module_t *initrd, -char *cmdline) +int __init construct_dom0 ( +struct domain *d,const struct boot_module *image, +struct boot_module *initrd,char *cmdline) {int rc;
@@ -588,9 +589,9 @@ int __init construct_dom0(struct domain *d, const module_t *image,process_pending_softirqs ();if (is_hvm_domain (d) ) - rc =dom0_construct_pvh (d, image, image_headroom, initrd, cmdline); + rc =dom0_construct_pvh (d, image, initrd, cmdline);else if (is_pv_domain (d) ) - rc =dom0_construct_pv (d, image, image_headroom, initrd, cmdline); + rc =dom0_construct_pv (d, image, initrd, cmdline);else panic ("Cannot construct Dom0. No guest interface available\n" );
xen/arch/x86/hvm/dom0_build.c +21 -21
94 lines not on this page · raw message
xen/arch/x86/include/asm/bootinfo.h +1 -1
@@ -24,7 +24,7 @@ struct __packed arch_boot_info {#define BOOTINFO_FLAG_X86_LOADERNAME 1U << 9#define BOOTINFO_FLAG_X86_APM 1U << 10 -bool xen_guest; +bool xenguest;char *boot_loader_name;char *kextra;
xen/arch/x86/include/asm/bzimage.h +3 -2
@@ -5,7 +5,8 @@unsigned long bzimage_headroom (void *image_start,unsigned long image_length); -int bzimage_parse (void *image_base,void **image_start, -unsigned long *image_len); +int bzimage_parse ( +void *image_base,void **image_start,unsigned int headroom, +unsigned long *image_len);#endif /* __X86_BZIMAGE_H__ */
xen/arch/x86/include/asm/dom0_build.h +7 -8
@@ -1,6 +1,7 @@#ifndef _DOM0_BUILD_H_#define _DOM0_BUILD_H_ +#include <xen/bootinfo.h> #include <xen/libelf.h> #include <xen/sched.h>
@@ -13,15 +14,13 @@ unsigned long dom0_compute_nr_pages(struct domain *d, unsigned long initrd_len);int dom0_setup_permissions (struct domain *d); -int dom0_construct_pv (struct domain *d,const module_t *image, -unsigned long image_headroom, -module_t *initrd, -char *cmdline); +int __init dom0_construct_pv ( +struct domain *d,const struct boot_module *image, +struct boot_module *initrd,char *cmdline); -int dom0_construct_pvh (struct domain *d,const module_t *image, -unsigned long image_headroom, -module_t *initrd, -char *cmdline); +int __init dom0_construct_pvh ( +struct domain *d,const struct boot_module *image, +struct boot_module *initrd,char *cmdline);unsigned long dom0_paging_pages (const struct domain *d,unsigned long nr_pages);
xen/arch/x86/include/asm/setup.h +6 -8
@@ -1,7 +1,8 @@#ifndef __X86_SETUP_H_#define __X86_SETUP_H_ -#include <xen/multiboot.h> +#include <xen/bootinfo.h> +#include <asm/numa.h> extern const char __2M_text_start[], __2M_text_end[];
@@ -33,20 +34,17 @@ static inline void vesa_init(void) {};#endif int construct_dom0 ( -struct domain *d, -const module_t *kernel,unsigned long kernel_headroom, -module_t *initrd, -char *cmdline); +struct domain *d,const struct boot_module *image, +struct boot_module *initrd,char *cmdline);void setup_io_bitmap (struct domain *d);unsigned long initial_images_nrpages (nodeid_t node);void discard_initial_images (void ); -void *bootstrap_map (const module_t *mod); +void *bootstrap_map (const struct boot_module *mod);int xen_in_range (unsigned long mfn); -void microcode_grab_module ( -unsigned long *,const multiboot_info_t *); +void microcode_grab_module (struct boot_info *bi);extern uint8_t kbd_shift_flags;
xen/arch/x86/pv/dom0_build.c +18 -16
@@ -4,6 +4,7 @@* Copyright (c) 2002-2005, K A Fraser */ +#include <xen/bootinfo.h> #include <xen/console.h> #include <xen/domain.h> #include <xen/domain_page.h>
@@ -294,11 +295,9 @@ static struct page_info * __init alloc_chunk(struct domain *d,return page; } -int __init dom0_construct_pv (struct domain *d, -const module_t *image, -unsigned long image_headroom, -module_t *initrd, -char *cmdline) +int __init dom0_construct_pv ( +struct domain *d,const struct boot_module *image, +struct boot_module *initrd,char *cmdline) {int i, rc, order, machine;bool compatible, compat;
@@ -314,9 +313,9 @@ int __init dom0_construct_pv(struct domain *d,start_info_t *si;struct vcpu *v = d->vcpu[0 ];void *image_base =bootstrap_map (image); -unsigned long image_len = image->mod_end; -void *image_start = image_base + image_headroom; -unsigned long initrd_len = initrd ? initrd->mod_end :0 ; +unsigned long image_len = image->size; +void *image_start = image_base + image->arch->headroom; +unsigned long initrd_len = initrd ? initrd->size :0 ;l4_pgentry_t *l4tab =NULL , *l4start =NULL ;l3_pgentry_t *l3tab =NULL , *l3start =NULL ;l2_pgentry_t *l2tab =NULL , *l2start =NULL ;
@@ -355,7 +354,9 @@ int __init dom0_construct_pv(struct domain *d, d->max_pages = ~0U ; -if ( (rc =bzimage_parse (image_base, &image_start, &image_len)) !=0 ) +if ( (rc = +bzimage_parse (image_base, &image_start, image->arch->headroom, + &image_len)) !=0 )return rc;if ( (rc =elf_init (&elf, image_start, image_len)) !=0 )
@@ -544,7 +545,7 @@ int __init dom0_construct_pv(struct domain *d, initrd_pfn = vinitrd_start ? (vinitrd_start - v_start) >>PAGE_SHIFT :domain_tot_pages (d); - initrd_mfn = mfn = initrd->mod_start; + initrd_mfn = mfn =mfn_x (initrd->mfn); count =PFN_UP (initrd_len);if ( d->arch.physaddr_bitsize && ((mfn + count -1 ) >> (d->arch.physaddr_bitsize -PAGE_SHIFT )) )
@@ -559,12 +560,13 @@ int __init dom0_construct_pv(struct domain *d,free_domheap_pages (page, order); page +=1UL << order; } -memcpy (page_to_virt (page),mfn_to_virt (initrd->mod_start), +memcpy (page_to_virt (page),maddr_to_virt (initrd->start), initrd_len); - mpt_alloc = (paddr_t )initrd->mod_start <<PAGE_SHIFT ; + mpt_alloc = initrd->start;init_domheap_pages (mpt_alloc, mpt_alloc +PAGE_ALIGN (initrd_len)); - initrd->mod_start = initrd_mfn =mfn_x (page_to_mfn (page)); +bootmodule_update_mfn (initrd,page_to_mfn (page)); + initrd_mfn =mfn_x (initrd->mfn); }else {
@@ -572,7 +574,7 @@ int __init dom0_construct_pv(struct domain *d,if (assign_pages (mfn_to_page (_mfn (mfn++)),1 , d,0 ) )BUG (); } - initrd->mod_end =0 ; + initrd->size =0 ; } printk("PHYSICAL MEMORY ARRANGEMENT:\n"
@@ -583,7 +585,7 @@ int __init dom0_construct_pv(struct domain *d, nr_pages - domain_tot_pages(d));if ( initrd ) { - mpt_alloc = (paddr_t )initrd->mod_start <<PAGE_SHIFT ; + mpt_alloc = initrd->start;printk ("\n Init. ramdisk: %" PRIpaddr"->%" PRIpaddr, mpt_alloc, mpt_alloc + initrd_len); }
@@ -804,7 +806,7 @@ int __init dom0_construct_pv(struct domain *d,if ( pfn >= initrd_pfn ) {if ( pfn < initrd_pfn +PFN_UP (initrd_len) ) - mfn = initrd->mod_start + (pfn - initrd_pfn); + mfn =mfn_x (initrd->mfn) + (pfn - initrd_pfn);else mfn -=PFN_UP (initrd_len); }
xen/arch/x86/setup.c +147 -98
477 lines not on this page · raw message
xen/include/xen/bootinfo.h +47 -0
51 lines not on this page · raw message
xen/include/xsm/xsm.h +12 -14
@@ -16,8 +16,8 @@#define __XSM_H__#include <xen/alternative-call.h> +#include <xen/bootinfo.h> #include <xen/sched.h> -#include <xen/multiboot.h> /* policy magic number (defined by XSM_MAGIC) */ typedef uint32_t xsm_magic_t ;
@@ -776,15 +776,14 @@ static inline int xsm_argo_send(const struct domain *d, const struct domain *t)#endif /* XSM_NO_WRAPPERS */ -#ifdef CONFIG_MULTIBOOT -int xsm_multiboot_init ( -unsigned long *module_map,const multiboot_info_t *mbi); -int xsm_multiboot_policy_init ( -unsigned long *module_map,const multiboot_info_t *mbi, -void **policy_buffer,size_t *policy_size); -#endif +#ifndef CONFIG_HAS_DEVICE_TREE +int xsm_bootmodule_init (const struct boot_info *bi); +int xsm_bootmodule_policy_init ( +const struct boot_info *bi,const unsigned char **policy_buffer, +size_t *policy_size); + +#else -#ifdef CONFIG_HAS_DEVICE_TREE /* * Initialize XSM *
@@ -826,15 +825,14 @@ static const inline struct xsm_ops *silo_init(void)#include <xsm/dummy.h> -#ifdef CONFIG_MULTIBOOT -static inline int xsm_multiboot_init ( -unsigned long *module_map,const multiboot_info_t *mbi) +#ifndef CONFIG_HAS_DEVICE_TREE +static inline int xsm_bootmodule_init (const struct boot_info *bi) {return 0 ; } -#endif -#ifdef CONFIG_HAS_DEVICE_TREE +#else +static inline int xsm_dt_init (void ) {return 0 ;
xen/xsm/xsm_core.c +27 -16
77 lines not on this page · raw message
xen/xsm/xsm_policy.c +28 -28
89 lines not on this page · raw message
Signature · 1 line
2.20.1
Re: [PATCH v1 03/18] x86: adopt new boot info structures
Quoted 5 lines · Daniel P. Smith
This commit replaces the use of the multiboot v1 structures starting at
__start_xen(). The majority of this commit is converting the fields being accessed for the startup calculations. While adapting the ucode boot module location logic, this code was refactored to reduce some of the unnecessary complexity.
Things like this or ...
Quoted 43 lines
xen/arch/x86/bzimage.c +8
-8@@ -69,10 +69,8 @@ static __init int bzimage_check(struct setup_header *hdr, unsigned long len)return 1 ; } -static unsigned long __initdata orig_image_len; - -unsigned long __init bzimage_headroom (void *image_start, -unsigned long image_length) +unsigned long __init bzimage_headroom ( +void *image_start,unsigned long image_length) {struct setup_header *hdr = (struct setup_header *)image_start;int err;@@ -91,7 +89,6 @@ unsigned long __init bzimage_headroom(void *image_start,if (elf_is_elfbinary (image_start, image_length) )return 0 ; - orig_image_len = image_length; headroom =output_length (image_start, image_length);if (gzip_check (image_start, image_length)) {@@ -104,12 +101,15 @@ unsigned long __init bzimage_headroom(void *image_start,return headroom; } -int __init bzimage_parse (void *image_base,void **image_start, -unsigned long *image_len) +int __init bzimage_parse ( +void *image_base,void **image_start,unsigned int headroom, +unsigned long *image_len) {struct setup_header *hdr = (struct setup_header *)(*image_start);int err =bzimage_check (hdr, *image_len); -unsigned long output_len; +unsigned long output_len, orig_image_len; + + orig_image_len = *image_len - headroom;if ( err <0 )return err;
xen/arch/x86/bzimage.c int __init bzimage_parse(void *image_base, void **image_start, · line 131
Quoted 8 lines
xen/arch/x86/bzimage.c +1
-1@@ -125,7 +125,7 @@ int __init bzimage_parse(void *image_base, void **image_start,BUG_ON (!(image_base < *image_start)); - output_len =output_length (*image_start, orig_image_len); + output_len =output_length (*image_start, *image_len);if ( (err =perform_gunzip (image_base, *image_start, orig_image_len)) >0 ) err =decompress (*image_start, orig_image_len, image_base);
... whatever the deal is here want factoring out. Also you want to avoid making formatting changes (like in the function headers here) in an already large patch, when you don't otherwise touch the functions. I'm not even convinced the formatting changes are desirable here, so I'd like to ask that even on code you do touch for other reasons you do so only if the existing layout ends up really awkward.
I have not looked in any further detail at this patch, sorry. Together with my comment on the earlier patch I conclude that it might be best if you moved things to the new representation field by field (or set of related fields), introducing the new fields in the abstraction struct as they are being made use of.
Jan
Re: [PATCH v1 03/18] x86: adopt new boot info structures
Quoted 58 lines · Jan Beulich
Quoted 5 lines · Daniel P. Smith
This commit replaces the use of the multiboot v1 structures starting at
__start_xen(). The majority of this commit is converting the fields being accessed for the startup calculations. While adapting the ucode boot module location logic, this code was refactored to reduce some of the unnecessary complexity.Things like this or ...
Quoted 49 lines
xen/arch/x86/bzimage.c +9
-9@@ -69,10 +69,8 @@ static __init int bzimage_check(struct setup_header *hdr, unsigned long len)return 1 ; } -static unsigned long __initdata orig_image_len; - -unsigned long __init bzimage_headroom (void *image_start, -unsigned long image_length) +unsigned long __init bzimage_headroom ( +void *image_start,unsigned long image_length) {struct setup_header *hdr = (struct setup_header *)image_start;int err;@@ -91,7 +89,6 @@ unsigned long __init bzimage_headroom(void *image_start,if (elf_is_elfbinary (image_start, image_length) )return 0 ; - orig_image_len = image_length; headroom =output_length (image_start, image_length);if (gzip_check (image_start, image_length)) {@@ -104,12 +101,15 @@ unsigned long __init bzimage_headroom(void *image_start,return headroom; } -int __init bzimage_parse (void *image_base,void **image_start, -unsigned long *image_len) +int __init bzimage_parse ( +void *image_base,void **image_start,unsigned int headroom, +unsigned long *image_len) {struct setup_header *hdr = (struct setup_header *)(*image_start);int err =bzimage_check (hdr, *image_len); -unsigned long output_len; +unsigned long output_len, orig_image_len; + + orig_image_len = *image_len - headroom;if ( err <0 )return err;@@ -125,7 +125,7 @@ int __init bzimage_parse(void *image_base, void **image_start,BUG_ON (!(image_base < *image_start)); - output_len =output_length (*image_start, orig_image_len); + output_len =output_length (*image_start, *image_len);
Quoted 9 lines
xen/arch/x86/bzimage.c int __init bzimage_parse(void *image_base, void **image_start, · line 131
Quoted 2 lines
xen/arch/x86/bzimage.c +0
-0if ( (err =perform_gunzip (image_base, *image_start, orig_image_len)) >0 ) err =decompress (*image_start, orig_image_len, image_base);... whatever the deal is here want factoring out. Also you want to avoid making formatting changes (like in the function headers here) in an already large patch, when you don't otherwise touch the functions. I'm not even convinced the formatting changes are desirable here, so I'd like to ask that even on code you do touch for other reasons you do so only if the existing layout ends up really awkward.
Ack. As I mentioned, I tried dropping these based on past reviews. I will do another pass to try to catch just formatting and drop them.
Quoted 5 lines
I have not looked in any further detail at this patch, sorry. Together with my comment on the earlier patch I conclude that it might be best if you moved things to the new representation field by field (or set of related fields), introducing the new fields in the abstraction struct as they are being made use of.
I am not sure whether it is possible to do this field by field. This is why I was asking on IRC on about dealing with this kind of situation. As soon as multiboot_info_t/module_t are replaced with struct boot_info{}/struct boot_module{} a wholesale replacement must be done.
v/r,
dps
PATCH 4/18 x86: refactor entrypoints to new boot info
In reply to Daniel P. SmithThis previous commit added a transition point from multiboot v1 structures to the new boot info structures at the earliest common point for all the x86 entrypoints. The result is that each of the entrypoints would construct a multiboot v1 structure from the structures used by each entrypoint. This meant that multiboot2, EFI, and PVH all converted their structures over to mutliboot v1 to only be converted again upon entering __start_xen().
This commit drops the translation function and moves the population of the new boot info structures down into the various entrypoints.
--- xen/arch/x86/boot/boot_info32.h | 94 +++++++++++ xen/arch/x86/boot/defs.h | 17 +- xen/arch/x86/boot/reloc.c | 187 +++++++++++++++------- xen/arch/x86/efi/efi-boot.h | 96 ++++++----- xen/arch/x86/guest/xen/pvh-boot.c | 64 +++++--- xen/arch/x86/include/asm/guest/pvh-boot.h | 6 +- xen/arch/x86/setup.c | 71 +++----- xen/common/efi/boot.c | 4 +- 8 files changed, 359 insertions(+), 180 deletions(-) create mode 100644 xen/arch/x86/boot/boot_info32.h
xen/arch/x86/boot/boot_info32.h new file +94 -0
@@ -0,0 +1,94 @@ +#ifndef __BOOT_INFO32_H__ +#define __BOOT_INFO32_H__ + +#include "defs.h" + +typedef enum { +BOOTMOD_UNKNOWN , +BOOTMOD_XEN , +BOOTMOD_FDT , +BOOTMOD_KERNEL , +BOOTMOD_RAMDISK , +BOOTMOD_XSM , +BOOTMOD_UCODE , +BOOTMOD_GUEST_DTB , +}bootmodule_kind ; + +typedef enum { +BOOTSTR_EMPTY , +BOOTSTR_STRING , +BOOTSTR_CMDLINE , +}bootstring_kind ; + +#define BOOTMOD_MAX_STRING 1024 +struct __packed boot_string { +u32 kind; +u64 arch; + +char bytes[BOOTMOD_MAX_STRING ]; +u64 len; +}; + +struct __packed arch_bootmodule { +bool relocated; +u32 flags; +#define BOOTMOD_FLAG_X86_RELOCATED 1U << 0 +u32 headroom; +}; + +struct __packed boot_module { +u32 kind; +u64 start; +u64 mfn; +u64 size; + +u64 arch; +struct boot_string string; +}; + +struct __packed arch_boot_info { +/* uint32_t */ +u32 flags; +#define BOOTINFO_FLAG_X86_MEMLIMITS 1U << 0 +#define BOOTINFO_FLAG_X86_BOOTDEV 1U << 1 +#define BOOTINFO_FLAG_X86_CMDLINE 1U << 2 +#define BOOTINFO_FLAG_X86_MODULES 1U << 3 +#define BOOTINFO_FLAG_X86_AOUT_SYMS 1U << 4 +#define BOOTINFO_FLAG_X86_ELF_SYMS 1U << 5 +#define BOOTINFO_FLAG_X86_MEMMAP 1U << 6 +#define BOOTINFO_FLAG_X86_DRIVES 1U << 7 +#define BOOTINFO_FLAG_X86_BIOSCONFIG 1U << 8 +#define BOOTINFO_FLAG_X86_LOADERNAME 1U << 9 +#define BOOTINFO_FLAG_X86_APM 1U << 10 + +/* bool */ +u8 xen_guest; + +/* char* */ +u64 boot_loader_name; +u64 kextra; + +/* uint32_t */ +u32 mem_lower; +u32 mem_upper; + +/* uint32_t */ +u32 mmap_length; +/* paddr_t */ +u64 mmap_addr; +}; + +struct __packed boot_info { +/* char* */ +u64 cmdline; + +/* uint32_t */ +u32 nr_mods; +/* struct boot_module* */ +u64 mods; + +/* struct arch_boot_info* */ +u64 arch; +}; + +#endif
xen/arch/x86/boot/defs.h +9 -8
@@ -22,11 +22,11 @@#include "../../../include/xen/stdbool.h" -#define __maybe_unused __attribute__((__unused__)) -#define __packed __attribute__((__packed__)) -#define __stdcall __attribute__((__stdcall__)) +#define __maybe_unused __attribute__((__unused__)) +#define __packed __attribute__((__packed__)) +#define __stdcall __attribute__((__stdcall__)) -#define NULL ((void *)0) +#define NULL ((void *)0)#define ALIGN_UP (arg, align) \ (((arg) + (align) - 1) & ~((typeof(arg))(align) - 1))
@@ -43,9 +43,10 @@ (void ) (&_x == &_y); \ _x > _y ? _x : _y; }) -#define _p (val) ((void *)(unsigned long)(val)) +#define _p (val) ((void *)(unsigned long)(val)) +#define _addr (val) ((unsigned long)(void *)(val)) -#define tolower (c) ((c) | 0x20) +#define tolower (c) ((c) | 0x20)typedef unsigned char u8 ;typedef unsigned short u16 ;
@@ -57,7 +58,7 @@ typedef u16 uint16_t;typedef u32 uint32_t ;typedef u64 uint64_t ; -#define U16_MAX ((u16)(~0U)) -#define UINT_MAX (~0U) +#define U16_MAX ((u16)(~0U)) +#define UINT_MAX (~0U)#endif /* __BOOT_DEFS_H__ */
xen/arch/x86/boot/reloc.c +129 -58
285 lines not on this page · raw message
xen/arch/x86/efi/efi-boot.h +58 -38
160 lines not on this page · raw message
xen/arch/x86/guest/xen/pvh-boot.c +43 -21
112 lines not on this page · raw message
xen/arch/x86/include/asm/guest/pvh-boot.h +3 -3
@@ -19,13 +19,13 @@#ifndef __X86_PVH_BOOT_H__#define __X86_PVH_BOOT_H__ -#include <xen/multiboot.h> +#include <xen/bootinfo.h> #ifdef CONFIG_PVH_GUEST extern bool pvh_boot; -void pvh_init (multiboot_info_t **mbi,module_t **mod); +void __init pvh_init (struct boot_info **bi);void pvh_print_info (void );#else
@@ -34,7 +34,7 @@ void pvh_print_info(void);#define pvh_boot 0 -static inline void pvh_init (multiboot_info_t **mbi,module_t **mod) +static inline void __init pvh_init (struct boot_info **bi) {ASSERT_UNREACHABLE (); }
xen/arch/x86/setup.c +20 -51
105 lines not on this page · raw message
xen/common/efi/boot.c +3 -1
@@ -3,6 +3,7 @@#include <efi/efipciio.h> #include <public/xen.h> #include <xen/bitops.h> +#include <xen/bootinfo.h> #include <xen/compile.h> #include <xen/ctype.h> #include <xen/dmi.h>
@@ -11,7 +12,6 @@#include <xen/keyhandler.h> #include <xen/lib.h> #include <xen/mm.h> -#include <xen/multiboot.h> #include <xen/param.h> #include <xen/pci_regs.h> #include <xen/pfn.h>
@@ -1222,6 +1222,8 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)efi_arch_relocate_image (0 ); +efi_arch_bootinfo_init (); +if ( use_cfg_file ) {EFI_FILE_HANDLE dir_handle;
Signature · 1 line
2.20.1
RE: [PATCH v1 04/18] x86: refactor entrypoints to new boot info
Hi Daniel,
I hope outlook gets this reply right.
Quoted 14 lines
-----Original Message-----
Subject: [PATCH v1 04/18] x86: refactor entrypoints to new boot infoxen/arch/x86/guest/xen/pvh-boot.c +0
-1@@ -99,13 +118,16 @@ static void __init get_memory_map(void)sanitize_e820_map (e820_raw.map, &e820_raw.nr_map); } -void __init pvh_init (multiboot_info_t **mbi,module_t **mod)
xen/arch/x86/guest/xen/pvh-boot.c static void __init get_memory_map(void) · line 129
Quoted 10 lines
xen/arch/x86/guest/xen/pvh-boot.c +4
-1+void __init pvh_init (struct boot_info **bi) { -convert_pvh_info (mbi, mod); + *bi =init_pvh_info (); +convert_pvh_info (*bi);hypervisor_probe ();ASSERT (xen_guest); + (*bi)->arch->xen_guest = xen_guest;
I think you may have a typo/missed refactoring here?
I changed this line to "(*bi)->arch->xenguest = xen_guest;" to get the patchset to build.
The arch_boot_info struct in boot_info32.h has a field 'xen_guest' but the same field in asm/bootinfo.h was re-named from 'xen_guest' to 'xenguest' in the 'x86: adopt new boot info structures' commit.
What was your intent?
xen/arch/x86/guest/xen/pvh-boot.c static void __init get_memory_map(void) · line 133
Quoted 4 lines
xen/arch/x86/guest/xen/pvh-boot.c +1
-0+get_memory_map (); }
Thanks,
Jackson Smith
Re: [PATCH v1 04/18] x86: refactor entrypoints to new boot info
Quoted 3 lines · Jackson
Hi Daniel,
I hope outlook gets this reply right.
Looks good to me, thank you for taking the time to review.
Quoted 20 lines
Quoted 20 lines
-----Original Message-----
Subject: [PATCH v1 04/18] x86: refactor entrypoints to new boot infoxen/arch/x86/guest/xen/pvh-boot.c +3
-2@@ -99,13 +118,16 @@ static void __init get_memory_map(void)sanitize_e820_map (e820_raw.map, &e820_raw.nr_map); } -void __init pvh_init (multiboot_info_t **mbi,module_t **mod) +void __init pvh_init (struct boot_info **bi) { -convert_pvh_info (mbi, mod); + *bi =init_pvh_info (); +convert_pvh_info (*bi);
Quoted 8 lines
xen/arch/x86/guest/xen/pvh-boot.c static void __init get_memory_map(void) · line 129
Quoted 4 lines
xen/arch/x86/guest/xen/pvh-boot.c +1
-0hypervisor_probe ();ASSERT (xen_guest); + (*bi)->arch->xen_guest = xen_guest;I think you may have a typo/missed refactoring here?
I changed this line to "(*bi)->arch->xenguest =xen_guest;" to get the patchset to build.
Hmm, I guess I missed one. I originally was going to mimic the name xen_guest in the structure definition but when xen guest support is disable the xen_guest global turns into a #define which replaces the reference resulting in a compilation error.
Quoted 5 lines
The
arch_boot_infostruct inboot_info32.hhas a field 'xen_guest' but the same field inasm/bootinfo.hwas re-named from 'xen_guest' to 'xenguest' in the 'x86: adopt new boot info structures' commit.What was your intent?
As I mentioned above, the renaming was intentional, and it looks like I do a poor job catching everywhere where the renaming need to be done.
Quoted 7 lines
xen/arch/x86/guest/xen/pvh-boot.c static void __init get_memory_map(void) · line 133
Quoted 4 lines
xen/arch/x86/guest/xen/pvh-boot.c +1
-0+get_memory_map (); }Thanks,
Jackson Smith
v/r,
dps
PATCH 5/18 x86: refactor xen cmdline into general framework
In reply to Daniel P. SmithThis refactors xen cmdline processing into a general framework under the new boot info abstraction.
--- xen/arch/x86/include/asm/bootinfo.h | 49 ++++++++++++++++++++++++ xen/arch/x86/setup.c | 58 ++++------------------------- xen/include/xen/bootinfo.h | 11 ++++++ 3 files changed, 68 insertions(+), 50 deletions(-)
xen/arch/x86/include/asm/bootinfo.h +49 -0
@@ -45,4 +45,53 @@ struct __packed mb_memmap {uint32_t type; }; +static inline bool loader_is_grub2 (const char *loader_name) +{ +/* GRUB1="GNU GRUB 0.xx"; GRUB2="GRUB 1.xx" */ +const char *p =strstr (loader_name,"GRUB " ); +return (p !=NULL ) && (p[5 ] !='0' ); +} + +static inline char *arch_prepare_cmdline (char *p,struct arch_boot_info *arch) +{ + p = p ? :"" ; + +/* Strip leading whitespace. */ +while ( *p ==' ' ) + p++; + +/* GRUB2 and PVH don't not include image name as first item on command line. */ +if ( !(arch->xenguest ||loader_is_grub2 (arch->boot_loader_name)) ) + { +/* Strip image name plus whitespace. */ +while ( (*p !=' ' ) && (*p !='\0' ) ) + p++; +while ( *p ==' ' ) + p++; + } + +return p; +} + +static inline char *arch_bootinfo_prepare_cmdline ( +char *cmdline,struct arch_boot_info *arch) +{ +if ( !cmdline ) +return "" ; + +if ( (arch->kextra =strstr (cmdline," -- " )) !=NULL ) + { +/* +* Options after ' -- ' separator belong to dom0. +* 1. Orphan dom0's options from Xen's command line. +* 2. Skip all but final leading space from dom0's options. +*/ + *arch->kextra ='\0' ; + arch->kextra +=3 ; +while ( arch->kextra[1 ] ==' ' ) arch->kextra++; + } + + +return arch_prepare_cmdline (cmdline, arch); +}#endif
xen/arch/x86/setup.c +8 -50
100 lines not on this page · raw message
xen/include/xen/bootinfo.h +11 -0
@@ -53,6 +53,17 @@ struct __packed boot_info {extern struct boot_info *boot_info; +static inline char *bootinfo_prepare_cmdline (struct boot_info *bi) +{ + bi->cmdline =arch_bootinfo_prepare_cmdline (bi->cmdline, bi->arch); + +if ( *bi->cmdline ==' ' ) +printk (XENLOG_WARNING "%s: leading whitespace left on cmdline\n" , + __func__); + +return bi->cmdline; +} +static inline unsigned long bootmodule_next_idx_by_kind (const struct boot_info *bi,bootmodule_kind kind,unsigned long start) {
Signature · 1 line
2.20.1
Re: [PATCH v1 05/18] x86: refactor xen cmdline into general framework
xen/include/xen/bootinfo.h struct __packed boot_info { · line 62
Quoted 13 lines · Daniel P. Smith
xen/include/xen/bootinfo.h +7
-0@@ -53,6 +53,17 @@ struct __packed boot_info {extern struct boot_info *boot_info; +static inline char *bootinfo_prepare_cmdline (struct boot_info *bi) +{ + bi->cmdline =arch_bootinfo_prepare_cmdline (bi->cmdline, bi->arch); + +if ( *bi->cmdline ==' ' ) +printk (XENLOG_WARNING "%s: leading whitespace left on cmdline\n" , + __func__);
Just a remark and a question on this one: I don't view the use of __func__ here (and in fact in many other cases as well) as very useful. And why do we need such a warning all of the sudden in the first place?
Jan
Re: [PATCH v1 05/18] x86: refactor xen cmdline into general framework
Quoted 7 lines · Jan Beulich
Quoted 6 lines · Daniel P. Smith
xen/include/xen/bootinfo.h +0
-0@@ -53,6 +53,17 @@ struct __packed boot_info {extern struct boot_info *boot_info;
Quoted 12 lines
xen/include/xen/bootinfo.h struct __packed boot_info { · line 62
Quoted 7 lines
xen/include/xen/bootinfo.h +7
-0+static inline char *bootinfo_prepare_cmdline (struct boot_info *bi) +{ + bi->cmdline =arch_bootinfo_prepare_cmdline (bi->cmdline, bi->arch); + +if ( *bi->cmdline ==' ' ) +printk (XENLOG_WARNING "%s: leading whitespace left on cmdline\n" , + __func__);Just a remark and a question on this one: I don't view the use of
__func__here (and in fact in many other cases as well) as very useful. And why do we need such a warning all of the sudden in the first place?
This started as just a debug print, thus why __func__ is in place, but later decided to leave it. This is because after this point, the code assumes that all leading space was stripped, but there was never a check that logic did the job correct. I don't believe a failure to do so warranted a halt to the boot process, but at least provide a warning into the log should the trimming fail. Doing so would allow an admin to have a clue should an unexpected behavior occur as a result of leading space making it through and breaking the fully trimmed assumption made elsewhere.
v/r,
dps
Re: [PATCH v1 05/18] x86: refactor xen cmdline into general framework
Quoted 21 lines · Daniel P. Smith
Quoted 7 lines · Jan Beulich
Quoted 6 lines · Daniel P. Smith
xen/include/xen/bootinfo.h +0
-0@@ -53,6 +53,17 @@ struct __packed boot_info {extern struct boot_info *boot_info;Quoted 12 lines
xen/include/xen/bootinfo.h struct __packed boot_info { · line 62
Quoted 7 lines
xen/include/xen/bootinfo.h +7
-0+static inline char *bootinfo_prepare_cmdline (struct boot_info *bi) +{ + bi->cmdline =arch_bootinfo_prepare_cmdline (bi->cmdline, bi->arch); + +if ( *bi->cmdline ==' ' ) +printk (XENLOG_WARNING "%s: leading whitespace left on cmdline\n" , + __func__);Just a remark and a question on this one: I don't view the use of
__func__here (and in fact in many other cases as well) as very useful. And why do we need such a warning all of the sudden in the first place?
Quoted 9 lines
This started as just a debug print, thus why
__func__is in place, but later decided to leave it. This is because after this point, the code assumes that all leading space was stripped, but there was never a check that logic did the job correct. I don't believe a failure to do so warranted a halt to the boot process, but at least provide a warning into the log should the trimming fail. Doing so would allow an admin to have a clue should an unexpected behavior occur as a result of leading space making it through and breaking the fully trimmed assumption made elsewhere.
All fine, but then such a change wants doing on its own, not in the middle of pretty involved refactoring work.
Jan
PATCH 6/18 fdt: make fdt handling reusable across arch
In reply to Daniel P. SmithThis refactors reusable code from Arm's bootfdt.c and device-tree.h that is general fdt handling code. The Kconfig parameter CORE_DEVICE_TREE is introduced for when the ability of parsing DTB files is needed by a capability such as hyperlaunch.
--- xen/arch/arm/bootfdt.c | 115 +---------------------------- xen/common/Kconfig | 4 ++ xen/common/Makefile | 3 +- xen/common/fdt.c | 131 ++++++++++++++++++++++++++++++++++ xen/include/xen/device_tree.h | 50 +------------ xen/include/xen/fdt.h | 79 ++++++++++++++++++++ 6 files changed, 218 insertions(+), 164 deletions(-) create mode 100644 xen/common/fdt.c create mode 100644 xen/include/xen/fdt.h
xen/arch/arm/bootfdt.c +1 -114
@@ -14,53 +14,11 @@#include <xen/efi.h> #include <xen/device_tree.h> #include <xen/libfdt/libfdt.h> +#include <xen/fdt.h> #include <xen/sort.h> #include <xsm/xsm.h> #include <asm/setup.h> -static bool __init device_tree_node_matches (const void *fdt,int node, -const char *match) -{ -const char *name; -size_t match_len; - - name =fdt_get_name (fdt, node,NULL ); - match_len =strlen (match); - -/* Match both "match" and "match@..." patterns but not -"match-foo". */ -return strncmp (name, match, match_len) ==0 - && (name[match_len] =='@' || name[match_len] =='\0' ); -} - -static bool __init device_tree_node_compatible (const void *fdt,int node, -const char *match) -{ -int len, l; -const void *prop; - - prop =fdt_getprop (fdt, node,"compatible" , &len); -if ( prop ==NULL ) -return false; - -while ( len >0 ) { -if ( !dt_compat_cmp (prop, match) ) -return true; - l =strlen (prop) +1 ; - prop += l; - len -= l; - } - -return false; -} - -void __init device_tree_get_reg (const __be32 **cell,u32 address_cells, -u32 size_cells,u64 *start,u64 *size) -{ - *start =dt_next_cell (address_cells, cell); - *size =dt_next_cell (size_cells, cell); -} - static int __init device_tree_get_meminfo(const void *fdt, int node, const char *prop_name, u32 address_cells, u32 size_cells,
@@ -108,77 +66,6 @@ static int __init device_tree_get_meminfo(const void *fdt, int node,return 0 ; } -u32 __init device_tree_get_u32 (const void *fdt,int node, -const char *prop_name,u32 dflt) -{ -const struct fdt_property *prop; - - prop =fdt_get_property (fdt, node, prop_name,NULL ); -if ( !prop || prop->len <sizeof (u32) ) -return dflt; - -return fdt32_to_cpu (*(uint32_t *)prop->data); -} - -/** -* device_tree_for_each_node - iterate over all device tree sub-nodes -* @fdt: flat device tree. -* @node: parent node to start the search from -* @func: function to call for each sub-node. -* @data: data to pass to @func. -* -* Any nodes nested at DEVICE_TREE_MAX_DEPTH or deeper are ignored. -* -* Returns 0 if all nodes were iterated over successfully. If @func -* returns a value different from 0, that value is returned immediately. -*/ -int __init device_tree_for_each_node (const void *fdt,int node, -device_tree_node_func func, -void *data) -{ -/* -* We only care about relative depth increments, assume depth of -* node is 0 for simplicity. -*/ -int depth =0 ; -const int first_node = node; -u32 address_cells[DEVICE_TREE_MAX_DEPTH ]; -u32 size_cells[DEVICE_TREE_MAX_DEPTH ]; -int ret; - -do { -const char *name =fdt_get_name (fdt, node,NULL ); -u32 as, ss; - -if ( depth >=DEVICE_TREE_MAX_DEPTH ) - { -printk ("Warning: device tree node `%s' is nested too deep\n" , - name); -continue ; - } - - as = depth >0 ? address_cells[depth-1 ] :DT_ROOT_NODE_ADDR_CELLS_DEFAULT ; - ss = depth >0 ? size_cells[depth-1 ] :DT_ROOT_NODE_SIZE_CELLS_DEFAULT ; - - address_cells[depth] =device_tree_get_u32 (fdt, node, -"#address-cells" , as); - size_cells[depth] =device_tree_get_u32 (fdt, node, -"#size-cells" , ss); - -/* skip the first node */ -if ( node != first_node ) - { - ret =func (fdt, node, name, depth, as, ss, data); -if ( ret !=0 ) -return ret; - } - - node =fdt_next_node (fdt, node, &depth); - }while ( node >=0 && depth >0 ); - -return 0 ; -} - static int __init process_memory_node(const void *fdt, int node, const char *name, int depth, u32 address_cells, u32 size_cells,
xen/common/Kconfig +4 -0
@@ -31,8 +31,12 @@ config HAS_ALTERNATIVE config HAS_COMPAT bool +config CORE_DEVICE_TREE + bool + config HAS_DEVICE_TREE bool + select CORE_DEVICE_TREE config HAS_EX_TABLE bool
xen/common/Makefile +2 -1
@@ -10,6 +10,7 @@ obj-y += domain.o obj-y += event_2l.o obj-y += event_channel.o obj-y += event_fifo.o +obj-$(CONFIG_CORE_DEVICE_TREE) += fdt.o obj-$(CONFIG_CRASH_DEBUG) += gdbstub.o obj-$(CONFIG_GRANT_TABLE) += grant_table.o obj-y += guestcopy.o
@@ -73,7 +74,7 @@ obj-y += sched/ obj-$(CONFIG_UBSAN) += ubsan/ obj-$(CONFIG_NEEDS_LIBELF) += libelf/ -obj-$(CONFIG_HAS_DEVICE_TREE) += libfdt/ +obj-$(CONFIG_CORE_DEVICE_TREE) += libfdt/ CONF_FILE := $(if $(patsubst /%,,$(KCONFIG_CONFIG)),$(objtree)/)$(KCONFIG_CONFIG) $(obj)/config.gz: $(CONF_FILE)
xen/common/fdt.c new file +131 -0
131 lines not on this page · raw message
xen/include/xen/device_tree.h +1 -49
81 lines not on this page · raw message
xen/include/xen/fdt.h new file +79 -0
79 lines not on this page · raw message
Signature · 1 line
2.20.1
RE: [PATCH v1 06/18] fdt: make fdt handling reusable across arch
Hi Daniel,
Quoted 10 lines
-----Original Message-----
Subject: [PATCH v1 06/18] fdt: make fdt handling reusable across archThis refactors reusable code from Arm's
bootfdt.canddevice-tree.hthat is general fdt handling code. TheKconfigparameterCORE_DEVICE_TREEis introduced for when the ability of parsing DTB files is needed by a capability such as hyperlaunch.Signed-off-by Daniel P. Smith <[email protected]>Reviewed-by Christopher Clark <[email protected]>
Kind regards,
Henry
Quoted 7 lines
--- xen/arch/arm/bootfdt.c | 115 +---------------------------- xen/common/Kconfig | 4 ++ xen/common/Makefile | 3 +- xen/common/fdt.c | 131 ++++++++++++++++++++++++++++++++++ xen/include/xen/device_tree.h | 50 +------------ xen/include/xen/fdt.h | 79 ++++++++++++++++++++
Re: [PATCH v1 06/18] fdt: make fdt handling reusable across arch
In reply to Daniel P. SmithQuoted 7 lines · Daniel P. Smith
This refactors reusable code from Arm's
bootfdt.canddevice-tree.hthat is general fdt handling code. TheKconfigparameterCORE_DEVICE_TREEis introduced for when the ability of parsing DTB files is needed by a capability such as hyperlaunch.Signed-off-by Daniel P. Smith <[email protected]>Reviewed-by Christopher Clark <[email protected]>
Quoted 10 lines
--- xen/arch/arm/bootfdt.c | 115 +---------------------------- xen/common/Kconfig | 4 ++ xen/common/Makefile | 3 +- xen/common/fdt.c | 131 ++++++++++++++++++++++++++++++++++ xen/include/xen/device_tree.h | 50 +------------ xen/include/xen/fdt.h | 79 ++++++++++++++++++++ 6 files changed, 218 insertions(+), 164 deletions(-) create mode 100644 xen/common/fdt.c create mode 100644 xen/include/xen/fdt.h
I think this wants to be accompanied by an update to ./MAINTAINERS, so maintainership doesn't silently transition to THE REST.
I further think that the moved code would want to have style adjusted to match present guidelines - I've noticed a number of u<N> uses which should be uint<N>_t. I didn't look closely to see whether other style violations are also retained in the moved code.
Jan
Re: [PATCH v1 06/18] fdt: make fdt handling reusable across arch
Quoted 19 lines · Jan Beulich
Quoted 7 lines · Daniel P. Smith
This refactors reusable code from Arm's
bootfdt.canddevice-tree.hthat is general fdt handling code. TheKconfigparameterCORE_DEVICE_TREEis introduced for when the ability of parsing DTB files is needed by a capability such as hyperlaunch.Signed-off-by Daniel P. Smith <[email protected]>Reviewed-by Christopher Clark <[email protected]>Quoted 10 lines
--- xen/arch/arm/bootfdt.c | 115 +---------------------------- xen/common/Kconfig | 4 ++ xen/common/Makefile | 3 +- xen/common/fdt.c | 131 ++++++++++++++++++++++++++++++++++ xen/include/xen/device_tree.h | 50 +------------ xen/include/xen/fdt.h | 79 ++++++++++++++++++++ 6 files changed, 218 insertions(+), 164 deletions(-) create mode 100644 xen/common/fdt.c create mode 100644 xen/include/xen/fdt.h
Quoted 2 lines
I think this wants to be accompanied by an update to ./MAINTAINERS, so maintainership doesn't silently transition to THE REST.
ack
Quoted 4 lines
I further think that the moved code would want to have style adjusted to match present guidelines - I've noticed a number of u<N> uses which should be uint<N>_t. I didn't look closely to see whether other style violations are also retained in the moved code.
ack
v/r,
dps
PATCH 7/18 docs: update hyperlaunch device tree documentation
In reply to Daniel P. SmithThis commit is to update the hyperlaunch device tree documentation to align with the DTB parsing implementation.
--- .../designs/launch/hyperlaunch-devicetree.rst | 497 +++++++++++------- 1 file changed, 306 insertions(+), 191 deletions(-)
docs/designs/launch/hyperlaunch-devicetree.rst +306 -191
@@ -13,12 +13,268 @@ difference is the introduction of the ``hypervisor`` node that is under the 2. Allows for the domain construction information to easily be sanitized by simple removing the ``/chosen/hypervisor`` node. + +The Hypervisor node +------------------- + +The ``hypervisor`` node is a top level container for the domains that will be built +by hypervisor on start up. The node will be named ``hypervisor`` with a ``compatible`` +property to identify which hypervisors the configuration is intended. The hypervisor +node will consist of one or more config nodes and one or more domain nodes. + +Properties +"""""""""" + +compatible + Identifies which hypervisors the configuration is compatible. Required. + + Format: "hypervisor,<hypervisor name>", e.g "hypervisor,xen" + +Child Nodes +""""""""""" + +* config +* domain + +Config Node +----------- + +A ``config`` node is for passing configuration data and identifying any boot +modules that is of interest to the hypervisor. For example this would be where +Xen would be informed of microcode or XSM policy locations. Each ``config`` +node will require a unique device-tree compliant name as there may be one or +more ``config`` nodes present in a single dtb file. To identify which +hypervisor the configuration is intended, the required ``compatible`` property +must be present. + +While the config node is not meant to replace the hypervisor commandline, there +may be cases where it is better suited for passing configuration details at +boot time. This additional information may be carried in properties assigned +to a ``config`` node. If there are any boot modules that are intended for the +hypervisor, then a ``module`` child node should be provided to identify the +boot module. + +Properties +"""""""""" + +compatible + Identifies the hypervisor the confiugration is intended. Required. + + Format: "<hypervisor name>,config", e.g "xen,config" + +bootargs + This is used to provide the boot params for Xen. + + Format: String, e.g. "flask=silo" + +Child Nodes +""""""""""" + +* module + +Domain Node +----------- + +A ``domain`` node is for describing the construction of a domain. Since there +may be one or more domain nodes, each one requires a unique, DTB compliant name +and a ``compatible`` property to identify as a domain node. + +A ``domain`` node may provide a ``domid`` property which will be used as the +requested domain id for the domain with a value of “0” signifying to use the +next available domain id, which is the default behavior if omitted. It should +be noted that a domain configuration is not able to request a domid of “0”. +Beyond that a domain node may have any of the following optional properties. + +Properties +"""""""""" + +compatible + Identifies the node as a domain node and for which hypervisor. Required. + + Format: "<hypervisor name>,domain", e.g "xen,domain" + +domid + Identifies the domid requested to assign to the domain. + + Format: Integer, e.g <0> + +permissions + This sets what Discretionary Access Control permissions + a domain is assigned. Optional, default is none. + + Format: Bitfield, e.g <3> or <0x00000003> + + PERMISSION_NONE (0) + PERMISSION_CONTROL (1 << 0) + PERMISSION_HARDWARE (1 << 1) + +functions + This identifies what system functions a domain will fulfill. + Optional, the default is none. + + Format: Bitfield, e.g <3221225487> or <0xC0000007> + + FUNCTION_NONE (0) + FUNCTION_BOOT (1 << 0) + FUNCTION_CRASH (1 << 1) + FUNCTION_CONSOLE (1 << 2) + FUNCTION_XENSTORE (1 << 30) + FUNCTION_LEGACY_DOM0 (1 << 31) + +.. note:: The `functions` bits that have been selected to indicate + ``FUNCTION_XENSTORE`` and ``FUNCTION_LEGACY_DOM0`` are the last two bits + (30, 31) such that should these features ever be fully replaced or retired, + the flags may be dropped without leaving a gap in the flag set. + +mode + The mode the domain will be executed under. Required. + + Format: Bitfield, e.g <5> or <0x00000005> + + MODE_PARAVIRTUALIZED (1 << 0) PV | PVH/HVM + MODE_ENABLE_DEVICE_MODEL (1 << 1) HVM | PVH + MODE_LONG (1 << 2) 64 BIT | 32 BIT + +domain-uuid + A globally unique identifier for the domain. Optional, + the default is NULL. + + Format: Byte Array, e.g [B3 FB 98 FB 8F 9F 67 A3] + +cpus + The number of vCPUs to be assigned to the domain. Optional, + the default is “1”. + + Format: Integer, e.g <0> + +memory + The amount of memory to assign to the domain, in KBs. This field uses a DTB + Reg which contains a start and size. For memory allocation start may or may + not have significance but size will always be used for the amount of memory + Required. + + Format: String min:<sz> | max:<sz> | <sz>, e.g. "256M" + +security-id + The security identity to be assigned to the domain when XSM + is the access control mechanism being used. Optional, + the default is “system_u:system_r:domU_t”. + + Format: string, e.g. "system_u:system_r:domU_t" + +Child Nodes +""""""""""" + +* module + +Module node +----------- + +This node describes a boot module loaded by the boot loader. A ``module`` node +will often appear repeatedly and will require a unique and DTB compliant name +for each instance. The compatible property is required to identify that the +node is a ``module`` node, the type of boot module, and what it represents. + +Depending on the type of boot module, the ``module`` node will require either a +``module-index`` or ``module-addr`` property must be present. They provide the +boot module specific way of locating the boot module in memory. + +Properties +"""""""""" + +compatible + This identifies what the module is and thus what the hypervisor + should use the module for during domain construction. Required. + + Format: "module,<module type>"[, "module,<locating type>"] + module type: kernel, ramdisk, device-tree, microcode, xsm-policy, + config + + locating type: index, addr + +module-index + This identifies the index for this module when in a module chain. + Required for multiboot environments. + + Format: Integer, e.g. <0> + +module-addr + This identifies where in memory this module is located. Required for + non-multiboot environments. + + Format: DTB Reg <start size>, e.g. <0x0 0x20000> + +bootargs + This is used to provide the boot params to kernel modules. + + Format: String, e.g. "ro quiet" + +.. note:: The bootargs property is intended for situations where the same kernel multiboot module is used for more than one domain. + Example Configuration --------------------- -Below are two example device tree definitions for the hypervisor node. The -first is an example of a multiboot-based configuration for x86 and the second -is a module-based configuration for Arm. +Below are examples device tree definitions for the hypervisor node. The first +is an example of booting a dom0 only configuration. Afterh that are a +multiboot-based configuration for x86 and a module-based configuration for Arm. + +Multiboot x86 Configuration Dom0-only: +"""""""""""""""""""""""""""""""""""""" +The following dts file can be provided to the Device Tree compiler, ``dtc``, to +produce a dtb file. +:: + + /dts-v1/; + + / { + chosen { + hypervisor { + compatible = "hypervisor,xen"; + + dom0 { + compatible = "xen,domain"; + + domid = <0>; + + permissions = <3>; + functions = <0xC000000F>; + mode = <5>; + + domain-uuid = [B3 FB 98 FB 8F 9F 67 A3 8A 6E 62 5A 09 13 F0 8C]; + + cpus = <1>; + memory = <0x0 0x20000000>; + + kernel { + compatible = "module,kernel", "module,index"; + module-index = <1>; + }; + }; + + }; + }; + }; + +The resulting dtb file, in this case dom0-only.dtb, can then be used with a +GRUB menuentry as such, +:: + + menuentry 'Devuan GNU/Linux, with Xen hyperlaunch' { + insmod part_gpt + insmod ext2 + set root='hd0,gpt2' + + echo 'Loading Xen hyperlaunch ...' + + multiboot2 /xen.gz placeholder sync_console + echo 'Loading Dom0 hyperlaunch dtb ...' + module2 --nounzip /dom0-only.dtb + echo 'Loading Linux 5.4.36+ ...' + module2 /vmlinuz-5.4.36+ placeholder root=/dev/mapper/test01--vg-root ro quiet + echo 'Loading initial ramdisk ...' + module2 --nounzip /initrd.img-5.4.36+ + } + Multiboot x86 Configuration: """"""""""""""""""""""""""""
@@ -31,89 +287,70 @@ Multiboot x86 Configuration: compatible = “hypervisor,xen” // Configuration container - config { + xen-config { compatible = "xen,config"; - module { - compatible = "module,microcode", "multiboot,module"; - mb-index = <1>; + bootargs = "console=com1,vga com1=115200,8n1 loglvl=all"; + + microcode { + compatible = "module,microcode", "module,index"; + module-index = <1>; }; - module { - compatible = "module,xsm-policy", "multiboot,module"; - mb-index = <2>; + policy { + compatible = "module,xsm-policy", "module,index"; + module-index = <2>; }; }; // Boot Domain definition - domain { + domB { compatible = "xen,domain"; domid = <0x7FF5>; - // FUNCTION_NONE (0) - // FUNCTION_BOOT (1 << 0) - // FUNCTION_CRASH (1 << 1) - // FUNCTION_CONSOLE (1 << 2) - // FUNCTION_XENSTORE (1 << 30) - // FUNCTION_LEGACY_DOM0 (1 << 31) functions = <0x00000001>; memory = <0x0 0x20000>; cpus = <1>; - module { - compatible = "module,kernel", "multiboot,module"; - mb-index = <3>; - }; - module { - compatible = "module,ramdisk", "multiboot,module"; - mb-index = <4>; + kernel { + compatible = "module,kernel", "module,index"; + module-index = <3>; }; - module { - compatible = "module,config", "multiboot,module"; - mb-index = <5>; + initrd { + compatible = "module,ramdisk", "module,index"; + module-index = <4>; + }; + dom-config { + compatible = "module,config", "module,index"; + module-index = <5>; }; // Classic Dom0 definition - domain { + dom0 { compatible = "xen,domain"; domid = <0>; - // PERMISSION_NONE (0) - // PERMISSION_CONTROL (1 << 0) - // PERMISSION_HARDWARE (1 << 1) permissions = <3>; - - // FUNCTION_NONE (0) - // FUNCTION_BOOT (1 << 0) - // FUNCTION_CRASH (1 << 1) - // FUNCTION_CONSOLE (1 << 2) - // FUNCTION_XENSTORE (1 << 30) - // FUNCTION_LEGACY_DOM0 (1 << 31) functions = <0xC0000006>; - - // MODE_PARAVIRTUALIZED (1 << 0) /* PV | PVH/HVM */ - // MODE_ENABLE_DEVICE_MODEL (1 << 1) /* HVM | PVH */ - // MODE_LONG (1 << 2) /* 64 BIT | 32 BIT */ mode = <5>; /* 64 BIT, PV */ - // UUID domain-uuid = [B3 FB 98 FB 8F 9F 67 A3]; cpus = <1>; memory = <0x0 0x20000>; - security-id = “dom0_t; + security-id = “system_u:system_r:dom0_t; - module { - compatible = "module,kernel", "multiboot,module"; - mb-index = <6>; + kernel { + compatible = "module,kernel", "module,index"; + module-index = <6>; bootargs = "console=hvc0"; }; - module { - compatible = "module,ramdisk", "multiboot,module"; - mb-index = <7>; + initrd { + compatible = "module,ramdisk", "module,index"; + module-index = <7>; }; };
@@ -137,89 +374,68 @@ Module Arm Configuration: compatible = “hypervisor,xen” // Configuration container - config { + xen-config { compatible = "xen,config"; - module { - compatible = "module,microcode”; + microcode { + compatible = "module,microcode”,"module,addr"; module-addr = <0x0000ff00 0x80>; }; - module { - compatible = "module,xsm-policy"; + policy { + compatible = "module,xsm-policy","module,addr"; module-addr = <0x0000ff00 0x80>; }; }; // Boot Domain definition - domain { + domB { compatible = "xen,domain"; domid = <0x7FF5>; - // FUNCTION_NONE (0) - // FUNCTION_BOOT (1 << 0) - // FUNCTION_CRASH (1 << 1) - // FUNCTION_CONSOLE (1 << 2) - // FUNCTION_XENSTORE (1 << 30) - // FUNCTION_LEGACY_DOM0 (1 << 31) functions = <0x00000001>; memory = <0x0 0x20000>; cpus = <1>; - module { - compatible = "module,kernel"; + + kernel { + compatible = "module,kernel","module,addr"; module-addr = <0x0000ff00 0x80>; }; - - module { - compatible = "module,ramdisk"; + initrd { + compatible = "module,ramdisk","module,addr"; module-addr = <0x0000ff00 0x80>; }; - module { - compatible = "module,config"; + dom-config { + compatible = "module,config","module,addr"; module-addr = <0x0000ff00 0x80>; }; // Classic Dom0 definition - domain@0 { + dom0 { compatible = "xen,domain"; domid = <0>; - // PERMISSION_NONE (0) - // PERMISSION_CONTROL (1 << 0) - // PERMISSION_HARDWARE (1 << 1) permissions = <3>; - - // FUNCTION_NONE (0) - // FUNCTION_BOOT (1 << 0) - // FUNCTION_CRASH (1 << 1) - // FUNCTION_CONSOLE (1 << 2) - // FUNCTION_XENSTORE (1 << 30) - // FUNCTION_LEGACY_DOM0 (1 << 31) functions = <0xC0000006>; - - // MODE_PARAVIRTUALIZED (1 << 0) /* PV | PVH/HVM */ - // MODE_ENABLE_DEVICE_MODEL (1 << 1) /* HVM | PVH */ - // MODE_LONG (1 << 2) /* 64 BIT | 32 BIT */ mode = <5>; /* 64 BIT, PV */ - // UUID domain-uuid = [B3 FB 98 FB 8F 9F 67 A3]; cpus = <1>; memory = <0x0 0x20000>; - security-id = “dom0_t”; + security-id = “system_u:system_r:dom0_t”; - module { - compatible = "module,kernel"; + kernel { + compatible = "module,kernel","module,addr"; module-addr = <0x0000ff00 0x80>; bootargs = "console=hvc0"; }; - module { - compatible = "module,ramdisk"; + intird { + compatible = "module,ramdisk","module,addr"; module-addr = <0x0000ff00 0x80>; }; };
@@ -240,104 +456,3 @@ provided to Xen using the standard method currently in use. The remaining modules would need to be loaded in the respective addresses specified in the `module-addr` property. - -The Hypervisor node -------------------- - -The hypervisor node is a top level container for the domains that will be built -by hypervisor on start up. On the ``hypervisor`` node the ``compatible`` -property is used to identify the type of hypervisor node present.. - -compatible - Identifies the type of node. Required. - -The Config node ---------------- - -A config node is for detailing any modules that are of interest to Xen itself. -For example this would be where Xen would be informed of microcode or XSM -policy locations. If the modules are multiboot modules and are able to be -located by index within the module chain, the ``mb-index`` property should be -used to specify the index in the multiboot module chain.. If the module will be -located by physical memory address, then the ``module-addr`` property should be -used to identify the location and size of the module. - -compatible - Identifies the type of node. Required. - -The Domain node ---------------- - -A domain node is for describing the construction of a domain. It may provide a -domid property which will be used as the requested domain id for the domain -with a value of “0” signifying to use the next available domain id, which is -the default behavior if omitted. A domain configuration is not able to request -a domid of “0”. After that a domain node may have any of the following -parameters, - -compatible - Identifies the type of node. Required. - -domid - Identifies the domid requested to assign to the domain. Required. - -permissions - This sets what Discretionary Access Control permissions - a domain is assigned. Optional, default is none. - -functions - This identifies what system functions a domain will fulfill. - Optional, the default is none. - -.. note:: The `functions` bits that have been selected to indicate - ``FUNCTION_XENSTORE`` and ``FUNCTION_LEGACY_DOM0`` are the last two bits - (30, 31) such that should these features ever be fully retired, the flags may - be dropped without leaving a gap in the flag set. - -mode - The mode the domain will be executed under. Required. - -domain-uuid - A globally unique identifier for the domain. Optional, - the default is NULL. - -cpus - The number of vCPUs to be assigned to the domain. Optional, - the default is “1”. - -memory - The amount of memory to assign to the domain, in KBs. - Required. - -security-id - The security identity to be assigned to the domain when XSM - is the access control mechanism being used. Optional, - the default is “domu_t”. - -The Module node ---------------- - -This node describes a boot module loaded by the boot loader. The required -compatible property follows the format: module,<type> where type can be -“kernel”, “ramdisk”, “device-tree”, “microcode”, “xsm-policy” or “config”. In -the case the module is a multiboot module, the additional property string -“multiboot,module” may be present. One of two properties is required and -identifies how to locate the module. They are the mb-index, used for multiboot -modules, and the module-addr for memory address based location. - -compatible - This identifies what the module is and thus what the hypervisor - should use the module for during domain construction. Required. - -mb-index - This identifies the index for this module in the multiboot module chain. - Required for multiboot environments. - -module-addr - This identifies where in memory this module is located. Required for - non-multiboot environments. - -bootargs - This is used to provide the boot params to kernel modules. - -.. note:: The bootargs property is intended for situations where the same kernel multiboot module is used for more than one domain.
Signature · 1 line
2.20.1
RE: [PATCH v1 07/18] docs: update hyperlaunch device tree documentation
Hi Daniel,
Quoted 21 lines
-----Original Message-----
Subject: [PATCH v1 07/18] docs: update hyperlaunch device tree documentationdocs/designs/launch/hyperlaunch-devicetree.rst +6
-0@@ -13,12 +13,268 @@ difference is the introduction of the ``hypervisor`` + +The Hypervisor node +------------------- + +The ``hypervisor`` node is a top level container for the domains that +will bebuilt
docs/designs/launch/hyperlaunch-devicetree.rst +2
-0+by hypervisor on start up. The node will be named ``hypervisor`` with +a
Quoted 2 lines
``compatible``
+property to identify which hypervisors the configuration is intended.
^^^ Should there be a note here that hypervisor node also needs a compatible "xen,<arch>"?
docs/designs/launch/hyperlaunch-devicetree.rst difference is the introduction of the ``hypervisor`` node that is under the · line 31
Quoted 12 lines
+The
hypervisor
+node will consist of one or more config nodes and one or more domain nodes.docs/designs/launch/hyperlaunch-devicetree.rst +8
-0+ +Properties +"""""""""" + +compatible + Identifies which hypervisors the configuration is compatible. Required. + + Format: "hypervisor,<hypervisor name>", e.g "hypervisor,xen"
^^^ Same here: compatible "<hypervisor name>,<arch>"?
Quoted 8 lines
Example Configuration
---------------------docs/designs/launch/hyperlaunch-devicetree.rst +5
-0+ +Multiboot x86 Configuration Dom0-only: +"""""""""""""""""""""""""""""""""""""" +The following dts file can be provided to the Device Tree compiler, +``dtc``,to
docs/designs/launch/hyperlaunch-devicetree.rst difference is the introduction of the ``hypervisor`` node that is under the · line 232
Quoted 9 lines
docs/designs/launch/hyperlaunch-devicetree.rst +9
-0+produce a dtb file. +:: + + /dts-v1/; + + / { + chosen { + hypervisor { + compatible = "hypervisor,xen";
^^^^^^^^ compatible = "hypervisor,xen", "xen,x86";
Quoted 13 lines
docs/designs/launch/hyperlaunch-devicetree.rst +12
-0+ + dom0 { + compatible = "xen,domain"; + + domid = <0>; + + permissions = <3>; + functions = <0xC000000F>; + mode = <5>; + + domain-uuid = [B3 FB 98 FB 8F 9F 67 A3 8A 6E 62 5A 09 + 13 F08C];
docs/designs/launch/hyperlaunch-devicetree.rst difference is the introduction of the ``hypervisor`` node that is under the · line 246
Quoted 3 lines
docs/designs/launch/hyperlaunch-devicetree.rst +3
-0+ + cpus = <1>; + memory = <0x0 0x20000000>;
^^^^^^^^^^ memory = "2048M";
Needs to be updated to new format for mem.
docs/designs/launch/hyperlaunch-devicetree.rst difference is the introduction of the ``hypervisor`` node that is under the · line 257
Quoted 11 lines
docs/designs/launch/hyperlaunch-devicetree.rst +11
-0+ + kernel { + compatible = "module,kernel", "module,index"; + module-index = <1>; + }; + }; + + }; + }; + }; +
Similar adjustments are needed for the rest of the examples I believe.
Also, two typos:
Line 287 is missing a line ending semi-colon. Line 82 has a double space between 'node' and 'may'.
Best,
Jackson
Re: [PATCH v1 07/18] docs: update hyperlaunch device tree documentation
Quoted 14 lines · Jackson
Hi Daniel,
Quoted 11 lines
-----Original Message-----
Subject: [PATCH v1 07/18] docs: update hyperlaunch device tree documentationdocs/designs/launch/hyperlaunch-devicetree.rst +0
-0@@ -13,12 +13,268 @@ difference is the introduction of the ``hypervisor``
Quoted 13 lines
Quoted 11 lines
docs/designs/launch/hyperlaunch-devicetree.rst +6
-0+ +The Hypervisor node +------------------- + +The ``hypervisor`` node is a top level container for the domains that +will bebuilt
docs/designs/launch/hyperlaunch-devicetree.rst +2
-0+by hypervisor on start up. The node will be named ``hypervisor`` with +a``compatible``
+property to identify which hypervisors the configuration is intended.^^^ Should there be a note here that hypervisor node also needs a compatible "xen,<arch>"?
Ack.
Quoted 5 lines
Quoted 5 lines
+The
hypervisor
+node will consist of one or more config nodes and one or more domain nodes.
+
Quoted 8 lines
docs/designs/launch/hyperlaunch-devicetree.rst difference is the introduction of the ``hypervisor`` node that is under the · line 31
Quoted 7 lines
docs/designs/launch/hyperlaunch-devicetree.rst +7
-0+Properties +"""""""""" + +compatible + Identifies which hypervisors the configuration is compatible. Required. + + Format: "hypervisor,<hypervisor name>", e.g "hypervisor,xen"^^^ Same here: compatible "<hypervisor name>,<arch>"?
Ack.
Quoted 10 lines
Quoted 10 lines
Example Configuration
---------------------docs/designs/launch/hyperlaunch-devicetree.rst +5
-0+ +Multiboot x86 Configuration Dom0-only: +"""""""""""""""""""""""""""""""""""""" +The following dts file can be provided to the Device Tree compiler, +``dtc``,to
+produce a dtb file.
+::
Quoted 8 lines
docs/designs/launch/hyperlaunch-devicetree.rst difference is the introduction of the ``hypervisor`` node that is under the · line 232
Quoted 7 lines
docs/designs/launch/hyperlaunch-devicetree.rst +7
-0+ + /dts-v1/; + + / { + chosen { + hypervisor { + compatible = "hypervisor,xen";^^^^^^^^ compatible = "hypervisor,xen", "xen,x86";
Ack.
Quoted 10 lines
Quoted 10 lines
docs/designs/launch/hyperlaunch-devicetree.rst +10
-0+ + dom0 { + compatible = "xen,domain"; + + domid = <0>; + + permissions = <3>; + functions = <0xC000000F>; + mode = <5>; +
Quoted 8 lines
docs/designs/launch/hyperlaunch-devicetree.rst difference is the introduction of the ``hypervisor`` node that is under the · line 246
Quoted 6 lines
+ domain-uuid = [B3 FB 98 FB 8F 9F 67 A3 8A 6E 62 5A 09
+ 13 F0
8C];docs/designs/launch/hyperlaunch-devicetree.rst +3
-0+ + cpus = <1>; + memory = <0x0 0x20000000>;^^^^^^^^^^ memory = "2048M";
Needs to be updated to new format for mem.
Ack.
Quoted 14 lines
docs/designs/launch/hyperlaunch-devicetree.rst difference is the introduction of the ``hypervisor`` node that is under the · line 257
Quoted 11 lines
docs/designs/launch/hyperlaunch-devicetree.rst +11
-0+ + kernel { + compatible = "module,kernel", "module,index"; + module-index = <1>; + }; + }; + + }; + }; + }; +Similar adjustments are needed for the rest of the examples I believe.
Quoted 3 lines
Also, two typos:
Line 287 is missing a line ending semi-colon. Line 82 has a double space between 'node' and 'may'.
Ack.
v/r,
dps
PATCH 8/18 kconfig: introduce domain builder config option
In reply to Daniel P. SmithHyperlaunch domain builder is the consolidated boot time domain building logic framework. This commit introduces the first config option for the domain builder to control support for loading the domain configurations via the flattened device tree.
--- xen/common/Kconfig | 1 + xen/common/domain-builder/Kconfig | 15 +++++++++++++++ 2 files changed, 16 insertions(+) create mode 100644 xen/common/domain-builder/Kconfig
xen/common/Kconfig +1 -0
@@ -355,6 +355,7 @@ config ARGO If unsure, say N. +source "common/domain-builder/Kconfig" source "common/sched/Kconfig" config CRYPTO
xen/common/domain-builder/Kconfig new file +15 -0
@@ -0,0 +1,15 @@ + +menu "Domain Builder Features" + +config BUILDER_FDT + bool "Domain builder device tree (UNSUPPORTED)" if UNSUPPORTED + select CORE_DEVICE_TREE + ---help--- + Enables the ability to configure the domain builder using a + flattened device tree. + + This feature is currently experimental. + + If unsure, say N. + +endmenu
Signature · 1 line
2.20.1
RE: [PATCH v1 08/18] kconfig: introduce domain builder config option
Hi Daniel,
Quoted 11 lines
-----Original Message-----
Subject: [PATCH v1 08/18] kconfig: introduce domain builder config optionHyperlaunch domain builder is the consolidated boot time domain building logic
framework. This commit introduces the first config option for the domain builder to control support for loading the domain configurations via the flattened device tree.Signed-off-by Daniel P. Smith <[email protected]>Reviewed-by Christopher Clark <[email protected]>
Kind regards,
Henry
Quoted 3 lines
--- xen/common/Kconfig | 1 + xen/common/domain-builder/Kconfig | 15 +++++++++++++++
Re: [PATCH v1 08/18] kconfig: introduce domain builder config option
In reply to Daniel P. Smithxen/common/domain-builder/Kconfig line 7
Quoted 10 lines · Daniel P. Smith
xen/common/domain-builder/Kconfig new file +7
-0@@ -0,0 +1,15 @@ + +menu "Domain Builder Features" + +config BUILDER_FDT + bool "Domain builder device tree (UNSUPPORTED)" if UNSUPPORTED + select CORE_DEVICE_TREE + ---help---
Nit: No new ---help--- please anymore.
xen/common/domain-builder/Kconfig line 9
Quoted 2 lines
xen/common/domain-builder/Kconfig new file +2
-0+ Enables the ability to configure the domain builder using a + flattened device tree.
Is this about both Dom0 and DomU? Especially if not, this wants making explicit. But perhaps even if so it wants saying, for the avoidance of doubt.
Jan
Re: [PATCH v1 08/18] kconfig: introduce domain builder config option
Quoted 13 lines · Jan Beulich
xen/common/domain-builder/Kconfig line 7
Quoted 10 lines · Daniel P. Smith
xen/common/domain-builder/Kconfig new file +7
-0@@ -0,0 +1,15 @@ + +menu "Domain Builder Features" + +config BUILDER_FDT + bool "Domain builder device tree (UNSUPPORTED)" if UNSUPPORTED + select CORE_DEVICE_TREE + ---help---Nit: No new ---help--- please anymore.
Ack.
Quoted 6 lines
xen/common/domain-builder/Kconfig line 9
Quoted 2 lines
xen/common/domain-builder/Kconfig new file +2
-0+ Enables the ability to configure the domain builder using a + flattened device tree.Is this about both Dom0 and DomU? Especially if not, this wants making explicit. But perhaps even if so it wants saying, for the avoidance of doubt.
The following patches will end with full conversion of both Dom0 and DomU construction to be handled by a core domain construction framework. If a device tree configuration is not present or this Kconfig option is not set, then the domain builder will construct a Dom0 as it does today. Turning this option on enables controlling the domain builder using a device tree configuration, which (eventually) will be able to construct any combination of Dom0, HWDom, CtlDom, DomU, etc. So I can add a qualifier of, 'configure what domains will be constructed at boot by the domain builder using a flattened device tree'. I can even add an explanation that on x86 the FDT must be provided as the first multiboot module.
v/r,
dps
PATCH 9/18 x86: introduce abstractions for domain builder
In reply to Daniel P. SmithThis commit expands the new boot info structs to provide the initial abstractions for domain builder. Additionally, it reuses the memory allocation structures previously used for dom0, bring the structures and helper functions under the domain builder.
--- xen/arch/x86/boot/boot_info32.h | 3 ++ xen/arch/x86/dom0_build.c | 21 +--------- xen/arch/x86/include/asm/bootdomain.h | 30 +++++++++++++++ xen/arch/x86/include/asm/bootinfo.h | 2 + xen/include/xen/bootdomain.h | 52 +++++++++++++++++++++++++ xen/include/xen/bootinfo.h | 4 ++ xen/include/xen/domain_builder.h | 55 +++++++++++++++++++++++++++ 7 files changed, 147 insertions(+), 20 deletions(-) create mode 100644 xen/arch/x86/include/asm/bootdomain.h create mode 100644 xen/include/xen/bootdomain.h create mode 100644 xen/include/xen/domain_builder.h
xen/arch/x86/boot/boot_info32.h +3 -0
@@ -87,6 +87,9 @@ struct __packed boot_info {/* struct boot_module* */ u64 mods; +/* struct domain_builder* */ +u64 builder; +/* struct arch_boot_info* */ u64 arch; };
xen/arch/x86/dom0_build.c +1 -20
@@ -4,6 +4,7 @@* Copyright (c) 2002-2005, K A Fraser */ +#include <xen/bootdomain.h> #include <xen/bootinfo.h> #include <xen/init.h> #include <xen/iocap.h>
@@ -22,31 +23,11 @@#include <asm/setup.h> #include <asm/spec_ctrl.h> -struct memsize { -long nr_pages; -unsigned int percent; -bool minus; -}; -static struct memsize __initdata dom0_size;static struct memsize __initdata dom0_min_size;static struct memsize __initdata dom0_max_size = { .nr_pages =LONG_MAX };static bool __initdata dom0_mem_set; -static bool __init memsize_gt_zero (const struct memsize *sz) -{ -return !sz->minus && sz->nr_pages; -} - -static unsigned long __init get_memsize (const struct memsize *sz, -unsigned long avail) -{ -unsigned long pages; - - pages = sz->nr_pages + sz->percent * avail /100 ; -return sz->minus ? avail - pages : pages; -} -/* * dom0_mem=[min:<min_amt>,][max:<max_amt>,][<amt>] *
xen/arch/x86/include/asm/bootdomain.h new file +30 -0
@@ -0,0 +1,30 @@ +#ifndef __ARCH_X86_BOOTDOMAIN_H__ +#define __ARCH_X86_BOOTDOMAIN_H__ + +struct memsize { +long nr_pages; +unsigned int percent; +bool minus; +}; + +static inline bool memsize_gt_zero (const struct memsize *sz) +{ +return !sz->minus && sz->nr_pages; +} + +static inline unsigned long get_memsize ( +const struct memsize *sz,unsigned long avail) +{ +unsigned long pages; + + pages = sz->nr_pages + sz->percent * avail /100 ; +return sz->minus ? avail - pages : pages; +} + +struct arch_domain_mem { +struct memsize mem_size; +struct memsize mem_min; +struct memsize mem_max; +}; + +#endif
xen/arch/x86/include/asm/bootinfo.h +2 -0
@@ -45,6 +45,8 @@ struct __packed mb_memmap {uint32_t type; }; +struct arch_domain_builder { }; +static inline bool loader_is_grub2 (const char *loader_name) {/* GRUB1="GNU GRUB 0.xx"; GRUB2="GRUB 1.xx" */
xen/include/xen/bootdomain.h new file +52 -0
52 lines not on this page · raw message
xen/include/xen/bootinfo.h +4 -0
@@ -1,6 +1,7 @@#ifndef __XEN_BOOTINFO_H__#define __XEN_BOOTINFO_H__ +#include <xen/bootdomain.h> #include <xen/mm.h> #include <xen/types.h>
@@ -15,6 +16,7 @@ typedef enum {BOOTMOD_XSM ,BOOTMOD_UCODE ,BOOTMOD_GUEST_DTB , +BOOTMOD_GUEST_CONF , } bootmodule_kind; typedef enum {
@@ -48,6 +50,8 @@ struct __packed boot_info {uint32_t nr_mods;struct boot_module *mods; +struct domain_builder *builder; +struct arch_boot_info *arch; };
xen/include/xen/domain_builder.h new file +55 -0
55 lines not on this page · raw message
Signature · 1 line
2.20.1
Re: [PATCH v1 09/18] x86: introduce abstractions for domain builder
Quoted 15 lines · Daniel P. Smith
xen/arch/x86/include/asm/bootdomain.h new file +12
-0@@ -0,0 +1,30 @@ +#ifndef __ARCH_X86_BOOTDOMAIN_H__ +#define __ARCH_X86_BOOTDOMAIN_H__ + +struct memsize { +long nr_pages; +unsigned int percent; +bool minus; +}; + +static inline bool memsize_gt_zero (const struct memsize *sz) +{ +return !sz->minus && sz->nr_pages;
xen/arch/x86/include/asm/bootdomain.h line 22
Quoted 10 lines
xen/arch/x86/include/asm/bootdomain.h new file +10
-0+} + +static inline unsigned long get_memsize ( +const struct memsize *sz,unsigned long avail) +{ +unsigned long pages; + + pages = sz->nr_pages + sz->percent * avail /100 ; +return sz->minus ? avail - pages : pages; +}
For both functions I think you should retain the __init, just in case the compiler decides against actually inlining them (according to my observations Clang frequently won't).
xen/arch/x86/include/asm/bootdomain.h line 28
Quoted 5 lines
xen/arch/x86/include/asm/bootdomain.h new file +5
-0+struct arch_domain_mem { +struct memsize mem_size; +struct memsize mem_min; +struct memsize mem_max; +};
How come this is introduced here without the three respective Dom0 variables being replaced by an instance of this struct? At which point a further question would be: What about dom0_mem_set?
xen/include/xen/bootdomain.h line 10
Quoted 13 lines
xen/include/xen/bootdomain.h new file +10
-0@@ -0,0 +1,52 @@ +#ifndef __XEN_BOOTDOMAIN_H__ +#define __XEN_BOOTDOMAIN_H__ + +#include <xen/bootinfo.h> +#include <xen/types.h> + +#include <public/xen.h> +#include <asm/bootdomain.h> + +struct domain ;
Why the forward decl? There's no function being declared here, and this is not C++.
xen/include/xen/bootdomain.h line 17
Quoted 5 lines
xen/include/xen/bootdomain.h new file +5
-0+struct boot_domain { +#define BUILD_PERMISSION_NONE (0) +#define BUILD_PERMISSION_CONTROL (1 << 0) +#define BUILD_PERMISSION_HARDWARE (1 << 1) +uint32_t permissions;
Why a fixed width type? And why no 'u' suffixes on the 1s being left shifted above? (Same further down from here.)
xen/include/xen/bootdomain.h line 30
Quoted 12 lines
xen/include/xen/bootdomain.h new file +12
-0+#define BUILD_FUNCTION_NONE (0) +#define BUILD_FUNCTION_BOOT (1 << 0) +#define BUILD_FUNCTION_CRASH (1 << 1) +#define BUILD_FUNCTION_CONSOLE (1 << 2) +#define BUILD_FUNCTION_STUBDOM (1 << 3) +#define BUILD_FUNCTION_XENSTORE (1 << 30) +#define BUILD_FUNCTION_INITIAL_DOM (1 << 31) +uint32_t functions; +/* On | Off */ +#define BUILD_MODE_PARAVIRTUALIZED (1 << 0)/* PV | PVH/HVM */ +#define BUILD_MODE_ENABLE_DEVICE_MODEL (1 << 1)/* HVM | PVH */ +#define BUILD_MODE_LONG (1 << 2)/* 64 BIT | 32 BIT */
I guess bitness would better not be a boolean-like value (and "LONG" is kind of odd anyway) - see RISC-V having provisions right away for 128-bit mode.
Quoted 13 lines
xen/include/xen/domain_builder.h new file +10
-0@@ -0,0 +1,55 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +#ifndef XEN_DOMAIN_BUILDER_H +#define XEN_DOMAIN_BUILDER_H + +#include <xen/bootdomain.h> +#include <xen/bootinfo.h> + +#include <asm/setup.h> +
xen/include/xen/domain_builder.h line 20
Quoted 10 lines
xen/include/xen/domain_builder.h new file +10
-0+struct domain_builder { +bool fdt_enabled; +#define BUILD_MAX_BOOT_DOMAINS 64 +uint16_t nr_doms; +struct boot_domain domains[BUILD_MAX_BOOT_DOMAINS ]; + +struct arch_domain_builder *arch; +}; + +static inline bool builder_is_initdom (struct boot_domain *bd)
const wherever possible, please.
xen/include/xen/domain_builder.h line 28
Quoted 8 lines
xen/include/xen/domain_builder.h new file +8
-0+{ +return bd->functions &BUILD_FUNCTION_INITIAL_DOM ; +} + +static inline bool builder_is_ctldom (struct boot_domain *bd) +{ +return (bd->functions &BUILD_FUNCTION_INITIAL_DOM || + bd->permissions &BUILD_PERMISSION_CONTROL );
Please parenthesize the operands of &, |, or ^ inside && or ||.
xen/include/xen/domain_builder.h line 39
Quoted 11 lines
xen/include/xen/domain_builder.h new file +11
-0+} + +static inline bool builder_is_hwdom (struct boot_domain *bd) +{ +return (bd->functions &BUILD_FUNCTION_INITIAL_DOM || + bd->permissions &BUILD_PERMISSION_HARDWARE ); +} + +static inline struct domain *builder_get_hwdom (struct boot_info *info) +{ +int i;
unsigned int please when the value can't go negative.
xen/include/xen/domain_builder.h line 53
Quoted 13 lines
xen/include/xen/domain_builder.h new file +13
-0+for ( i =0 ; i < info->builder->nr_doms; i++ ) + { +struct boot_domain *d = &info->builder->domains[i]; + +if (builder_is_hwdom (d) ) +return d->domain; + } + +return NULL ; +} + +void builder_init (struct boot_info *info); +uint32_t builder_create_domains (struct boot_info *info);
Both for these and for the inline functions - how is one to judge they are (a) needed and (b) fit their purpose without seeing even a single caller. And for the prototypes not even the implementation is there:
What's wrong with adding those at the time they're actually implemented (and hopefully also used)?
Jan
PATCH 10/18 x86: introduce the domain builder
In reply to Daniel P. SmithThis commit introduces the domain builder configuration FDT parser along with the domain builder core for domain creation. To enable domain builder to be a cross architecture internal API, a new arch domain creation call is introduced for use by the domain builder.
--- xen/arch/x86/setup.c | 9 + xen/common/Makefile | 1 + xen/common/domain-builder/Makefile | 2 + xen/common/domain-builder/core.c | 96 ++++++++++ xen/common/domain-builder/fdt.c | 295 +++++++++++++++++++++++++++++ xen/common/domain-builder/fdt.h | 7 + xen/include/xen/bootinfo.h | 16 ++ xen/include/xen/domain_builder.h | 1 + 8 files changed, 427 insertions(+) create mode 100644 xen/common/domain-builder/Makefile create mode 100644 xen/common/domain-builder/core.c create mode 100644 xen/common/domain-builder/fdt.c create mode 100644 xen/common/domain-builder/fdt.h
xen/arch/x86/setup.c +9 -0
@@ -1,4 +1,6 @@ +#include <xen/bootdomain.h> #include <xen/bootinfo.h> +#include <xen/domain_builder.h> #include <xen/init.h> #include <xen/lib.h> #include <xen/err.h>
@@ -826,6 +828,13 @@ static struct domain *__init create_dom0(const struct boot_info *bi)return d; } +void __init arch_create_dom ( +const struct boot_info *bi,struct boot_domain *bd) +{ +if (builder_is_initdom (bd) ) +create_dom0 (bi); +} +/* How much of the directmap is prebuilt at compile time. */ #define PREBUILT_MAP_LIMIT (1 << L2_PAGETABLE_SHIFT)
xen/common/Makefile +1 -0
@@ -72,6 +72,7 @@ extra-y := symbols-dummy.o obj-$(CONFIG_COVERAGE) += coverage/ obj-y += sched/ obj-$(CONFIG_UBSAN) += ubsan/ +obj-y += domain-builder/ obj-$(CONFIG_NEEDS_LIBELF) += libelf/ obj-$(CONFIG_CORE_DEVICE_TREE) += libfdt/
xen/common/domain-builder/Makefile new file +2 -0
@@ -0,0 +1,2 @@ +obj-$(CONFIG_BUILDER_FDT) += fdt.o +obj-y += core.o
xen/common/domain-builder/core.c new file +96 -0
96 lines not on this page · raw message
xen/common/domain-builder/fdt.c new file +295 -0
295 lines not on this page · raw message
xen/common/domain-builder/fdt.h new file +7 -0
@@ -0,0 +1,7 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +#ifndef COMMON_BUILDER_FDT_H +#define COMMON_BUILDER_FDT_H + +int __init check_fdt (struct boot_info *info,void *fdt); +#endif
xen/include/xen/bootinfo.h +16 -0
@@ -101,6 +101,22 @@ static inline struct boot_module *bootmodule_next_by_kind(return NULL ; } +static inline struct boot_module *bootmodule_next_by_addr ( +const struct boot_info *bi,paddr_t addr,struct boot_module *start) +{ +/* point end at the entry for xen */ +struct boot_module *end = &bi->mods[bi->nr_mods]; + +if ( !start ) + start = bi->mods; + +for ( ; start < end; start++ ) +if ( start->start == addr ) +return start; + +return NULL ; +} +static inline void bootmodule_update_start (struct boot_module *b,paddr_t new_start) { b->start = new_start;
xen/include/xen/domain_builder.h +1 -0
@@ -51,5 +51,6 @@ static inline struct domain *builder_get_hwdom(struct boot_info *info)void builder_init (struct boot_info *info);uint32_t builder_create_domains (struct boot_info *info); +void arch_create_dom (const struct boot_info *bi,struct boot_domain *bd);#endif /* XEN_DOMAIN_BUILDER_H */
Signature · 1 line
2.20.1
RE: [PATCH v1 10/18] x86: introduce the domain builder
Hi Daniel,
Quoted 6 lines
-----Original Message-----
Subject: [PATCH v1 10/18] x86: introduce the domain builderThis commit introduces the domain builder configuration FDT parser along with the domain builder core for domain creation. To enable domain builder to be a cross architecture internal API, a new arch domain creation call
is
Quoted 35 lines
introduced for use by the domain builder.
xen/common/domain-builder/core.c new file +23
-0+void __init builder_init (struct boot_info *info) { +struct boot_domain *d =NULL ; + + info->builder = &builder; + +if (IS_ENABLED (CONFIG_BUILDER_FDT ) ) + { + } + +/* +* No FDT config support or an FDT wasn't present, do an initial +* domain construction +*/ +printk ("Domain Builder: falling back to initial domain build\n" ); + info->builder->nr_doms =1 ; + d = &info->builder->domains[0 ]; + + d->mode = opt_dom0_pvh ?0 :BUILD_MODE_PARAVIRTUALIZED ; + + d->kernel = &info->mods[0 ]; + d->kernel->kind =BOOTMOD_KERNEL ; + + d->permissions = BUILD_PERMISSION_CONTROL |
BUILD_PERMISSION_HARDWARE;
+d->functions=BUILD_FUNCTION_CONSOLE|BUILD_FUNCTION_XENSTORE|xen/common/domain-builder/core.c new file +3
-0+BUILD_FUNCTION_INITIAL_DOM ; + + d->kernel->arch->headroom = bzimage_headroom(bootstrap_map(d-Quoted 1 line
kernel),
xen/common/domain-builder/core.c
Quoted 5 lines
xen/common/domain-builder/core.c new file +5
-0+ d->kernel->size); +bootstrap_map (NULL ); + +if ( d->kernel->string.len ) + d->kernel->string.kind =BOOTSTR_CMDLINE ; }
Forgive me if I'm incorrect, but I believe there is an issue with this fallback logic for the case where no FDT was provided.
If dom0_mem is not supplied to the xen cmd line, then d->meminfo is never initialized. (See dom0_compute_nr_pages/dom0_build.c:335)
This was giving me trouble because bd->meminfo.mem_max.nr_pages was left at 0, effectivity clamping dom0 to 0 pages of ram.
I'm not sure what the best solution is but one (easy) possibility is just initializing meminfo to the dom0 defaults near the end of this function:
d->meminfo.mem_size = dom0_size;
d->meminfo.mem_min = dom0_min_size;
d->meminfo.mem_max = dom0_max_size;Thanks,
Jackson
Re: [PATCH v1 10/18] x86: introduce the domain builder
Quoted 44 lines · Jackson
Hi Daniel,
Quoted 6 lines
-----Original Message-----
Subject: [PATCH v1 10/18] x86: introduce the domain builderThis commit introduces the domain builder configuration FDT parser along with the domain builder core for domain creation. To enable domain builder to be a cross architecture internal API, a new arch domain creation call
is
Quoted 35 lines
introduced for use by the domain builder.
xen/common/domain-builder/core.c new file +23
-0+void __init builder_init (struct boot_info *info) { +struct boot_domain *d =NULL ; + + info->builder = &builder; + +if (IS_ENABLED (CONFIG_BUILDER_FDT ) ) + { + } + +/* +* No FDT config support or an FDT wasn't present, do an initial +* domain construction +*/ +printk ("Domain Builder: falling back to initial domain build\n" ); + info->builder->nr_doms =1 ; + d = &info->builder->domains[0 ]; + + d->mode = opt_dom0_pvh ?0 :BUILD_MODE_PARAVIRTUALIZED ; + + d->kernel = &info->mods[0 ]; + d->kernel->kind =BOOTMOD_KERNEL ; + + d->permissions = BUILD_PERMISSION_CONTROL |
BUILD_PERMISSION_HARDWARE;
+d->functions=BUILD_FUNCTION_CONSOLE|BUILD_FUNCTION_XENSTORE|xen/common/domain-builder/core.c new file +3
-0+BUILD_FUNCTION_INITIAL_DOM ; + + d->kernel->arch->headroom = bzimage_headroom(bootstrap_map(d-Quoted 1 line
kernel),
Quoted 8 lines
xen/common/domain-builder/core.c
Quoted 5 lines
xen/common/domain-builder/core.c new file +5
-0+ d->kernel->size); +bootstrap_map (NULL ); + +if ( d->kernel->string.len ) + d->kernel->string.kind =BOOTSTR_CMDLINE ; }Forgive me if I'm incorrect, but I believe there is an issue with this fallback logic for the case where no FDT was provided.
IIUC, the issue at hand has to deal with patch #15.
Quoted 10 lines
If
dom0_memis not supplied to the xen cmd line, thend->meminfois never initialized. (Seedom0_compute_nr_pages/dom0_build.c:335)
This was giving me trouble becausebd->meminfo.mem_max.nr_pageswas left at 0, effectivity clamping dom0 to 0 pages of ram.I'm not sure what the best solution is but one (easy) possibility is just initializing meminfo to the dom0 defaults near the end of this function:
d->meminfo.mem_size = dom0_size; d->meminfo.mem_min = dom0_min_size; d->meminfo.mem_max = dom0_max_size;
I believe the correct fix is to this hunk,
@@ -416,7 +379,12 @@ unsigned long __init dom0_compute_nr_pages(
}
}- d->max_pages = min_t(unsigned long, max_pages, UINT_MAX); + /* Clamp according to min/max limits and available memory (final). */ + nr_pages = max(nr_pages, min_pages); + nr_pages = min(nr_pages, max_pages); + nr_pages = min(nr_pages, avail);
+
+ bd->domain->max_pages = min_t(unsigned long, max_pages, UINT_MAX);
Before that last line, there should be a clamp up of max_pages, e.g.
nr_pages = max(nr_pages, min_pages);
nr_pages = min(nr_pages, max_pages);
nr_pages = min(nr_pages, avail);
max_pages = max(nr_pages, max_pages);
bd->domain->max_pages = min_t(unsigned long, max_pages, UINT_MAX);v/r,
dps
RE: [PATCH v1 10/18] x86: introduce the domain builder
Quoted 55 lines
-----Original Message-----
From: Daniel P. Smith <[email protected]>Quoted 51 lines · Jackson
Hi Daniel,
Quoted 7 lines
-----Original Message-----
Subject: [PATCH v1 10/18] x86: introduce the domain builderThis commit introduces the domain builder configuration FDT parser along with the domain builder core for domain creation. To enable domain builder to be a cross architecture internal API, a new arch domain creation call
is
Quoted 40 lines
introduced for use by the domain builder.
xen/common/domain-builder/core.c new file +23
-0+void __init builder_init (struct boot_info *info) { +struct boot_domain *d =NULL ; + + info->builder = &builder; + +if (IS_ENABLED (CONFIG_BUILDER_FDT ) ) + { + } + +/* +* No FDT config support or an FDT wasn't present, do an initial +* domain construction +*/ +printk ("Domain Builder: falling back to initial domain build\n" ); + info->builder->nr_doms =1 ; + d = &info->builder->domains[0 ]; + + d->mode = opt_dom0_pvh ?0 :BUILD_MODE_PARAVIRTUALIZED ; + + d->kernel = &info->mods[0 ]; + d->kernel->kind =BOOTMOD_KERNEL ; + + d->permissions = BUILD_PERMISSION_CONTROL |
BUILD_PERMISSION_HARDWARE;
+d->functions=BUILD_FUNCTION_CONSOLE|BUILD_FUNCTION_XENSTORE|xen/common/domain-builder/core.c new file +3
-0+BUILD_FUNCTION_INITIAL_DOM ; + + d->kernel->arch->headroom = bzimage_headroom(bootstrap_map(d-Quoted 1 line
kernel),
xen/common/domain-builder/core.c new file +5
-0+ d->kernel->size); +bootstrap_map (NULL ); + +if ( d->kernel->string.len ) + d->kernel->string.kind =BOOTSTR_CMDLINE ; }
Quoted 10 lines
Quoted 2 lines
Forgive me if I'm incorrect, but I believe there is an issue with this fallback logic for the case where no FDT was provided.
IIUC, the issue at hand has to deal with patch #15.
Quoted 5 lines
If
dom0_memis not supplied to the xen cmd line, thend->meminfois never initialized. (Seedom0_compute_nr_pages/dom0_build.c:335) This was giving me trouble becausebd->meminfo.mem_max.nr_pageswas left at 0, effectivity clamping dom0 to 0 pages of ram.
I realize I never shared the exact panic message I was experiencing. Sorry about that. It's "Domain 0 allocation is too small for kernel image" on xen/arch/x86/pv/domain_builder.c:534
I think you should be able to consistently reproduce what I'm seeing as long as these two conditions are met:
- the dom0_mem cmdline option is _not_ set
- no domain builder device tree is passed to xen (the fallback case I identified above)
Quoted 30 lines
Quoted 5 lines
I'm not sure what the best solution is but one (easy) possibility is just initializing meminfo to the dom0 defaults near the end of this function:
d->meminfo.mem_size = dom0_size; d->meminfo.mem_min = dom0_min_size; d->meminfo.mem_max = dom0_max_size;I believe the correct fix is to this hunk,
(quoted diff) +6
-1@@ -416,7 +379,12 @@ unsigned long __init dom0_compute_nr_pages( } } - d->max_pages = min_t(unsigned long, max_pages, UINT_MAX); + /* Clamp according to min/max limits and available memory (final). */ + nr_pages = max(nr_pages, min_pages); + nr_pages = min(nr_pages, max_pages); + nr_pages = min(nr_pages, avail); + + bd->domain->max_pages = min_t(unsigned long, max_pages, UINT_MAX);Before that last line, there should be a clamp up of
max_pages, e.g.nr_pages = max(nr_pages, min_pages); nr_pages = min(nr_pages, max_pages); nr_pages = min(nr_pages, avail); max_pages = max(nr_pages, max_pages); bd->domain->max_pages = min_t(unsigned long, max_pages, UINT_MAX);
Quoted 2 lines
v/r,
dps
I don't believe this resolves my issue.
If max_pages is 0 before these 5 lines, then the second line will still clamp nr_pages to 0 and the panic on line 534 will be hit.
Before patch 15, this max limit came directly from dom0_max_size, which has a default value of { .nr_pages = LONG_MAX }, so no clamping will occur unless overridden by the cmd line.
After patch 15, bd->meminfo.mem_max is used as the max limit. (unless overridden by the cmdline)
I'm assuming it will eventually be specified in the device tree, but for now, the max limit just set to equal to the size (xen/common/domain-builder/fdt.c:155) so no down-clamping will occur.
The only exception is the initial domain construction fallback. In this case, there is no device tree and bd->meminfo is never initialized.
If bd->meminfo.mem_size is zero, the code will try to compute a reasonable default for nr_pages, but there is no such logic max_pages. It remains 0, and clamps nr_pages to zero.
Does this help clarify?
The core issue is that without a device tree or command line option to specify the max limit, the max limit is left uninitialized, which clamps dom0's memory to 0. I think it should be initialized to LONG_MAX in that case, like it was before this patch set.
Thanks,
Jackson
Re: [PATCH v1 10/18] x86: introduce the domain builder
Quoted 60 lines · Jackson
Quoted 60 lines
-----Original Message-----
From: Daniel P. Smith <[email protected]>Quoted 45 lines · Jackson
Hi Daniel,
Quoted 7 lines
-----Original Message-----
Subject: [PATCH v1 10/18] x86: introduce the domain builderThis commit introduces the domain builder configuration FDT parser along with the domain builder core for domain creation. To enable domain builder to be a cross architecture internal API, a new arch domain creation call
is
Quoted 35 lines
introduced for use by the domain builder.
xen/common/domain-builder/core.c new file +23
-0+void __init builder_init (struct boot_info *info) { +struct boot_domain *d =NULL ; + + info->builder = &builder; + +if (IS_ENABLED (CONFIG_BUILDER_FDT ) ) + { + } + +/* +* No FDT config support or an FDT wasn't present, do an initial +* domain construction +*/ +printk ("Domain Builder: falling back to initial domain build\n" ); + info->builder->nr_doms =1 ; + d = &info->builder->domains[0 ]; + + d->mode = opt_dom0_pvh ?0 :BUILD_MODE_PARAVIRTUALIZED ; + + d->kernel = &info->mods[0 ]; + d->kernel->kind =BOOTMOD_KERNEL ; + + d->permissions = BUILD_PERMISSION_CONTROL |
BUILD_PERMISSION_HARDWARE;
+d->functions=BUILD_FUNCTION_CONSOLE|BUILD_FUNCTION_XENSTORE|xen/common/domain-builder/core.c new file +3
-0+BUILD_FUNCTION_INITIAL_DOM ; + + d->kernel->arch->headroom = bzimage_headroom(bootstrap_map(d-Quoted 1 line
kernel),
Quoted 8 lines
xen/common/domain-builder/core.c
Quoted 5 lines
xen/common/domain-builder/core.c new file +5
-0+ d->kernel->size); +bootstrap_map (NULL ); + +if ( d->kernel->string.len ) + d->kernel->string.kind =BOOTSTR_CMDLINE ; }Forgive me if I'm incorrect, but I believe there is an issue with this fallback logic for the case where no FDT was provided.
IIUC, the issue at hand has to deal with patch #15.
Quoted 8 lines
Quoted 5 lines
Quoted 5 lines
If
dom0_memis not supplied to the xen cmd line, thend->meminfois never initialized. (Seedom0_compute_nr_pages/dom0_build.c:335) This was giving me trouble becausebd->meminfo.mem_max.nr_pageswas left at 0, effectivity clamping dom0 to 0 pages of ram.I realize I never shared the exact panic message I was experiencing. Sorry about that. It's "Domain 0 allocation is too small for kernel image" on
xen/arch/x86/pv/domain_builder.c:534
Yep, I ran into this one before and thought I had it addressed.
Quoted 3 lines
I think you should be able to consistently reproduce what I'm seeing as long as these two conditions are met:
- thedom0_memcmdline option is _not_ set
- no domain builder device tree is passed to xen (the fallback case I identified above)
Ack
Quoted 39 lines
Quoted 30 lines
Quoted 5 lines
I'm not sure what the best solution is but one (easy) possibility is just initializing meminfo to the dom0 defaults near the end of this function:
d->meminfo.mem_size = dom0_size; d->meminfo.mem_min = dom0_min_size; d->meminfo.mem_max = dom0_max_size;I believe the correct fix is to this hunk,
(quoted diff) +6
-1@@ -416,7 +379,12 @@ unsigned long __init dom0_compute_nr_pages( } } - d->max_pages = min_t(unsigned long, max_pages, UINT_MAX); + /* Clamp according to min/max limits and available memory (final). */ + nr_pages = max(nr_pages, min_pages); + nr_pages = min(nr_pages, max_pages); + nr_pages = min(nr_pages, avail); + + bd->domain->max_pages = min_t(unsigned long, max_pages, UINT_MAX);Before that last line, there should be a clamp up of
max_pages, e.g.nr_pages = max(nr_pages, min_pages); nr_pages = min(nr_pages, max_pages); nr_pages = min(nr_pages, avail); max_pages = max(nr_pages, max_pages); bd->domain->max_pages = min_t(unsigned long, max_pages, UINT_MAX);Quoted 2 lines
v/r,
dpsI don't believe this resolves my issue.
If
max_pagesis 0 before these 5 lines, then the second line will still clampnr_pagesto 0 and the panic on line 534 will be hit.Before patch 15, this max limit came directly from
dom0_max_size, which has a default value of { .nr_pages =LONG_MAX}, so no clamping will occur unless overridden by the cmd line.
Quoted 8 lines
After patch 15,
bd->meminfo.mem_maxis used as the max limit. (unless overridden by the cmdline)
I'm assuming it will eventually be specified in the device tree, but for now, the max limit just set to equal to the size (xen/common/domain-builder/fdt.c:155) so no down-clamping will occur.The only exception is the initial domain construction fallback. In this case, there is no device tree and
bd->meminfois never initialized.
Ifbd->meminfo.mem_sizeis zero, the code will try to compute a reasonable default fornr_pages, but there is no such logicmax_pages. It remains 0, and clampsnr_pagesto zero.Does this help clarify?
The core issue is that without a device tree or command line option to specify the max limit, the max limit is left uninitialized, which clamps dom0's memory to 0. I think it should be initialized toLONG_MAXin that case, like it was before this patch set.
You are correct, my apologies. Thank you!
Quoted 2 lines
Thanks,
Jackson
Re: [PATCH v1 10/18] x86: introduce the domain builder
In reply to Daniel P. SmithQuoted 7 lines · Daniel P. Smith
This commit introduces the domain builder configuration FDT parser along with the domain builder core for domain creation. To enable domain builder to be a cross architecture internal API, a new arch domain creation call is introduced for use by the domain builder.
Signed-off-by Daniel P. Smith <[email protected]>Reviewed-by Christopher Clark <[email protected]>
Quoted 10 lines
--- xen/arch/x86/setup.c | 9 + xen/common/Makefile | 1 + xen/common/domain-builder/Makefile | 2 + xen/common/domain-builder/core.c | 96 ++++++++++ xen/common/domain-builder/fdt.c | 295 +++++++++++++++++++++++++++++ xen/common/domain-builder/fdt.h | 7 + xen/include/xen/bootinfo.h | 16 ++ xen/include/xen/domain_builder.h | 1 + 8 files changed, 427 insertions(+)
With this diffstat - why the x86: prefix in the subject?
Also note the naming inconsistency: domain-builder/ (preferred) vs domain_builder.h (adjustment would require touching earlier patches).
Quoted 10 lines
xen/arch/x86/setup.c +2
-0@@ -1,4 +1,6 @@ +#include <xen/bootdomain.h> #include <xen/bootinfo.h> +#include <xen/domain_builder.h> #include <xen/init.h> #include <xen/lib.h> #include <xen/err.h>
xen/arch/x86/setup.c static struct domain *__init create_dom0(const struct boot_info *bi) · line 836
Quoted 9 lines
xen/arch/x86/setup.c +6
-0@@ -826,6 +828,13 @@ static struct domain *__init create_dom0(const struct boot_info *bi)return d; } +void __init arch_create_dom ( +const struct boot_info *bi,struct boot_domain *bd) +{ +if (builder_is_initdom (bd) ) +create_dom0 (bi); +}
You're not removing any code in exchange - is Dom0 now being built twice? Or is the function above effectively dead code?
xen/common/Makefile extra-y := symbols-dummy.o · line 75
Quoted 7 lines
xen/common/Makefile +1
-0@@ -72,6 +72,7 @@ extra-y := symbols-dummy.o obj-$(CONFIG_COVERAGE) += coverage/ obj-y += sched/ obj-$(CONFIG_UBSAN) += ubsan/ +obj-y += domain-builder/
At least as long as all of this is still experimental I would really like to see a way to disable all of it via Kconfig.
Quoted 16 lines
xen/common/domain-builder/core.c new file +13
-0@@ -0,0 +1,96 @@ +#include <xen/bootdomain.h> +#include <xen/bootinfo.h> +#include <xen/domain_builder.h> +#include <xen/init.h> +#include <xen/types.h> + +#include <asm/bzimage.h> +#include <asm/setup.h> + +#include "fdt.h" + +static struct domain_builder __initdata builder; +
xen/common/domain-builder/core.c line 23
Quoted 10 lines
xen/common/domain-builder/core.c new file +10
-0+void __init builder_init (struct boot_info *info) +{ +struct boot_domain *d =NULL ; + + info->builder = &builder; + +if (IS_ENABLED (CONFIG_BUILDER_FDT ) ) + { +/* fdt is required to be module 0 */ + switch ( check_fdt(info, __va(info->mods[0].start)) )
Besides requiring fixed order looking inflexible to me, what guarantees there is at least one module? (Perhaps there is, but once again - without seeing where this function is being called from, how am I to judge?)
xen/common/domain-builder/core.c line 31
Quoted 8 lines
xen/common/domain-builder/core.c new file +8
-0+ { +case 0 : +printk ("Domain Builder: initialized from config\n" ); + info->builder->fdt_enabled = true; +return ; +case -EINVAL : + info->builder->fdt_enabled = false; +break ;
Aiui this is the case where no FDT is present. I'd strongly suggest to use a less common / ambiguous error code to cover that case. Maybe -ENODEV or -EOPNOTSUPP or ...
xen/common/domain-builder/core.c line 32
Quoted 1 line
xen/common/domain-builder/core.c new file +1
-0+case -ENODATA :
... -ENODATA, albeit you having that here suggests this has some other specific meaning already.
Quoted 11 lines
xen/common/domain-builder/core.c new file +11
-0+default : +panic ("%s: error occured processing DTB\n" , __func__); + } + } + +/* +* No FDT config support or an FDT wasn't present, do an initial +* domain construction +*/ +printk ("Domain Builder: falling back to initial domain build\n" ); + info->builder->nr_doms =1 ;
xen/common/domain-builder/core.c line 53
Quoted 10 lines
xen/common/domain-builder/core.c new file +10
-0+ d = &info->builder->domains[0 ]; + + d->mode = opt_dom0_pvh ?0 :BUILD_MODE_PARAVIRTUALIZED ; + + d->kernel = &info->mods[0 ]; + d->kernel->kind =BOOTMOD_KERNEL ; + + d->permissions =BUILD_PERMISSION_CONTROL |BUILD_PERMISSION_HARDWARE ; + d->functions =BUILD_FUNCTION_CONSOLE |BUILD_FUNCTION_XENSTORE | +BUILD_FUNCTION_INITIAL_DOM ;
Nit: Indentation.
xen/common/domain-builder/core.c line 56
Quoted 2 lines
xen/common/domain-builder/core.c new file +2
-0+ d->kernel->arch->headroom =bzimage_headroom (bootstrap_map (d->kernel), + d->kernel->size);
bzimage isn't an arch-agnostic concept afaict, so I don't see this function legitimately being called from here.
And nit again: Indentation. (And at least one more further down.)
Quoted 4 lines
xen/common/domain-builder/core.c new file +4
-0+bootstrap_map (NULL ); + +if ( d->kernel->string.len ) + d->kernel->string.kind =BOOTSTR_CMDLINE ;
xen/common/domain-builder/core.c line 70
Quoted 10 lines
xen/common/domain-builder/core.c new file +10
-0+} + +uint32_t __init builder_create_domains (struct boot_info *info) +{ +uint32_t build_count =0 , functions_built =0 ; +int i; + +for ( i =0 ; i < info->builder->nr_doms; i++ ) + { +struct boot_domain *d = &info->builder->domains[i];
Can variables of this type please not be named "d", but e.g. "bd"?
xen/common/domain-builder/core.c line 73
Quoted 2 lines
xen/common/domain-builder/core.c new file +2
-0+ if ( ! IS_ENABLED(CONFIG_MULTIDOM_BUILDER) && + ! builder_is_initdom(d) &&
Nit: Stray blanks after ! .
Quoted 44 lines
xen/common/domain-builder/fdt.c new file +41
-0@@ -0,0 +1,295 @@ +#include <xen/bootdomain.h> +#include <xen/bootinfo.h> +#include <xen/domain_builder.h> +#include <xen/fdt.h> +#include <xen/init.h> +#include <xen/lib.h> +#include <xen/libfdt/libfdt.h> +#include <xen/page-size.h> +#include <xen/pfn.h> +#include <xen/types.h> + +#include <asm/bzimage.h> +#include <asm/setup.h> + +#include "fdt.h" + +#define BUILDER_FDT_TARGET_UNK 0 +#define BUILDER_FDT_TARGET_X86 1 +#define BUILDER_FDT_TARGET_ARM 2 +static int __initdata target_arch =BUILDER_FDT_TARGET_UNK ; + +static struct boot_module *read_module ( +const void *fdt,int node,uint32_t address_cells,uint32_t size_cells, +struct boot_info *info) +{ +const struct fdt_property *prop; +const __be32 *cell; +struct boot_module *bm; +bootmodule_kind kind =BOOTMOD_UNKNOWN ; +int len; + +if (device_tree_node_compatible (fdt, node,"module,kernel" ) ) + kind =BOOTMOD_KERNEL ; + +if (device_tree_node_compatible (fdt, node,"module,ramdisk" ) ) + kind =BOOTMOD_RAMDISK ; + +if (device_tree_node_compatible (fdt, node,"module,microcode" ) ) + kind =BOOTMOD_UCODE ; + +if (device_tree_node_compatible (fdt, node,"module,xsm-policy" ) )
xen/common/domain-builder/fdt.c line 51
Quoted 10 lines
xen/common/domain-builder/fdt.c new file +10
-0+ kind =BOOTMOD_XSM ; + +if (device_tree_node_compatible (fdt, node,"module,config" ) ) + kind =BOOTMOD_GUEST_CONF ; + +if (device_tree_node_compatible (fdt, node,"module,index" ) ) + { +uint32_t idx; + + idx = (uint32_t )device_tree_get_u32 (fdt, node,"module-index" ,0 );
Why the cast?
xen/common/domain-builder/fdt.c line 109
Quoted 1 line
xen/common/domain-builder/fdt.c new file +1
-0+static int process_domain_node(
__init?
Quoted 106 lines
xen/common/domain-builder/fdt.c new file +106
-0+ const void *fdt, int node, const char *name, int depth, + uint32_t address_cells, uint32_t size_cells, void *data) +{ + struct boot_info *info = (struct boot_info *)data; +const struct fdt_property *prop; +struct boot_domain *domain; +int node_next, i, plen; + +if ( !info ) +return -1 ; + +if ( info->builder->nr_doms >=BUILD_MAX_BOOT_DOMAINS ) +return -1 ; + + domain = &info->builder->domains[info->builder->nr_doms]; + + domain->domid = (domid_t )device_tree_get_u32 (fdt, node,"domid" ,0 ); + domain->permissions =device_tree_get_u32 (fdt, node,"permissions" ,0 ); + domain->functions =device_tree_get_u32 (fdt, node,"functions" ,0 ); + domain->mode =device_tree_get_u32 (fdt, node,"mode" ,0 ); + + prop =fdt_get_property (fdt, node,"domain-uuid" , &plen); +if ( prop ) +for ( i=0 ; i <sizeof (domain->uuid) %sizeof (uint32_t ); i++ ) + *(domain->uuid + i) =fdt32_to_cpu ((uint32_t )prop->data[i]); + + domain->ncpus =device_tree_get_u32 (fdt, node,"cpus" ,1 ); + +if ( target_arch ==BUILDER_FDT_TARGET_X86 ) + { + prop =fdt_get_property (fdt, node,"memory" , &plen); +if ( prop ) + { +int sz =fdt32_to_cpu (prop->len); +char s[64 ]; +unsigned long val; + +if ( sz >=64 ) +panic ("node %s invalid `memory' property\n" , name); + +memcpy (s, prop->data, sz); + s[sz] ='\0' ; + val =parse_size_and_unit (s,NULL ); + + domain->meminfo.mem_size.nr_pages =PFN_UP (val); + domain->meminfo.mem_max.nr_pages =PFN_UP (val); + } +else +panic ("node %s missing `memory' property\n" , name); + } +else +panic ("%s: only x86 memory parsing supported\n" , __func__); + + prop =fdt_get_property (fdt, node,"security-id" , + &plen); +if ( prop ) + { +int sz =fdt32_to_cpu (prop->len); + sz = sz >BUILD_MAX_SECID_LEN ?BUILD_MAX_SECID_LEN : sz; +memcpy (domain->secid, prop->data, sz); + } + +for ( node_next =fdt_first_subnode (fdt, node); + node_next >0 ; + node_next =fdt_next_subnode (fdt, node_next)) + { +struct boot_module *bm =read_module (fdt, node_next, address_cells, + size_cells, info); + +switch ( bm->kind ) + { +case BOOTMOD_KERNEL : +/* kernel was already found */ +if ( domain->kernel !=NULL ) +continue ; + + bm->arch->headroom =bzimage_headroom (bootstrap_map (bm), bm->size); +bootstrap_map (NULL ); + +if ( bm->string.len ) + bm->string.kind =BOOTSTR_CMDLINE ; +else + { + prop =fdt_get_property (fdt, node_next,"bootargs" , &plen); +if ( prop ) + { +int size =fdt32_to_cpu (prop->len); + size = size >BOOTMOD_MAX_STRING ? +BOOTMOD_MAX_STRING : size; +memcpy (bm->string.bytes, prop->data, size); + bm->string.kind =BOOTSTR_CMDLINE ; + } + } + + domain->kernel = bm; + +break ; +case BOOTMOD_RAMDISK : +/* ramdisk was already found */ +if ( domain->ramdisk !=NULL ) +continue ; + + domain->ramdisk = bm; + +break ; +case BOOTMOD_GUEST_CONF :
xen/common/domain-builder/fdt.c line 225
Quoted 10 lines
xen/common/domain-builder/fdt.c new file +10
-0+/* guest config was already found */ +if ( domain->configs[BUILD_DOM_CONF_IDX ] !=NULL ) +continue ; + + domain->configs[BUILD_DOM_CONF_IDX ] = bm; + +break ; +default : +continue ; + }
For larger switch() statements please have blank lines between non-fall- through case blocks.
Quoted 23 lines
xen/common/domain-builder/fdt.c new file +23
-0+/* check_fdt +* Attempts to initialize hyperlaunch config +* +* Returns: +* -EINVAL: Not a valid DTB +* -ENODATA: Valid DTB but not a valid hyperlaunch device tree +* 0: Valid hyperlaunch device tree +*/ +int __init check_fdt (struct boot_info *info,void *fdt) +{ +int hv_node, ret; + + ret =fdt_check_header (fdt); +if ( ret <0 ) +return -EINVAL ; + + hv_node =fdt_path_offset (fdt,"/chosen/hypervisor" ); +if ( hv_node <0 ) +return -ENODATA ; + +if ( !device_tree_node_compatible (fdt, hv_node,"hypervisor,xen" ) ) +return -EINVAL ; +
xen/common/domain-builder/fdt.c line 288
Quoted 10 lines
xen/common/domain-builder/fdt.c new file +10
-0+if (IS_ENABLED (CONFIG_X86 ) && +device_tree_node_compatible (fdt, hv_node,"xen,x86" ) ) + target_arch =BUILDER_FDT_TARGET_X86 ; +else if (IS_ENABLED (CONFIG_ARM ) && +device_tree_node_compatible (fdt, hv_node,"xen,arm" ) ) + target_arch =BUILDER_FDT_TARGET_ARM ; + +if ( target_arch !=BUILDER_FDT_TARGET_X86 && + target_arch !=BUILDER_FDT_TARGET_ARM ) +return -EINVAL ;
So you'd happily accept BUILDER_FDT_TARGET_ARM on x86 or BUILDER_FDT_TARGET_X86 on Arm? And there's no distinction between Arm32 and Arm64?
xen/common/domain-builder/fdt.h line 7
Quoted 10 lines
xen/common/domain-builder/fdt.h new file +7
-0@@ -0,0 +1,7 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +#ifndef COMMON_BUILDER_FDT_H +#define COMMON_BUILDER_FDT_H + +int __init check_fdt (struct boot_info *info,void *fdt); +#endif
Nit: Please put another blank line before #endif.
Jan
PATCH 11/18 x86: initial conversion to domain builder
In reply to Daniel P. SmithThis commit is the first step in adopting domain builder. It goes through the dom0 creation and construction functions, converting them over to consume struct boot_domaain and changes the startup sequence to use the domain builder to create and construct dom0.
--- xen/arch/x86/dom0_build.c | 30 +++---- xen/arch/x86/hvm/dom0_build.c | 10 +-- xen/arch/x86/include/asm/dom0_build.h | 8 +- xen/arch/x86/include/asm/setup.h | 5 +- xen/arch/x86/pv/dom0_build.c | 39 ++++----- xen/arch/x86/setup.c | 114 +++++++++++++++----------- 6 files changed, 109 insertions(+), 97 deletions(-)
xen/arch/x86/dom0_build.c +16 -14
@@ -6,6 +6,7 @@#include <xen/bootdomain.h> #include <xen/bootinfo.h> +#include <xen/domain_builder.h> #include <xen/init.h> #include <xen/iocap.h> #include <xen/libelf.h>
@@ -556,31 +557,32 @@ int __init dom0_setup_permissions(struct domain *d)return rc; } -int __init construct_dom0 ( -struct domain *d,const struct boot_module *image, -struct boot_module *initrd,char *cmdline) +int __init construct_domain (struct boot_domain *bd) { -int rc; +int rc =0 ;/* Sanity! */ -BUG_ON (!pv_shim && d->domain_id !=0 ); -BUG_ON (d->vcpu[0 ] ==NULL ); -BUG_ON (d->vcpu[0 ]->is_initialised); +BUG_ON (!pv_shim && bd->domid !=0 ); +BUG_ON (bd->domain->vcpu[0 ] ==NULL ); +BUG_ON (bd->domain->vcpu[0 ]->is_initialised);process_pending_softirqs (); -if (is_hvm_domain (d) ) - rc =dom0_construct_pvh (d, image, initrd, cmdline); -else if (is_pv_domain (d) ) - rc =dom0_construct_pv (d, image, initrd, cmdline); -else -panic ("Cannot construct Dom0. No guest interface available\n" ); +if (builder_is_initdom (bd) ) + { +if (is_hvm_domain (bd->domain) ) + rc =dom0_construct_pvh (bd); +else if (is_pv_domain (bd->domain) ) + rc =dom0_construct_pv (bd); +else +panic ("Cannot construct Dom0. No guest interface available\n" ); + }if ( rc )return rc;/* Sanity! */ -BUG_ON (!d->vcpu[0 ]->is_initialised); +BUG_ON (!bd->domain->vcpu[0 ]->is_initialised);return 0 ; }
xen/arch/x86/hvm/dom0_build.c +5 -5
@@ -19,6 +19,7 @@*/ #include <xen/acpi.h> +#include <xen/bootdomain.h> #include <xen/bootinfo.h> #include <xen/init.h> #include <xen/libelf.h>
@@ -1217,10 +1218,9 @@ static void __hwdom_init pvh_setup_mmcfg(struct domain *d) } } -int __init dom0_construct_pvh ( -struct domain *d,const struct boot_module *image, -struct boot_module *initrd,char *cmdline) +int __init dom0_construct_pvh (struct boot_domain *bd) { +struct domain *d = bd->domain;paddr_t entry, start_info;int rc;
@@ -1249,8 +1249,8 @@ int __init dom0_construct_pvh(return rc; } - rc =pvh_load_kernel (d, image, initrd,bootstrap_map (image), - cmdline, &entry, &start_info); + rc =pvh_load_kernel (d, bd->kernel, bd->ramdisk,bootstrap_map (bd->kernel), + bd->kernel->string.bytes, &entry, &start_info);if ( rc ) {printk ("Failed to load Dom0 kernel\n" );
xen/arch/x86/include/asm/dom0_build.h +2 -6
@@ -14,13 +14,9 @@ unsigned long dom0_compute_nr_pages(struct domain *d, unsigned long initrd_len);int dom0_setup_permissions (struct domain *d); -int __init dom0_construct_pv ( -struct domain *d,const struct boot_module *image, -struct boot_module *initrd,char *cmdline); +int dom0_construct_pv (struct boot_domain *bd); -int __init dom0_construct_pvh ( -struct domain *d,const struct boot_module *image, -struct boot_module *initrd,char *cmdline); +int dom0_construct_pvh (struct boot_domain *bd);unsigned long dom0_paging_pages (const struct domain *d,unsigned long nr_pages);
xen/arch/x86/include/asm/setup.h +2 -3
@@ -33,9 +33,8 @@ void vesa_init(void);static inline void vesa_init (void ) {};#endif -int construct_dom0 ( -struct domain *d,const struct boot_module *image, -struct boot_module *initrd,char *cmdline); +int construct_domain (struct boot_domain *bd); +void setup_io_bitmap (struct domain *d);unsigned long initial_images_nrpages (nodeid_t node);
xen/arch/x86/pv/dom0_build.c +20 -19
@@ -4,6 +4,7 @@* Copyright (c) 2002-2005, K A Fraser */ +#include <xen/bootdomain.h> #include <xen/bootinfo.h> #include <xen/console.h> #include <xen/domain.h>
@@ -295,9 +296,7 @@ static struct page_info * __init alloc_chunk(struct domain *d,return page; } -int __init dom0_construct_pv ( -struct domain *d,const struct boot_module *image, -struct boot_module *initrd,char *cmdline) +int __init dom0_construct_pv (struct boot_domain *bd) {int i, rc, order, machine;bool compatible, compat;
@@ -311,11 +310,12 @@ int __init dom0_construct_pv(unsigned long count;struct page_info *page =NULL ;start_info_t *si; +struct domain *d = bd->domain;struct vcpu *v = d->vcpu[0 ]; -void *image_base =bootstrap_map (image); -unsigned long image_len = image->size; -void *image_start = image_base + image->arch->headroom; -unsigned long initrd_len = initrd ? initrd->size :0 ; +void *image_base =bootstrap_map (bd->kernel); +unsigned long image_len = bd->kernel->size; +void *image_start = image_base + bd->kernel->arch->headroom; +unsigned long initrd_len = bd->ramdisk ? bd->ramdisk->size :0 ;l4_pgentry_t *l4tab =NULL , *l4start =NULL ;l3_pgentry_t *l3tab =NULL , *l3start =NULL ;l2_pgentry_t *l2tab =NULL , *l2start =NULL ;
@@ -355,7 +355,7 @@ int __init dom0_construct_pv( d->max_pages = ~0U ;if ( (rc = -bzimage_parse (image_base, &image_start, image->arch->headroom, +bzimage_parse (image_base, &image_start, bd->kernel->arch->headroom, &image_len)) !=0 )return rc;
@@ -545,7 +545,7 @@ int __init dom0_construct_pv( initrd_pfn = vinitrd_start ? (vinitrd_start - v_start) >>PAGE_SHIFT :domain_tot_pages (d); - initrd_mfn = mfn =mfn_x (initrd->mfn); + initrd_mfn = mfn =mfn_x (bd->ramdisk->mfn); count =PFN_UP (initrd_len);if ( d->arch.physaddr_bitsize && ((mfn + count -1 ) >> (d->arch.physaddr_bitsize -PAGE_SHIFT )) )
@@ -560,13 +560,13 @@ int __init dom0_construct_pv(free_domheap_pages (page, order); page +=1UL << order; } -memcpy (page_to_virt (page),maddr_to_virt (initrd->start), +memcpy (page_to_virt (page),maddr_to_virt (bd->ramdisk->start), initrd_len); - mpt_alloc = initrd->start; + mpt_alloc = bd->ramdisk->start;init_domheap_pages (mpt_alloc, mpt_alloc +PAGE_ALIGN (initrd_len)); -bootmodule_update_mfn (initrd,page_to_mfn (page)); - initrd_mfn =mfn_x (initrd->mfn); +bootmodule_update_mfn (bd->ramdisk,page_to_mfn (page)); + initrd_mfn =mfn_x (bd->ramdisk->mfn); }else {
@@ -574,7 +574,7 @@ int __init dom0_construct_pv(if (assign_pages (mfn_to_page (_mfn (mfn++)),1 , d,0 ) )BUG (); } - initrd->size =0 ; + bd->ramdisk->size =0 ; } printk("PHYSICAL MEMORY ARRANGEMENT:\n"
@@ -583,9 +583,9 @@ int __init dom0_construct_pv(if (domain_tot_pages (d) < nr_pages )printk (" (%lu pages to be allocated)" , nr_pages -domain_tot_pages (d)); -if ( initrd ) +if ( bd->ramdisk ) { - mpt_alloc = initrd->start; + mpt_alloc = bd->ramdisk->start;printk ("\n Init. ramdisk: %" PRIpaddr"->%" PRIpaddr, mpt_alloc, mpt_alloc + initrd_len); }
@@ -806,7 +806,7 @@ int __init dom0_construct_pv(if ( pfn >= initrd_pfn ) {if ( pfn < initrd_pfn +PFN_UP (initrd_len) ) - mfn =mfn_x (initrd->mfn) + (pfn - initrd_pfn); + mfn =mfn_x (bd->ramdisk->mfn) + (pfn - initrd_pfn);else mfn -=PFN_UP (initrd_len); }
@@ -866,8 +866,9 @@ int __init dom0_construct_pv( }memset (si->cmd_line,0 ,sizeof (si->cmd_line)); -if ( cmdline !=NULL ) -strlcpy ((char *)si->cmd_line, cmdline,sizeof (si->cmd_line)); +if (strlen (bd->kernel->string.bytes) >0 ) +strlcpy ((char *)si->cmd_line, bd->kernel->string.bytes, +sizeof (si->cmd_line));#ifdef CONFIG_VIDEO if ( !pv_shim &&fill_console_start_info ((void *)(si +1 )) )
xen/arch/x86/setup.c +64 -50
207 lines not on this page · raw message
Signature · 1 line
2.20.1
Re: [PATCH v1 11/18] x86: initial conversion to domain builder
Quoted 55 lines · Daniel P. Smith
This commit is the first step in adopting domain builder. It goes through the dom0 creation and construction functions, converting them over to consume struct
boot_domaainand changes the startup sequence to use the domain builder to create and construct dom0.Signed-off-by Daniel P. Smith <[email protected]>Reviewed-by Christopher Clark <[email protected]>--- xen/arch/x86/dom0_build.c | 30 +++---- xen/arch/x86/hvm/dom0_build.c | 10 +-- xen/arch/x86/include/asm/dom0_build.h | 8 +- xen/arch/x86/include/asm/setup.h | 5 +- xen/arch/x86/pv/dom0_build.c | 39 ++++----- xen/arch/x86/setup.c | 114 +++++++++++++++----------- 6 files changed, 109 insertions(+), 97 deletions(-)xen/arch/x86/dom0_build.c +6
-12@@ -6,6 +6,7 @@#include <xen/bootdomain.h> #include <xen/bootinfo.h> +#include <xen/domain_builder.h> #include <xen/init.h> #include <xen/iocap.h> #include <xen/libelf.h> @@ -556,31 +557,32 @@ int __init dom0_setup_permissions(struct domain *d)return rc; } -int __init construct_dom0 ( -struct domain *d,const struct boot_module *image, -struct boot_module *initrd,char *cmdline) +int __init construct_domain (struct boot_domain *bd) { -int rc; +int rc =0 ;/* Sanity! */ -BUG_ON (!pv_shim && d->domain_id !=0 ); -BUG_ON (d->vcpu[0 ] ==NULL ); -BUG_ON (d->vcpu[0 ]->is_initialised); +BUG_ON (!pv_shim && bd->domid !=0 ); +BUG_ON (bd->domain->vcpu[0 ] ==NULL ); +BUG_ON (bd->domain->vcpu[0 ]->is_initialised);process_pending_softirqs (); -if (is_hvm_domain (d) ) - rc =dom0_construct_pvh (d, image, initrd, cmdline); -else if (is_pv_domain (d) ) - rc =dom0_construct_pv (d, image, initrd, cmdline); -else
xen/arch/x86/dom0_build.c int __init dom0_setup_permissions(struct domain *d) · line 579
Quoted 10 lines
xen/arch/x86/dom0_build.c +9
-1-panic ("Cannot construct Dom0. No guest interface available\n" ); +if (builder_is_initdom (bd) ) + { +if (is_hvm_domain (bd->domain) ) + rc =dom0_construct_pvh (bd); +else if (is_pv_domain (bd->domain) ) + rc =dom0_construct_pv (bd); +else +panic ("Cannot construct Dom0. No guest interface available\n" ); + }
Isn't there an "else" missing here, even if just to ASSERT_UNREACHABLE() and set rc to, say, -EOPNOTSUPP?
Quoted 54 lines
xen/arch/x86/dom0_build.c +11
-10@@ -311,11 +310,12 @@ int __init dom0_construct_pv(unsigned long count;struct page_info *page =NULL ;start_info_t *si; +struct domain *d = bd->domain;struct vcpu *v = d->vcpu[0 ]; -void *image_base =bootstrap_map (image); -unsigned long image_len = image->size; -void *image_start = image_base + image->arch->headroom; -unsigned long initrd_len = initrd ? initrd->size :0 ; +void *image_base =bootstrap_map (bd->kernel); +unsigned long image_len = bd->kernel->size; +void *image_start = image_base + bd->kernel->arch->headroom; +unsigned long initrd_len = bd->ramdisk ? bd->ramdisk->size :0 ;l4_pgentry_t *l4tab =NULL , *l4start =NULL ;l3_pgentry_t *l3tab =NULL , *l3start =NULL ;l2_pgentry_t *l2tab =NULL , *l2start =NULL ;@@ -355,7 +355,7 @@ int __init dom0_construct_pv( d->max_pages = ~0U ;if ( (rc = -bzimage_parse (image_base, &image_start, image->arch->headroom, +bzimage_parse (image_base, &image_start, bd->kernel->arch->headroom, &image_len)) !=0 )return rc;@@ -545,7 +545,7 @@ int __init dom0_construct_pv( initrd_pfn = vinitrd_start ? (vinitrd_start - v_start) >>PAGE_SHIFT :domain_tot_pages (d); - initrd_mfn = mfn =mfn_x (initrd->mfn); + initrd_mfn = mfn =mfn_x (bd->ramdisk->mfn); count =PFN_UP (initrd_len);if ( d->arch.physaddr_bitsize && ((mfn + count -1 ) >> (d->arch.physaddr_bitsize -PAGE_SHIFT )) )@@ -560,13 +560,13 @@ int __init dom0_construct_pv(free_domheap_pages (page, order); page +=1UL << order; } -memcpy (page_to_virt (page),maddr_to_virt (initrd->start), +memcpy (page_to_virt (page),maddr_to_virt (bd->ramdisk->start), initrd_len); - mpt_alloc = initrd->start; + mpt_alloc = bd->ramdisk->start;init_domheap_pages (mpt_alloc, mpt_alloc +PAGE_ALIGN (initrd_len)); -bootmodule_update_mfn (initrd,page_to_mfn (page)); - initrd_mfn =mfn_x (initrd->mfn); +bootmodule_update_mfn (bd->ramdisk,page_to_mfn (page)); + initrd_mfn =mfn_x (bd->ramdisk->mfn); }else {
xen/arch/x86/dom0_build.c int __init dom0_construct_pv( · line 577
Quoted 5 lines
xen/arch/x86/dom0_build.c +1
-1@@ -574,7 +574,7 @@ int __init dom0_construct_pv(if (assign_pages (mfn_to_page (_mfn (mfn++)),1 , d,0 ) )BUG (); } - initrd->size =0 ; + bd->ramdisk->size =0 ;
From an abstract pov: Is it legitimate to alter values under bd-> ? I would have assumed bd and everything under it is r/o at this point (and could/should be const-qualified).
Quoted 11 lines
xen/arch/x86/dom0_build.c +7
-0@@ -272,6 +271,24 @@ static int __init cf_check parse_acpi_param(const char *s) }custom_param ("acpi" , parse_acpi_param); +void __init arch_builder_apply_cmdline ( +struct boot_info *info,struct boot_domain *bd) +{ +if ( skip_ioapic_setup && !strstr (bd->kernel->string.bytes,"noapic" ) ) +strlcat (bd->kernel->string.bytes," noapic" ,MAX_GUEST_CMDLINE ); +if ( (strlen (acpi_param) ==0 ) && acpi_disabled ) + {
xen/arch/x86/dom0_build.c static int __init cf_check parse_acpi_param(const char *s) · line 290
Quoted 10 lines
xen/arch/x86/dom0_build.c +10
-0+printk ("ACPI is disabled, notifying Domain 0 (acpi=off)\n" ); +strlcpy (acpi_param,"off" ,sizeof (acpi_param)); + } +if ( (strlen (acpi_param) !=0 ) && + !strstr (bd->kernel->string.bytes,"acpi=" ) ) + { +strlcat (bd->kernel->string.bytes," acpi=" ,MAX_GUEST_CMDLINE ); +strlcat (bd->kernel->string.bytes, acpi_param,MAX_GUEST_CMDLINE ); + } +}
This duplicates existing code rather than replacing it. How do you envision the two to remain in sync? Such things should live in exactly one place imo.
xen/arch/x86/dom0_build.c static struct domain *__init create_dom0(const struct boot_info *bi) · line 835
Quoted 7 lines
xen/arch/x86/dom0_build.c +1
-1@@ -816,7 +831,7 @@ static struct domain *__init create_dom0(const struct boot_info *bi)write_cr4 (read_cr4 () & ~X86_CR4_SMAP ); } -if (construct_dom0 (d, image, initrd, cmdline) !=0 ) +if (construct_domain (bd) !=0 )panic ("Could not construct domain 0\n" );
You leave the log message text in place here, but ...
Quoted 30 lines
xen/arch/x86/dom0_build.c +12
-12@@ -1905,22 +1912,29 @@ void __init noreturn __start_xen(unsigned long bi_p) cpu_has_nx ?XENLOG_INFO :XENLOG_WARNING "Warning: " , cpu_has_nx ? "" : "not "); - initrdidx =bootmodule_next_idx_by_kind (boot_info,BOOTMOD_UNKNOWN ,0 ); -if ( initrdidx < boot_info->nr_mods ) - boot_info->mods[initrdidx].kind =BOOTMOD_RAMDISK ; - -if (bootmodule_count_by_kind (boot_info,BOOTMOD_UNKNOWN ) >1 ) -printk (XENLOG_WARNING -"Multiple initrd candidates, picking module #%u\n" , - initrdidx); -/* -* We're going to setup domain0 using the module(s) that we stashed safely -* above our heap. The second module, if present, is an initrd ramdisk. +* Boot description not provided, check to see if there are any remaining +* boot modules, the first one found will be provided as the ramdisk. */ - dom0 =create_dom0 (boot_info); +if ( ! boot_info->builder->fdt_enabled ) + { + initrdidx =bootmodule_next_idx_by_kind (boot_info,BOOTMOD_UNKNOWN ,0 ); +if ( initrdidx < boot_info->nr_mods ) + { + boot_info->builder->domains[0 ].ramdisk = &boot_info->mods[initrdidx]; + boot_info->mods[initrdidx].kind =BOOTMOD_RAMDISK ; + } +if (bootmodule_count_by_kind (boot_info,BOOTMOD_UNKNOWN ) >1 ) + printk(XENLOG_WARNING
xen/arch/x86/dom0_build.c void __init noreturn __start_xen(unsigned long bi_p) · line 1937
Quoted 10 lines
xen/arch/x86/dom0_build.c +8
-1+"Multiple initrd candidates, picking module #%u\n" , + initrdidx); + } + +builder_create_domains (boot_info); + + dom0 =builder_get_hwdom (boot_info);if ( !dom0 ) -panic ("Could not set up DOM0 guest OS\n" ); +panic ("No hardware domain was built\n" );
... you change it here, neglecting that in the late-hwdom case what is being built here is only Dom0, not hwdom. This may also affect the name of the function that you call.
Jan
PATCH 12/18 x86: convert dom0 creation to domain builder
In reply to Daniel P. SmithThis commit begins the transtion over to domain builder by coverting the dom0 creation logic into a generalized domain creation logic.
--- xen/arch/x86/Makefile | 1 + xen/arch/x86/domain_builder.c | 128 +++++++++++++++++++++++++++++++ xen/arch/x86/include/asm/setup.h | 1 + xen/arch/x86/setup.c | 109 +++----------------------- 4 files changed, 141 insertions(+), 98 deletions(-) create mode 100644 xen/arch/x86/domain_builder.c
xen/arch/x86/Makefile +1 -0
@@ -27,6 +27,7 @@ obj-y += desc.o obj-bin-y += dmi_scan.init.o obj-y += domain.o obj-bin-y += dom0_build.init.o +obj-bin-y += domain_builder.init.o obj-y += domain_page.o obj-y += e820.o obj-y += emul-i8254.o
xen/arch/x86/domain_builder.c new file +128 -0
128 lines not on this page · raw message
xen/arch/x86/include/asm/setup.h +1 -0
@@ -33,6 +33,7 @@ void vesa_init(void);static inline void vesa_init (void ) {};#endif +void apply_xen_cmdline (char *cmdline);int construct_domain (struct boot_domain *bd);void setup_io_bitmap (struct domain *d);
xen/arch/x86/setup.c +11 -98
126 lines not on this page · raw message
Signature · 1 line
2.20.1
Re: [PATCH v1 12/18] x86: convert dom0 creation to domain builder
Quoted 6 lines · Daniel P. Smith
xen/arch/x86/domain_builder.c new file +3
-0@@ -0,0 +1,128 @@ +#include <xen/bootdomain.h> +#include <xen/bootinfo.h> +#include <xen/domain.h>
xen/arch/x86/domain_builder.c line 13
Quoted 10 lines
xen/arch/x86/domain_builder.c new file +10
-0+#include <xen/domain_builder.h> +#include <xen/err.h> +#include <xen/grant_table.h> +#include <xen/iommu.h> +#include <xen/sched.h> + +#include <asm/pv/shim.h> +#include <asm/setup.h> + +extern unsigned long cr4_pv32_mask;
Such declarations need to go in a header which both producer and consumer(s) include.
xen/arch/x86/domain_builder.c line 23
Quoted 9 lines
xen/arch/x86/domain_builder.c new file +9
-0+static unsigned int __init dom_max_vcpus (struct boot_domain *bd) +{ +unsigned int limit; + +if (builder_is_initdom (bd) ) +return dom0_max_vcpus (); + + limit = bd->mode &BUILD_MODE_PARAVIRTUALIZED ? +MAX_VIRT_CPUS :HVM_MAX_VCPUS ;
Nit: Indentation.
xen/arch/x86/domain_builder.c line 28
Quoted 4 lines
xen/arch/x86/domain_builder.c new file +4
-0+if ( bd->ncpus > limit ) +return limit; +else +return bd->ncpus;
return min(bd->ncpus, limit);
Quoted 8 lines
xen/arch/x86/domain_builder.c new file +8
-0+} + +void __init arch_create_dom ( +const struct boot_info *bi,struct boot_domain *bd) +{ +struct xen_domctl_createdomain dom_cfg = { + .flags =IS_ENABLED (CONFIG_TBOOT ) ? XEN_DOMCTL_CDF_s3_integrity :0 , + .max_evtchn_port =-1 ,
xen/arch/x86/domain_builder.c line 46
Quoted 10 lines
xen/arch/x86/domain_builder.c new file +10
-0+ .max_grant_frames = -1, + .max_maptrack_frames = -1, + .grant_opts = XEN_DOMCTL_GRANT_version(opt_gnttab_max_version), + .max_vcpus = dom_max_vcpus(bd), + .arch = { + .misc_flags = bd->functions & BUILD_FUNCTION_INITIAL_DOM && + opt_dom0_msr_relaxed ? XEN_X86_MSR_RELAXED : 0, + }, + }; +unsigned int is_privileged =0 ;
Either this is bool and retains its name, or it remains unsigned int and changes its name (to e.g. "cdf").
xen/arch/x86/domain_builder.c line 50
Quoted 4 lines
xen/arch/x86/domain_builder.c new file +4
-0+char *cmdline; + +if ( bd->kernel ==NULL ) +panic ("Error creating d%uv0\n" , bd->domid);
This gives too little information and, by mentioning vCPU 0, is imo actively misleading.
xen/arch/x86/domain_builder.c line 54
Quoted 3 lines
xen/arch/x86/domain_builder.c new file +3
-0+/* mask out PV and device model bits, if 0 then the domain is PVH */ +if ( !(bd->mode & + (BUILD_MODE_PARAVIRTUALIZED |BUILD_MODE_ENABLE_DEVICE_MODEL )) )
Shouldn't you outright reject BUILD_MODE_ENABLE_DEVICE_MODEL, since you can't fulfill that request?
xen/arch/x86/domain_builder.c line 65
Quoted 11 lines
xen/arch/x86/domain_builder.c new file +11
-0+ { + dom_cfg.flags |= (XEN_DOMCTL_CDF_hvm | + (hvm_hap_supported () ? XEN_DOMCTL_CDF_hap :0 )); + +/* +* If shadow paging is enabled for the initial domain, mask out +* HAP if it was just enabled. +*/ +if (builder_is_initdom (bd) ) +if ( opt_dom0_shadow ) + dom_cfg.flags |= ~XEN_DOMCTL_CDF_hap;
Please combine such if()s into a single one using &&. And I suppose you mean &= ? Furthermore - how would a DomU be started without using HAP when HAP is available?
xen/arch/x86/domain_builder.c line 73
Quoted 7 lines
xen/arch/x86/domain_builder.c new file +7
-0+/* TODO: review which flags should be present */ + dom_cfg.arch.emulation_flags |= +XEN_X86_EMU_LAPIC |XEN_X86_EMU_IOAPIC |XEN_X86_EMU_VPCI ; + } + +if ( iommu_enabled &&builder_is_hwdom (bd) ) + dom_cfg.flags |= XEN_DOMCTL_CDF_iommu;
Why would only hwdom get an IOMMU?
xen/arch/x86/domain_builder.c line 78
Quoted 4 lines
xen/arch/x86/domain_builder.c new file +4
-0+if ( !pv_shim &&builder_is_ctldom (bd) ) + is_privileged = CDF_privileged; + +/* Create initial domain. Not d0 for pvshim. */
Up to here I was assuming this function would deal with more than just Dom0, based on conditionals seen. What's the intention? Mixing things is at best confusing.
xen/arch/x86/domain_builder.c line 79
Quoted 1 line
xen/arch/x86/domain_builder.c new file +1
-0+ bd->domid =get_initial_domain_id ();
Higher up in the panic() invocation you did use bd->domid already.
Quoted 12 lines
xen/arch/x86/domain_builder.c new file +12
-0+ bd->domain =domain_create (bd->domid, &dom_cfg, is_privileged); +if (IS_ERR (bd->domain) ) +panic ("Error creating d%u: %ld\n" , bd->domid,PTR_ERR (bd->domain)); + +init_dom0_cpuid_policy (bd->domain); + +if (alloc_dom0_vcpu0 (bd->domain) ==NULL ) +panic ("Error creating d%uv0\n" , bd->domid); + +/* Grab the DOM0 command line. */ + cmdline = (bd->kernel->string.kind ==BOOTSTR_CMDLINE ) ? + bd->kernel->string.bytes :NULL ;
xen/arch/x86/domain_builder.c line 101
Quoted 10 lines
xen/arch/x86/domain_builder.c new file +10
-0+if ( cmdline || bi->arch->kextra ) + { +char dom_cmdline[MAX_GUEST_CMDLINE ]; + + cmdline =arch_prepare_cmdline (cmdline, bi->arch); +strlcpy (dom_cmdline, cmdline,MAX_GUEST_CMDLINE ); + +if ( bi->arch->kextra ) +/* kextra always includes exactly one leading space. */ +strlcat (dom_cmdline, bi->arch->kextra,MAX_GUEST_CMDLINE );
I don't think the comment belongs here - there's no insertion of a blank in sight anywhere.
xen/arch/x86/domain_builder.c line 105
Quoted 3 lines
xen/arch/x86/domain_builder.c new file +3
-0+apply_xen_cmdline (dom_cmdline); + +strlcpy (bd->kernel->string.bytes, dom_cmdline,MAX_GUEST_CMDLINE );
Further up using MAX_GUEST_CMDLINE is acceptable, because it's easy to see that this is the array's size. But here this isn't the case - you want to use ARRAY_SIZE() at least in this one case (ideally everywhere).
xen/arch/x86/domain_builder.c line 109
Quoted 4 lines
xen/arch/x86/domain_builder.c new file +4
-0+ } + +/* +* Temporarily clear SMAP in CR4 to allow user-accesses in construct_dom0().
dom0 here, but ...
xen/arch/x86/domain_builder.c line 119
Quoted 10 lines
xen/arch/x86/domain_builder.c new file +10
-0+* This saves a large number of corner cases interactions with +* copy_from_user(). +*/ +if ( cpu_has_smap ) + { + cr4_pv32_mask &= ~X86_CR4_SMAP ; +write_cr4 (read_cr4 () & ~X86_CR4_SMAP ); + } + +if (construct_domain (bd) !=0 )
... domain here, yet then ...
xen/arch/x86/domain_builder.c line 120
Quoted 1 line
xen/arch/x86/domain_builder.c new file +1
-0+panic ("Could not construct domain 0\n" );
... domain 0 again here.
Quoted 108 lines
xen/arch/x86/domain_builder.c new file +10
-89@@ -745,109 +746,21 @@ static unsigned int __init copy_bios_e820(struct e820entry *map, unsigned int lireturn n; } -static struct domain *__init create_dom0 ( -const struct boot_info *bi,struct boot_domain *bd) +void __init apply_xen_cmdline (char *cmdline) { -struct xen_domctl_createdomain dom0_cfg = { - .flags =IS_ENABLED (CONFIG_TBOOT ) ? XEN_DOMCTL_CDF_s3_integrity :0 , - .max_evtchn_port =-1 , - .max_grant_frames =-1 , - .max_maptrack_frames =-1 , - .grant_opts =XEN_DOMCTL_GRANT_version (opt_gnttab_max_version), - .max_vcpus =dom0_max_vcpus (), - .arch = { - .misc_flags = opt_dom0_msr_relaxed ?XEN_X86_MSR_RELAXED :0 , - }, - }; -char *cmdline; - -if ( bd->kernel ==NULL ) -panic ("Error creating d%uv0\n" , bd->domid); - -if ( opt_dom0_pvh ) - { - dom0_cfg.flags |= (XEN_DOMCTL_CDF_hvm | - ((hvm_hap_supported () && !opt_dom0_shadow) ? - XEN_DOMCTL_CDF_hap :0 )); - - dom0_cfg.arch.emulation_flags |= -XEN_X86_EMU_LAPIC |XEN_X86_EMU_IOAPIC |XEN_X86_EMU_VPCI ; - } - -if ( iommu_enabled ) - dom0_cfg.flags |= XEN_DOMCTL_CDF_iommu; - -/* Create initial domain. Not d0 for pvshim. */ - bd->domid =get_initial_domain_id (); - bd->domain =domain_create (bd->domid, &dom0_cfg, pv_shim ? -0 : CDF_privileged); -if (IS_ERR (bd->domain) ) -panic ("Error creating d%u: %ld\n" , bd->domid,PTR_ERR (bd->domain)); - -init_dom0_cpuid_policy (bd->domain); - -if (alloc_dom0_vcpu0 (bd->domain) ==NULL ) -panic ("Error creating d%uv0\n" , bd->domid); - -/* Grab the DOM0 command line. */ - cmdline = (bd->kernel->string.kind ==BOOTSTR_CMDLINE ) ? - bd->kernel->string.bytes :NULL ; -if ( cmdline || bi->arch->kextra ) - { -char dom0_cmdline[MAX_GUEST_CMDLINE ]; - - cmdline =arch_prepare_cmdline (cmdline, bi->arch); -strlcpy (dom0_cmdline, cmdline,MAX_GUEST_CMDLINE ); - -if ( bi->arch->kextra ) -/* kextra always includes exactly one leading space. */ -strlcat (dom0_cmdline, bi->arch->kextra,MAX_GUEST_CMDLINE ); - -/* Append any extra parameters. */ -if ( skip_ioapic_setup && !strstr (dom0_cmdline,"noapic" ) ) -strlcat (dom0_cmdline," noapic" ,MAX_GUEST_CMDLINE ); -if ( (strlen (acpi_param) ==0 ) && acpi_disabled ) - { -printk ("ACPI is disabled, notifying Domain 0 (acpi=off)\n" ); -strlcpy (acpi_param,"off" ,sizeof (acpi_param)); - } -if ( (strlen (acpi_param) !=0 ) && !strstr (dom0_cmdline,"acpi=" ) ) - { -strlcat (dom0_cmdline," acpi=" ,MAX_GUEST_CMDLINE ); -strlcat (dom0_cmdline, acpi_param,MAX_GUEST_CMDLINE ); - } - -strlcpy (bd->kernel->string.bytes, dom0_cmdline,MAX_GUEST_CMDLINE ); - } - -/* -* Temporarily clear SMAP in CR4 to allow user-accesses in construct_dom0(). -* This saves a large number of corner cases interactions with -* copy_from_user(). -*/ -if ( cpu_has_smap ) +if ( skip_ioapic_setup && !strstr (cmdline,"noapic" ) ) +strlcat (cmdline," noapic" ,MAX_GUEST_CMDLINE ); +if ( (strlen (acpi_param) ==0 ) && acpi_disabled ) { - cr4_pv32_mask &= ~X86_CR4_SMAP ; -write_cr4 (read_cr4 () & ~X86_CR4_SMAP ); +printk ("ACPI is disabled, notifying Domain 0 (acpi=off)\n" ); +strlcpy (acpi_param,"off" ,sizeof (acpi_param)); } - -if (construct_domain (bd) !=0 ) -panic ("Could not construct domain 0\n" ); - -if ( cpu_has_smap ) +if ( (strlen (acpi_param) !=0 ) && + !strstr (cmdline,"acpi=" ) ) { -write_cr4 (read_cr4 () |X86_CR4_SMAP ); - cr4_pv32_mask |=X86_CR4_SMAP ; +strlcat (cmdline," acpi=" ,MAX_GUEST_CMDLINE ); +strlcat (cmdline, acpi_param,MAX_GUEST_CMDLINE ); }
xen/arch/x86/domain_builder.c static unsigned int __init copy_bios_e820(struct e820entry *map, unsigned int li · line 764
Quoted 10 lines
xen/arch/x86/domain_builder.c new file +0
-9- -return bd->domain; -} - -void __init arch_create_dom ( -const struct boot_info *bi,struct boot_domain *bd) -{ -if (builder_is_initdom (bd) ) -create_dom0 (bi, bd); }
Earlier on a function was introduced to deal with this cmdline handling. And now you introduce a 2nd such function?
Jan
PATCH 13/18 x86: generalize physmap logic
In reply to Daniel P. SmithThe existing physmap code is specific to dom0. In this commit, the dom0 physmap code is generalized for any domain and functions are renamed to reflect their new general nature.
--- xen/arch/x86/include/asm/dom0_build.h | 2 +- xen/arch/x86/pv/dom0_build.c | 10 +++++----- xen/arch/x86/pv/shim.c | 4 ++-- 3 files changed, 8 insertions(+), 8 deletions(-)
xen/arch/x86/include/asm/dom0_build.h +1 -1
@@ -21,7 +21,7 @@ int dom0_construct_pvh(struct boot_domain *bd);unsigned long dom0_paging_pages (const struct domain *d,unsigned long nr_pages); -void dom0_update_physmap (bool compat,unsigned long pfn, +void dom_update_physmap (bool compat,unsigned long pfn,unsigned long mfn,unsigned long vphysmap_s);#endif /* _DOM0_BUILD_H_ */
xen/arch/x86/pv/dom0_build.c +5 -5
@@ -34,8 +34,8 @@#define L3_PROT (BASE_PROT|_PAGE_DIRTY)#define L4_PROT (BASE_PROT|_PAGE_DIRTY) -void __init dom0_update_physmap (bool compat,unsigned long pfn, -unsigned long mfn,unsigned long vphysmap_s) +void __init dom_update_physmap ( +bool compat,unsigned long pfn,unsigned long mfn,unsigned long vphysmap_s) {if ( !compat ) ((unsigned long *)vphysmap_s)[pfn] = mfn;
@@ -815,7 +815,7 @@ int __init dom0_construct_pv(struct boot_domain *bd)if ( pfn >REVERSE_START && (vinitrd_start || pfn < initrd_pfn) ) mfn = alloc_epfn - (pfn -REVERSE_START );#endif -dom0_update_physmap (compat, pfn, mfn, vphysmap_start); +dom_update_physmap (compat, pfn, mfn, vphysmap_start);if ( !(pfn &0xfffff ) )process_pending_softirqs (); }
@@ -831,7 +831,7 @@ int __init dom0_construct_pv(struct boot_domain *bd) !get_page_and_type(page, d, PGT_writable_page) )BUG (); -dom0_update_physmap (compat, pfn, mfn, vphysmap_start); +dom_update_physmap (compat, pfn, mfn, vphysmap_start); ++pfn;if ( !(pfn &0xfffff ) )process_pending_softirqs ();
@@ -851,7 +851,7 @@ int __init dom0_construct_pv(struct boot_domain *bd)#ifndef NDEBUG #define pfn (nr_pages - 1 - (pfn - (alloc_epfn - alloc_spfn)))#endif -dom0_update_physmap (compat, pfn, mfn, vphysmap_start); +dom_update_physmap (compat, pfn, mfn, vphysmap_start);#undef pfn page++; pfn++;if ( !(pfn &0xfffff ) )
xen/arch/x86/pv/shim.c +2 -2
@@ -210,7 +210,7 @@ void __init pv_shim_setup_dom(struct domain *d, l4_pgentry_t *l4start, { \share_xen_page_with_guest (mfn_to_page (_mfn (param)), d, SHARE_rw); \replace_va_mapping (d, l4start, va,_mfn (param)); \ -dom0_update_physmap (compat, \ +dom_update_physmap (compat, \PFN_DOWN ((va) - va_start), param, vphysmap); \ } \ else \
@@ -238,7 +238,7 @@ void __init pv_shim_setup_dom(struct domain *d, l4_pgentry_t *l4start, si->console.domU.mfn =mfn_x (console_mfn);share_xen_page_with_guest (mfn_to_page (console_mfn), d, SHARE_rw);replace_va_mapping (d, l4start, console_va, console_mfn); -dom0_update_physmap (compat, (console_va - va_start) >>PAGE_SHIFT , +dom_update_physmap (compat, (console_va - va_start) >>PAGE_SHIFT ,mfn_x (console_mfn), vphysmap);consoled_set_ring_addr (page); }
Signature · 1 line
2.20.1
Re: [PATCH v1 13/18] x86: generalize physmap logic
Quoted 1 line · Daniel P. Smith
The existing physmap code is specific to dom0.
I think this needs better wording. Either you name the function or you explain what piece of code you're talking about. "physmap" alone is just not meaningful enough. (Also applies to the title.)
xen/arch/x86/include/asm/dom0_build.h int dom0_construct_pvh(struct boot_domain *bd); · line 25
Quoted 9 lines
xen/arch/x86/include/asm/dom0_build.h +1
-1@@ -21,7 +21,7 @@ int dom0_construct_pvh(struct boot_domain *bd);unsigned long dom0_paging_pages (const struct domain *d,unsigned long nr_pages); -void dom0_update_physmap (bool compat,unsigned long pfn, +void dom_update_physmap (bool compat,unsigned long pfn,unsigned long mfn,unsigned long vphysmap_s);
So my initial inclination was to suggest domain_ as a name prefix, matching what we have elsewhere. But when we're already giving the thing a new name, its PV-only nature also wants expressing. Hence I'd like to suggest pv_update_physmap(). And then please fix indentation of the continuation lines here and below.
xen/arch/x86/pv/dom0_build.c line 39
Quoted 11 lines
xen/arch/x86/pv/dom0_build.c +2
-2@@ -34,8 +34,8 @@#define L3_PROT (BASE_PROT|_PAGE_DIRTY)#define L4_PROT (BASE_PROT|_PAGE_DIRTY) -void __init dom0_update_physmap (bool compat,unsigned long pfn, -unsigned long mfn,unsigned long vphysmap_s) +void __init dom_update_physmap ( +bool compat,unsigned long pfn,unsigned long mfn,unsigned long vphysmap_s) {
Personally I dislike this further change to re-flow the parameter list, as I see no particular reason for doing so.
Jan
PATCH 14/18 x86: generalize vcpu for domain building
In reply to Daniel P. SmithHere, the vcpu initialization code for dom0 creation is generalized for use for other domains.
--- xen/arch/x86/domain_builder.c | 14 +++++++++++++- xen/arch/x86/hvm/dom0_build.c | 7 ++++--- xen/arch/x86/pv/dom0_build.c | 2 +- xen/common/sched/core.c | 25 ++++++++++++++++--------- xen/include/xen/sched.h | 3 ++- 5 files changed, 36 insertions(+), 15 deletions(-)
xen/arch/x86/domain_builder.c +13 -1
@@ -28,6 +28,18 @@ static unsigned int __init dom_max_vcpus(struct boot_domain *bd)return bd->ncpus; } +struct vcpu *__init alloc_dom_vcpu0 (struct boot_domain *bd) +{ +if ( bd->functions &BUILD_FUNCTION_INITIAL_DOM ) +return alloc_dom0_vcpu0 (bd->domain); + + bd->domain->node_affinity = node_online_map; + bd->domain->auto_node_affinity = true; + +return vcpu_create (bd->domain,0 ); +} + +void __init arch_create_dom (const struct boot_info *bi,struct boot_domain *bd) {
@@ -83,7 +95,7 @@ void __init arch_create_dom(init_dom0_cpuid_policy (bd->domain); -if (alloc_dom0_vcpu0 (bd->domain) ==NULL ) +if (alloc_dom_vcpu0 (bd) ==NULL )panic ("Error creating d%uv0\n" , bd->domid);/* Grab the DOM0 command line. */
xen/arch/x86/hvm/dom0_build.c +4 -3
@@ -696,9 +696,10 @@ static int __init pvh_load_kernel(return 0 ; } -static int __init pvh_setup_cpus (struct domain *d,paddr_t entry, +static int __init pvh_setup_cpus (struct boot_domain *bd,paddr_t entry,paddr_t start_info) { +struct domain *d = bd->domain;struct vcpu *v = d->vcpu[0 ];int rc;/*
@@ -722,7 +723,7 @@ static int __init pvh_setup_cpus(struct domain *d, paddr_t entry, .cpu_regs.x86_32.tr_ar = 0x8b, }; -sched_setup_dom0_vcpus (d); +sched_setup_dom_vcpus (bd); rc =arch_set_info_hvm_guest (v, &cpu_ctx);if ( rc )
@@ -1257,7 +1258,7 @@ int __init dom0_construct_pvh(struct boot_domain *bd)return rc; } - rc =pvh_setup_cpus (d, entry, start_info); + rc =pvh_setup_cpus (bd, entry, start_info);if ( rc ) {printk ("Failed to setup Dom0 CPUs: %d\n" , rc);
xen/arch/x86/pv/dom0_build.c +1 -1
@@ -729,7 +729,7 @@ int __init dom0_construct_pv(struct boot_domain *bd)printk ("Dom%u has maximum %u VCPUs\n" , d->domain_id, d->max_vcpus); -sched_setup_dom0_vcpus (d); +sched_setup_dom_vcpus (bd); d->arch.paging.mode =0 ;
xen/common/sched/core.c +16 -9
@@ -14,6 +14,8 @@*/ #ifndef COMPAT +#include <xen/bootdomain.h> +#include <xen/domain_builder.h> #include <xen/init.h> #include <xen/lib.h> #include <xen/param.h>
@@ -3399,13 +3401,13 @@ void wait(void) }#ifdef CONFIG_X86 -void __init sched_setup_dom0_vcpus (struct domain *d) +void __init sched_setup_dom_vcpus (struct boot_domain *bd) {unsigned int i;struct sched_unit *unit; -for ( i =1 ; i < d->max_vcpus; i++ ) -vcpu_create (d, i); +for ( i =1 ; i < bd->domain->max_vcpus; i++ ) +vcpu_create (bd->domain, i);/* * PV-shim: vcpus are pinned 1:1.
@@ -3413,19 +3415,24 @@ void __init sched_setup_dom0_vcpus(struct domain *d)* onlining them. This avoids pinning a vcpu to a not yet online cpu here. */ if ( pv_shim ) -sched_set_affinity (d->vcpu[0 ]->sched_unit, +sched_set_affinity (bd->domain->vcpu[0 ]->sched_unit,cpumask_of (0 ),cpumask_of (0 ));else { - for_each_sched_unit ( d, unit ) + for_each_sched_unit ( bd->domain, unit ) { -if ( !opt_dom0_vcpus_pin && !dom0_affinity_relaxed ) -sched_set_affinity (unit, &dom0_cpus,NULL ); -sched_set_affinity (unit,NULL , &dom0_cpus); +if (builder_is_initdom (bd) ) + { +if ( !opt_dom0_vcpus_pin && !dom0_affinity_relaxed ) +sched_set_affinity (unit, &dom0_cpus,NULL ); +sched_set_affinity (unit,NULL , &dom0_cpus); + } +else +sched_set_affinity (unit,NULL ,cpupool_valid_cpus (cpupool0)); } } - domain_update_node_affinity(d); + domain_update_node_affinity(bd->domain); }#endif
xen/include/xen/sched.h +2 -1
@@ -2,6 +2,7 @@#ifndef __SCHED_H__#define __SCHED_H__ +#include <xen/bootdomain.h> #include <xen/types.h> #include <xen/spinlock.h> #include <xen/rwlock.h>
@@ -1003,7 +1004,7 @@ static inline bool sched_has_urgent_vcpu(void) }void vcpu_set_periodic_timer (struct vcpu *v,s_time_t value); -void sched_setup_dom0_vcpus (struct domain *d); +void sched_setup_dom_vcpus (struct boot_domain *d);int vcpu_temporary_affinity (struct vcpu *v,unsigned int cpu,uint8_t reason);int vcpu_set_hard_affinity (struct vcpu *v,const cpumask_t *affinity);void restore_vcpu_affinity (struct domain *d);
Signature · 1 line
2.20.1
Re: [PATCH v1 14/18] x86: generalize vcpu for domain building
Quoted 2 lines · Daniel P. Smith
Here, the vcpu initialization code for dom0 creation is generalized for use for other domains.
Yet with "other domains" still only ones created during boot, aiui. Imo such details want spelling out.
The title also is too generic / imprecise.
xen/arch/x86/domain_builder.c static unsigned int __init dom_max_vcpus(struct boot_domain *bd) · line 31
Quoted 7 lines
xen/arch/x86/domain_builder.c +1
-0@@ -28,6 +28,18 @@ static unsigned int __init dom_max_vcpus(struct boot_domain *bd)return bd->ncpus; } +struct vcpu *__init alloc_dom_vcpu0 (struct boot_domain *bd)
domain_alloc_vcpu0()?
xen/arch/x86/domain_builder.c static unsigned int __init dom_max_vcpus(struct boot_domain *bd) · line 37
Quoted 6 lines
xen/arch/x86/domain_builder.c +6
-0+{ +if ( bd->functions &BUILD_FUNCTION_INITIAL_DOM ) +return alloc_dom0_vcpu0 (bd->domain); + + bd->domain->node_affinity = node_online_map; + bd->domain->auto_node_affinity = true;
I can spot neither consumers of nor code being replaced by this.
xen/arch/x86/domain_builder.c static unsigned int __init dom_max_vcpus(struct boot_domain *bd) · line 43
Quoted 5 lines
xen/arch/x86/domain_builder.c +4
-0+return vcpu_create (bd->domain,0 ); +} + + void __init arch_create_dom(
No double blank lines please.
Quoted 12 lines
xen/common/sched/core.c +2
-0@@ -14,6 +14,8 @@*/ #ifndef COMPAT +#include <xen/bootdomain.h> +#include <xen/domain_builder.h> #include <xen/init.h> #include <xen/lib.h> #include <xen/param.h>
xen/common/sched/core.c void wait(void) · line 3404
Quoted 5 lines
xen/common/sched/core.c +1
-1@@ -3399,13 +3401,13 @@ void wait(void) }#ifdef CONFIG_X86 -void __init sched_setup_dom0_vcpus (struct domain *d) +void __init sched_setup_dom_vcpus (struct boot_domain *bd)
Perhaps simply drop the original _dom0 infix?
xen/common/sched/core.c void wait(void) · line 3410
Quoted 8 lines
xen/common/sched/core.c +2
-2{unsigned int i;struct sched_unit *unit; -for ( i =1 ; i < d->max_vcpus; i++ ) -vcpu_create (d, i); +for ( i =1 ; i < bd->domain->max_vcpus; i++ ) +vcpu_create (bd->domain, i);
Seeing the further uses below, perhaps better introduce a local variable "d", like you do elsewhere?
Quoted 13 lines
xen/common/sched/core.c +2
-3@@ -3413,19 +3415,24 @@ void __init sched_setup_dom0_vcpus(struct domain *d)* onlining them. This avoids pinning a vcpu to a not yet online cpu here. */ if ( pv_shim ) -sched_set_affinity (d->vcpu[0 ]->sched_unit, +sched_set_affinity (bd->domain->vcpu[0 ]->sched_unit,cpumask_of (0 ),cpumask_of (0 ));else { - for_each_sched_unit ( d, unit ) + for_each_sched_unit ( bd->domain, unit ) { -if ( !opt_dom0_vcpus_pin && !dom0_affinity_relaxed )
xen/common/sched/core.c void __init sched_setup_dom0_vcpus(struct domain *d) · line 3431
Quoted 10 lines
xen/common/sched/core.c +8
-2-sched_set_affinity (unit, &dom0_cpus,NULL ); -sched_set_affinity (unit,NULL , &dom0_cpus); +if (builder_is_initdom (bd) ) + { +if ( !opt_dom0_vcpus_pin && !dom0_affinity_relaxed ) +sched_set_affinity (unit, &dom0_cpus,NULL ); +sched_set_affinity (unit,NULL , &dom0_cpus); + } +else +sched_set_affinity (unit,NULL ,cpupool_valid_cpus (cpupool0));
Hard-coded cpupool0?
xen/include/xen/sched.h line 5
Quoted 7 lines
xen/include/xen/sched.h +1
-0@@ -2,6 +2,7 @@#ifndef __SCHED_H__#define __SCHED_H__ +#include <xen/bootdomain.h>
Please don't - this header has already too many dependencies. All you really need ...
xen/include/xen/sched.h static inline bool sched_has_urgent_vcpu(void) · line 1007
Quoted 6 lines
xen/include/xen/sched.h +1
-1@@ -1003,7 +1004,7 @@ static inline bool sched_has_urgent_vcpu(void) }void vcpu_set_periodic_timer (struct vcpu *v,s_time_t value); -void sched_setup_dom0_vcpus (struct domain *d); +void sched_setup_dom_vcpus (struct boot_domain *d);
... for this is a forward declaration of struct boot_domain.
Jan
PATCH 15/18 x86: rework domain page allocation
In reply to Daniel P. SmithThis reworks all the dom0 page allocation functions for general domain construction. Where possible, common logic between the two was split into a separate function for reuse by the two functions.
--- xen/arch/x86/dom0_build.c | 76 ++++++------------- xen/arch/x86/domain_builder.c | 102 ++++++++++++++++++++++++++ xen/arch/x86/hvm/dom0_build.c | 11 +-- xen/arch/x86/include/asm/dom0_build.h | 14 +++- xen/arch/x86/pv/dom0_build.c | 2 +- 5 files changed, 142 insertions(+), 63 deletions(-)
xen/arch/x86/dom0_build.c +22 -54
@@ -320,69 +320,31 @@ static unsigned long __init default_nr_pages(unsigned long avail) }unsigned long __init dom0_compute_nr_pages ( -struct domain *d,struct elf_dom_parms *parms,unsigned long initrd_len) +struct boot_domain *bd,struct elf_dom_parms *parms, +unsigned long initrd_len) { -nodeid_t node; -unsigned long avail =0 , nr_pages, min_pages, max_pages, iommu_pages =0 ; +unsigned long avail, nr_pages, min_pages, max_pages;/* The ordering of operands is to work around a clang5 issue. */ if (CONFIG_DOM0_MEM [0 ] && !dom0_mem_set )parse_dom0_mem (CONFIG_DOM0_MEM ); -for_each_node_mask ( node, dom0_nodes ) - avail += avail_domheap_pages_region(node, 0, 0) + -initial_images_nrpages (node); - -/* Reserve memory for further dom0 vcpu-struct allocations... */ - avail -= (d->max_vcpus -1UL ) - <<get_order_from_bytes (sizeof (struct vcpu )); -/* ...and compat_l4's, if needed. */ -if (is_pv_32bit_domain (d) ) - avail -= d->max_vcpus -1 ; - -/* Reserve memory for iommu_dom0_init() (rough estimate). */ -if (is_iommu_enabled (d) && !iommu_hwdom_passthrough ) - { -unsigned int s; - -for ( s =9 ; s <BITS_PER_LONG ; s +=9 ) - iommu_pages += max_pdx >> s; - - avail -= iommu_pages; - } + avail =dom_avail_nr_pages (bd, dom0_nodes); -if (paging_mode_enabled (d) || opt_dom0_shadow || opt_pv_l1tf_hwdom ) +/* command line overrides configuration */ +if ( dom0_mem_set ) { -unsigned long cpu_pages; - - nr_pages =get_memsize (&dom0_size, avail) ?:default_nr_pages (avail); - -/* -* Clamp according to min/max limits and available memory -* (preliminary). -*/ - nr_pages =max (nr_pages,get_memsize (&dom0_min_size, avail)); - nr_pages =min (nr_pages,get_memsize (&dom0_max_size, avail)); - nr_pages =min (nr_pages, avail); - - cpu_pages =dom0_paging_pages (d, nr_pages); - -if ( !iommu_use_hap_pt (d) ) - avail -= cpu_pages; -else if ( cpu_pages > iommu_pages ) - avail -= cpu_pages - iommu_pages; + bd->meminfo.mem_size = dom0_size; + bd->meminfo.mem_min = dom0_min_size; + bd->meminfo.mem_max = dom0_max_size; } - nr_pages =get_memsize (&dom0_size, avail) ?:default_nr_pages (avail); - min_pages =get_memsize (&dom0_min_size, avail); - max_pages =get_memsize (&dom0_max_size, avail); - -/* Clamp according to min/max limits and available memory (final). */ - nr_pages =max (nr_pages, min_pages); - nr_pages =min (nr_pages, max_pages); - nr_pages =min (nr_pages, avail); + nr_pages =get_memsize (&bd->meminfo.mem_size, avail) ? + :default_nr_pages (avail); + min_pages =get_memsize (&bd->meminfo.mem_min, avail); + max_pages =get_memsize (&bd->meminfo.mem_max, avail); -if (is_pv_domain (d) && +if (is_pv_domain (bd->domain) && (parms->p2m_base ==UNSET_ADDR ) && !memsize_gt_zero (&dom0_size) && (!memsize_gt_zero (&dom0_min_size) || (nr_pages > min_pages)) ) {
@@ -395,7 +357,8 @@ unsigned long __init dom0_compute_nr_pages(* available between params.virt_base and the address space end. */ unsigned long vstart, vend, end; -size_t sizeof_long =is_pv_32bit_domain (d) ?sizeof (int ) :sizeof (long ); +size_t sizeof_long =is_pv_32bit_domain (bd->domain) ? +sizeof (int ) :sizeof (long ); vstart = parms->virt_base; vend =round_pgup (parms->virt_kend);
@@ -416,7 +379,12 @@ unsigned long __init dom0_compute_nr_pages( } } - d->max_pages = min_t(unsigned long, max_pages, UINT_MAX); +/* Clamp according to min/max limits and available memory (final). */ + nr_pages =max (nr_pages, min_pages); + nr_pages =min (nr_pages, max_pages); + nr_pages =min (nr_pages, avail); + + bd->domain->max_pages = min_t(unsigned long, max_pages, UINT_MAX);return nr_pages; }
xen/arch/x86/domain_builder.c +102 -0
115 lines not on this page · raw message
xen/arch/x86/hvm/dom0_build.c +6 -5
@@ -412,15 +412,16 @@ static __init void pvh_setup_e820(struct domain *d, unsigned long nr_pages)ASSERT (cur_pages == nr_pages); } -static void __init pvh_init_p2m (struct domain *d) +static void __init pvh_init_p2m (struct boot_domain *bd) { -unsigned long nr_pages =dom0_compute_nr_pages (d,NULL ,0 ); +unsigned long nr_pages =dom_compute_nr_pages (bd,NULL ,0 );bool preempted; -pvh_setup_e820 (d, nr_pages); +pvh_setup_e820 (bd->domain, nr_pages);do { preempted = false; -paging_set_allocation (d,dom0_paging_pages (d, nr_pages), +paging_set_allocation (bd->domain, +dom0_paging_pages (bd->domain, nr_pages), &preempted);process_pending_softirqs (); }while ( preempted );
@@ -1239,7 +1240,7 @@ int __init dom0_construct_pvh(struct boot_domain *bd)* be done before the iommu initializion, since iommu initialization code * will likely add mappings required by devices to the p2m (ie: RMRRs). */ -pvh_init_p2m (d); +pvh_init_p2m (bd);iommu_hwdom_init (d);
xen/arch/x86/include/asm/dom0_build.h +11 -3
@@ -9,9 +9,17 @@extern unsigned int dom0_memflags; -unsigned long dom0_compute_nr_pages (struct domain *d, -struct elf_dom_parms *parms, -unsigned long initrd_len); +unsigned long dom_avail_nr_pages ( +struct boot_domain *bd,nodemask_t nodes); + +unsigned long dom0_compute_nr_pages ( +struct boot_domain *bd,struct elf_dom_parms *parms, +unsigned long initrd_len); + +unsigned long dom_compute_nr_pages ( +struct boot_domain *bd,struct elf_dom_parms *parms, +unsigned long initrd_len); +int dom0_setup_permissions (struct domain *d);int dom0_construct_pv (struct boot_domain *bd);
xen/arch/x86/pv/dom0_build.c +1 -1
@@ -428,7 +428,7 @@ int __init dom0_construct_pv(struct boot_domain *bd) } } - nr_pages =dom0_compute_nr_pages (d, &parms, initrd_len); + nr_pages =dom_compute_nr_pages (bd, &parms, initrd_len);if ( parms.pae ==XEN_PAE_EXTCR3 )set_bit (VMASST_TYPE_pae_extended_cr3, &d->vm_assist);
Signature · 1 line
2.20.1
Re: [PATCH v1 15/18] x86: rework domain page allocation
Quoted 3 lines · Daniel P. Smith
This reworks all the dom0 page allocation functions for general domain construction. Where possible, common logic between the two was split into a separate function for reuse by the two functions.
You absolutely need to mention what behavioral / functional changes there are (intended), even in case it is "none".
Quoted 34 lines
xen/arch/x86/dom0_build.c +3
-19@@ -320,69 +320,31 @@ static unsigned long __init default_nr_pages(unsigned long avail) }unsigned long __init dom0_compute_nr_pages ( -struct domain *d,struct elf_dom_parms *parms,unsigned long initrd_len) +struct boot_domain *bd,struct elf_dom_parms *parms, +unsigned long initrd_len) { -nodeid_t node; -unsigned long avail =0 , nr_pages, min_pages, max_pages, iommu_pages =0 ; +unsigned long avail, nr_pages, min_pages, max_pages;/* The ordering of operands is to work around a clang5 issue. */ if (CONFIG_DOM0_MEM [0 ] && !dom0_mem_set )parse_dom0_mem (CONFIG_DOM0_MEM ); -for_each_node_mask ( node, dom0_nodes ) - avail += avail_domheap_pages_region(node, 0, 0) + -initial_images_nrpages (node); - -/* Reserve memory for further dom0 vcpu-struct allocations... */ - avail -= (d->max_vcpus -1UL ) - <<get_order_from_bytes (sizeof (struct vcpu )); -/* ...and compat_l4's, if needed. */ -if (is_pv_32bit_domain (d) ) - avail -= d->max_vcpus -1 ; - -/* Reserve memory for iommu_dom0_init() (rough estimate). */ -if (is_iommu_enabled (d) && !iommu_hwdom_passthrough ) - { -unsigned int s; -
xen/arch/x86/dom0_build.c static unsigned long __init default_nr_pages(unsigned long avail) · line 335
Quoted 10 lines
xen/arch/x86/dom0_build.c +3
-6-for ( s =9 ; s <BITS_PER_LONG ; s +=9 ) - iommu_pages += max_pdx >> s; - - avail -= iommu_pages; - } + avail =dom_avail_nr_pages (bd, dom0_nodes); -if (paging_mode_enabled (d) || opt_dom0_shadow || opt_pv_l1tf_hwdom ) +/* command line overrides configuration */ +if ( dom0_mem_set )
Nit: Stray double blanks.
Quoted 9 lines
xen/arch/x86/dom0_build.c +0
-8{ -unsigned long cpu_pages; - - nr_pages =get_memsize (&dom0_size, avail) ?:default_nr_pages (avail); - -/* -* Clamp according to min/max limits and available memory -* (preliminary). -*/
xen/arch/x86/dom0_build.c static unsigned long __init default_nr_pages(unsigned long avail) · line 373
Quoted 10 lines
xen/arch/x86/dom0_build.c +0
-10- nr_pages =max (nr_pages,get_memsize (&dom0_min_size, avail)); - nr_pages =min (nr_pages,get_memsize (&dom0_max_size, avail)); - nr_pages =min (nr_pages, avail); - - cpu_pages =dom0_paging_pages (d, nr_pages); - -if ( !iommu_use_hap_pt (d) ) - avail -= cpu_pages; -else if ( cpu_pages > iommu_pages ) - avail -= cpu_pages - iommu_pages;
I can't see any of this represented in the new code. Have you gone through the history of this code, to understand why things are the way they are, and hence what (corner) cases need to remain behaviorally unchanged?
Quoted 10 lines
xen/arch/x86/dom0_build.c +6
-0@@ -40,6 +42,106 @@ struct vcpu *__init alloc_dom_vcpu0(struct boot_domain *bd) } +unsigned long __init dom_avail_nr_pages ( +struct boot_domain *bd,nodemask_t nodes) +{ +unsigned long avail =0 , iommu_pages =0 ; +bool is_ctldom = false, is_hwdom = false; +unsigned long nr_pages = bd->meminfo.mem_size.nr_pages;
xen/arch/x86/dom0_build.c struct vcpu *__init alloc_dom_vcpu0(struct boot_domain *bd) · line 60
Quoted 10 lines
xen/arch/x86/dom0_build.c +10
-0+nodeid_t node; + +if (builder_is_ctldom (bd) ) + is_ctldom = true; +if (builder_is_hwdom (bd) ) + is_hwdom = true; + +for_each_node_mask ( node, nodes ) + avail += avail_domheap_pages_region(node, 0, 0) + +initial_images_nrpages (node);
I don't think this is suitable for other than Dom0, so I question the splitting out and generalizing of this logic. For "ordinary" domains their memory size should be well-defined rather than inferred from host capacity.
Starting from host capacity also means you become ordering dependent when it comes to creating (not starting) all the domains: Which one is to come first? And even with this limited to just Dom0 - is its size calculated before or after all the other domains were created?
xen/arch/x86/dom0_build.c struct vcpu *__init alloc_dom_vcpu0(struct boot_domain *bd) · line 61
Quoted 1 line
xen/arch/x86/dom0_build.c +1
-0+/* Reserve memory for further dom0 vcpu-struct allocations... */
dom0?
xen/arch/x86/dom0_build.c struct vcpu *__init alloc_dom_vcpu0(struct boot_domain *bd) · line 69
Quoted 8 lines
xen/arch/x86/dom0_build.c +8
-0+ avail -= (bd->domain->max_vcpus -1UL ) + <<get_order_from_bytes (sizeof (struct vcpu )); +/* ...and compat_l4's, if needed. */ +if (is_pv_32bit_domain (bd->domain) ) + avail -= bd->domain->max_vcpus -1 ; + +/* Reserve memory for iommu_dom0_init() (rough estimate). */ +if ( is_hwdom &&is_iommu_enabled (bd->domain) && !iommu_hwdom_passthrough )
Again the question why this would be Dom0-only.
xen/arch/x86/dom0_build.c struct vcpu *__init alloc_dom_vcpu0(struct boot_domain *bd) · line 81
Quoted 12 lines
xen/arch/x86/dom0_build.c +12
-0+ { +unsigned int s; + +for ( s =9 ; s <BITS_PER_LONG ; s +=9 ) + iommu_pages += max_pdx >> s; + + avail -= iommu_pages; + } + +if (paging_mode_enabled (bd->domain) || + (is_ctldom && opt_dom0_shadow) || + (is_hwdom && opt_pv_l1tf_hwdom) )
An interesting combination of conditions. It (again) looks to me as if it first needs properly separating Dom0 from hwdom, in an abstract sense.
Quoted 23 lines
xen/arch/x86/dom0_build.c +23
-0+ { +unsigned long cpu_pages =dom0_paging_pages (bd->domain, nr_pages); + +if ( !iommu_use_hap_pt (bd->domain) ) + avail -= cpu_pages; +else if ( cpu_pages > iommu_pages ) + avail -= cpu_pages - iommu_pages; + } + +return avail; +} + +unsigned long __init dom_compute_nr_pages ( +struct boot_domain *bd,struct elf_dom_parms *parms, +unsigned long initrd_len) +{ +unsigned long avail, nr_pages = bd->meminfo.mem_size.nr_pages; + +if (builder_is_initdom (bd) ) +return dom0_compute_nr_pages (bd, parms, initrd_len); + + avail =dom_avail_nr_pages (bd, node_online_map); +
xen/arch/x86/dom0_build.c struct vcpu *__init alloc_dom_vcpu0(struct boot_domain *bd) · line 114
Quoted 10 lines
xen/arch/x86/dom0_build.c +10
-0+if (is_pv_domain (bd->domain) && (parms->p2m_base ==UNSET_ADDR ) ) + { +/* +* Legacy Linux kernels (i.e. such without a XEN_ELFNOTE_INIT_P2M +* note) require that there is enough virtual space beyond the initial +* allocation to set up their initial page tables. This space is +* roughly the same size as the p2m table, so make sure the initial +* allocation doesn't consume more than about half the space that's +* available between params.virt_base and the address space end. +*/
This duplicates an existing comment (and hence below likely also existing code) rather than replacing / moving the original. As in an earlier case - how are the two going to remain in sync?
Jan
PATCH 16/18 x86: add pv multidomain construction
In reply to Daniel P. SmithThis adds the ability to domain builder for the construction of multiple pv domains at boot.
---
xen/arch/x86/dom0_build.c | 31 -----
xen/arch/x86/domain_builder.c | 58 ++++++--
xen/arch/x86/include/asm/dom0_build.h | 2 -
xen/arch/x86/include/asm/setup.h | 2 +
xen/arch/x86/pv/Makefile | 2 +-
.../x86/pv/{dom0_build.c => domain_builder.c} | 86 +++++++-----
xen/common/domain-builder/Kconfig | 10 ++
xen/common/domain-builder/core.c | 130 ++++++++++++++++--
xen/include/xen/bootdomain.h | 6 +
xen/include/xen/domain_builder.h | 19 +++
10 files changed, 259 insertions(+), 87 deletions(-)
rename xen/arch/x86/pv/{dom0_build.c => domain_builder.c} (92%)xen/arch/x86/dom0_build.c +0 -31
@@ -524,37 +524,6 @@ int __init dom0_setup_permissions(struct domain *d)return rc; } - -int __init construct_domain (struct boot_domain *bd) -{ -int rc =0 ; - -/* Sanity! */ -BUG_ON (!pv_shim && bd->domid !=0 ); -BUG_ON (bd->domain->vcpu[0 ] ==NULL ); -BUG_ON (bd->domain->vcpu[0 ]->is_initialised); - -process_pending_softirqs (); - -if (builder_is_initdom (bd) ) - { -if (is_hvm_domain (bd->domain) ) - rc =dom0_construct_pvh (bd); -else if (is_pv_domain (bd->domain) ) - rc =dom0_construct_pv (bd); -else -panic ("Cannot construct Dom0. No guest interface available\n" ); - } - -if ( rc ) -return rc; - -/* Sanity! */ -BUG_ON (!bd->domain->vcpu[0 ]->is_initialised); - -return 0 ; -} -/* * Local variables: * mode: C
xen/arch/x86/domain_builder.c +50 -8
92 lines not on this page · raw message
xen/arch/x86/include/asm/dom0_build.h +0 -2
@@ -22,8 +22,6 @@ unsigned long dom_compute_nr_pages(int dom0_setup_permissions (struct domain *d); -int dom0_construct_pv (struct boot_domain *bd); -int dom0_construct_pvh (struct boot_domain *bd); unsigned long dom0_paging_pages(const struct domain *d,
xen/arch/x86/include/asm/setup.h +2 -0
@@ -36,6 +36,8 @@ static inline void vesa_init(void) {};void apply_xen_cmdline (char *cmdline);int construct_domain (struct boot_domain *bd); +int dom_construct_pv (struct boot_domain *bd); +void setup_io_bitmap (struct domain *d);unsigned long initial_images_nrpages (nodeid_t node);
xen/arch/x86/pv/Makefile +1 -1
@@ -15,5 +15,5 @@ obj-$(CONFIG_PV_SHIM) += shim.o obj-$(CONFIG_TRACEBUFFER) += trace.o obj-y += traps.o -obj-bin-y += dom0_build.init.o +obj-bin-y += domain_builder.init.o obj-bin-y += gpr_switch.o
xen/arch/x86/pv/domain_builder.c renamed from xen/arch/x86/pv/dom0_build.c +52 -34
224 lines not on this page · raw message
xen/common/domain-builder/Kconfig +10 -0
@@ -12,4 +12,14 @@ config BUILDER_FDT If unsure, say N. +config MULTIDOM_BUILDER + bool "Multidomain building (UNSUPPORTED)" if UNSUPPORTED + depends on BUILDER_FDT + ---help--- + Enables the domain builder to construct multiple domains. + + This feature is currently experimental. + + If unsure, say N. + endmenu
xen/common/domain-builder/core.c +119 -11
162 lines not on this page · raw message
xen/include/xen/bootdomain.h +6 -0
@@ -47,6 +47,12 @@ struct boot_domain {struct boot_module *configs[BUILD_MAX_CONF_MODS ];struct domain *domain; +struct { +xen_pfn_t mfn; +unsigned int evtchn; + } store, console; + bool constructed; + };#endif
xen/include/xen/domain_builder.h +19 -0
@@ -34,6 +34,22 @@ static inline bool builder_is_hwdom(struct boot_domain *bd) bd->permissions & BUILD_PERMISSION_HARDWARE ); } +static inline struct boot_domain *builder_dom_by_function ( +const struct boot_info *info,uint32_t func) +{ +int i; + +for ( i =0 ; i < info->builder->nr_doms; i++ ) + { +struct boot_domain *bd = &info->builder->domains[i]; + +if ( bd->functions & func ) +return bd; + } + +return NULL ; +} +static inline struct domain *builder_get_hwdom (struct boot_info *info) {int i;
@@ -51,6 +67,9 @@ static inline struct domain *builder_get_hwdom(struct boot_info *info)void builder_init (struct boot_info *info);uint32_t builder_create_domains (struct boot_info *info); +domid_t get_next_domid (void ); +int alloc_system_evtchn ( +const struct boot_info *info,struct boot_domain *bd);void arch_create_dom (const struct boot_info *bi,struct boot_domain *bd);#endif /* XEN_DOMAIN_BUILDER_H */
Signature · 1 line
2.20.1
Re: [PATCH v1 16/18] x86: add pv multidomain construction
Quoted 26 lines · Daniel P. Smith
xen/arch/x86/dom0_build.c +0
-20@@ -524,37 +524,6 @@ int __init dom0_setup_permissions(struct domain *d)return rc; } - -int __init construct_domain (struct boot_domain *bd) -{ -int rc =0 ; - -/* Sanity! */ -BUG_ON (!pv_shim && bd->domid !=0 ); -BUG_ON (bd->domain->vcpu[0 ] ==NULL ); -BUG_ON (bd->domain->vcpu[0 ]->is_initialised); - -process_pending_softirqs (); - -if (builder_is_initdom (bd) ) - { -if (is_hvm_domain (bd->domain) ) - rc =dom0_construct_pvh (bd); -else if (is_pv_domain (bd->domain) ) - rc =dom0_construct_pv (bd); -else -panic ("Cannot construct Dom0. No guest interface available\n" );
xen/arch/x86/dom0_build.c int __init dom0_setup_permissions(struct domain *d) · line 556
Quoted 10 lines
xen/arch/x86/dom0_build.c +0
-10- } - -if ( rc ) -return rc; - -/* Sanity! */ -BUG_ON (!bd->domain->vcpu[0 ]->is_initialised); - -return 0 ; -}
Iirc this function was introduced earlier in the series. Just for it to now be moved around? Why can't it be introduced in the intended source file right away?
xen/arch/x86/dom0_build.c void __init arch_create_dom( · line 197
Quoted 11 lines
xen/arch/x86/dom0_build.c +5
-2@@ -189,18 +190,22 @@ void __init arch_create_dom(if ( !pv_shim &&builder_is_ctldom (bd) ) is_privileged = CDF_privileged; -/* Create initial domain. Not d0 for pvshim. */ - bd->domid =get_initial_domain_id (); +/* Determine proper domain id. */ +if (builder_is_initdom (bd) ) + bd->domid =get_initial_domain_id (); +else + bd->domid = bd->domid ? bd->domid :get_next_domid ();
We prefer to omit the middle operand in such cases.
Where to you guarantee no two domains would use the same domain ID? I can't help thinking that a predetermined one may have been assigned earlier on to a domain which got it from get_next_domid().
xen/arch/x86/dom0_build.c void __init arch_create_dom( · line 203
Quoted 7 lines
xen/arch/x86/dom0_build.c +2
-1bd->domain =domain_create (bd->domid, &dom_cfg, is_privileged);if (IS_ERR (bd->domain) )panic ("Error creating d%u: %ld\n" , bd->domid,PTR_ERR (bd->domain)); -init_dom0_cpuid_policy (bd->domain); +if (builder_is_initdom (bd) ) +init_dom0_cpuid_policy (bd->domain);
What about other than Dom0?
Quoted 6 lines
xen/arch/x86/dom0_build.c +0
-2@@ -210,15 +215,23 @@ void __init arch_create_dom( cmdline =arch_prepare_cmdline (cmdline, bi->arch);strlcpy (dom_cmdline, cmdline,MAX_GUEST_CMDLINE ); -if ( bi->arch->kextra ) -/* kextra always includes exactly one leading space. */
xen/arch/x86/dom0_build.c void __init arch_create_dom( · line 225
Quoted 10 lines
xen/arch/x86/dom0_build.c +7
-2-strlcat (dom_cmdline, bi->arch->kextra,MAX_GUEST_CMDLINE ); +if (builder_is_initdom (bd) ) + { +if ( bi->arch->kextra ) +/* kextra always includes exactly one leading space. */ +strlcat (dom_cmdline, bi->arch->kextra,MAX_GUEST_CMDLINE ); -apply_xen_cmdline (dom_cmdline); +apply_xen_cmdline (dom_cmdline); + }
Why is kextra applicable to Dom0 only? Shouldn't each domain have a way to append to its command line?
xen/arch/x86/dom0_build.c void __init arch_create_dom( · line 232
Quoted 7 lines
xen/arch/x86/dom0_build.c +4
-0strlcpy (bd->kernel->string.bytes, dom_cmdline,MAX_GUEST_CMDLINE ); } +if (alloc_system_evtchn (bi, bd) !=0 ) +printk (XENLOG_WARNING "%s: " +"unable set up system event channels for Dom%d\n" , + __func__, bd->domid);
So if Dom0 is created after e.g. a separate xenstore domain, it'll also have a xenstore event channel assigned (changing behavior from what we have today)?
Quoted 11 lines
xen/arch/x86/dom0_build.c +7
-0@@ -240,3 +253,32 @@ void __init arch_create_dom( } } +int __init construct_domain (struct boot_domain *bd) +{ +int rc =0 ; + +/* Sanity! */ +BUG_ON (bd->domid != bd->domain->domain_id); +BUG_ON (bd->domain->vcpu[0 ] ==NULL );
xen/arch/x86/dom0_build.c void __init arch_create_dom( · line 272
Quoted 10 lines
xen/arch/x86/dom0_build.c +10
-0+BUG_ON (bd->domain->vcpu[0 ]->is_initialised); + +process_pending_softirqs (); + +if (is_hvm_domain (bd->domain) ) +if (builder_is_initdom (bd) ) + rc =dom0_construct_pvh (bd); +else +panic ("Cannot construct HVM DomU. Not supported.\n" ); +else if (is_pv_domain (bd->domain) )
Please properly use braces to enclose the inner if/else pair.
xen/arch/x86/dom0_build.c void __init arch_create_dom( · line 275
Quoted 3 lines
xen/arch/x86/dom0_build.c +3
-0+ rc =dom_construct_pv (bd); + else + panic("Cannot construct Dom0. No guest interface available\n");
Dom0?
Quoted 27 lines
xen/arch/x86/pv/domain_builder.c renamed from xen/arch/x86/pv/dom0_build.c +3
-2@@ -1,5 +1,5 @@/****************************************************************************** -* pv/dom0_build.c +* pv/domain_builder.c * * Copyright (c) 2002-2005, K A Fraser */ @@ -8,6 +8,7 @@#include <xen/bootinfo.h> #include <xen/console.h> #include <xen/domain.h> +#include <xen/domain_builder.h> #include <xen/domain_page.h> #include <xen/init.h> #include <xen/libelf.h> @@ -296,7 +297,7 @@ static struct page_info * __init alloc_chunk(struct domain *d,return page; } -int __init dom0_construct_pv (struct boot_domain *bd) +int __init dom_construct_pv (struct boot_domain *bd) {int i, rc, order, machine;bool compatible, compat;
xen/arch/x86/pv/domain_builder.c int __init dom0_construct_pv(struct boot_domain *bd) · line 354
Quoted 5 lines
xen/arch/x86/pv/domain_builder.c renamed from xen/arch/x86/pv/dom0_build.c +1
-1@@ -350,7 +351,7 @@ int __init dom0_construct_pv(struct boot_domain *bd)/* Machine address of next candidate page-table page. */ paddr_t mpt_alloc; -printk (XENLOG_INFO "*** Building a PV Dom%d ***\n" , d->domain_id); +printk (XENLOG_INFO "*** Constructing a PV Dom%d ***\n" , d->domain_id);
When touching things like here and even more so when ...
xen/arch/x86/pv/domain_builder.c int __init dom0_construct_pv(struct boot_domain *bd) · line 389
Quoted 7 lines
xen/arch/x86/pv/domain_builder.c renamed from xen/arch/x86/pv/dom0_build.c +2
-1@@ -384,7 +385,8 @@ int __init dom0_construct_pv(struct boot_domain *bd) {if (unlikely (rc =switch_compat (d)) ) { -printk ("Dom0 failed to switch to compat: %d\n" , rc); +printk ("Dom%d failed to switch to compat: %d\n" , + d->domain_id, rc);
... adding new logging of domain IDs, please use %pd whenever possible.
Quoted 41 lines
xen/arch/x86/pv/domain_builder.c renamed from xen/arch/x86/pv/dom0_build.c +9
-7@@ -404,22 +406,23 @@ int __init dom0_construct_pv(struct boot_domain *bd)if (elf_msb (&elf) ) compatible = false; -printk (" Dom0 kernel: %s-bit%s, %s, paddr %#" PRIx64" -> %#" PRIx64"\n" , -elf_64bit (&elf) ?"64" :elf_32bit (&elf) ?"32" :"??" , +printk (" Dom%d kernel: %s-bit%s, %s, paddr %#" PRIx64" -> %#" PRIx64"\n" , + d->domain_id,elf_64bit (&elf) ?"64" :elf_32bit (&elf) ?"32" :"??" , parms.pae ?", PAE" :"" ,elf_msb (&elf) ?"msb" :"lsb" , elf.pstart, elf.pend);if ( elf.bsd_symtab_pstart ) -printk (" Dom0 symbol map %#" PRIx64" -> %#" PRIx64"\n" , - elf.bsd_symtab_pstart, elf.bsd_symtab_pend); +printk (" Dom%d symbol map %#" PRIx64" -> %#" PRIx64"\n" , + d->domain_id, elf.bsd_symtab_pstart, elf.bsd_symtab_pend);if ( !compatible ) { -printk ("Mismatch between Xen and DOM0 kernel\n" ); +printk ("Mismatch between Xen and DOM%d kernel\n" , d->domain_id);return -EINVAL ; } -if ( parms.elf_notes[XEN_ELFNOTE_SUPPORTED_FEATURES ].type !=XEN_ENT_NONE ) +if (builder_is_initdom (bd) && + parms.elf_notes[XEN_ELFNOTE_SUPPORTED_FEATURES ].type !=XEN_ENT_NONE ) {if ( !pv_shim && !test_bit (XENFEAT_dom0, parms.f_supported) ) {@@ -443,7 +446,8 @@ int __init dom0_construct_pv(struct boot_domain *bd)if ( value > __HYPERVISOR_COMPAT_VIRT_START ) { -printk ("Dom0 expects too high a hypervisor start address\n" ); +printk ("Dom%d expects too high a hypervisor start address\n" , + d->domain_id);return -ERANGE ; } HYPERVISOR_COMPAT_VIRT_START(d) =
xen/arch/x86/pv/domain_builder.c int __init dom0_construct_pv(struct boot_domain *bd) · line 494
Quoted 5 lines
xen/arch/x86/pv/domain_builder.c renamed from xen/arch/x86/pv/dom0_build.c +1
-1@@ -487,7 +491,7 @@ int __init dom0_construct_pv(struct boot_domain *bd) vstartinfo_start =round_pgup (vphysmap_end); vstartinfo_end = vstartinfo_start +sizeof (struct start_info ); -if ( pv_shim ) +if ( pv_shim || !builder_is_initdom (bd) )
As elsewhere - stray blank after ! . Also wouldn't it make sense for builder_is_initdom() to return false in the pv_shim case, thus making the || here (and again below) unnecessary?
Quoted 6 lines
xen/arch/x86/pv/domain_builder.c renamed from xen/arch/x86/pv/dom0_build.c +2
-0@@ -789,6 +790,19 @@ int __init dom0_construct_pv(struct boot_domain *bd)snprintf (si->magic,sizeof (si->magic),"xen-3.0-x86_%d%s" ,elf_64bit (&elf) ?64 :32 , parms.pae ?"p" :"" ); +if ( !builder_is_initdom (bd) ) + {
xen/arch/x86/pv/domain_builder.c int __init dom0_construct_pv(struct boot_domain *bd) · line 804
Quoted 10 lines
xen/arch/x86/pv/domain_builder.c renamed from xen/arch/x86/pv/dom0_build.c +10
-0+ si->store_mfn = ((vxenstore_start - v_start) >>PAGE_SHIFT ) + + alloc_spfn; + bd->store.mfn = si->store_mfn; + si->store_evtchn = bd->store.evtchn; + + si->console.domU.mfn = ((vconsole_start - v_start) >>PAGE_SHIFT ) + + alloc_spfn; + bd->console.mfn = si->console.domU.mfn; + si->console.domU.evtchn = bd->console.evtchn; + }
While elsewhere you allow separate hwdom and ctrldom, aiui only one of the two would fail the entry condition to this if(). Which one would that be? And in how far are there kernels knowing how to deal with the situation? I'm not even certain this can be properly expressed in the present start_info structure, as a non-Dom0 control domain would e.g. need to have xenstore coordinates passed, but might act as the domain handling consoles.
xen/arch/x86/pv/domain_builder.c int __init dom0_construct_pv(struct boot_domain *bd) · line 889
Quoted 11 lines
xen/arch/x86/pv/domain_builder.c renamed from xen/arch/x86/pv/dom0_build.c +2
-5@@ -871,23 +885,24 @@ int __init dom0_construct_pv(struct boot_domain *bd) sizeof(si->cmd_line));#ifdef CONFIG_VIDEO -if ( !pv_shim &&fill_console_start_info ((void *)(si +1 )) ) - { - si->console.dom0.info_off =sizeof (struct start_info ); - si->console.dom0.info_size =sizeof (struct dom0_vga_console_info ); - } +if (builder_is_hwdom (bd) ) +if ( !pv_shim &&fill_console_start_info ((void *)(si +1 )) )
As before - please combine such if()s.
xen/arch/x86/pv/domain_builder.c int __init dom0_construct_pv(struct boot_domain *bd) · line 893
Quoted 4 lines
xen/arch/x86/pv/domain_builder.c renamed from xen/arch/x86/pv/dom0_build.c +4
-0+ { + si->console.dom0.info_off =sizeof (struct start_info ); + si->console.dom0.info_size =sizeof (struct dom0_vga_console_info ); + }
I don't view it as a given that hwdom is the domain to have access to the physical VGA. While it may follow from its name, it may be more useful to have the control domain direct its output there.
xen/arch/x86/pv/domain_builder.c int __init dom0_construct_pv(struct boot_domain *bd) · line 902
Quoted 10 lines
xen/arch/x86/pv/domain_builder.c renamed from xen/arch/x86/pv/dom0_build.c +1
-1#endif /* * TODO: provide an empty stub for fill_console_start_info in the * !CONFIG_VIDEO case so the logic here can be simplified. */ -if ( pv_shim ) +if (builder_is_hwdom (bd) && pv_shim )pv_shim_setup_dom (d, l4start, v_start, vxenstore_start, vconsole_start, vphysmap_start, si);
???
xen/arch/x86/pv/domain_builder.c int __init dom0_construct_pv(struct boot_domain *bd) · line 907
Quoted 5 lines
xen/arch/x86/pv/domain_builder.c renamed from xen/arch/x86/pv/dom0_build.c +1
-1#ifdef CONFIG_COMPAT -if ( compat ) +if (builder_is_hwdom (bd) && compat )xlat_start_info (si, pv_shim ? XLAT_start_info_console_domU : XLAT_start_info_console_dom0);
Even more so here: ??? (This is a clear sign that your commit messages are lacking helpful detail.)
xen/arch/x86/pv/domain_builder.c int __init dom0_construct_pv(struct boot_domain *bd) · line 948
Quoted 11 lines
xen/arch/x86/pv/domain_builder.c renamed from xen/arch/x86/pv/dom0_build.c +5
-2@@ -926,15 +941,18 @@ int __init dom0_construct_pv(struct boot_domain *bd)if (test_bit (XENFEAT_supervisor_mode_kernel, parms.f_required) )panic ("Dom0 requires supervisor-mode execution\n" ); - rc =dom0_setup_permissions (d); -BUG_ON (rc !=0 ); +if (builder_is_hwdom (bd) ) + { + rc =dom0_setup_permissions (d); +BUG_ON (rc !=0 ); + }
What about other domains?
xen/arch/x86/pv/domain_builder.c int __init dom0_construct_pv(struct boot_domain *bd) · line 955
Quoted 7 lines
xen/arch/x86/pv/domain_builder.c renamed from xen/arch/x86/pv/dom0_build.c +1
-1if ( d->domain_id == hardware_domid )iommu_hwdom_init (d);#ifdef CONFIG_SHADOW_PAGING /* Fill the shadow pool if necessary. */ -if ( opt_dom0_shadow || opt_pv_l1tf_hwdom ) +if (builder_is_hwdom (bd) && (opt_dom0_shadow || opt_pv_l1tf_hwdom) )
With this I'd like to refer you back to my "An interesting combination of conditions" comment on patch 15.
xen/common/domain-builder/Kconfig config BUILDER_FDT · line 17
Quoted 9 lines
xen/common/domain-builder/Kconfig +3
-0@@ -12,4 +12,14 @@ config BUILDER_FDT If unsure, say N. +config MULTIDOM_BUILDER + bool "Multidomain building (UNSUPPORTED)" if UNSUPPORTED + depends on BUILDER_FDT
Shouldn't this be "select", with that other option perhaps not even needing a prompt?
Quoted 11 lines
xen/common/domain-builder/core.c +1
-0@@ -1,6 +1,7 @@#include <xen/bootdomain.h> #include <xen/bootinfo.h> #include <xen/domain_builder.h> +#include <xen/event.h> #include <xen/init.h> #include <xen/types.h>
xen/common/domain-builder/core.c void __init builder_init(struct boot_info *info) · line 66
Quoted 6 lines
xen/common/domain-builder/core.c +3
-0@@ -60,37 +61,144 @@ void __init builder_init(struct boot_info *info) d->kernel->string.kind =BOOTSTR_CMDLINE ; } +static bool __init build_domain (struct boot_info *info,struct boot_domain *bd) +{ +if ( bd->constructed == true )
Please omit "== true" / replace "== false" or alike.
Quoted 29 lines
xen/common/domain-builder/core.c +24
-0+return true; + +if ( bd->kernel ==NULL ) +return false; + +printk (XENLOG_INFO "*** Building Dom%d ***\n" , bd->domid); + +arch_create_dom (info, bd); +if ( bd->domain ) + { + bd->constructed = true; +return true; + } + +return false; +} +uint32_t __init builder_create_domains (struct boot_info *info) {uint32_t build_count =0 , functions_built =0 ; +struct boot_domain *bd;int i; +if (IS_ENABLED (CONFIG_MULTIDOM_BUILDER ) ) + { + bd =builder_dom_by_function (info,BUILD_FUNCTION_XENSTORE ); +if (build_domain (info, bd) ) + { + functions_built |= bd->functions;
xen/common/domain-builder/core.c void __init builder_init(struct boot_info *info) · line 105
Quoted 10 lines
xen/common/domain-builder/core.c +10
-0+ build_count++; + } +else +printk (XENLOG_WARNING "Xenstore build failed, system may be unusable\n" ); + + bd =builder_dom_by_function (info,BUILD_FUNCTION_CONSOLE ); +if (build_domain (info, bd) ) + { + functions_built |= bd->functions; + build_count++;
If both are the same domain, you'll end up with a count of 2 here even though only one domain was built. This looks misleading.
xen/common/domain-builder/core.c void __init builder_init(struct boot_info *info) · line 108
Quoted 3 lines
xen/common/domain-builder/core.c +3
-0+ } +else +printk (XENLOG_WARNING "Console build failed, system may be unusable\n" );
I think this and the similar earlier message want to include the word "domain". You also want to split the lines at the start of the literal strings.
xen/common/domain-builder/core.c void __init builder_init(struct boot_info *info) · line 115
Quoted 8 lines
xen/common/domain-builder/core.c +3
-1+ } +for ( i =0 ; i < info->builder->nr_doms; i++ ) { -struct boot_domain *d = &info->builder->domains[i]; + bd = &info->builder->domains[i]; if ( ! IS_ENABLED(CONFIG_MULTIDOM_BUILDER) &&
Interesting - this config option is being introduced only here.
Quoted 17 lines
xen/common/domain-builder/core.c +4
-5- !builder_is_initdom (d) && + !builder_is_initdom (bd) && functions_built & BUILD_FUNCTION_INITIAL_DOM ) continue; -if ( d->kernel ==NULL ) +if ( !build_domain (info, bd) ) { -if (builder_is_initdom (d) ) +if (builder_is_initdom (bd) )panic ("%s: intial domain missing kernel\n" , __func__); -printk (XENLOG_ERR "%s:Dom%d definiton has no kernel\n" , __func__, - d->domid); +printk (XENLOG_WARNING "Dom%d build failed, skipping\n" , bd->domid);continue ; }
xen/common/domain-builder/core.c void __init builder_init(struct boot_info *info) · line 135
Quoted 10 lines
xen/common/domain-builder/core.c +7
-2-arch_create_dom (info, d); -if ( d->domain ) + functions_built |= bd->functions; + build_count++; + } + +if (IS_ENABLED (CONFIG_X86 ) ) +/* Free temporary buffers. */ +discard_initial_images ();
I guess this won't build on Arm. Plus Arm has a similarly named function (discard_initial_modules()) which likely wants calling here (or rather:
adding suitable abstraction for the right function to be called).
Quoted 16 lines
xen/common/domain-builder/core.c +13
-2+return build_count; +} + +domid_t __init get_next_domid (void ) +{ +static domid_t __initdata last_domid =0 ; +domid_t next; + +for ( next = last_domid +1 ; next <DOMID_FIRST_RESERVED ; next++ ) + { +struct domain *d; + +if ( (d =rcu_lock_domain_by_id (next)) ==NULL ) { - functions_built |= d->functions; - build_count++;
xen/common/domain-builder/core.c void __init builder_init(struct boot_info *info) · line 159
Quoted 10 lines
xen/common/domain-builder/core.c +6
-1+ last_domid = next; +return next; } + +rcu_unlock_domain (d); } -return build_count; +return 0 ; +}
This looks suspiciously similar to code in common/domctl.c. Perhaps you want to make a function usable by both (introduced in a separate patch)?
xen/include/xen/bootdomain.h struct boot_domain { · line 56
Quoted 13 lines
xen/include/xen/bootdomain.h +6
-0@@ -47,6 +47,12 @@ struct boot_domain {struct boot_module *configs[BUILD_MAX_CONF_MODS ];struct domain *domain; +struct { +xen_pfn_t mfn; +unsigned int evtchn; + } store, console; +bool constructed; + };
Stray blank line? Or maybe you meant it to go in front of "constructed"?
Jan
PATCH 17/18 builder: introduce domain builder hypfs tree
In reply to Daniel P. SmithThis enables domain builder to construct a hypfs tree to expose relevant domain creation information for use by the boot domain and/or the runtime system.
--- xen/common/domain-builder/Kconfig | 11 ++ xen/common/domain-builder/Makefile | 1 + xen/common/domain-builder/core.c | 3 + xen/common/domain-builder/hypfs.c | 193 +++++++++++++++++++++++++++++ xen/include/xen/domain_builder.h | 13 ++ 5 files changed, 221 insertions(+) create mode 100644 xen/common/domain-builder/hypfs.c
xen/common/domain-builder/Kconfig +11 -0
@@ -22,4 +22,15 @@ config MULTIDOM_BUILDER If unsure, say N. +config BUILDER_HYPFS + bool "Domain builder hypfs support (UNSUPPORTED)" if UNSUPPORTED + depends on HYPFS + ---help--- + Exposes the domain builder construction information + through hypfs. + + This feature is currently experimental. + + If unsure, say N. + endmenu
xen/common/domain-builder/Makefile +1 -0
@@ -1,2 +1,3 @@ obj-$(CONFIG_BUILDER_FDT) += fdt.o +obj-$(CONFIG_HYPFS) += hypfs.o obj-y += core.o
xen/common/domain-builder/core.c +3 -0
@@ -134,6 +134,9 @@ uint32_t __init builder_create_domains(struct boot_info *info)/* Free temporary buffers. */ discard_initial_images (); +if (IS_ENABLED (CONFIG_BUILDER_HYPFS ) ) +builder_hypfs (info); +return build_count; }
xen/common/domain-builder/hypfs.c new file +193 -0
193 lines not on this page · raw message
xen/include/xen/domain_builder.h +13 -0
@@ -72,4 +72,17 @@ int alloc_system_evtchn( const struct boot_info *info, struct boot_domain *bd);void arch_create_dom (const struct boot_info *bi,struct boot_domain *bd); +#ifdef CONFIG_HYPFS + +void builder_hypfs (struct boot_info *info); + +#else + +static inline void builder_hypfs (struct boot_info *info) +{ +return ; +} + +#endif +#endif /* XEN_DOMAIN_BUILDER_H */
Signature · 1 line
2.20.1
Re: [PATCH v1 17/18] builder: introduce domain builder hypfs tree
xen/common/domain-builder/core.c uint32_t __init builder_create_domains(struct boot_info *info) · line 138
Quoted 8 lines · Daniel P. Smith
xen/common/domain-builder/core.c +2
-0@@ -134,6 +134,9 @@ uint32_t __init builder_create_domains(struct boot_info *info)/* Free temporary buffers. */ discard_initial_images (); +if (IS_ENABLED (CONFIG_BUILDER_HYPFS ) ) +builder_hypfs (info);
No need for the if() here when you provide a stub in the header file. Just that of course the stub vs prototype there needs to depend on CONFIG_BUILDER_HYPFS, not CONFIG_HYPFS.
xen/common/domain-builder/core.c uint32_t __init builder_create_domains(struct boot_info *info) · line 142
Quoted 4 lines
xen/common/domain-builder/core.c +4
-0+static int __init alloc_hypfs (struct boot_info *info) +{ +if ( !(builder_dir = (struct hypfs_entry_dir *)xmalloc_bytes ( +sizeof (struct hypfs_entry_dir ))) )
Why not xmalloc() (or xzalloc()), which specifically exists to avoid open-coded casts like the one here?
Quoted 6 lines
xen/common/domain-builder/core.c +6
-0+ { +printk (XENLOG_WARNING "%s: unable to allocate hypfs dir\n" , __func__); +return -ENOMEM ; + } + + builder_dir->e.type =XEN_HYPFS_TYPE_DIR ;
xen/common/domain-builder/core.c uint32_t __init builder_create_domains(struct boot_info *info)
Quoted 10 lines
xen/common/domain-builder/core.c +10
-0+ builder_dir->e.encoding =XEN_HYPFS_ENC_PLAIN ; + builder_dir->e.name ="builder" ; + builder_dir->e.size =0 ; + builder_dir->e.max_size =0 ; +INIT_LIST_HEAD (&builder_dir->e.list); + builder_dir->e.funcs = &hypfs_dir_funcs; +INIT_LIST_HEAD (&builder_dir->dirlist); + +if ( !(entries = (struct domain_node *)xmalloc_bytes ( +sizeof (struct domain_node ) * info->builder->nr_doms)) )
xmalloc_array()?
xen/common/domain-builder/core.c
Quoted 13 lines
xen/common/domain-builder/core.c +13
-0+ { +printk (XENLOG_WARNING "%s: unable to allocate hypfs nodes\n" , __func__); +return -ENOMEM ; + } + +return 0 ; +} + +void __init builder_hypfs (struct boot_info *info) +{ +int i; + +printk ("Domain Builder: creating hypfs nodes\n" );
If at all, then dprintk().
Quoted 5 lines
xen/common/domain-builder/core.c +5
-0+if (alloc_hypfs (info) !=0 ) +return ; + +for ( i =0 ; i < info->builder->nr_doms; i++ ) + {
xen/common/domain-builder/core.c
Quoted 10 lines
xen/common/domain-builder/core.c +10
-0+struct domain_node *e = &entries[i]; +struct boot_domain *bd = &info->builder->domains[i]; +uint8_t *uuid = bd->uuid; + +snprintf (e->dir_name,sizeof (e->dir_name),"%d" , bd->domid); + +snprintf (e->uuid,sizeof (e->uuid),"%08x-%04x-%04x-%04x-%04x%08x" , + *(uint32_t *)uuid, *(uint16_t *)(uuid+4 ), + *(uint16_t *)(uuid+6 ), *(uint16_t *)(uuid+8 ), + *(uint16_t *)(uuid+10 ), *(uint32_t *)(uuid+12 ));
Perhaps better introduce a properly typed structure? Endian-ness-wise I'm also unsure about the last 12 nibbles: Isn't this an array of bytes really? Actually the second-to-last 16-bit item is an array of two bytes as well, if Linux'es %pU vsprintf() formatting is to be trusted.
xen/common/domain-builder/core.c
Quoted 6 lines
xen/common/domain-builder/core.c +6
-0+ e->functions = bd->functions; + e->constructed = bd->constructed; + + e->ncpus = bd->ncpus; + e->mem_size = (bd->meminfo.mem_size.nr_pages *PAGE_SIZE )/1024 ; + e->mem_max = (bd->meminfo.mem_max.nr_pages *PAGE_SIZE )/1024 ;
Nit: Blanks around / please.
Quoted 8 lines
xen/common/domain-builder/core.c +8
-0+ e->xs.evtchn = bd->store.evtchn; + e->xs.mfn = bd->store.mfn; + + e->con_dev.evtchn = bd->console.evtchn; + e->con_dev.mfn = bd->console.mfn; + +/* Initialize and construct builder hypfs tree */ +INIT_HYPFS_DIR (e->dir, e->dir_name);
xen/common/domain-builder/core.c
Quoted 10 lines
xen/common/domain-builder/core.c +10
-0+INIT_HYPFS_DIR (e->xs.dir,"xenstore" ); +INIT_HYPFS_DIR (e->dev_dir,"devices" ); +INIT_HYPFS_DIR (e->con_dev.dir,"console" ); + +INIT_HYPFS_STRING (e->uuid_leaf,"uuid" ); +hypfs_string_set_reference (&e->uuid_leaf, e->uuid); +INIT_HYPFS_UINT (e->func_leaf,"functions" , e->functions); +INIT_HYPFS_UINT (e->ncpus_leaf,"ncpus" , e->ncpus); +INIT_HYPFS_UINT (e->mem_sz_leaf,"mem_size" , e->mem_size); +INIT_HYPFS_UINT (e->mem_mx_leaf,"mem_max" , e->mem_max);
May I suggest to prefer - over _ in node names?
Quoted 8 lines
xen/include/xen/domain_builder.h +2
-0@@ -72,4 +72,17 @@ int alloc_system_evtchn( const struct boot_info *info, struct boot_domain *bd);void arch_create_dom (const struct boot_info *bi,struct boot_domain *bd); +#ifdef CONFIG_HYPFS +
xen/include/xen/domain_builder.h int alloc_system_evtchn( · line 86
Quoted 10 lines
xen/include/xen/domain_builder.h +10
-0+void builder_hypfs (struct boot_info *info); + +#else + +static inline void builder_hypfs (struct boot_info *info) +{ +return ; +} + +#endif
This would better go in a private header in xen/common/domain-builder/.
Jan
PATCH 18/18 tools: introduce example late pv helper
In reply to Daniel P. SmithThe late pv helper is an example helper tool for late setup of Xenstore for a domain that was created by the hypervisor using hyperlaunch.
--- .gitignore | 1 + tools/helpers/Makefile | 11 ++ tools/helpers/builder-hypfs.c | 253 ++++++++++++++++++++++++++++++ tools/helpers/hypfs-helpers.h | 9 ++ tools/helpers/late-init-pv.c | 287 ++++++++++++++++++++++++++++++++++ tools/helpers/late-init-pv.h | 29 ++++ tools/helpers/xs-helpers.c | 117 ++++++++++++++ tools/helpers/xs-helpers.h | 27 ++++ 8 files changed, 734 insertions(+) create mode 100644 tools/helpers/builder-hypfs.c create mode 100644 tools/helpers/hypfs-helpers.h create mode 100644 tools/helpers/late-init-pv.c create mode 100644 tools/helpers/late-init-pv.h create mode 100644 tools/helpers/xs-helpers.c create mode 100644 tools/helpers/xs-helpers.h
.gitignore +1 -0
@@ -206,6 +206,7 @@ tools/fuzz/x86_instruction_emulator/x86_emulate tools/fuzz/x86_instruction_emulator/x86-emulate.[ch] tools/helpers/init-xenstore-domain tools/helpers/xen-init-dom0 +tools/helpers/late-init-pv tools/hotplug/common/hotplugpath.sh tools/hotplug/FreeBSD/rc.d/xencommons tools/hotplug/FreeBSD/rc.d/xendriverdomain
tools/helpers/Makefile +11 -0
@@ -14,6 +14,7 @@ ifeq ($(CONFIG_ARM),y) PROGS += init-dom0less endif endif +PROGS += late-init-pv XEN_INIT_DOM0_OBJS = xen-init-dom0.o init-dom-json.o $(XEN_INIT_DOM0_OBJS): CFLAGS += $(CFLAGS_libxentoollog)
@@ -36,6 +37,13 @@ $(INIT_DOM0LESS_OBJS): CFLAGS += $(CFLAGS_libxenlight) $(INIT_DOM0LESS_OBJS): CFLAGS += $(CFLAGS_libxenctrl) $(INIT_DOM0LESS_OBJS): CFLAGS += $(CFLAGS_libxenevtchn) +LATE_INIT_PV_OBJS = late-init-pv.o builder-hypfs.o xs-helpers.o +$(LATE_INIT_PV_OBJS): CFLAGS += $(CFLAGS_libxentoollog) +$(LATE_INIT_PV_OBJS): CFLAGS += $(CFLAGS_libxenguest) +$(LATE_INIT_PV_OBJS): CFLAGS += $(CFLAGS_libxenctrl) +$(LATE_INIT_PV_OBJS): CFLAGS += $(CFLAGS_libxenhypfs) +$(LATE_INIT_PV_OBJS): CFLAGS += $(CFLAGS_libxenstore) + .PHONY: all all: $(PROGS)
@@ -48,6 +56,9 @@ init-xenstore-domain: $(INIT_XENSTORE_DOMAIN_OBJS) init-dom0less: $(INIT_DOM0LESS_OBJS) $(CC) $(LDFLAGS) -o $@ $(INIT_DOM0LESS_OBJS) $(LDLIBS_libxenctrl) $(LDLIBS_libxenevtchn) $(LDLIBS_libxentoollog) $(LDLIBS_libxenstore) $(LDLIBS_libxenlight) $(LDLIBS_libxenguest) $(LDLIBS_libxenforeignmemory) $(APPEND_LDFLAGS) +late-init-pv: $(LATE_INIT_PV_OBJS) + $(CC) $(LDFLAGS) -o $@ $(LATE_INIT_PV_OBJS) $(LDLIBS_libxentoollog) $(LDLIBS_libxenstore) $(LDLIBS_libxenctrl) $(LDLIBS_libxenguest) $(LDLIBS_libxenhypfs) $(APPEND_LDFLAGS) + .PHONY: install install: all $(INSTALL_DIR) $(DESTDIR)$(LIBEXEC_BIN)
tools/helpers/builder-hypfs.c new file +253 -0
253 lines not on this page · raw message
tools/helpers/hypfs-helpers.h new file +9 -0
@@ -0,0 +1,9 @@ +#ifndef __HYPFS_HELPERS_H +#define __HYPFS_HELPERS_H + +#include "late-init-pv.h" + +bool has_builder_hypfs (xenhypfs_handle *hdl,uint32_t domid); +int read_hypfs_tree (xenhypfs_handle *hdl,struct domain_info *di); + +#endif
tools/helpers/late-init-pv.c new file +287 -0
287 lines not on this page · raw message
tools/helpers/late-init-pv.h new file +29 -0
@@ -0,0 +1,29 @@ +#ifndef __LATE_INIT_PV_H +#define __LATE_INIT_PV_H + +struct domain_info { +uint16_t domid; +bool is_hvm; +bool override_uuid; +const char *uuid; +uint32_t num_cpu; +uint32_t max_cpu; +struct { +uint32_t target; +uint32_t max; +uint32_t video; + } mem_info; +struct { +uint16_t be_domid; +uint32_t evtchn_port; +uint64_t mfn; + } xs_info; +struct { +bool enable; +uint16_t be_domid; +uint32_t evtchn_port; +uint64_t mfn; + } cons_info; +}; + +#endif
tools/helpers/xs-helpers.c new file +117 -0
117 lines not on this page · raw message
tools/helpers/xs-helpers.h new file +27 -0
@@ -0,0 +1,27 @@ +#ifndef __XS_HELPERS_H +#define __XS_HELPERS_H + +#include <xenstore.h> + +int do_xs_start_transaction (struct xs_handle *xsh); +int do_xs_end_transaction (struct xs_handle *xsh); + +int do_xs_write (struct xs_handle *xsh,char *path,char *val); +int do_xs_perms ( +struct xs_handle *xsh,char *path,struct xs_permissions *perms, +uint32_t num_perms); + +int do_xs_write_dir_node_with_perm ( +struct xs_handle *xsh,char *dir,char *node,char *val, +struct xs_permissions *perms,uint32_t num_perms); +int do_xs_write_dir_node ( +struct xs_handle *xsh,char *dir,char *node,char *val); + +int do_xs_write_dom_with_perm ( +struct xs_handle *xsh,uint32_t domid,char *path,char *val, +struct xs_permissions *perms,uint32_t num_perms); +int do_xs_write_dom ( +struct xs_handle *xsh,uint32_t domid,char *path,char *val); + +#endif +
Signature · 1 line
2.20.1
Hi Daniel,
Quoted 2 lines
-----Original Message-----
Subject: [PATCH v1 00/18] Hyperlaunch
With the adjustments that I suggested in other messages, this patch builds and boots for me on x86 (including a device tree with a domU). I will continue to poke around and see if I discover any other rough edges.
One strange behavior I see is that xen fails to start the Dom0 kernel on a warm reboot. I'm using qemu_system_x86 with the KVM backend to test out the patch. After starting qemu, xen will boot correctly only once. If I attempt to reboot the virtual system (through the 'reboot' command in dom0 or the 'system_reset' qemu monitor command) without exiting/starting a new qemu process on the host machine, xen panics while booting after printing this:
(XEN) *** Building Dom0 ***
(XEN) Dom0 has maximum 856 PIRQs
(XEN) *** Constructing a PV Dom0 *** (XEN) ELF: not an ELF binary
(XEN)
(XEN) **************************************** (XEN) Panic on CPU 0:
(XEN) Could not construct domain 0 (XEN) ****************************************
This happens with the BUILDER_FDT config option on and off, and regardless of what dtb (if any) I pass to xen. I don't see this behavior if I switch back to xen's master branch.
Hopefully that explanation made sense. Let me know if I can provide any further information about my setup.
Thanks,
Jackson
Also, I apologize that my last messages included a digital signature. Should be fixed now.
Quoted 6 lines · Jackson
Hi Daniel,
Quoted 2 lines
-----Original Message-----
Subject: [PATCH v1 00/18] HyperlaunchWith the adjustments that I suggested in other messages, this patch builds and boots for me on x86 (including a device tree with a domU). I will continue to poke around and see if I discover any other rough edges.
Thank you so much for reviewing and testing!
Quoted 12 lines
One strange behavior I see is that xen fails to start the Dom0 kernel on a warm reboot. I'm using
qemu_system_x86with the KVM backend to test out the patch. After starting qemu, xen will boot correctly only once. If I attempt to reboot the virtual system (through the 'reboot' command in dom0 or the 'system_reset' qemu monitor command) without exiting/starting a new qemu process on the host machine, xen panics while booting after printing this:(XEN) *** Building Dom0 ***
(XEN) Dom0 has maximum 856 PIRQs
(XEN) *** Constructing a PV Dom0 *** (XEN) ELF: not an ELF binary
(XEN)
(XEN) **************************************** (XEN) Panic on CPU 0:
(XEN) Could not construct domain 0 (XEN) ****************************************
Quoted 3 lines
This happens with the
BUILDER_FDTconfig option on and off, and regardless of what dtb (if any) I pass to xen. I don't see this behavior if I switch back to xen's master branch.Hopefully that explanation made sense. Let me know if I can provide any further information about my setup.
That is certainly a very strange behavior. I never tested reboot as I assumed it should just go through the same process as could boot. I will add this to my tests to run and see if I can track down why it is happening.
V/r,
Daniel P. Smith