Skip to contentSkip to conversation

Add support for ISO9660 image compatible with Legacy and EFI BIOS

PATCH Applied Kory Maincent 6 participants 45 messages Sep 23, 2021 – Oct 20, 2021

lore
Share & tools
lore
#0 lore raw

v3 cover Add support for ISO9660 image compatible with Legacy and EFI BIOS

This series of patches aims to support the generation of an ISO9660 hybrid image compatible with Legacy BIOS and EFI BIOS. To implement this, we need to improve the grub2 package and modify the ISO9660 image support

Grub2 was written to build only one configuration at a time. For the hybrid image we need to have several configuration of Grub2 in the same image. For example we might want to have Grub2 built for BIOS, EFI 32 bits and EFI 64 bits in the same image. To support this, we chose to fill a list of configuration name tuples, and then build each Grub2 configuration in a separate build-$(tuple) folder. It seems simpler than having multiple duplicated grub2 packages for each possible configuration.

The generation of ISO9660 image was only supporting bootloaders based on Legacy BIOS boot. We first change the ISO9660 image generation to use xorriso instead of genimageiso, in order to be able to build an image compatible with both legacy and EFI BIOS. Then we add the generation of an EFI System Partition in iso9660 so that we can install the EFI-compatible bootloader."

In detail:

- PATCH 1 drop the usage of the useless startup.nsh file
- PATCH 2 implements simultaneous build of GRUB for different configurations
- PATCH 3 implements the generation of ISO9660 image booting on a EFI BIOS
- PATCH 4 implements the generation of hybrid image compatible with Legacy and EFI BIOS
- PATCH 5 updates the encoding of the text return from testing emulator
- PATCH 6 add support to i386 architecture to edk2 package
- PATCH 7 updates iso9660 tests and implements a test for EFI image and hybrid image.

Changes in v2:
- Update the typo of Grub2 configuration tuples to make more legible code
- Expand explanation in few commit messages.
- Fix typos.
- Fix Grub2 Legacy builtin configurations.
- Add mkfs and mcopy parameters in iso9660 package to build reproducible images.
- Remove the implementation of host-efi-bios package.
- Add support for i386 architecture to edk2 package.

Changes in v3:
- Add a patch to drop all the startup.nsh usage in Buildroot. The EFI payload naming is sufficient to choose the right boot payload.
- Update the Grub2 tuples management to have a more readable package.
- Fix ISO9660 EFI image generation options.
- Fix Kconfig options between EFI and BIOS bootloader in the ISO9660 package.

Thanks in advance for your review and feedback

Kory Maincent (7):
board, boot, package: remove usage of startup.nsh in EFI partition boot/grub2: add support to build multiple Grub2 configurations in the
same build
fs/iso9660: add support to Grub EFI bootloader in the image fs/iso9660: add support for hybrid image using Grub bootloader on BIOS
and EFI
support/testing/infra/emulator.py: update encoding when calling qemu boot/edk2: add support to i386 architecture support/testing/tests/fs/test_iso9660.py: add support to test using
EFI BIOS

 Config.in.legacy                         |  24 +++
 board/aarch64-efi/genimage-efi.cfg       |   3 -
 board/intel/galileo/genimage.cfg         |   3 -
 board/minnowboard/genimage.cfg           |   3 -
 board/pc/genimage-efi.cfg                |   3 -
 board/pc/post-build.sh                   |   2 +-
 board/qemu/aarch64-sbsa/genimage.cfg     |   3 -
 boot/edk2/Config.in                      |  12 +-
 boot/edk2/edk2.mk                        |  12 +-
 boot/grub2/Config.in                     |  53 +++++--
 boot/grub2/grub2.mk                      | 181 +++++++++++++----------
 boot/gummiboot/gummiboot.mk              |   2 -
 fs/iso9660/Config.in                     |  27 +++-
 fs/iso9660/iso9660.mk                    |  64 +++++++-
 package/systemd/systemd.mk               |   2 -
 support/testing/conf/grub2-efi.cfg       |   2 +
 support/testing/infra/emulator.py        |   2 +-
 support/testing/tests/fs/test_iso9660.py |  80 +++++++++-
 18 files changed, 342 insertions(+), 136 deletions(-)
 create mode 100644 support/testing/conf/grub2-efi.cfg
Signature · 6 lines
2.25.1

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#1 lore raw

PATCH v3 1/7 board, boot, package: remove usage of startup.nsh in EFI partition

The startup.nsh file is useless to boot EFI payloads. We just need to follow the naming detection specified in the UEFI spec.
The EFI payload need to be placed in the boot/efi folder in the EFI partition and follow the architecture naming as described below:
32bit : bootia32.efi
x64 : bootx64.efi
aarch32 : bootarm.efi
aarch64 : bootaa64.efi

This naming is already right in the packages involved (systemd, grub2, gummiboot), therefore we just need to drop the generation of the startup.nsh file.

The usage of the startup.nsh in genimage is also dropped to avoid errors in the image generation.

---
 board/aarch64-efi/genimage-efi.cfg   | 3 ---
 board/intel/galileo/genimage.cfg     | 3 ---
 board/minnowboard/genimage.cfg       | 3 ---
 board/pc/genimage-efi.cfg            | 3 ---
 board/pc/post-build.sh               | 2 +-
 board/qemu/aarch64-sbsa/genimage.cfg | 3 ---
 boot/grub2/grub2.mk                  | 8 --------
 boot/gummiboot/gummiboot.mk          | 2 --
 package/systemd/systemd.mk           | 2 --
 9 files changed, 1 insertion(+), 28 deletions(-)
board/aarch64-efi/genimage-efi.cfg +0 -3
index 34cca1d64e..2daa70c361 100644
@@ -1,8 +1,5 @@
 image efi-part.vfat {
   vfat {
-    file startup.nsh {
-      image = "efi-part/startup.nsh"
-    }
     file EFI {
       image = "efi-part/EFI"
     }
board/intel/galileo/genimage.cfg +0 -3
index 31add21e67..b930c0deb9 100644
@@ -1,9 +1,6 @@
 # Create an image of the efi partition
 image efi-part.vfat {
 	vfat {
-		file startup.nsh {
-			image = "efi-part/startup.nsh"
-		}
 		file EFI {
 			image = "efi-part/EFI"
 		}
board/minnowboard/genimage.cfg +0 -3
index c5b07179b4..521385aadd 100644
@@ -1,9 +1,6 @@
 # Create an image of the efi partition
 image efi-part.vfat {
 	vfat {
-		file startup.nsh {
-			image = "efi-part/startup.nsh"
-		}
 		file EFI {
 			image = "efi-part/EFI"
 		}
board/pc/genimage-efi.cfg +0 -3
index ec7e85b06c..34befb7d69 100644
@@ -1,8 +1,5 @@
 image efi-part.vfat {
   vfat {
-    file startup.nsh {
-      image = "efi-part/startup.nsh"
-    }
     file EFI {
       image = "efi-part/EFI"
     }
board/pc/post-build.sh +1 -1
index ed37b3b87e..db23795e27 100755
@@ -5,7 +5,7 @@ set -e
 BOARD_DIR=$(dirname "$0")
 
 # Detect boot strategy, EFI or BIOS
-if [ -f "$BINARIES_DIR/efi-part/startup.nsh" ]; then
+if [ -d "$BINARIES_DIR/efi-part/" ]; then
     cp -f "$BOARD_DIR/grub-efi.cfg" "$BINARIES_DIR/efi-part/EFI/BOOT/grub.cfg"
 else
     cp -f "$BOARD_DIR/grub-bios.cfg" "$TARGET_DIR/boot/grub/grub.cfg"
board/qemu/aarch64-sbsa/genimage.cfg +0 -3
index 285b308d90..c75d9fcca1 100644
@@ -1,8 +1,5 @@
 image efi-part.vfat {
   vfat {
-    file startup.nsh {
-      image = "efi-part/startup.nsh"
-    }
     file EFI {
       image = "efi-part/EFI"
     }
boot/grub2/grub2.mk +0 -8
index 52e9199ae9..ad2edf17aa 100644
@@ -168,13 +168,5 @@ define GRUB2_INSTALL_IMAGES_CMDS
 	$(GRUB2_IMAGE_INSTALL_ELTORITO)
 endef
 
-ifeq ($(GRUB2_PLATFORM),efi)
-define GRUB2_EFI_STARTUP_NSH
-	echo $(notdir $(GRUB2_IMAGE)) > \
-		$(BINARIES_DIR)/efi-part/startup.nsh
-endef
-GRUB2_POST_INSTALL_IMAGES_HOOKS += GRUB2_EFI_STARTUP_NSH
-endif
-
 $(eval $(autotools-package))
 $(eval $(host-autotools-package))
boot/gummiboot/gummiboot.mk +0 -2
index 748e87030e..eb1f3da78c 100644
@@ -32,8 +32,6 @@ GUMMIBOOT_CONF_OPTS = \
 define GUMMIBOOT_INSTALL_IMAGES_CMDS
 	$(INSTALL) -D -m 0644 $(@D)/gummiboot$(GUMMIBOOT_IMGARCH).efi \
 		$(BINARIES_DIR)/efi-part/EFI/BOOT/boot$(GUMMIBOOT_IMGARCH).efi
-	echo "boot$(GUMMIBOOT_IMGARCH).efi" > \
-		$(BINARIES_DIR)/efi-part/startup.nsh
 	$(INSTALL) -D -m 0644 boot/gummiboot/loader.conf \
 		$(BINARIES_DIR)/efi-part/loader/loader.conf
 	$(INSTALL) -D -m 0644 boot/gummiboot/buildroot.conf \
package/systemd/systemd.mk +0 -2
index 5a76f2f246..c403896759 100644
@@ -514,8 +514,6 @@ SYSTEMD_BOOT_EFI_ARCH = $(call qstrip,$(BR2_PACKAGE_SYSTEMD_BOOT_EFI_ARCH))
 define SYSTEMD_INSTALL_BOOT_FILES
 	$(INSTALL) -D -m 0644 $(@D)/build/src/boot/efi/systemd-boot$(SYSTEMD_BOOT_EFI_ARCH).efi \
 		$(BINARIES_DIR)/efi-part/EFI/BOOT/boot$(SYSTEMD_BOOT_EFI_ARCH).efi
-	echo "boot$(SYSTEMD_BOOT_EFI_ARCH).efi" > \
-		$(BINARIES_DIR)/efi-part/startup.nsh
 	$(INSTALL) -D -m 0644 $(SYSTEMD_PKGDIR)/boot-files/loader.conf \
 		$(BINARIES_DIR)/efi-part/loader/loader.conf
 	$(INSTALL) -D -m 0644 $(SYSTEMD_PKGDIR)/boot-files/buildroot.conf \
Signature · 6 lines
2.25.1

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#2 lore raw

Re: [Buildroot] [PATCH v3 1/7] board, boot, package: remove usage of startup.nsh in EFI partition

Köry, All,

+Dick, Erico, Gerome, Nicholas, Peter, Romain:

This patch touches boards for which you are registered in the DEVELOPERS file. Could you verify that those boards still boot with this patch applied, please?

Otherwise, I have no comment on the patch; good! :-)

Regards,
Yann E. MORIN.

On 2021-09-23 17:57 +0200, Kory Maincent spake thusly:

Quoted 156 lines

The startup.nsh file is useless to boot EFI payloads. We just need to follow the naming detection specified in the UEFI spec.
The EFI payload need to be placed in the boot/efi folder in the EFI partition and follow the architecture naming as described below:
32bit : bootia32.efi
x64 : bootx64.efi
aarch32 : bootarm.efi
aarch64 : bootaa64.efi

This naming is already right in the packages involved (systemd, grub2, gummiboot), therefore we just need to drop the generation of the startup.nsh file.

The usage of the startup.nsh in genimage is also dropped to avoid errors in the image generation.

---
 board/aarch64-efi/genimage-efi.cfg   | 3 ---
 board/intel/galileo/genimage.cfg     | 3 ---
 board/minnowboard/genimage.cfg       | 3 ---
 board/pc/genimage-efi.cfg            | 3 ---
 board/pc/post-build.sh               | 2 +-
 board/qemu/aarch64-sbsa/genimage.cfg | 3 ---
 boot/grub2/grub2.mk                  | 8 --------
 boot/gummiboot/gummiboot.mk          | 2 --
 package/systemd/systemd.mk           | 2 --
 9 files changed, 1 insertion(+), 28 deletions(-)
board/aarch64-efi/genimage-efi.cfg +0 -3
index 34cca1d64e..2daa70c361 100644
@@ -1,8 +1,5 @@
 image efi-part.vfat {
   vfat {
-    file startup.nsh {
-      image = "efi-part/startup.nsh"
-    }
     file EFI {
       image = "efi-part/EFI"
     }
board/intel/galileo/genimage.cfg +0 -3
index 31add21e67..b930c0deb9 100644
@@ -1,9 +1,6 @@
 # Create an image of the efi partition
 image efi-part.vfat {
 	vfat {
-		file startup.nsh {
-			image = "efi-part/startup.nsh"
-		}
 		file EFI {
 			image = "efi-part/EFI"
 		}
board/minnowboard/genimage.cfg +0 -3
index c5b07179b4..521385aadd 100644
@@ -1,9 +1,6 @@
 # Create an image of the efi partition
 image efi-part.vfat {
 	vfat {
-		file startup.nsh {
-			image = "efi-part/startup.nsh"
-		}
 		file EFI {
 			image = "efi-part/EFI"
 		}
board/pc/genimage-efi.cfg +0 -3
index ec7e85b06c..34befb7d69 100644
@@ -1,8 +1,5 @@
 image efi-part.vfat {
   vfat {
-    file startup.nsh {
-      image = "efi-part/startup.nsh"
-    }
     file EFI {
       image = "efi-part/EFI"
     }
board/pc/post-build.sh +1 -1
index ed37b3b87e..db23795e27 100755
@@ -5,7 +5,7 @@ set -e
 BOARD_DIR=$(dirname "$0")
 
 # Detect boot strategy, EFI or BIOS
-if [ -f "$BINARIES_DIR/efi-part/startup.nsh" ]; then
+if [ -d "$BINARIES_DIR/efi-part/" ]; then
     cp -f "$BOARD_DIR/grub-efi.cfg" "$BINARIES_DIR/efi-part/EFI/BOOT/grub.cfg"
 else
     cp -f "$BOARD_DIR/grub-bios.cfg" "$TARGET_DIR/boot/grub/grub.cfg"
board/qemu/aarch64-sbsa/genimage.cfg +0 -3
index 285b308d90..c75d9fcca1 100644
@@ -1,8 +1,5 @@
 image efi-part.vfat {
   vfat {
-    file startup.nsh {
-      image = "efi-part/startup.nsh"
-    }
     file EFI {
       image = "efi-part/EFI"
     }
boot/grub2/grub2.mk +0 -8
index 52e9199ae9..ad2edf17aa 100644
@@ -168,13 +168,5 @@ define GRUB2_INSTALL_IMAGES_CMDS
 	$(GRUB2_IMAGE_INSTALL_ELTORITO)
 endef
 
-ifeq ($(GRUB2_PLATFORM),efi)
-define GRUB2_EFI_STARTUP_NSH
-	echo $(notdir $(GRUB2_IMAGE)) > \
-		$(BINARIES_DIR)/efi-part/startup.nsh
-endef
-GRUB2_POST_INSTALL_IMAGES_HOOKS += GRUB2_EFI_STARTUP_NSH
-endif
-
 $(eval $(autotools-package))
 $(eval $(host-autotools-package))
boot/gummiboot/gummiboot.mk +0 -2
index 748e87030e..eb1f3da78c 100644
@@ -32,8 +32,6 @@ GUMMIBOOT_CONF_OPTS = \
 define GUMMIBOOT_INSTALL_IMAGES_CMDS
 	$(INSTALL) -D -m 0644 $(@D)/gummiboot$(GUMMIBOOT_IMGARCH).efi \
 		$(BINARIES_DIR)/efi-part/EFI/BOOT/boot$(GUMMIBOOT_IMGARCH).efi
-	echo "boot$(GUMMIBOOT_IMGARCH).efi" > \
-		$(BINARIES_DIR)/efi-part/startup.nsh
 	$(INSTALL) -D -m 0644 boot/gummiboot/loader.conf \
 		$(BINARIES_DIR)/efi-part/loader/loader.conf
 	$(INSTALL) -D -m 0644 boot/gummiboot/buildroot.conf \
package/systemd/systemd.mk +0 -2
index 5a76f2f246..c403896759 100644
@@ -514,8 +514,6 @@ SYSTEMD_BOOT_EFI_ARCH = $(call qstrip,$(BR2_PACKAGE_SYSTEMD_BOOT_EFI_ARCH))
 define SYSTEMD_INSTALL_BOOT_FILES
 	$(INSTALL) -D -m 0644 $(@D)/build/src/boot/efi/systemd-boot$(SYSTEMD_BOOT_EFI_ARCH).efi \
 		$(BINARIES_DIR)/efi-part/EFI/BOOT/boot$(SYSTEMD_BOOT_EFI_ARCH).efi
-	echo "boot$(SYSTEMD_BOOT_EFI_ARCH).efi" > \
-		$(BINARIES_DIR)/efi-part/startup.nsh
 	$(INSTALL) -D -m 0644 $(SYSTEMD_PKGDIR)/boot-files/loader.conf \
 		$(BINARIES_DIR)/efi-part/loader/loader.conf
 	$(INSTALL) -D -m 0644 $(SYSTEMD_PKGDIR)/boot-files/buildroot.conf \
Signature · 1 line
2.25.1
Signature · 10 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#3 lore raw

Re: [Buildroot] [PATCH v3 1/7] board, boot, package: remove usage of startup.nsh in EFI partition

Quoted 8 lines · Yann E. MORIN

Köry, All,

+Dick, Erico, Gerome, Nicholas, Peter, Romain:

This patch touches boards for which you are registered in the DEVELOPERS file. Could you verify that those boards still boot with this patch applied, please?

The patch looks ok and on a quick test, both aarch64_efi_defconfig and pc_x86_64_efi_defconfig continue to work as usual. So looks good to me.

Erico
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

#4 lore raw

Re: [Buildroot] [PATCH v3 1/7] board, boot, package: remove usage of startup.nsh in EFI partition

Erico, All,

On 2021-09-24 22:28 +0200, Erico Nunes spake thusly:

Quoted 9 lines · Yann E. MORIN
Quoted 5 lines

+Dick, Erico, Gerome, Nicholas, Peter, Romain:

This patch touches boards for which you are registered in the DEVELOPERS file. Could you verify that those boards still boot with this patch applied, please?

The patch looks ok and on a quick test, both aarch64_efi_defconfig and pc_x86_64_efi_defconfig continue to work as usual. So looks good to me.

Thanks for the feedback; I've made that a Tested-by tag from you when applying. :-)

Regards,
Yann E. MORIN.

Signature · 10 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#5 lore raw

Re: [Buildroot] [PATCH v3 1/7] board, boot, package: remove usage of startup.nsh in EFI partition

In reply to Kory Maincent

Köry, All,

On 2021-09-23 17:57 +0200, Kory Maincent spake thusly:

Quoted 9 lines

The startup.nsh file is useless to boot EFI payloads. We just need to follow the naming detection specified in the UEFI spec.
The EFI payload need to be placed in the boot/efi folder in the EFI partition and follow the architecture naming as described below:
32bit : bootia32.efi
x64 : bootx64.efi
aarch32 : bootarm.efi
aarch64 : bootaa64.efi

Quoted 8 lines

This naming is already right in the packages involved (systemd, grub2, gummiboot), therefore we just need to drop the generation of the startup.nsh file.

The usage of the startup.nsh in genimage is also dropped to avoid errors in the image generation.

Applied to master, thanks.

Regards,
Yann E. MORIN.

Quoted 143 lines
---
 board/aarch64-efi/genimage-efi.cfg   | 3 ---
 board/intel/galileo/genimage.cfg     | 3 ---
 board/minnowboard/genimage.cfg       | 3 ---
 board/pc/genimage-efi.cfg            | 3 ---
 board/pc/post-build.sh               | 2 +-
 board/qemu/aarch64-sbsa/genimage.cfg | 3 ---
 boot/grub2/grub2.mk                  | 8 --------
 boot/gummiboot/gummiboot.mk          | 2 --
 package/systemd/systemd.mk           | 2 --
 9 files changed, 1 insertion(+), 28 deletions(-)
board/aarch64-efi/genimage-efi.cfg +0 -3
index 34cca1d64e..2daa70c361 100644
@@ -1,8 +1,5 @@
 image efi-part.vfat {
   vfat {
-    file startup.nsh {
-      image = "efi-part/startup.nsh"
-    }
     file EFI {
       image = "efi-part/EFI"
     }
board/intel/galileo/genimage.cfg +0 -3
index 31add21e67..b930c0deb9 100644
@@ -1,9 +1,6 @@
 # Create an image of the efi partition
 image efi-part.vfat {
 	vfat {
-		file startup.nsh {
-			image = "efi-part/startup.nsh"
-		}
 		file EFI {
 			image = "efi-part/EFI"
 		}
board/minnowboard/genimage.cfg +0 -3
index c5b07179b4..521385aadd 100644
@@ -1,9 +1,6 @@
 # Create an image of the efi partition
 image efi-part.vfat {
 	vfat {
-		file startup.nsh {
-			image = "efi-part/startup.nsh"
-		}
 		file EFI {
 			image = "efi-part/EFI"
 		}
board/pc/genimage-efi.cfg +0 -3
index ec7e85b06c..34befb7d69 100644
@@ -1,8 +1,5 @@
 image efi-part.vfat {
   vfat {
-    file startup.nsh {
-      image = "efi-part/startup.nsh"
-    }
     file EFI {
       image = "efi-part/EFI"
     }
board/pc/post-build.sh +1 -1
index ed37b3b87e..db23795e27 100755
@@ -5,7 +5,7 @@ set -e
 BOARD_DIR=$(dirname "$0")
 
 # Detect boot strategy, EFI or BIOS
-if [ -f "$BINARIES_DIR/efi-part/startup.nsh" ]; then
+if [ -d "$BINARIES_DIR/efi-part/" ]; then
     cp -f "$BOARD_DIR/grub-efi.cfg" "$BINARIES_DIR/efi-part/EFI/BOOT/grub.cfg"
 else
     cp -f "$BOARD_DIR/grub-bios.cfg" "$TARGET_DIR/boot/grub/grub.cfg"
board/qemu/aarch64-sbsa/genimage.cfg +0 -3
index 285b308d90..c75d9fcca1 100644
@@ -1,8 +1,5 @@
 image efi-part.vfat {
   vfat {
-    file startup.nsh {
-      image = "efi-part/startup.nsh"
-    }
     file EFI {
       image = "efi-part/EFI"
     }
boot/grub2/grub2.mk +0 -8
index 52e9199ae9..ad2edf17aa 100644
@@ -168,13 +168,5 @@ define GRUB2_INSTALL_IMAGES_CMDS
 	$(GRUB2_IMAGE_INSTALL_ELTORITO)
 endef
 
-ifeq ($(GRUB2_PLATFORM),efi)
-define GRUB2_EFI_STARTUP_NSH
-	echo $(notdir $(GRUB2_IMAGE)) > \
-		$(BINARIES_DIR)/efi-part/startup.nsh
-endef
-GRUB2_POST_INSTALL_IMAGES_HOOKS += GRUB2_EFI_STARTUP_NSH
-endif
-
 $(eval $(autotools-package))
 $(eval $(host-autotools-package))
boot/gummiboot/gummiboot.mk +0 -2
index 748e87030e..eb1f3da78c 100644
@@ -32,8 +32,6 @@ GUMMIBOOT_CONF_OPTS = \
 define GUMMIBOOT_INSTALL_IMAGES_CMDS
 	$(INSTALL) -D -m 0644 $(@D)/gummiboot$(GUMMIBOOT_IMGARCH).efi \
 		$(BINARIES_DIR)/efi-part/EFI/BOOT/boot$(GUMMIBOOT_IMGARCH).efi
-	echo "boot$(GUMMIBOOT_IMGARCH).efi" > \
-		$(BINARIES_DIR)/efi-part/startup.nsh
 	$(INSTALL) -D -m 0644 boot/gummiboot/loader.conf \
 		$(BINARIES_DIR)/efi-part/loader/loader.conf
 	$(INSTALL) -D -m 0644 boot/gummiboot/buildroot.conf \
package/systemd/systemd.mk +0 -2
index 5a76f2f246..c403896759 100644
@@ -514,8 +514,6 @@ SYSTEMD_BOOT_EFI_ARCH = $(call qstrip,$(BR2_PACKAGE_SYSTEMD_BOOT_EFI_ARCH))
 define SYSTEMD_INSTALL_BOOT_FILES
 	$(INSTALL) -D -m 0644 $(@D)/build/src/boot/efi/systemd-boot$(SYSTEMD_BOOT_EFI_ARCH).efi \
 		$(BINARIES_DIR)/efi-part/EFI/BOOT/boot$(SYSTEMD_BOOT_EFI_ARCH).efi
-	echo "boot$(SYSTEMD_BOOT_EFI_ARCH).efi" > \
-		$(BINARIES_DIR)/efi-part/startup.nsh
 	$(INSTALL) -D -m 0644 $(SYSTEMD_PKGDIR)/boot-files/loader.conf \
 		$(BINARIES_DIR)/efi-part/loader/loader.conf
 	$(INSTALL) -D -m 0644 $(SYSTEMD_PKGDIR)/boot-files/buildroot.conf \
Signature · 6 lines
2.25.1

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
Signature · 10 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#6 lore raw

PATCH v3 2/7 boot/grub2: add support to build multiple Grub2 configurations in the same build

In reply to Kory Maincent

When Grub2 is build it is configured only for one boot set-up, BIOS Legacy, EFI 32 bit or EFI 64 bit. It can not deal with several boot set-up on the same image.

This patch allows to build Grub2 for different configurations simultaneously. To cover Grub2 configuration of legacy BIOS platforms (32-bit), 32-bit EFI BIOS and 64-bit EFI BIOS in the same build, multi-build system felt much more reasonable to just extend the grub2 package into 3 packages.

We can no longer use autotools-package as a consequence of this multi-build, and we have to resort to generic-package and a partial duplication of the autotools-infra. Grub2 was already using custom option like --prefix or --exec-prefix so this won't add much more weirdness.

We use a GRUB2_TUPLES list to describe all the configurations selected. For each boot case described in the GRUB2_TUPLES list, it configures and builds Grub2 in a separate folder named build-$(tuple).
We use a foreach loop to make actions on each tuple selected.

We have to separate the BR2_TARGET_GRUB2_BUILTIN_MODULES and the BR2_TARGET_GRUB2_BUILTIN_CONFIG for each BIOS or EFI boot cases.

---

Changes in v2:
Following the review from Yann E. Morin:
- Update the tuples configurations to makes the code more readable.
- Move "--target" and "--with-platform" parameters into the GRUB2_CONFIGURE_CMD.
- Fix startup.nsh installation.
- Fix Grub2 configuration to be sure that at least one platform is always enabled.
- Fix legacy builtin configurations to string in place of bool.

Changes in v3:
Following the review from Yann E. Morin:
- Update the tuples configurations to remove the conditions and makes it more readable.
- Update the tuples configurations for builtin config and modules options.
- Fix typo.
- Remove the generation of startup.nsh.

 Config.in.legacy                         |  24 ++++
 boot/grub2/Config.in                     |  53 +++++--
 boot/grub2/grub2.mk                      | 175 +++++++++++++----------
 support/testing/tests/fs/test_iso9660.py |   6 +-
 4 files changed, 169 insertions(+), 89 deletions(-)
Config.in.legacy +24 -0
index 35a11f4dc6..33cee0202b 100644
@@ -168,6 +168,30 @@ config BR2_KERNEL_HEADERS_5_12
 
 comment "Legacy options removed in 2021.08"
 
+config BR2_TARGET_GRUB2_BUILTIN_MODULES
+	string "the grub2 builtin modules has been renamed"
+	help
+	  This option has been split to separate the builtin modules
+	  between BR2_TARGET_GRUB2_BUILTIN_MODULES_PC and
+	  BR2_TARGET_GRUB2_BUILTIN_MODULES_EFI.
+
+config BR2_TARGET_GRUB2_BUILTIN_MODULES_WRAP
+	bool
+	default y if BR2_TARGET_GRUB2_BUILTIN_MODULES != ""
+	select BR2_LEGACY
+
+config BR2_TARGET_GRUB2_BUILTIN_CONFIG
+	string "the grub2 builtin configuration has been renamed"
+	help
+	  This option has been split to separate the builtin configuration
+	  between BR2_TARGET_GRUB2_BUILTIN_CONFIG_PC and
+	  BR2_TARGET_GRUB2_BUILTIN_CONFIG_EFI.
+
+config BR2_TARGET_GRUB2_BUILTIN_CONFIG_WRAP
+	bool
+	default y if BR2_TARGET_GRUB2_BUILTIN_CONFIG != ""
+	select BR2_LEGACY
+
 config BR2_PACKAGE_LIBMCRYPT
 	bool "libmcrypt package was removed"
 	select BR2_LEGACY
boot/grub2/Config.in +41 -12

109 lines not on this page · raw message

boot/grub2/grub2.mk +101 -74

198 lines not on this page · raw message

support/testing/tests/fs/test_iso9660.py +3 -3
index 68f4840852..1b699e1fef 100644
@@ -54,7 +54,7 @@ class TestIso9660Grub2External(infra.basetest.BRTest):
         # BR2_TARGET_ROOTFS_ISO9660_INITRD is not set
         BR2_TARGET_GRUB2=y
         BR2_TARGET_GRUB2_BOOT_PARTITION="cd"
-        BR2_TARGET_GRUB2_BUILTIN_MODULES="boot linux ext2 fat part_msdos part_gpt normal biosdisk iso9660"
+        BR2_TARGET_GRUB2_BUILTIN_MODULES_PC="boot linux ext2 fat part_msdos part_gpt normal biosdisk iso9660"
         BR2_TARGET_ROOTFS_ISO9660_BOOT_MENU="{}"
         """.format(infra.filepath("conf/grub2.cfg"))
 
@@ -75,7 +75,7 @@ class TestIso9660Grub2ExternalCompress(infra.basetest.BRTest):
         BR2_TARGET_ROOTFS_ISO9660_TRANSPARENT_COMPRESSION=y
         BR2_TARGET_GRUB2=y
         BR2_TARGET_GRUB2_BOOT_PARTITION="cd"
-        BR2_TARGET_GRUB2_BUILTIN_MODULES="boot linux ext2 fat part_msdos part_gpt normal biosdisk iso9660"
+        BR2_TARGET_GRUB2_BUILTIN_MODULES_PC="boot linux ext2 fat part_msdos part_gpt normal biosdisk iso9660"
         BR2_TARGET_ROOTFS_ISO9660_BOOT_MENU="{}"
         """.format(infra.filepath("conf/grub2.cfg"))
 
@@ -95,7 +95,7 @@ class TestIso9660Grub2Internal(infra.basetest.BRTest):
         BR2_TARGET_ROOTFS_ISO9660_INITRD=y
         BR2_TARGET_GRUB2=y
         BR2_TARGET_GRUB2_BOOT_PARTITION="cd"
-        BR2_TARGET_GRUB2_BUILTIN_MODULES="boot linux ext2 fat part_msdos part_gpt normal biosdisk iso9660"
+        BR2_TARGET_GRUB2_BUILTIN_MODULES_PC="boot linux ext2 fat part_msdos part_gpt normal biosdisk iso9660"
         BR2_TARGET_ROOTFS_ISO9660_BOOT_MENU="{}"
         """.format(infra.filepath("conf/grub2.cfg"))
 
Signature · 6 lines
2.25.1

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#7 lore raw

Re: [Buildroot] [PATCH v3 2/7] boot/grub2: add support to build multiple Grub2 configurations in the same build

Köry, All,

On 2021-09-23 17:57 +0200, Kory Maincent spake thusly:

Quoted 4 lines

This patch allows to build Grub2 for different configurations simultaneously. To cover Grub2 configuration of legacy BIOS platforms (32-bit), 32-bit EFI BIOS and 64-bit EFI BIOS in the same build, multi-build system felt much more reasonable to just extend the grub2 package into 3 packages.

[--SNIP--]

Quoted 14 lines
Config.in.legacy +6 -0
index 35a11f4dc6..33cee0202b 100644
@@ -168,6 +168,30 @@ config BR2_KERNEL_HEADERS_5_12
 
 comment "Legacy options removed in 2021.08"
 
+config BR2_TARGET_GRUB2_BUILTIN_MODULES
+	string "the grub2 builtin modules has been renamed"
+	help
+	  This option has been split to separate the builtin modules
+	  between BR2_TARGET_GRUB2_BUILTIN_MODULES_PC and
+	  BR2_TARGET_GRUB2_BUILTIN_MODULES_EFI.

Config.in.legacy config BR2_KERNEL_HEADERS_5_12 · line 186

Quoted 10 lines
Config.in.legacy +10 -0
+
+config BR2_TARGET_GRUB2_BUILTIN_MODULES_WRAP
+	bool
+	default y if BR2_TARGET_GRUB2_BUILTIN_MODULES != ""
+	select BR2_LEGACY
+
+config BR2_TARGET_GRUB2_BUILTIN_CONFIG
+	string "the grub2 builtin configuration has been renamed"
+	help
+	  This option has been split to separate the builtin configuration
    $ make check-package
    Config.in.legacy:186: help text: <tab><2 spaces><62 chars>
    (http://nightly.buildroot.org/#writing-rules-config-in)

[--SNIP--]

Quoted 19 lines
boot/grub2/Config.in +7 -0
index e45133999e..10bba6c5e4 100644
@@ -10,6 +10,13 @@ config BR2_TARGET_GRUB2
 	bool "grub2"
 	depends on BR2_TARGET_GRUB2_ARCH_SUPPORTS
 	depends on BR2_USE_WCHAR
+	select BR2_TARGET_GRUB2_I386_PC if \
+		!BR2_TARGET_GRUB2_HAS_PTF && \
+		(BR2_i386 || BR2_x86_64)
+	select BR2_TARGET_GRUB2_ARM_UBOOT if \
+		!BR2_TARGET_GRUB2_HAS_PTF && \
+		BR2_arm
+	select BR2_TARGET_GRUB2_ARM64_EFI if BR2_aarch64
 	help
 	  GNU GRUB is a Multiboot boot loader. It was derived from
 	  GRUB, the GRand Unified Bootloader, which was originally

boot/grub2/Config.in config BR2_TARGET_GRUB2 · line 36

Quoted 6 lines
boot/grub2/Config.in +2 -1
@@ -25,10 +32,10 @@ config BR2_TARGET_GRUB2
 
 	  http://www.gnu.org/software/grub/
 
-if BR2_TARGET_GRUB2
+config BR2_TARGET_GRUB2_HAS_PTF
+	bool

I've moved this option inside the 'if' conditional, otherwise the suboptions are not properly indented.

Applied to master with the above two fixed, thanks.

Regards,
Yann E. MORIN.

Signature · 10 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#8 lore raw

Re: [Buildroot] [PATCH v3 2/7] boot/grub2: add support to build multiple Grub2 configurations in the same build

In reply to Kory Maincent

Köry, All,

On 2021-09-23 17:57 +0200, Kory Maincent spake thusly:
[--SNIP--]

Quoted 4 lines

We can no longer use autotools-package as a consequence of this multi-build, and we have to resort to generic-package and a partial duplication of the autotools-infra. Grub2 was already using custom option like --prefix or --exec-prefix so this won't add much more weirdness.

But this had an unfortunate side effect: the grub2 tools, like grub-editenv, enabled with BR2_TARGET_GRUB2_INSTALL_TOOLS, are no longer installed in the target now.

Indeed, it worked so far because there is a default target install command for autotools packages, which does basivally call

    make install DESTDIR=$(TARGET_DIR)

But generic-package has no default install command, so declaring that the package installs in target does nothing.

I see two options:

1. provide a custom command for one of the enabled platforms; e.g. arbitrarily choose the first in the tuple and install in target from that one tuple

2. revert to using an autotools package, but build for no platform at all and just install the tools. For the platforms, move the current foreach loops into post-build and post-install-image hooks

Care to look into that, please?

Regards,
Yann E. MORIN.

Quoted 432 lines

We use a GRUB2_TUPLES list to describe all the configurations selected. For each boot case described in the GRUB2_TUPLES list, it configures and builds Grub2 in a separate folder named build-$(tuple).
We use a foreach loop to make actions on each tuple selected.

We have to separate the BR2_TARGET_GRUB2_BUILTIN_MODULES and the BR2_TARGET_GRUB2_BUILTIN_CONFIG for each BIOS or EFI boot cases.

---

Changes in v2:
Following the review from Yann E. Morin:
- Update the tuples configurations to makes the code more readable.
- Move "--target" and "--with-platform" parameters into the GRUB2_CONFIGURE_CMD.
- Fix startup.nsh installation.
- Fix Grub2 configuration to be sure that at least one platform is always enabled.
- Fix legacy builtin configurations to string in place of bool.

Changes in v3:
Following the review from Yann E. Morin:
- Update the tuples configurations to remove the conditions and makes it more readable.
- Update the tuples configurations for builtin config and modules options.
- Fix typo.
- Remove the generation of startup.nsh.

 Config.in.legacy                         |  24 ++++
 boot/grub2/Config.in                     |  53 +++++--
 boot/grub2/grub2.mk                      | 175 +++++++++++++----------
 support/testing/tests/fs/test_iso9660.py |   6 +-
 4 files changed, 169 insertions(+), 89 deletions(-)
Config.in.legacy +24 -0
index 35a11f4dc6..33cee0202b 100644
@@ -168,6 +168,30 @@ config BR2_KERNEL_HEADERS_5_12
 
 comment "Legacy options removed in 2021.08"
 
+config BR2_TARGET_GRUB2_BUILTIN_MODULES
+	string "the grub2 builtin modules has been renamed"
+	help
+	  This option has been split to separate the builtin modules
+	  between BR2_TARGET_GRUB2_BUILTIN_MODULES_PC and
+	  BR2_TARGET_GRUB2_BUILTIN_MODULES_EFI.
+
+config BR2_TARGET_GRUB2_BUILTIN_MODULES_WRAP
+	bool
+	default y if BR2_TARGET_GRUB2_BUILTIN_MODULES != ""
+	select BR2_LEGACY
+
+config BR2_TARGET_GRUB2_BUILTIN_CONFIG
+	string "the grub2 builtin configuration has been renamed"
+	help
+	  This option has been split to separate the builtin configuration
+	  between BR2_TARGET_GRUB2_BUILTIN_CONFIG_PC and
+	  BR2_TARGET_GRUB2_BUILTIN_CONFIG_EFI.
+
+config BR2_TARGET_GRUB2_BUILTIN_CONFIG_WRAP
+	bool
+	default y if BR2_TARGET_GRUB2_BUILTIN_CONFIG != ""
+	select BR2_LEGACY
+
 config BR2_PACKAGE_LIBMCRYPT
 	bool "libmcrypt package was removed"
 	select BR2_LEGACY
boot/grub2/Config.in +41 -12
index e45133999e..10bba6c5e4 100644
@@ -10,6 +10,13 @@ config BR2_TARGET_GRUB2
 	bool "grub2"
 	depends on BR2_TARGET_GRUB2_ARCH_SUPPORTS
 	depends on BR2_USE_WCHAR
+	select BR2_TARGET_GRUB2_I386_PC if \
+		!BR2_TARGET_GRUB2_HAS_PTF && \
+		(BR2_i386 || BR2_x86_64)
+	select BR2_TARGET_GRUB2_ARM_UBOOT if \
+		!BR2_TARGET_GRUB2_HAS_PTF && \
+		BR2_arm
+	select BR2_TARGET_GRUB2_ARM64_EFI if BR2_aarch64
 	help
 	  GNU GRUB is a Multiboot boot loader. It was derived from
 	  GRUB, the GRand Unified Bootloader, which was originally
@@ -25,10 +32,10 @@ config BR2_TARGET_GRUB2
 
 	  http://www.gnu.org/software/grub/
 
-if BR2_TARGET_GRUB2
+config BR2_TARGET_GRUB2_HAS_PTF
+	bool
 
-choice
-	prompt "Platform"
+if BR2_TARGET_GRUB2
 
 config BR2_TARGET_GRUB2_I386_PC
 	bool "i386-pc"
@@ -40,6 +47,7 @@ config BR2_TARGET_GRUB2_I386_PC
 config BR2_TARGET_GRUB2_I386_EFI
 	bool "i386-efi"
 	depends on BR2_i386 || BR2_x86_64
+	select BR2_TARGET_GRUB2_HAS_PTF
 	help
 	  Select this option if the platform you're targetting has a
 	  32 bits EFI BIOS. Note that some x86-64 platforms use a 32
@@ -48,6 +56,7 @@ config BR2_TARGET_GRUB2_I386_EFI
 config BR2_TARGET_GRUB2_X86_64_EFI
 	bool "x86-64-efi"
 	depends on BR2_x86_64
+	select BR2_TARGET_GRUB2_HAS_PTF
 	help
 	  Select this option if the platform you're targetting has a
 	  64 bits EFI BIOS.
@@ -63,6 +72,7 @@ config BR2_TARGET_GRUB2_ARM_UBOOT
 config BR2_TARGET_GRUB2_ARM_EFI
 	bool "arm-efi"
 	depends on BR2_arm
+	select BR2_TARGET_GRUB2_HAS_PTF
 	help
 	  Select this option if the platform you're targetting is an
 	  ARM platform and you want to boot Grub 2 as an EFI
@@ -76,10 +86,10 @@ config BR2_TARGET_GRUB2_ARM64_EFI
 	  Aarch64 platform and you want to boot Grub 2 as an EFI
 	  application.
 
-endchoice
-
 if BR2_TARGET_GRUB2_I386_PC || BR2_TARGET_GRUB2_ARM_UBOOT
 
+comment "Options for the x86 legacy BIOS or ARM U-Boot support"
+
 config BR2_TARGET_GRUB2_BOOT_PARTITION
 	string "boot partition"
 	default "hd0,msdos1"
@@ -89,24 +99,43 @@ config BR2_TARGET_GRUB2_BOOT_PARTITION
 	  first disk if using a legacy partition table, or 'hd0,gpt1'
 	  if using GPT partition table.
 
-endif # BR2_TARGET_GRUB2_I386_PC || BR2_TARGET_GRUB2_ARM_UBOOT
-
-config BR2_TARGET_GRUB2_BUILTIN_MODULES
+config BR2_TARGET_GRUB2_BUILTIN_MODULES_PC
 	string "builtin modules"
+	default BR2_TARGET_GRUB2_BUILTIN_MODULES if BR2_TARGET_GRUB2_BUILTIN_MODULES != "" # legacy
 	default "boot linux ext2 fat squash4 part_msdos part_gpt normal biosdisk" if BR2_TARGET_GRUB2_I386_PC
-	default "boot linux ext2 fat squash4 part_msdos part_gpt normal efi_gop" \
-		if BR2_TARGET_GRUB2_I386_EFI || BR2_TARGET_GRUB2_X86_64_EFI || \
-		   BR2_TARGET_GRUB2_ARM_EFI  || BR2_TARGET_GRUB2_ARM64_EFI
 	default "linux ext2 fat part_msdos normal" if BR2_TARGET_GRUB2_ARM_UBOOT
 
-config BR2_TARGET_GRUB2_BUILTIN_CONFIG
+config BR2_TARGET_GRUB2_BUILTIN_CONFIG_PC
+	string "builtin config"
+	default BR2_TARGET_GRUB2_BUILTIN_CONFIG if BR2_TARGET_GRUB2_BUILTIN_CONFIG != "" # legacy
+	help
+	  Path to a Grub 2 configuration file that will be embedded
+	  into the Grub image itself. This allows to set the root
+	  device and other configuration parameters, but however menu
+	  entries cannot be described in this embedded configuration.
+
+endif # BR2_TARGET_GRUB2_I386_PC || BR2_TARGET_GRUB2_ARM_UBOOT
+
+if BR2_TARGET_GRUB2_I386_EFI || BR2_TARGET_GRUB2_X86_64_EFI || \
+	BR2_TARGET_GRUB2_ARM_EFI  || BR2_TARGET_GRUB2_ARM64_EFI
+
+comment "Options for the EFI BIOS or ARM EFI support"
+
+config BR2_TARGET_GRUB2_BUILTIN_MODULES_EFI
+	string "builtin modules"
+	default BR2_TARGET_GRUB2_BUILTIN_MODULES if BR2_TARGET_GRUB2_BUILTIN_MODULES != "" # legacy
+	default "boot linux ext2 fat squash4 part_msdos part_gpt normal efi_gop"
+
+config BR2_TARGET_GRUB2_BUILTIN_CONFIG_EFI
 	string "builtin config"
+	default BR2_TARGET_GRUB2_BUILTIN_CONFIG if BR2_TARGET_GRUB2_BUILTIN_CONFIG != "" # legacy
 	help
 	  Path to a Grub 2 configuration file that will be embedded
 	  into the Grub image itself. This allows to set the root
 	  device and other configuration parameters, but however menu
 	  entries cannot be described in this embedded configuration.
 
+endif
 config BR2_TARGET_GRUB2_INSTALL_TOOLS
 	bool "install tools"
 	help
boot/grub2/grub2.mk +101 -74
index ad2edf17aa..e01ebb2edb 100644
@@ -57,53 +57,65 @@ GRUB2_INSTALL_TARGET = NO
 endif
 GRUB2_CPE_ID_VENDOR = gnu
 
-GRUB2_BUILTIN_MODULES = $(call qstrip,$(BR2_TARGET_GRUB2_BUILTIN_MODULES))
-GRUB2_BUILTIN_CONFIG = $(call qstrip,$(BR2_TARGET_GRUB2_BUILTIN_CONFIG))
+GRUB2_BUILTIN_MODULES_PC = $(call qstrip,$(BR2_TARGET_GRUB2_BUILTIN_MODULES_PC))
+GRUB2_BUILTIN_MODULES_EFI = $(call qstrip,$(BR2_TARGET_GRUB2_BUILTIN_MODULES_EFI))
+GRUB2_BUILTIN_CONFIG_PC = $(call qstrip,$(BR2_TARGET_GRUB2_BUILTIN_CONFIG_PC))
+GRUB2_BUILTIN_CONFIG_EFI = $(call qstrip,$(BR2_TARGET_GRUB2_BUILTIN_CONFIG_EFI))
 GRUB2_BOOT_PARTITION = $(call qstrip,$(BR2_TARGET_GRUB2_BOOT_PARTITION))
 
-ifeq ($(BR2_TARGET_GRUB2_I386_PC),y)
-GRUB2_IMAGE = $(BINARIES_DIR)/grub.img
-GRUB2_CFG = $(TARGET_DIR)/boot/grub/grub.cfg
-GRUB2_PREFIX = ($(GRUB2_BOOT_PARTITION))/boot/grub
-GRUB2_TUPLE = i386-pc
-GRUB2_TARGET = i386
-GRUB2_PLATFORM = pc
-else ifeq ($(BR2_TARGET_GRUB2_I386_EFI),y)
-GRUB2_IMAGE = $(BINARIES_DIR)/efi-part/EFI/BOOT/bootia32.efi
-GRUB2_CFG = $(BINARIES_DIR)/efi-part/EFI/BOOT/grub.cfg
-GRUB2_PREFIX = /EFI/BOOT
-GRUB2_TUPLE = i386-efi
-GRUB2_TARGET = i386
-GRUB2_PLATFORM = efi
-else ifeq ($(BR2_TARGET_GRUB2_X86_64_EFI),y)
-GRUB2_IMAGE = $(BINARIES_DIR)/efi-part/EFI/BOOT/bootx64.efi
-GRUB2_CFG = $(BINARIES_DIR)/efi-part/EFI/BOOT/grub.cfg
-GRUB2_PREFIX = /EFI/BOOT
-GRUB2_TUPLE = x86_64-efi
-GRUB2_TARGET = x86_64
-GRUB2_PLATFORM = efi
-else ifeq ($(BR2_TARGET_GRUB2_ARM_UBOOT),y)
-GRUB2_IMAGE = $(BINARIES_DIR)/boot-part/grub/grub.img
-GRUB2_CFG = $(BINARIES_DIR)/boot-part/grub/grub.cfg
-GRUB2_PREFIX = ($(GRUB2_BOOT_PARTITION))/boot/grub
-GRUB2_TUPLE = arm-uboot
-GRUB2_TARGET = arm
-GRUB2_PLATFORM = uboot
-else ifeq ($(BR2_TARGET_GRUB2_ARM_EFI),y)
-GRUB2_IMAGE = $(BINARIES_DIR)/efi-part/EFI/BOOT/bootarm.efi
-GRUB2_CFG = $(BINARIES_DIR)/efi-part/EFI/BOOT/grub.cfg
-GRUB2_PREFIX = /EFI/BOOT
-GRUB2_TUPLE = arm-efi
-GRUB2_TARGET = arm
-GRUB2_PLATFORM = efi
-else ifeq ($(BR2_TARGET_GRUB2_ARM64_EFI),y)
-GRUB2_IMAGE = $(BINARIES_DIR)/efi-part/EFI/BOOT/bootaa64.efi
-GRUB2_CFG = $(BINARIES_DIR)/efi-part/EFI/BOOT/grub.cfg
-GRUB2_PREFIX = /EFI/BOOT
-GRUB2_TUPLE = arm64-efi
-GRUB2_TARGET = aarch64
-GRUB2_PLATFORM = efi
-endif
+GRUB2_IMAGE_i386-pc = $(BINARIES_DIR)/grub.img
+GRUB2_CFG_i386-pc = $(TARGET_DIR)/boot/grub/grub.cfg
+GRUB2_PREFIX_i386-pc = ($(GRUB2_BOOT_PARTITION))/boot/grub
+GRUB2_TARGET_i386-pc = i386
+GRUB2_PLATFORM_i386-pc = pc
+GRUB2_BUILTIN_CONFIG_i386-pc = $(GRUB2_BUILTIN_CONFIG_PC)
+GRUB2_BUILTIN_MODULES_i386-pc = $(GRUB2_BUILTIN_MODULES_PC)
+GRUB2_TUPLES-$(BR2_TARGET_GRUB2_I386_PC) += i386-pc
+
+GRUB2_IMAGE_i386-efi = $(BINARIES_DIR)/efi-part/EFI/BOOT/bootia32.efi
+GRUB2_CFG_i386-efi = $(BINARIES_DIR)/efi-part/EFI/BOOT/grub.cfg
+GRUB2_PREFIX_i386-efi = /EFI/BOOT
+GRUB2_TARGET_i386-efi = i386
+GRUB2_PLATFORM_i386-efi = efi
+GRUB2_BUILTIN_CONFIG_i386-efi = $(GRUB2_BUILTIN_CONFIG_EFI)
+GRUB2_BUILTIN_MODULES_i386-efi = $(GRUB2_BUILTIN_MODULES_EFI)
+GRUB2_TUPLES-$(BR2_TARGET_GRUB2_I386_EFI) += i386-efi
+
+GRUB2_IMAGE_x86_64-efi = $(BINARIES_DIR)/efi-part/EFI/BOOT/bootx64.efi
+GRUB2_CFG_x86_64-efi = $(BINARIES_DIR)/efi-part/EFI/BOOT/grub.cfg
+GRUB2_PREFIX_x86_64-efi = /EFI/BOOT
+GRUB2_TARGET_x86_64-efi = x86_64
+GRUB2_PLATFORM_x86_64-efi = efi
+GRUB2_BUILTIN_CONFIG_x86_64-efi = $(GRUB2_BUILTIN_CONFIG_EFI)
+GRUB2_BUILTIN_MODULES_x86_64-efi = $(GRUB2_BUILTIN_MODULES_EFI)
+GRUB2_TUPLES-$(BR2_TARGET_GRUB2_X86_64_EFI) += x86_64-efi
+
+GRUB2_IMAGE_arm-uboot = $(BINARIES_DIR)/boot-part/grub/grub.img
+GRUB2_CFG_arm-uboot = $(BINARIES_DIR)/boot-part/grub/grub.cfg
+GRUB2_PREFIX_arm-uboot = ($(GRUB2_BOOT_PARTITION))/boot/grub
+GRUB2_TARGET_arm-uboot = arm
+GRUB2_PLATFORM_arm-uboot = uboot
+GRUB2_BUILTIN_CONFIG_arm-uboot = $(GRUB2_BUILTIN_CONFIG_PC)
+GRUB2_BUILTIN_MODULES_arm-uboot = $(GRUB2_BUILTIN_MODULES_PC)
+GRUB2_TUPLES-$(BR2_TARGET_GRUB2_ARM_UBOOT) += arm-uboot
+
+GRUB2_IMAGE_arm-efi = $(BINARIES_DIR)/efi-part/EFI/BOOT/bootarm.efi
+GRUB2_CFG_arm-efi = $(BINARIES_DIR)/efi-part/EFI/BOOT/grub.cfg
+GRUB2_PREFIX_arm-efi = /EFI/BOOT
+GRUB2_TARGET_arm-efi = arm
+GRUB2_PLATFORM_arm-efi = efi
+GRUB2_BUILTIN_CONFIG_arm-efi = $(GRUB2_BUILTIN_CONFIG_EFI)
+GRUB2_BUILTIN_MODULES_arm-efi = $(GRUB2_BUILTIN_MODULES_EFI)
+GRUB2_TUPLES-$(BR2_TARGET_GRUB2_ARM_EFI) += arm-efi
+
+GRUB2_IMAGE_arm64-efi = $(BINARIES_DIR)/efi-part/EFI/BOOT/bootaa64.efi
+GRUB2_CFG_arm64-efi = $(BINARIES_DIR)/efi-part/EFI/BOOT/grub.cfg
+GRUB2_PREFIX_arm64-efi = /EFI/BOOT
+GRUB2_TARGET_arm64-efi = aarch64
+GRUB2_PLATFORM_arm64-efi = efi
+GRUB2_BUILTIN_CONFIG_arm64-efi = $(GRUB2_BUILTIN_CONFIG_EFI)
+GRUB2_BUILTIN_MODULES_arm64-efi = $(GRUB2_BUILTIN_MODULES_EFI)
+GRUB2_TUPLES-$(BR2_TARGET_GRUB2_ARM64_EFI) += arm64-efi
 
 # Grub2 is kind of special: it considers CC, LD and so on to be the
 # tools to build the host programs and uses TARGET_CC, TARGET_CFLAGS,
@@ -127,18 +139,6 @@ GRUB2_CONF_ENV = \
 	TARGET_OBJCOPY="$(TARGET_OBJCOPY)" \
 	TARGET_STRIP="$(TARGET_CROSS)strip"
 
-GRUB2_CONF_OPTS = \
-	--target=$(GRUB2_TARGET) \
-	--with-platform=$(GRUB2_PLATFORM) \
-	--prefix=/ \
-	--exec-prefix=/ \
-	--disable-grub-mkfont \
-	--enable-efiemu=no \
-	ac_cv_lib_lzma_lzma_code=no \
-	--enable-device-mapper=no \
-	--enable-libzfs=no \
-	--disable-werror
-
 HOST_GRUB2_CONF_OPTS = \
 	--disable-grub-mkfont \
 	--enable-efiemu=no \
@@ -147,26 +147,53 @@ HOST_GRUB2_CONF_OPTS = \
 	--enable-libzfs=no \
 	--disable-werror
 
-ifeq ($(BR2_TARGET_GRUB2_I386_PC),y)
-define GRUB2_IMAGE_INSTALL_ELTORITO
-	cat $(HOST_DIR)/lib/grub/$(GRUB2_TUPLE)/cdboot.img $(GRUB2_IMAGE) > \
-		$(BINARIES_DIR)/grub-eltorito.img
+define GRUB2_CONFIGURE_CMDS
+	$(foreach tuple, $(GRUB2_TUPLES-y), \
+		mkdir -p $(@D)/build-$(tuple) ; \
+		cd $(@D)/build-$(tuple) ; \
+		$(TARGET_CONFIGURE_OPTS) \
+		$(TARGET_CONFIGURE_ARGS) \
+		$(GRUB2_CONF_ENV) \
+		../configure \
+			--target=$(GRUB2_TARGET_$(tuple)) \
+			--with-platform=$(GRUB2_PLATFORM_$(tuple)) \
+			--host=$(GNU_TARGET_NAME) \
+			--build=$(GNU_HOST_NAME) \
+			--prefix=/ \
+			--exec-prefix=/ \
+			--disable-grub-mkfont \
+			--enable-efiemu=no \
+			ac_cv_lib_lzma_lzma_code=no \
+			--enable-device-mapper=no \
+			--enable-libzfs=no \
+			--disable-werror
+	)
+endef
+
+define GRUB2_BUILD_CMDS
+	$(foreach tuple, $(GRUB2_TUPLES-y), \
+		$(TARGET_MAKE_ENV) $(MAKE) -C $(@D)/build-$(tuple)
+	)
 endef
-endif
 
 define GRUB2_INSTALL_IMAGES_CMDS
-	mkdir -p $(dir $(GRUB2_IMAGE))
-	$(HOST_DIR)/usr/bin/grub-mkimage \
-		-d $(@D)/grub-core/ \
-		-O $(GRUB2_TUPLE) \
-		-o $(GRUB2_IMAGE) \
-		-p "$(GRUB2_PREFIX)" \
-		$(if $(GRUB2_BUILTIN_CONFIG),-c $(GRUB2_BUILTIN_CONFIG)) \
-		$(GRUB2_BUILTIN_MODULES)
-	mkdir -p $(dir $(GRUB2_CFG))
-	$(INSTALL) -D -m 0644 boot/grub2/grub.cfg $(GRUB2_CFG)
-	$(GRUB2_IMAGE_INSTALL_ELTORITO)
+	$(foreach tuple, $(GRUB2_TUPLES-y), \
+		mkdir -p $(dir $(GRUB2_IMAGE_$(tuple))) ; \
+		$(HOST_DIR)/usr/bin/grub-mkimage \
+			-d $(@D)/build-$(tuple)/grub-core/ \
+			-O $(tuple) \
+			-o $(GRUB2_IMAGE_$(tuple)) \
+			-p "$(GRUB2_PREFIX_$(tuple))" \
+			$(if $(GRUB2_BUILTIN_CONFIG_$(tuple)), \
+				-c $(GRUB2_BUILTIN_CONFIG_$(tuple))) \
+			$(GRUB2_BUILTIN_MODULES_$(tuple)) ; \
+		$(INSTALL) -D -m 0644 boot/grub2/grub.cfg $(GRUB2_CFG_$(tuple)) ; \
+		$(if $(findstring $(GRUB2_PLATFORM_$(tuple)), pc), \
+			cat $(HOST_DIR)/lib/grub/$(tuple)/cdboot.img $(GRUB2_IMAGE_$(tuple)) > \
+				$(BINARIES_DIR)/grub-eltorito.img ; \
+		) \
+	)
 endef
 
-$(eval $(autotools-package))
+$(eval $(generic-package))
 $(eval $(host-autotools-package))
support/testing/tests/fs/test_iso9660.py +3 -3
index 68f4840852..1b699e1fef 100644
@@ -54,7 +54,7 @@ class TestIso9660Grub2External(infra.basetest.BRTest):
         # BR2_TARGET_ROOTFS_ISO9660_INITRD is not set
         BR2_TARGET_GRUB2=y
         BR2_TARGET_GRUB2_BOOT_PARTITION="cd"
-        BR2_TARGET_GRUB2_BUILTIN_MODULES="boot linux ext2 fat part_msdos part_gpt normal biosdisk iso9660"
+        BR2_TARGET_GRUB2_BUILTIN_MODULES_PC="boot linux ext2 fat part_msdos part_gpt normal biosdisk iso9660"
         BR2_TARGET_ROOTFS_ISO9660_BOOT_MENU="{}"
         """.format(infra.filepath("conf/grub2.cfg"))
 
@@ -75,7 +75,7 @@ class TestIso9660Grub2ExternalCompress(infra.basetest.BRTest):
         BR2_TARGET_ROOTFS_ISO9660_TRANSPARENT_COMPRESSION=y
         BR2_TARGET_GRUB2=y
         BR2_TARGET_GRUB2_BOOT_PARTITION="cd"
-        BR2_TARGET_GRUB2_BUILTIN_MODULES="boot linux ext2 fat part_msdos part_gpt normal biosdisk iso9660"
+        BR2_TARGET_GRUB2_BUILTIN_MODULES_PC="boot linux ext2 fat part_msdos part_gpt normal biosdisk iso9660"
         BR2_TARGET_ROOTFS_ISO9660_BOOT_MENU="{}"
         """.format(infra.filepath("conf/grub2.cfg"))
 
@@ -95,7 +95,7 @@ class TestIso9660Grub2Internal(infra.basetest.BRTest):
         BR2_TARGET_ROOTFS_ISO9660_INITRD=y
         BR2_TARGET_GRUB2=y
         BR2_TARGET_GRUB2_BOOT_PARTITION="cd"
-        BR2_TARGET_GRUB2_BUILTIN_MODULES="boot linux ext2 fat part_msdos part_gpt normal biosdisk iso9660"
+        BR2_TARGET_GRUB2_BUILTIN_MODULES_PC="boot linux ext2 fat part_msdos part_gpt normal biosdisk iso9660"
         BR2_TARGET_ROOTFS_ISO9660_BOOT_MENU="{}"
         """.format(infra.filepath("conf/grub2.cfg"))
 
Signature · 6 lines
2.25.1

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
Signature · 10 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#9 lore raw

Re: [Buildroot] [PATCH v3 2/7] boot/grub2: add support to build multiple Grub2 configurations in the same build

Quoted 4 lines · On Wed, 6 Oct 2021 20:11:04 +0200 "Yann E. MORIN" <[email protected]>…

Köry, All,

On 2021-09-23 17:57 +0200, Kory Maincent spake thusly:
[--SNIP--]

Quoted 8 lines
Quoted 4 lines

We can no longer use autotools-package as a consequence of this multi-build, and we have to resort to generic-package and a partial duplication of the autotools-infra. Grub2 was already using custom option like --prefix or --exec-prefix so this won't add much more weirdness.

But this had an unfortunate side effect: the grub2 tools, like grub-editenv, enabled with BR2_TARGET_GRUB2_INSTALL_TOOLS, are no longer installed in the target now.

Doh, didn't thought of that!

Quoted 9 lines

But generic-package has no default install command, so declaring that the package installs in target does nothing.

I see two options:

1. provide a custom command for one of the enabled platforms; e.g. arbitrarily choose the first in the tuple and install in target from that one tuple

Quoted 5 lines

2. revert to using an autotools package, but build for no platform at all and just install the tools. For the platforms, move the current foreach loops into post-build and post-install-image hooks

Care to look into that, please?

mmh that's tricky, because in the case we want to use the Grub modules it need to be generated for each platforms.
For example we can face a case where we will have these three Grub modules repositories for each platform:
target/lib/grub/i386-pc/
target/lib/grub/i386-efi/
target/lib/grub/x86_64-efi/

What about just adding the "make install" command in the foreach loop? This will overwrite the binaries, the locale and the /etc configurations for each "make" loop but we will have all the Grub modules repositories. I don't think the overwrites will cause issues because it is just tools.

Regards,

Köry
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

#10 lore raw

Re: [Buildroot] [PATCH v3 2/7] boot/grub2: add support to build multiple Grub2 configurations in the same build

Köry, All,

On 2021-10-07 10:23 +0200, Köry Maincent spake thusly:

Quoted 4 lines
Quoted 2 lines · On Wed, 6 Oct 2021 20:11:04 +0200 "Yann E. MORIN" <[email protected]>…

On 2021-09-23 17:57 +0200, Kory Maincent spake thusly:
[--SNIP--]

Quoted 8 lines
Quoted 7 lines
Quoted 4 lines

We can no longer use autotools-package as a consequence of this multi-build, and we have to resort to generic-package and a partial duplication of the autotools-infra. Grub2 was already using custom option like --prefix or --exec-prefix so this won't add much more weirdness.

But this had an unfortunate side effect: the grub2 tools, like grub-editenv, enabled with BR2_TARGET_GRUB2_INSTALL_TOOLS, are no longer installed in the target now.

Doh, didn't thought of that!

Neither did I...

[--SNIP--]

Quoted 10 lines
Quoted 1 line

Care to look into that, please?

mmh that's tricky, because in the case we want to use the Grub modules it need to be generated for each platforms.
For example we can face a case where we will have these three Grub modules repositories for each platform:
target/lib/grub/i386-pc/
target/lib/grub/i386-efi/
target/lib/grub/x86_64-efi/

Quoted 4 lines

What about just adding the "make install" command in the foreach loop? This will overwrite the binaries, the locale and the /etc configurations for each "make" loop but we will have all the Grub modules repositories. I don't think the overwrites will cause issues because it is just tools.

I was thinking about something along the lines of:

    diff --git a/boot/grub2/grub2.mk b/boot/grub2/grub2.mk
    index e01ebb2edb..bb853cace6 100644
    --- a/boot/grub2/grub2.mk
    +++ b/boot/grub2/grub2.mk
    @@ -127,6 +127,9 @@ GRUB2_TUPLES-$(BR2_TARGET_GRUB2_ARM64_EFI) +=
    arm64-efi
     HOST_GRUB2_CONF_ENV = \
        CPP="$(HOSTCC) -E"
     
    +# Main build: only build tools
    +GRUB2_CONF_OPTS = --with-platform=none --blabla...
    +
     GRUB2_CONF_ENV = \
        CPP="$(TARGET_CC) -E" \
        TARGET_CC="$(TARGET_CC)" \
    @@ -147,7 +150,7 @@ HOST_GRUB2_CONF_OPTS = \
        --enable-libzfs=no \
        --disable-werror
     
    -define GRUB2_CONFIGURE_CMDS
    +define GRUB2_CONFIGURE_CMDS_PTF
        $(foreach tuple, $(GRUB2_TUPLES-y), \
            mkdir -p $(@D)/build-$(tuple) ; \
            cd $(@D)/build-$(tuple) ; \
    @@ -169,14 +172,16 @@ define GRUB2_CONFIGURE_CMDS
                --disable-werror
        )
     endef
    +GRUB2_POST_CONFIGURE_HOOKS += GRUB2_CONFIGURE_CMDS_PTF
     
    -define GRUB2_BUILD_CMDS
    +define GRUB2_BUILD_CMDS_PTF
        $(foreach tuple, $(GRUB2_TUPLES-y), \
            $(TARGET_MAKE_ENV) $(MAKE) -C $(@D)/build-$(tuple)
        )
     endef
    +GRUB2_POST_BUILD_HOOKS += GRUB2_BUILD_CMDS_PTF
     
    -define GRUB2_INSTALL_IMAGES_CMDS
    +define GRUB2_INSTALL_IMAGES_CMDS_PTF
        $(foreach tuple, $(GRUB2_TUPLES-y), \
            mkdir -p $(dir $(GRUB2_IMAGE_$(tuple))) ; \
            $(HOST_DIR)/usr/bin/grub-mkimage \
    @@ -194,6 +199,7 @@ define GRUB2_INSTALL_IMAGES_CMDS
            ) \
        )
     endef
    +GRUB2_POST_INSTALL_IMAGES_HOOKS += GRUB2_INSTALL_IMAGES_CMDS_PTF
     
    -$(eval $(generic-package))
    +$(eval $(autotools-package))
     $(eval $(host-autotools-package))

By using --with-platform=none in the main CONF_OPTS, we should be able to disable building any of the "images", and just build the tools.

Totally untested, of course...

Alternatively, we could introduce a new package, grub2-tools, not unlike uboot-tools, which sole responsibility would be to build and install those tools.

Regards,
Yann E. MORIN.

Quoted 3 lines

Regards,

Köry

Signature · 10 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#11 lore raw

Re: [Buildroot] [PATCH v3 2/7] boot/grub2: add support to build multiple Grub2 configurations in the same build

Quoted 36 lines · On Thu, 7 Oct 2021 11:53:26 +0200 "Yann E. MORIN" <[email protected]>…

Köry, All,

On 2021-10-07 10:23 +0200, Köry Maincent spake thusly:

Quoted 9 lines
Quoted 7 lines · On Wed, 6 Oct 2021 20:11:04 +0200 "Yann E. MORIN" <[email protected]>…

On 2021-09-23 17:57 +0200, Kory Maincent spake thusly:
[--SNIP--]

Quoted 5 lines

We can no longer use autotools-package as a consequence of this multi-build, and we have to resort to generic-package and a partial duplication of the autotools-infra. Grub2 was already using custom option like --prefix or --exec-prefix so this won't add much more weirdness.

Quoted 4 lines
Quoted 3 lines

But this had an unfortunate side effect: the grub2 tools, like grub-editenv, enabled with BR2_TARGET_GRUB2_INSTALL_TOOLS, are no longer installed in the target now.

Doh, didn't thought of that!

Neither did I...

[--SNIP--]

Quoted 10 lines
Quoted 1 line

Care to look into that, please?

mmh that's tricky, because in the case we want to use the Grub modules it need to be generated for each platforms.
For example we can face a case where we will have these three Grub modules repositories for each platform:
target/lib/grub/i386-pc/
target/lib/grub/i386-efi/
target/lib/grub/x86_64-efi/

Quoted 4 lines

What about just adding the "make install" command in the foreach loop? This will overwrite the binaries, the locale and the /etc configurations for each "make" loop but we will have all the Grub modules repositories. I don't think the overwrites will cause issues because it is just tools.

Quoted 4 lines

By using --with-platform=none in the main CONF_OPTS, we should be able to disable building any of the "images", and just build the tools.

Totally untested, of course...

As I said if we do that we won't have all the Grub modules in the target directory. I think these modules are quiet important, an user that wants to use grub-mkimage will surely use them for example.

Regards,

Köry,

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

#12 lore raw

Re: [Buildroot] [PATCH v3 2/7] boot/grub2: add support to build multiple Grub2 configurations in the same build

Köry, All,

On 2021-10-07 14:43 +0200, Köry Maincent spake thusly:

Quoted 2 lines

On Thu, 7 Oct 2021 11:53:26 +0200
"Yann E. MORIN" <[email protected]> wrote:

[--SNIP--]

Quoted 7 lines
Quoted 4 lines

By using --with-platform=none in the main CONF_OPTS, we should be able to disable building any of the "images", and just build the tools.

Totally untested, of course...

As I said if we do that we won't have all the Grub modules in the target directory. I think these modules are quiet important, an user that wants to use grub-mkimage will surely use them for example.

OK, I had a hard time understanding what you meant... But indeed, BR2_TARGET_GRUB2_INSTALL_TOOLS *also* wants to install the modules in target/

In this case, this should not be too complex either; we just need a post-image hook that copies all modules into TARGET_DIR, like:

    diff --git a/boot/grub2/grub2.mk b/boot/grub2/grub2.mk
    index e01ebb2edb..0bb10d765d 100644
    --- a/boot/grub2/grub2.mk
    +++ b/boot/grub2/grub2.mk
    @@ -50,11 +50,6 @@ GRUB2_IGNORE_CVES += CVE-2019-14865
     # version available in Buildroot.
     GRUB2_IGNORE_CVES += CVE-2020-15705
     
    -ifeq ($(BR2_TARGET_GRUB2_INSTALL_TOOLS),y)
    -GRUB2_INSTALL_TARGET = YES
    -else
    -GRUB2_INSTALL_TARGET = NO
    -endif
     GRUB2_CPE_ID_VENDOR = gnu
     
     GRUB2_BUILTIN_MODULES_PC = $(call qstrip,$(BR2_TARGET_GRUB2_BUILTIN_MODULES_PC))
    @@ -127,6 +122,9 @@ GRUB2_TUPLES-$(BR2_TARGET_GRUB2_ARM64_EFI) += arm64-efi
     HOST_GRUB2_CONF_ENV = \
         CPP="$(HOSTCC) -E"
     
    +# Main build: only build tools
    +GRUB2_CONF_OPTS = --with-platform=none --blabla...
    +
     GRUB2_CONF_ENV = \
         CPP="$(TARGET_CC) -E" \
         TARGET_CC="$(TARGET_CC)" \
    @@ -147,7 +145,7 @@ HOST_GRUB2_CONF_OPTS = \
         --enable-libzfs=no \
         --disable-werror
     
    -define GRUB2_CONFIGURE_CMDS
    +define GRUB2_CONFIGURE_CMDS_PTF
         $(foreach tuple, $(GRUB2_TUPLES-y), \
             mkdir -p $(@D)/build-$(tuple) ; \
             cd $(@D)/build-$(tuple) ; \
    @@ -169,14 +167,16 @@ define GRUB2_CONFIGURE_CMDS
                 --disable-werror
         )
     endef
    +GRUB2_POST_CONFIGURE_HOOKS += GRUB2_CONFIGURE_CMDS_PTF
     
    -define GRUB2_BUILD_CMDS
    +define GRUB2_BUILD_CMDS_PTF
         $(foreach tuple, $(GRUB2_TUPLES-y), \
             $(TARGET_MAKE_ENV) $(MAKE) -C $(@D)/build-$(tuple)
         )
     endef
    +GRUB2_POST_BUILD_HOOKS += GRUB2_BUILD_CMDS_PTF
     
    -define GRUB2_INSTALL_IMAGES_CMDS
    +define GRUB2_INSTALL_IMAGES_CMDS_PTF
         $(foreach tuple, $(GRUB2_TUPLES-y), \
             mkdir -p $(dir $(GRUB2_IMAGE_$(tuple))) ; \
             $(HOST_DIR)/usr/bin/grub-mkimage \
    @@ -194,6 +194,23 @@ define GRUB2_INSTALL_IMAGES_CMDS
             ) \
         )
     endef
    +GRUB2_POST_INSTALL_IMAGES_HOOKS += GRUB2_INSTALL_IMAGES_CMDS_PTF
     
    -$(eval $(generic-package))
    +ifeq ($(BR2_TARGET_GRUB2_INSTALL_TOOLS),y)
    +GRUB2_INSTALL_TARGET = YES
    +define GRUB2_INSTALL_MODS_IN_TARGET
    +    mkdir -p $(TARGET_DIR)/boot
    +    $(foreach tuple, $(GRUB2_TUPLES-y), \
    +        cp -a $(GRUB2_IMAGE_$(tuple)) $(TARGET_DIR)/boot/
    +    )
    +endef
    +# **YES** this is a post-install-image hook that installs in target/
    +# **AND** we need it to be registered **LAST**, after all the per-tuple
    +# image-install hooks
    +GRUB2_POST_INSTALL_IMAGES_HOOKS += GRUB2_INSTALL_MODS_IN_TARGET
    +else
    +GRUB2_INSTALL_TARGET = NO
    +endif
    +
    +$(eval $(autotools-package))
     $(eval $(host-autotools-package))

NOTE: yes, I know that $(GRUB2_IMAGE_$(tuple)) is not exactly what we need to copy, and that it will need some additional tweaking.

But then, maybe installing all the per-tuple to target/ might just work in the end. That would not be nice, but as Thomas already said, grub2 is already not nice anyway...

Also, I just noticed that we copy the cdboot.img from the host dir! Are the modules all there?

Regards,
Yann E. MORIN.

Signature · 10 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#13 lore raw

Re: [Buildroot] [PATCH v3 2/7] boot/grub2: add support to build multiple Grub2 configurations in the same build

Quoted 9 lines · On Thu, 7 Oct 2021 18:29:31 +0200 "Yann E. MORIN" <[email protected]>…

Köry, All,

On 2021-10-07 14:43 +0200, Köry Maincent spake thusly:

Quoted 2 lines

On Thu, 7 Oct 2021 11:53:26 +0200
"Yann E. MORIN" <[email protected]> wrote:

[--SNIP--]

Quoted 3 lines
Quoted 3 lines

By using --with-platform=none in the main CONF_OPTS, we should be able to disable building any of the "images", and just build the tools.

Quoted 8 lines
Quoted 4 lines
Quoted 1 line

Totally untested, of course...

As I said if we do that we won't have all the Grub modules in the target directory. I think these modules are quiet important, an user that wants to use grub-mkimage will surely use them for example.

OK, I had a hard time understanding what you meant... But indeed, BR2_TARGET_GRUB2_INSTALL_TOOLS *also* wants to install the modules in target/

Sorry I was maybe not clear enough.
I was speaking about all the .mod and .module files installed in the /target/lib/grub/$(GRUB2_TUPLES-$(tuple)) directory when calling the "make install" command. If we use none as platform they are not generated, therefore installed.

Here is the list of installed file if I add "make install" in the foreach
loop: https://termbin.com/8b4w
Here is the list of installed file in the case of none
platform: https://termbin.com/x45h

Quoted 3 lines

In this case, this should not be too complex either; we just need a post-image hook that copies all modules into TARGET_DIR, like:

Yes it could works, but the installation of the module is not really proper, see below.

Quoted 61 lines
    diff --git a/boot/grub2/grub2.mk b/boot/grub2/grub2.mk
    index e01ebb2edb..0bb10d765d 100644
    --- a/boot/grub2/grub2.mk
    +++ b/boot/grub2/grub2.mk
    @@ -50,11 +50,6 @@ GRUB2_IGNORE_CVES += CVE-2019-14865
     # version available in Buildroot.
     GRUB2_IGNORE_CVES += CVE-2020-15705
     
    -ifeq ($(BR2_TARGET_GRUB2_INSTALL_TOOLS),y)
    -GRUB2_INSTALL_TARGET = YES
    -else
    -GRUB2_INSTALL_TARGET = NO
    -endif
     GRUB2_CPE_ID_VENDOR = gnu
     
     GRUB2_BUILTIN_MODULES_PC = $(call

qstrip,$(BR2_TARGET_GRUB2_BUILTIN_MODULES_PC)) @@ -127,6 +122,9 @@ GRUB2_TUPLES-$(BR2_TARGET_GRUB2_ARM64_EFI) += arm64-efi HOST_GRUB2_CONF_ENV = \ CPP="$(HOSTCC) -E"

    +# Main build: only build tools
    +GRUB2_CONF_OPTS = --with-platform=none --blabla...
    +
     GRUB2_CONF_ENV = \
         CPP="$(TARGET_CC) -E" \
         TARGET_CC="$(TARGET_CC)" \
    @@ -147,7 +145,7 @@ HOST_GRUB2_CONF_OPTS = \
         --enable-libzfs=no \
         --disable-werror
     
    -define GRUB2_CONFIGURE_CMDS
    +define GRUB2_CONFIGURE_CMDS_PTF
         $(foreach tuple, $(GRUB2_TUPLES-y), \
             mkdir -p $(@D)/build-$(tuple) ; \
             cd $(@D)/build-$(tuple) ; \
    @@ -169,14 +167,16 @@ define GRUB2_CONFIGURE_CMDS
                 --disable-werror
         )
     endef
    +GRUB2_POST_CONFIGURE_HOOKS += GRUB2_CONFIGURE_CMDS_PTF
     
    -define GRUB2_BUILD_CMDS
    +define GRUB2_BUILD_CMDS_PTF
         $(foreach tuple, $(GRUB2_TUPLES-y), \
             $(TARGET_MAKE_ENV) $(MAKE) -C $(@D)/build-$(tuple)
         )
     endef
    +GRUB2_POST_BUILD_HOOKS += GRUB2_BUILD_CMDS_PTF
     
    -define GRUB2_INSTALL_IMAGES_CMDS
    +define GRUB2_INSTALL_IMAGES_CMDS_PTF
         $(foreach tuple, $(GRUB2_TUPLES-y), \
             mkdir -p $(dir $(GRUB2_IMAGE_$(tuple))) ; \
             $(HOST_DIR)/usr/bin/grub-mkimage \
    @@ -194,6 +194,23 @@ define GRUB2_INSTALL_IMAGES_CMDS
             ) \
         )
     endef
    +GRUB2_POST_INSTALL_IMAGES_HOOKS += GRUB2_INSTALL_IMAGES_CMDS_PTF
Quoted 7 lines
    -$(eval $(generic-package))
    +ifeq ($(BR2_TARGET_GRUB2_INSTALL_TOOLS),y)
    +GRUB2_INSTALL_TARGET = YES
    +define GRUB2_INSTALL_MODS_IN_TARGET
    +    mkdir -p $(TARGET_DIR)/boot
    +    $(foreach tuple, $(GRUB2_TUPLES-y), \
    +        cp -a $(GRUB2_IMAGE_$(tuple)) $(TARGET_DIR)/boot/

Here you copy only the image generated, not the modules. It should more be like that:

	$(foreach tuple, $(GRUB2_TUPLES-y), \
		mkdir -p $(TARGET_DIR)/usr/lib/$(tuple) \
		cp -a $(@D)/build-$(tuple)/grub-core/{*.mod,*.module} \
			$(TARGET_DIR)/usr/lib/$(tuple)

We might also need *.img file because cdboot.img is needed to create CDROM drive image.

Or we also could "make install" each platform in a temporary folder then copy the content of tmp-folder-$(tuple)/usr/lib/$(tuple) in the $(TARGET_DIR) but I am not sure it is better than just "make install" directly in the $(TARGET_DIR).

Quoted 14 lines

+ )
+endef
+# **YES** this is a post-install-image hook that installs in target/ +# **AND** we need it to be registered **LAST**, after all the per-tuple +# image-install hooks
+GRUB2_POST_INSTALL_IMAGES_HOOKS += GRUB2_INSTALL_MODS_IN_TARGET +else
+GRUB2_INSTALL_TARGET = NO
+endif
+
+$(eval $(autotools-package))
$(eval $(host-autotools-package))

Quoted 6 lines

NOTE: yes, I know that $(GRUB2_IMAGE_$(tuple)) is not exactly what we need to copy, and that it will need some additional tweaking.

But then, maybe installing all the per-tuple to target/ might just work in the end. That would not be nice, but as Thomas already said, grub2 is already not nice anyway...

Yeah, not sure what is the better solution.

Quoted 2 lines

Also, I just noticed that we copy the cdboot.img from the host dir! Are the modules all there?

You may have copy a wrong code because it won't install cdboot.img in the $(TARGET_DIR) with the code below.

Regards,
Köry
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

#14 lore raw

Re: [Buildroot] [PATCH v3 2/7] boot/grub2: add support to build multiple Grub2 configurations in the same build

In reply to Yann E. MORIN

Hello Yann,

Quoted 9 lines · On Thu, 7 Oct 2021 18:29:31 +0200 "Yann E. MORIN" <[email protected]>…

Köry, All,

On 2021-10-07 14:43 +0200, Köry Maincent spake thusly:

Quoted 2 lines

On Thu, 7 Oct 2021 11:53:26 +0200
"Yann E. MORIN" <[email protected]> wrote:

[--SNIP--]

In this case, this should not be too complex either; we just need a post-image hook that copies all modules into TARGET_DIR, like:

Is something like the following code seems okay for you.
I use temporary install-$(tuple) folder for the different platforms. We need to do the CONFIGURE_CMD_PTF as a PRE_CONFIGURE_HOOK because the CONFIGURE_CMDS from autotools is configuring inside the GRUB2_SRCDIR folder. Then if the ./configure has been run in the GRUB2_SRCDIR I can not run other configure command without receive this error message:
configure: error: source directory already configured; run "make distclean" there first

To follow the logic I move all the *_CMDS_PTF as PRE_*_HOOK. I have tested it and it seems functioning.

boot/grub2/grub2.mk +29 -13
index e01ebb2edb..e339eed6de 100644
@@ -139,6 +139,22 @@ GRUB2_CONF_ENV = \
        TARGET_OBJCOPY="$(TARGET_OBJCOPY)" \
        TARGET_STRIP="$(TARGET_CROSS)strip"
 
+GRUB2_CONF_COMMON_OPTS = \
+       --host=$(GNU_TARGET_NAME) \
+       --build=$(GNU_HOST_NAME) \
+       --prefix=/ \
+       --exec-prefix=/ \
+       --disable-grub-mkfont \
+       --enable-efiemu=no \
+       ac_cv_lib_lzma_lzma_code=no \
+       --enable-device-mapper=no \
+       --enable-libzfs=no \
+       --disable-werror
+
+GRUB2_CONF_OPTS = \
+       $(GRUB2_CONF_COMMON_OPTS) \
+       --with-platform=none
+
 HOST_GRUB2_CONF_OPTS = \
        --disable-grub-mkfont \
        --enable-efiemu=no \
@@ -147,7 +163,7 @@ HOST_GRUB2_CONF_OPTS = \
        --enable-libzfs=no \
        --disable-werror
 
-define GRUB2_CONFIGURE_CMDS
+define GRUB2_CONFIGURE_CMDS_PTF
        $(foreach tuple, $(GRUB2_TUPLES-y), \
                mkdir -p $(@D)/build-$(tuple) ; \
                cd $(@D)/build-$(tuple) ; \
@@ -157,26 +173,19 @@ define GRUB2_CONFIGURE_CMDS
                ../configure \
                        --target=$(GRUB2_TARGET_$(tuple)) \
                        --with-platform=$(GRUB2_PLATFORM_$(tuple)) \
-                       --host=$(GNU_TARGET_NAME) \
-                       --build=$(GNU_HOST_NAME) \
-                       --prefix=/ \
-                       --exec-prefix=/ \
-                       --disable-grub-mkfont \
-                       --enable-efiemu=no \
-                       ac_cv_lib_lzma_lzma_code=no \
-                       --enable-device-mapper=no \
-                       --enable-libzfs=no \
-                       --disable-werror
+                       $(GRUB2_CONF_COMMON_OPTS)
        )
 endef
+GRUB2_PRE_CONFIGURE_HOOKS += GRUB2_CONFIGURE_CMDS_PTF
 
-define GRUB2_BUILD_CMDS
+define GRUB2_BUILD_CMDS_PTF
        $(foreach tuple, $(GRUB2_TUPLES-y), \
                $(TARGET_MAKE_ENV) $(MAKE) -C $(@D)/build-$(tuple)
        )
 endef
+GRUB2_PRE_BUILD_HOOKS += GRUB2_BUILD_CMDS_PTF
 
-define GRUB2_INSTALL_IMAGES_CMDS
+define GRUB2_INSTALL_IMAGES_CMDS_PTF
        $(foreach tuple, $(GRUB2_TUPLES-y), \
                mkdir -p $(dir $(GRUB2_IMAGE_$(tuple))) ; \
                $(HOST_DIR)/usr/bin/grub-mkimage \
@@ -194,6 +203,20 @@ define GRUB2_INSTALL_IMAGES_CMDS
                ) \
        )
 endef
+GRUB2_PRE_INSTALL_IMAGES_HOOKS += GRUB2_INSTALL_CMDS_PTF
+
+ifeq ($(BR2_TARGET_GRUB2_INSTALL_TOOLS),y)
+GRUB2_INSTALL_TARGET = YES
+define GRUB2_INSTALL_MODS_IN_TARGET
+       $(foreach tuple, $(GRUB2_TUPLES-y), \
+               $(TARGET_MAKE_ENV) $(MAKE) DESTDIR=$(@D)/$(install-$(tuple))

install -C $(@D)/build-$(tuple) ; \
+ cp -a $(@D)/$(install-$(tuple))/lib/* $(TARGET_DIR)/lib/ ;
+ )
+endef
+GRUB2_POST_INSTALL_IMAGES_HOOKS += GRUB2_INSTALL_MODS_IN_TARGET +else
+GRUB2_INSTALL_TARGET = NO
+endif

-$(eval $(generic-package))
+$(eval $(autotools-package))
$(eval $(host-autotools-package))

Regards,
Köry
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

#15 lore raw

Re: [Buildroot] [PATCH v3 2/7] boot/grub2: add support to build multiple Grub2 configurations in the same build

Köry, All,

Sorry for my late reply, I've been pretty busy here...

On 2021-10-11 12:27 +0200, Köry Maincent spake thusly:

Quoted 10 lines
Quoted 7 lines · On Thu, 7 Oct 2021 18:29:31 +0200 "Yann E. MORIN" <[email protected]>…

On 2021-10-07 14:43 +0200, Köry Maincent spake thusly:

Quoted 2 lines

On Thu, 7 Oct 2021 11:53:26 +0200
"Yann E. MORIN" <[email protected]> wrote:

[--SNIP--]

In this case, this should not be too complex either; we just need a post-image hook that copies all modules into TARGET_DIR, like:

Quoted 8 lines

Is something like the following code seems okay for you.
I use temporary install-$(tuple) folder for the different platforms. We need to do the CONFIGURE_CMD_PTF as a PRE_CONFIGURE_HOOK because the CONFIGURE_CMDS from autotools is configuring inside the GRUB2_SRCDIR folder. Then if the ./configure has been run in the GRUB2_SRCDIR I can not run other configure command without receive this error message:
configure: error: source directory already configured; run "make distclean" there first

Arg, I hadn't thought about this...

Quoted 2 lines

To follow the logic I move all the *_CMDS_PTF as PRE_*_HOOK. I have tested it and it seems functioning.

But I'm afraid it is going to fail if you try to reconfigure the thing, like:

    $ make grub2-reconfigure

So, in the end, your original proposal, to install each and every tuples into target/, is probably the best option we have, and it can be as simple as something like:

    diff --git a/boot/grub2/grub2.mk b/boot/grub2/grub2.mk
    index e01ebb2edb..a18696b6cc 100644
    --- a/boot/grub2/grub2.mk
    +++ b/boot/grub2/grub2.mk
    @@ -195,5 +195,13 @@ define GRUB2_INSTALL_IMAGES_CMDS
        )
     endef
     
    +ifeq ($(BR2_TARGET_GRUB2_INSTALL_TOOLS),y)
    +define GRUB2_INSTALL_TARGET_CMDS
    +	$(foreach tuple, $(GRUB2_TUPLES-y), \
    +		$(TARGET_MAKE_ENV) $(MAKE) -C $(@D)/build-$(tuple) DESTDIR=$(TARGET_DIR) install
    +	)
    +endef
    +endif
    +
     $(eval $(generic-package))
     $(eval $(host-autotools-package))

Yes, this will indeed install the same tools more than once, but this is not a problem: it is _not_ a case of a package overwriting files from another package, but overwriting its own files, so we don't care.

Test-built with:

    BR2_x86_pentium4=y
    BR2_TOOLCHAIN_EXTERNAL=y
    BR2_TARGET_GRUB2=y
    BR2_TARGET_GRUB2_I386_PC=y
    BR2_TARGET_GRUB2_I386_EFI=y
    BR2_TARGET_GRUB2_INSTALL_TOOLS=y

Side note, unrelated to this issue: we have a parallel build issue with grub2:

    2021-10-14 21:45:52 cd . && /bin/sh ./config.status config-util.h
    2021-10-14 21:45:52 config.status: creating config-util.h
    2021-10-14 21:45:52 bison -d -p grub_script_yy -b grub_script ../grub-core/script/parser.y
    2021-10-14 21:45:52 ../grub-core/script/parser.y:92.1-12: warning: deprecated directive: ‘%pure-parser’, use ‘%define api.pure’ [-Wdeprecated]
    2021-10-14 21:45:52    92 | %pure-parser
    2021-10-14 21:45:52       | ^~~~~~~~~~~~
    2021-10-14 21:45:52       | %define api.pure
    2021-10-14 21:45:52 ../grub-core/script/parser.y: warning: fix-its can be applied.  Rerun with option '--update'. [-Wother]
    2021-10-14 21:45:52 /home/ymorin/dev/buildroot/O/host/bin/i686-linux-gcc -E -DHAVE_CONFIG_H -I. -I..  -Wall -W -DGRUB_UTIL=1 -D_FILE_OFFSET_BITS=64 -I./include -DGRUB_FILE="util/grub-fstest.c" -I. -I.. -I. -I.. -I../include -I./include -I../grub-core/lib/libgcrypt-grub/src/ -I./grub-core/lib/gnulib -I../grub-core/lib/gnulib  -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -Os -fno-stack-protector -D_FILE_OFFSET_BITS=64   -D'GRUB_MOD_INIT(x)=@MARKER@x@' ../util/grub-fstest.c ../grub-core/kern/emu/hostfs.c ../ grub-core/disk/host.c ../grub-core/osdep/init.c > grub_fstest.pp || (rm -f grub_fstest.pp; exit 1)
    2021-10-14 21:45:52 ../grub-core/kern/emu/hostfs.c:20:10: fatal error: config-util.h: No such file or directory
    2021-10-14 21:45:52    20 | #include <config-util.h>
    2021-10-14 21:45:52       |          ^~~~~~~~~~~~~~~
    2021-10-14 21:45:52 compilation terminated.
    2021-10-14 21:45:52 make[2]: *** [Makefile:13107: grub_fstest.pp] Error 1
    2021-10-14 21:45:52 make[2]: *** Waiting for unfinished jobs....

And re-starting the build succeeds.

But just looking at how config-util.h is actually created (in Makefile.in) makes me cry:

   3389 config-util.h: stamp-h1
   3390     @test -f $@ || rm -f stamp-h1
   3391     @test -f $@ || $(MAKE) $(AM_MAKEFLAGS) stamp-h1
   3392
   3393 stamp-h1: $(srcdir)/config-util.h.in $(top_builddir)/config.status
   3394     @rm -f stamp-h1
   3395     cd $(top_builddir) && $(SHELL) ./config.status config-util.h

Whaaa?

But fortunately, it looks liek it is fixed with upstream commit 42f4054faf3c (Makefile: Make libgrub.pp depend on config-util.h).

Could you look into both the tools and the backport of the commit?

Regards,
Yann E. MORIN.

Signature · 10 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#16 lore raw

Re: [Buildroot] [PATCH v3 2/7] boot/grub2: add support to build multiple Grub2 configurations in the same build

Köry, All,

On 2021-10-14 22:02 +0200, Yann E. MORIN spake thusly:

Quoted 2 lines

Side note, unrelated to this issue: we have a parallel build issue with grub2:

[--SNIP--]

Quoted 2 lines

But fortunately, it looks liek it is fixed with upstream commit 42f4054faf3c (Makefile: Make libgrub.pp depend on config-util.h).

And of course, since this commit touches Makefile.am, but we avoid very hard re-running autoreconf for grub2, we also need to patch Makefile.in (the same hunk applies quite OK to both file, fortunately).

Regards,
Yann E. MORIN.

Signature · 10 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
Tested-by
#17 lore raw

Re: [Buildroot] [PATCH v3 2/7] boot/grub2: add support to build multiple Grub2 configurations in the same build

In reply to Yann E. MORIN

Hello;

Quoted 49 lines · Yann E. MORIN

Köry, All,

Sorry for my late reply, I've been pretty busy here...

On 2021-10-11 12:27 +0200, Köry Maincent spake thusly:

Quoted 10 lines
Quoted 7 lines · On Thu, 7 Oct 2021 18:29:31 +0200 "Yann E. MORIN" <[email protected]>…

On 2021-10-07 14:43 +0200, Köry Maincent spake thusly:

Quoted 2 lines

On Thu, 7 Oct 2021 11:53:26 +0200
"Yann E. MORIN" <[email protected]> wrote:

[--SNIP--]

In this case, this should not be too complex either; we just need a post-image hook that copies all modules into TARGET_DIR, like:

Quoted 8 lines

Is something like the following code seems okay for you.
I use temporary install-$(tuple) folder for the different platforms. We need to do the CONFIGURE_CMD_PTF as a PRE_CONFIGURE_HOOK because the CONFIGURE_CMDS from autotools is configuring inside the GRUB2_SRCDIR folder. Then if the ./configure has been run in the GRUB2_SRCDIR I can not run other configure command without receive this error message:
configure: error: source directory already configured; run "make distclean" there first

Arg, I hadn't thought about this...

Quoted 2 lines

To follow the logic I move all the *_CMDS_PTF as PRE_*_HOOK. I have tested it and it seems functioning.

But I'm afraid it is going to fail if you try to reconfigure the thing, like:

    $ make grub2-reconfigure

So, in the end, your original proposal, to install each and every tuples into target/, is probably the best option we have, and it can be as simple as something like:

    diff --git a/boot/grub2/grub2.mk b/boot/grub2/grub2.mk
    index e01ebb2edb..a18696b6cc 100644
    --- a/boot/grub2/grub2.mk
    +++ b/boot/grub2/grub2.mk
    @@ -195,5 +195,13 @@ define GRUB2_INSTALL_IMAGES_CMDS
        )
     endef

    +ifeq ($(BR2_TARGET_GRUB2_INSTALL_TOOLS),y)
    +define GRUB2_INSTALL_TARGET_CMDS
Quoted 9 lines
    +   $(foreach tuple, $(GRUB2_TUPLES-y), \
    +           $(TARGET_MAKE_ENV) $(MAKE) -C $(@D)/build-$(tuple) DESTDIR=$(TARGET_DIR) install
    +   )
    +endef
    +endif
    +
     $(eval $(generic-package))
     $(eval $(host-autotools-package))

This indeed works properly!

Quoted 55 lines

Yes, this will indeed install the same tools more than once, but this is not a problem: it is _not_ a case of a package overwriting files from another package, but overwriting its own files, so we don't care.

Test-built with:

    BR2_x86_pentium4=y
    BR2_TOOLCHAIN_EXTERNAL=y
    BR2_TARGET_GRUB2=y
    BR2_TARGET_GRUB2_I386_PC=y
    BR2_TARGET_GRUB2_I386_EFI=y
    BR2_TARGET_GRUB2_INSTALL_TOOLS=y

Side note, unrelated to this issue: we have a parallel build issue with grub2:

    2021-10-14 21:45:52 cd . && /bin/sh ./config.status config-util.h
    2021-10-14 21:45:52 config.status: creating config-util.h
    2021-10-14 21:45:52 bison -d -p grub_script_yy -b grub_script ../grub-core/script/parser.y
    2021-10-14 21:45:52 ../grub-core/script/parser.y:92.1-12: warning: deprecated directive: ‘%pure-parser’, use ‘%define api.pure’ [-Wdeprecated]
    2021-10-14 21:45:52    92 | %pure-parser
    2021-10-14 21:45:52       | ^~~~~~~~~~~~
    2021-10-14 21:45:52       | %define api.pure
    2021-10-14 21:45:52 ../grub-core/script/parser.y: warning: fix-its can be applied.  Rerun with option '--update'. [-Wother]
    2021-10-14 21:45:52 /home/ymorin/dev/buildroot/O/host/bin/i686-linux-gcc -E -DHAVE_CONFIG_H -I. -I..  -Wall -W -DGRUB_UTIL=1 -D_FILE_OFFSET_BITS=64 -I./include -DGRUB_FILE="util/grub-fstest.c" -I. -I.. -I. -I.. -I../include -I./include -I../grub-core/lib/libgcrypt-grub/src/ -I./grub-core/lib/gnulib -I../grub-core/lib/gnulib  -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -Os -fno-stack-protector -D_FILE_OFFSET_BITS=64   -D'GRUB_MOD_INIT(x)=@MARKER@x@' ../util/grub-fstest.c ../grub-core/kern/emu/hostfs.c ../ grub-core/disk/host.c ../grub-core/osdep/init.c > grub_fstest.pp || (rm -f grub_fstest.pp; exit 1)
    2021-10-14 21:45:52 ../grub-core/kern/emu/hostfs.c:20:10: fatal error: config-util.h: No such file or directory
    2021-10-14 21:45:52    20 | #include <config-util.h>
    2021-10-14 21:45:52       |          ^~~~~~~~~~~~~~~
    2021-10-14 21:45:52 compilation terminated.
    2021-10-14 21:45:52 make[2]: *** [Makefile:13107: grub_fstest.pp] Error 1
    2021-10-14 21:45:52 make[2]: *** Waiting for unfinished jobs....

And re-starting the build succeeds.

But just looking at how config-util.h is actually created (in Makefile.in) makes me cry:

   3389 config-util.h: stamp-h1
   3390     @test -f $@ || rm -f stamp-h1
   3391     @test -f $@ || $(MAKE) $(AM_MAKEFLAGS) stamp-h1
   3392
   3393 stamp-h1: $(srcdir)/config-util.h.in $(top_builddir)/config.status
   3394     @rm -f stamp-h1
   3395     cd $(top_builddir) && $(SHELL) ./config.status config-util.h

Whaaa?

But fortunately, it looks liek it is fixed with upstream commit 42f4054faf3c (Makefile: Make libgrub.pp depend on config-util.h).

Could you look into both the tools and the backport of the commit?

Regards,
Yann E. MORIN.

Quoted 7 lines

--
.-----------------.--------------------.------------------.--------------------.

|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |

'------------------------------^-------^------------------^--------------------'

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

#18 lore raw

Re: [Buildroot] [PATCH v3 2/7] boot/grub2: add support to build multiple Grub2 configurations in the same build

In reply to Yann E. MORIN

Köry, All,

On 2021-10-14 22:02 +0200, Yann E. MORIN spake thusly:

Quoted 2 lines

So, in the end, your original proposal, to install each and every tuples into target/, is probably the best option we have, [...]

Additionally, to test what was failing, I had instrumented the build with the following patch (note that it mixes fixes, like no semi-colons but separate lines, or using && not sime-colon, with the actual calls to MESSAGE):

    diff --git a/boot/grub2/grub2.mk b/boot/grub2/grub2.mk
    index e01ebb2edb..5400538837 100644
    --- a/boot/grub2/grub2.mk
    +++ b/boot/grub2/grub2.mk
    @@ -149,8 +149,9 @@ HOST_GRUB2_CONF_OPTS = \
     
     define GRUB2_CONFIGURE_CMDS
     	$(foreach tuple, $(GRUB2_TUPLES-y), \
    -		mkdir -p $(@D)/build-$(tuple) ; \
    -		cd $(@D)/build-$(tuple) ; \
    +		@$(call MESSAGE,Configuring $(tuple))
    +		mkdir -p $(@D)/build-$(tuple)
    +		cd $(@D)/build-$(tuple) && \
     		$(TARGET_CONFIGURE_OPTS) \
     				$(TARGET_CONFIGURE_ARGS) \
     		$(GRUB2_CONF_ENV) \
    @@ -172,13 +173,15 @@ endef
     
     define GRUB2_BUILD_CMDS
     	$(foreach tuple, $(GRUB2_TUPLES-y), \
    +		@$(call MESSAGE,Building $(tuple))
     		$(TARGET_MAKE_ENV) $(MAKE) -C $(@D)/build-$(tuple)
     	)
     endef
     
     define GRUB2_INSTALL_IMAGES_CMDS
     	$(foreach tuple, $(GRUB2_TUPLES-y), \
    -		mkdir -p $(dir $(GRUB2_IMAGE_$(tuple))) ; \
    +		@$(call MESSAGE,Installing $(tuple) to images directory)
    +		mkdir -p $(dir $(GRUB2_IMAGE_$(tuple)))
     		$(HOST_DIR)/usr/bin/grub-mkimage \
     			-d $(@D)/build-$(tuple)/grub-core/ \
     			-O $(tuple) \
    @@ -186,12 +189,12 @@ define GRUB2_INSTALL_IMAGES_CMDS
     			-p "$(GRUB2_PREFIX_$(tuple))" \
     			$(if $(GRUB2_BUILTIN_CONFIG_$(tuple)), \
     				-c $(GRUB2_BUILTIN_CONFIG_$(tuple))) \
    -				$(GRUB2_BUILTIN_MODULES_$(tuple)) ; \
    -		$(INSTALL) -D -m 0644 boot/grub2/grub.cfg $(GRUB2_CFG_$(tuple)) ; \
    +				$(GRUB2_BUILTIN_MODULES_$(tuple))
    +		$(INSTALL) -D -m 0644 boot/grub2/grub.cfg $(GRUB2_CFG_$(tuple))
     		$(if $(findstring $(GRUB2_PLATFORM_$(tuple)), pc), \
     			cat $(HOST_DIR)/lib/grub/$(tuple)/cdboot.img $(GRUB2_IMAGE_$(tuple)) > \
    -				$(BINARIES_DIR)/grub-eltorito.img ; \
    -		) \
    +				$(BINARIES_DIR)/grub-eltorito.img \
    +		)
     	)
     endef

That way, it is easier to see what is going on, and the output when filtered with brmake, looks nice ;-)

2021-10-14 22:52:36 >>> grub2 2.04 Extracting
2021-10-14 22:52:36 >>> grub2 2.04 Patching
2021-10-14 22:52:39 >>> grub2 2.04 Configuring
2021-10-14 22:52:39 >>> grub2 2.04 Configuring i386-pc
2021-10-14 22:52:56 >>> grub2 2.04 Configuring i386-efi
2021-10-14 22:53:13 >>> grub2 2.04 Building
2021-10-14 22:53:13 >>> grub2 2.04 Building i386-pc
2021-10-14 22:53:51 >>> grub2 2.04 Building i386-efi
2021-10-14 22:54:28 >>> grub2 2.04 Installing to target
2021-10-14 22:54:28 >>> grub2 2.04 Installing i386-pc to target
2021-10-14 22:54:31 >>> grub2 2.04 Installing i386-efi to target
2021-10-14 22:54:34 >>> grub2 2.04 Installing to images directory
2021-10-14 22:54:34 >>> grub2 2.04 Installing i386-pc to images directory
2021-10-14 22:54:34 >>> grub2 2.04 Installing i386-efi to images directory

Regards,
Yann E. MORIN.

Signature · 10 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#19 lore raw

Re: [Buildroot] [PATCH v3 2/7] boot/grub2: add support to build multiple Grub2 configurations in the same build

In reply to Yann E. MORIN

Hello Yann,

Quoted 3 lines · On Thu, 14 Oct 2021 22:02:07 +0200 "Yann E. MORIN" <[email protected]>…

Köry, All,

Sorry for my late reply, I've been pretty busy here...

No problem, be a maintainer is a full-time job done on your own time and you do great work! :)

Quoted 3 lines

So, in the end, your original proposal, to install each and every tuples into target/, is probably the best option we have, and it can be as simple as something like:

Alright

Quoted 32 lines

Test-built with:

    BR2_x86_pentium4=y
    BR2_TOOLCHAIN_EXTERNAL=y
    BR2_TARGET_GRUB2=y
    BR2_TARGET_GRUB2_I386_PC=y
    BR2_TARGET_GRUB2_I386_EFI=y
    BR2_TARGET_GRUB2_INSTALL_TOOLS=y

Side note, unrelated to this issue: we have a parallel build issue with grub2:

    2021-10-14 21:45:52 cd . && /bin/sh ./config.status config-util.h
    2021-10-14 21:45:52 config.status: creating config-util.h
    2021-10-14 21:45:52 bison -d -p grub_script_yy -b grub_script

../grub-core/script/parser.y 2021-10-14 21:45:52 ../grub-core/script/parser.y:92.1-12: warning: deprecated directive:
‘%pure-parser’, use ‘%define api.pure’ [-Wdeprecated] 2021-10-14 21:45:52
92 | %pure-parser 2021-10-14 21:45:52 | ^~~~~~~~~~~~ 2021-10-14
21:45:52 | %define api.pure 2021-10-14 21:45:52
../grub-core/script/parser.y: warning: fix-its can be applied. Rerun with option '--update'. [-Wother] 2021-10-14 21:45:52 /home/ymorin/dev/buildroot/O/host/bin/i686-linux-gcc -E -DHAVE_CONFIG_H -I.

(quoted diff) +0 -6
-I..  -Wall -W -DGRUB_UTIL=1 -D_FILE_OFFSET_BITS=64 -I./include
-DGRUB_FILE="util/grub-fstest.c" -I. -I.. -I. -I.. -I../include -I./include
-I../grub-core/lib/libgcrypt-grub/src/ -I./grub-core/lib/gnulib
-I../grub-core/lib/gnulib  -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE
-D_FILE_OFFSET_BITS=64 -Os -fno-stack-protector -D_FILE_OFFSET_BITS=64
-D'GRUB_MOD_INIT(x)=@MARKER@x@' ../util/grub-fstest.c

../grub-core/kern/emu/hostfs.c ../ grub-core/disk/host.c ../grub-core/osdep/init.c > grub_fstest.pp || (rm -f grub_fstest.pp; exit 1)

Quoted 8 lines
2021-10-14 21:45:52 ../grub-core/kern/emu/hostfs.c:20:10: fatal error:
config-util.h: No such file or directory 2021-10-14 21:45:52    20 | #include
<config-util.h> 2021-10-14 21:45:52       |          ^~~~~~~~~~~~~~~
2021-10-14 21:45:52 compilation terminated. 2021-10-14 21:45:52 make[2]: ***

[Makefile:13107: grub_fstest.pp] Error 1 2021-10-14 21:45:52 make[2]: *** Waiting for unfinished jobs....

And re-starting the build succeeds.

Weird, I do not face this build error with the same defconfig.

Regards,
Köry
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

#20 lore raw

Re: [Buildroot] [PATCH v3 2/7] boot/grub2: add support to build multiple Grub2 configurations in the same build

Quoted 3 lines · On Fri, 15 Oct 2021 11:19:39 +0200 Köry Maincent <[email protected]>…
Quoted 1 line

And re-starting the build succeeds.

Weird, I do not face this build error with the same defconfig.

It's a parallel build issue, which means it's a race condition. It will not happen all the time. Try on a build server with lots of CPU cores, and do the build in a loop multiple times.

Thomas

Signature · 7 lines
Thomas Petazzoni, co-owner and CEO, Bootlin
Embedded Linux and Kernel engineering and training
https://bootlin.com
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#21 lore raw

Re: [Buildroot] [PATCH v3 2/7] boot/grub2: add support to build multiple Grub2 configurations in the same build

In reply to Yann E. MORIN

Köry, All,

On 2021-10-14 22:02 +0200, Yann E. MORIN spake thusly:
[--SNIP--]

Quoted 6 lines

Side note, unrelated to this issue: we have a parallel build issue with grub2:

    2021-10-14 21:45:52 /home/ymorin/dev/buildroot/O/host/bin/i686-linux-gcc -E -DHAVE_CONFIG_H -I. -I..  -Wall -W -DGRUB_UTIL=1 -D_FILE_OFFSET_BITS=64 -I./include -DGRUB_FILE="util/grub-fstest.c" -I. -I.. -I. -I.. -I../include -I./include -I../grub-core/lib/libgcrypt-grub/src/ -I./grub-core/lib/gnulib -I../grub-core/lib/gnulib  -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -Os -fno-stack-protector -D_FILE_OFFSET_BITS=64   -D'GRUB_MOD_INIT(x)=@MARKER@x@' ../util/grub-fstest.c ../grub-core/kern/emu/hostfs.c ../ grub-core/disk/host.c ../grub-core/osdep/init.c > grub_fstest.pp || (rm -f grub_fstest.pp; exit 1)
    2021-10-14 21:45:52 ../grub-core/kern/emu/hostfs.c:20:10: fatal error: config-util.h: No such file or directory
    2021-10-14 21:45:52    20 | #include <config-util.h>
    2021-10-14 21:45:52       |          ^~~~~~~~~~~~~~~

[--SNIP--]

Quoted 2 lines

But fortunately, it looks liek it is fixed with upstream commit 42f4054faf3c (Makefile: Make libgrub.pp depend on config-util.h).

And apparently, this is not enough, because I still hit the issue, about one in five builds, always when building i386-efi... :-(

But given how inventive the grub buildsystem is, I dread to ever have to look at it once more... :-(

Regards,
Yann E. MORIN.

Signature · 10 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#22 lore raw

Re: [Buildroot] [PATCH v3 2/7] boot/grub2: add support to build multiple Grub2 configurations in the same build

Hey Kory;

Do you plan on submitting a patch to fix this? If not, I can do so. Just let me know!

Adam

Quoted 24 lines · Yann E. MORIN

Köry, All,

On 2021-10-14 22:02 +0200, Yann E. MORIN spake thusly:
[--SNIP--]

Quoted 6 lines

Side note, unrelated to this issue: we have a parallel build issue with grub2:

    2021-10-14 21:45:52 /home/ymorin/dev/buildroot/O/host/bin/i686-linux-gcc -E -DHAVE_CONFIG_H -I. -I..  -Wall -W -DGRUB_UTIL=1 -D_FILE_OFFSET_BITS=64 -I./include -DGRUB_FILE="util/grub-fstest.c" -I. -I.. -I. -I.. -I../include -I./include -I../grub-core/lib/libgcrypt-grub/src/ -I./grub-core/lib/gnulib -I../grub-core/lib/gnulib  -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -Os -fno-stack-protector -D_FILE_OFFSET_BITS=64   -D'GRUB_MOD_INIT(x)=@MARKER@x@' ../util/grub-fstest.c ../grub-core/kern/emu/hostfs.c ../ grub-core/disk/host.c ../grub-core/osdep/init.c > grub_fstest.pp || (rm -f grub_fstest.pp; exit 1)
    2021-10-14 21:45:52 ../grub-core/kern/emu/hostfs.c:20:10: fatal error: config-util.h: No such file or directory
    2021-10-14 21:45:52    20 | #include <config-util.h>
    2021-10-14 21:45:52       |          ^~~~~~~~~~~~~~~

[--SNIP--]

Quoted 2 lines

But fortunately, it looks liek it is fixed with upstream commit 42f4054faf3c (Makefile: Make libgrub.pp depend on config-util.h).

And apparently, this is not enough, because I still hit the issue, about one in five builds, always when building i386-efi... :-(

But given how inventive the grub buildsystem is, I dread to ever have to look at it once more... :-(

Regards,
Yann E. MORIN.

Quoted 7 lines

--
.-----------------.--------------------.------------------.--------------------.

|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |

'------------------------------^-------^------------------^--------------------'

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

#23 lore raw

Re: [Buildroot] [PATCH v3 2/7] boot/grub2: add support to build multiple Grub2 configurations in the same build

Hello Adam,

Quoted 4 lines · On Tue, 19 Oct 2021 09:30:10 -0700 Adam Duskett <[email protected]> wrote:

Hey Kory;

Do you plan on submitting a patch to fix this? If not, I can do so. Just let me know!

I was about to do it before Yann spot the parallel build error. As I did not managed to face the error I have stopped my action on the patch. I can however send the first part of it that fixing tools install, then wait the fix for the parallel build issue.

Regards,
Köry

Quoted 35 lines

Adam

Quoted 31 lines · Yann E. MORIN

Köry, All,

On 2021-10-14 22:02 +0200, Yann E. MORIN spake thusly:
[--SNIP--]

Quoted 15 lines

Side note, unrelated to this issue: we have a parallel build issue with grub2:

    2021-10-14 21:45:52

/home/ymorin/dev/buildroot/O/host/bin/i686-linux-gcc -E -DHAVE_CONFIG_H

(quoted diff) +0 -6
-I. -I..  -Wall -W -DGRUB_UTIL=1 -D_FILE_OFFSET_BITS=64 -I./include
-DGRUB_FILE="util/grub-fstest.c" -I. -I.. -I. -I.. -I../include
-I./include -I../grub-core/lib/libgcrypt-grub/src/
-I./grub-core/lib/gnulib -I../grub-core/lib/gnulib  -D_LARGEFILE_SOURCE
-D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -Os -fno-stack-protector
-D_FILE_OFFSET_BITS=64   -D'GRUB_MOD_INIT(x)=@MARKER@x@'

../util/grub-fstest.c ../grub-core/kern/emu/hostfs.c ../ grub-core/disk/host.c ../grub-core/osdep/init.c > grub_fstest.pp || (rm -f grub_fstest.pp; exit 1) 2021-10-14 21:45:52 ../grub-core/kern/emu/hostfs.c:20:10: fatal error: config-util.h: No such
file or directory 2021-10-14 21:45:52 20 | #include <config-util.h>

Quoted 1 line
2021-10-14 21:45:52       |          ^~~~~~~~~~~~~~~  

[--SNIP--]

Quoted 2 lines

But fortunately, it looks liek it is fixed with upstream commit 42f4054faf3c (Makefile: Make libgrub.pp depend on config-util.h).

And apparently, this is not enough, because I still hit the issue, about one in five builds, always when building i386-efi... :-(

But given how inventive the grub buildsystem is, I dread to ever have to look at it once more... :-(

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

#24 lore raw

PATCH v3 3/7 fs/iso9660: add support to Grub EFI bootloader in the image

In reply to Kory Maincent

Add support to boot the Grub bootloader from an EFI BIOS in the ISO9660 image.
For that we need to create EFI System Partition (ESP). The ESP is a vfat partition which contain the Grub2 binary at the /EFI/BOOT/ location. xorriso command will generate the iso image including the ESP.

We notice Grub can not read and mount the ESP, therefore we place the Grub configuration file in the ISO9660 partition. A Grub2 builtin configuration need to be used to tell Grub2 to search automatically its configuration file in the ISO9660 partition. Use 'set root=(cd0)' in the configuration file passed to BR2_TARGET_GRUB2_BUILTIN_CONFIG_EFI.

---

Changes in v2:
Following the review from Yann E. Morin:
- Add parameter to mkfs and mcopy to allow reproducible builds
- Fix typos and variable multi-line assignement

Changes in v3:
- Fix Kconfig dependencies.
- Add exclusive configuration choice between BIOS bootloader and EFI bootloader in Kconfig.
- Fix namings and reproducibility issues.
- Update image generation options to separate common options.

 fs/iso9660/Config.in  | 29 ++++++++++++++++++++++-------
 fs/iso9660/iso9660.mk | 43 ++++++++++++++++++++++++++++++++++++++++---
 2 files changed, 62 insertions(+), 10 deletions(-)
fs/iso9660/Config.in +22 -7
index 6f001c0640..f8f577506d 100644
@@ -2,8 +2,7 @@ config BR2_TARGET_ROOTFS_ISO9660
 	bool "iso image"
 	depends on (BR2_i386 || BR2_x86_64)
 	depends on BR2_LINUX_KERNEL
-	depends on BR2_TARGET_GRUB2_I386_PC || \
-		BR2_TARGET_SYSLINUX_ISOLINUX
+	depends on BR2_TARGET_GRUB2 || BR2_TARGET_SYSLINUX_ISOLINUX
 	select BR2_LINUX_KERNEL_INSTALL_TARGET \
 	       if (!BR2_TARGET_ROOTFS_ISO9660_INITRD && !BR2_TARGET_ROOTFS_INITRAMFS)
 	help
@@ -27,12 +26,15 @@ choice
 
 config BR2_TARGET_ROOTFS_ISO9660_GRUB2
 	bool "grub2"
-	depends on BR2_TARGET_GRUB2_I386_PC
+	depends on BR2_TARGET_GRUB2
 	help
 	  Use Grub 2 as the bootloader for the ISO9660 image. Make
 	  sure to enable the 'iso9660' module in
-	  BR2_TARGET_GRUB2_BUILTIN_MODULES and to use 'cd' as the boot
-	  partition in BR2_TARGET_GRUB2_BOOT_PARTITION=.
+	  BR2_TARGET_GRUB2_BUILTIN_MODULES_PC or
+	  BR2_TARGET_GRUB2_BUILTIN_MODULES_EFI. Use 'cd' as the boot
+	  partition in BR2_TARGET_GRUB2_BOOT_PARTITION= for GRUB on BIOS
+	  or 'set root=(cd0)' in the configuration file passed to
+	  BR2_TARGET_GRUB2_BUILTIN_CONFIG_EFI for GRUB on EFI.
 
 config BR2_TARGET_ROOTFS_ISO9660_ISOLINUX
 	bool "isolinux"
@@ -40,6 +42,19 @@ config BR2_TARGET_ROOTFS_ISO9660_ISOLINUX
 
 endchoice
 
+choice
+	prompt "Boot payload"
+
+config BR2_TARGET_ROOTFS_ISO9660_BIOS_BOOTLOADER
+	bool "legacy bios"
+	depends on BR2_TARGET_GRUB2_I386_PC || BR2_TARGET_ROOTFS_ISO9660_ISOLINUX
+
+config BR2_TARGET_ROOTFS_ISO9660_EFI_BOOTLOADER
+	bool "UEFI"
+	depends on BR2_TARGET_GRUB2_I386_EFI || BR2_TARGET_GRUB2_X86_64_EFI
+
+endchoice
+
 config BR2_TARGET_ROOTFS_ISO9660_BOOT_MENU
 	string "Boot menu config file"
 	default "fs/iso9660/grub.cfg" if BR2_TARGET_ROOTFS_ISO9660_GRUB2
@@ -83,7 +98,7 @@ config BR2_TARGET_ROOTFS_ISO9660_HYBRID
 
 endif
 
-comment "iso image needs a Linux kernel and either grub2 i386-pc or isolinux to be built"
+comment "iso image needs a Linux kernel and either grub2 or isolinux to be built"
 	depends on BR2_i386 || BR2_x86_64
 	depends on !BR2_LINUX_KERNEL || \
-		!(BR2_TARGET_GRUB2_I386_PC || BR2_TARGET_SYSLINUX_ISOLINUX)
+		!(BR2_TARGET_GRUB2 || BR2_TARGET_SYSLINUX_ISOLINUX)
fs/iso9660/iso9660.mk +40 -3

62 lines not on this page · raw message

Signature · 6 lines
2.25.1

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#25 lore raw

Re: [Buildroot] [PATCH v3 3/7] fs/iso9660: add support to Grub EFI bootloader in the image

Köry, All,

On 2021-09-23 17:57 +0200, Kory Maincent spake thusly:

Quoted 6 lines

Add support to boot the Grub bootloader from an EFI BIOS in the ISO9660 image.
For that we need to create EFI System Partition (ESP). The ESP is a vfat partition which contain the Grub2 binary at the /EFI/BOOT/ location. xorriso command will generate the iso image including the ESP.

Quoted 8 lines

We notice Grub can not read and mount the ESP, therefore we place the Grub configuration file in the ISO9660 partition. A Grub2 builtin configuration need to be used to tell Grub2 to search automatically its configuration file in the ISO9660 partition. Use 'set root=(cd0)' in the configuration file passed to BR2_TARGET_GRUB2_BUILTIN_CONFIG_EFI.

---

[--SNIP--]

fs/iso9660/iso9660.mk else · line 62

Quoted 12 lines
fs/iso9660/iso9660.mk +3 -1
index 23421a9a5c..921efa1b02 100644
@@ -57,7 +57,12 @@ else
 ROOTFS_ISO9660_TMP_TARGET_DIR = $(TARGET_DIR)
 endif
 
-ifeq ($(BR2_TARGET_ROOTFS_ISO9660_GRUB2),y)
+ifeq ($(BR2_REPRODUCIBLE),y)
+ROOTFS_ISO9660_VFAT_OPTS = --invariant
+ROOTFS_ISO9660_FIX_TIME = touch $(SOURCE_DATE_EPOCH)

I think you may have missed something about SOURCE_DATE_EPOCH: it is a UNIX timestamp, as a number of seconds elapsed since 1970-01-01T00:00:00+0000. Right now, it's around 1632775187.

So, what you wrote would create a file named '1632775187' in the Buildroot top-level directory... Probably not what is expected...

So I've changed to:

    ifeq ($(BR2_REPRODUCIBLE),y)
    ROOTFS_ISO9660_VFAT_OPTS = --invariant
    ROOTFS_ISO9660_FIX_TIME = touch -d @$(SOURCE_DATE_EPOCH)
    else
    ROOTFS_ISO9660_FIX_TIME = @:
    endif

... and then...

Quoted 11 lines
fs/iso9660/iso9660.mk +4 -0
+ifeq ($(BR2_TARGET_ROOTFS_ISO9660_GRUB2)$(BR2_TARGET_ROOTFS_ISO9660_BIOS_BOOTLOADER),yy)
 ROOTFS_ISO9660_DEPENDENCIES += grub2
 ROOTFS_ISO9660_BOOTLOADER_CONFIG_PATH = \
 	$(ROOTFS_ISO9660_TMP_TARGET_DIR)/boot/grub/grub.cfg
@@ -66,6 +71,22 @@ define ROOTFS_ISO9660_INSTALL_BOOTLOADER
 	$(INSTALL) -D -m 0644 $(BINARIES_DIR)/grub-eltorito.img \
 		$(ROOTFS_ISO9660_TMP_TARGET_DIR)/boot/grub/grub-eltorito.img
 endef
+else ifeq ($(BR2_TARGET_ROOTFS_ISO9660_GRUB2)$(BR2_TARGET_ROOTFS_ISO9660_EFI_BOOTLOADER),yy)
+ROOTFS_ISO9660_DEPENDENCIES += grub2 host-dosfstools host-mtools
+ROOTFS_ISO9660_EFI_PARTITION = boot/fat.efi

fs/iso9660/iso9660.mk define ROOTFS_ISO9660_INSTALL_BOOTLOADER · line 86

Quoted 10 lines
fs/iso9660/iso9660.mk +10 -0
+ROOTFS_ISO9660_EFI_PARTITION_PATH = $(ROOTFS_ISO9660_TMP_TARGET_DIR)/$(ROOTFS_ISO9660_EFI_PARTITION)
+ROOTFS_ISO9660_EFI_PARTITION_CONTENT = $(BINARIES_DIR)/efi-part
+ROOTFS_ISO9660_BOOTLOADER_CONFIG_PATH = \
+	$(ROOTFS_ISO9660_TMP_TARGET_DIR)/boot/grub/grub.cfg
+define ROOTFS_ISO9660_INSTALL_BOOTLOADER
+	rm -rf $(ROOTFS_ISO9660_EFI_PARTITION_PATH)
+	mkdir -p $(dir $(ROOTFS_ISO9660_EFI_PARTITION_PATH))
+	dd if=/dev/zero of=$(ROOTFS_ISO9660_EFI_PARTITION_PATH) bs=1M count=1
+	$(ROOTFS_ISO9660_FIX_TIME)
+	$(HOST_DIR)/sbin/mkfs.vfat $(ROOTFS_ISO9660_VFAT_OPTS) $(ROOTFS_ISO9660_EFI_PARTITION_PATH)

... here:

    $(ROOTFS_ISO9660_FIX_TIME) $(ROOTFS_ISO9660_EFI_PARTITION_CONTENT)/*

... and...

fs/iso9660/iso9660.mk define ROOTFS_ISO9660_INSTALL_BOOTLOADER · line 88

Quoted 2 lines
fs/iso9660/iso9660.mk +2 -0
+	$(HOST_DIR)/bin/mcopy -p -m -i $(ROOTFS_ISO9660_EFI_PARTITION_PATH) -s \
+		$(ROOTFS_ISO9660_EFI_PARTITION_CONTENT)/* ::/

... here:

    $(ROOTFS_ISO9660_FIX_TIME) $(ROOTFS_ISO9660_EFI_PARTITION_PATH)

Applied to master with the above changes, thanks.

Regards,
Yann E. MORIN.

Quoted 39 lines
fs/iso9660/iso9660.mk +19 -2
+endef
 else ifeq ($(BR2_TARGET_ROOTFS_ISO9660_ISOLINUX),y)
 ROOTFS_ISO9660_DEPENDENCIES += syslinux
 ROOTFS_ISO9660_BOOTLOADER_CONFIG_PATH = \
@@ -128,9 +149,25 @@ ROOTFS_ISO9660_PRE_GEN_HOOKS += ROOTFS_ISO9660_DISABLE_EXTERNAL_INITRD
 
 endif # ROOTFS_ISO9660_USE_INITRD
 
+ROOTFS_ISO9660_OPTS += \
+	-J \
+	-R \
+	-no-emul-boot
+
+ifeq ($(BR2_TARGET_ROOTFS_ISO9660_BIOS_BOOTLOADER),y)
+ROOTFS_ISO9660_OPTS += \
+	-boot-load-size 4 \
+	-boot-info-table \
+	-b $(ROOTFS_ISO9660_BOOT_IMAGE)
+endif
+
+ifeq ($(BR2_TARGET_ROOTFS_ISO9660_EFI_BOOTLOADER),y)
+ROOTFS_ISO9660_OPTS += \
+	--efi-boot $(ROOTFS_ISO9660_EFI_PARTITION)
+endif
+
 define ROOTFS_ISO9660_CMD
-	$(HOST_DIR)/bin/xorriso -as mkisofs -J -R -b $(ROOTFS_ISO9660_BOOT_IMAGE) \
-		-no-emul-boot -boot-load-size 4 -boot-info-table \
+	$(HOST_DIR)/bin/xorriso -as mkisofs \
 		$(ROOTFS_ISO9660_OPTS) \
 		-o $@ $(ROOTFS_ISO9660_TMP_TARGET_DIR)
 endef
Signature · 6 lines
2.25.1

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
Signature · 10 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#26 lore raw

Re: [Buildroot] [PATCH v3 3/7] fs/iso9660: add support to Grub EFI bootloader in the image

Köry, All,

On 2021-09-27 22:43 +0200, Yann E. MORIN spake thusly:

Quoted 3 lines

On 2021-09-23 17:57 +0200, Kory Maincent spake thusly:

Quoted 2 lines

Add support to boot the Grub bootloader from an EFI BIOS in the ISO9660 image.

[--SNIP--]

Quoted 4 lines

fs/iso9660/iso9660.mk else · line 62

Quoted 4 lines
fs/iso9660/iso9660.mk +3 -1
-ifeq ($(BR2_TARGET_ROOTFS_ISO9660_GRUB2),y)
+ifeq ($(BR2_REPRODUCIBLE),y)
+ROOTFS_ISO9660_VFAT_OPTS = --invariant
+ROOTFS_ISO9660_FIX_TIME = touch $(SOURCE_DATE_EPOCH)

[--SNIP--]

Quoted 6 lines

So I've changed to:

    ifeq ($(BR2_REPRODUCIBLE),y)
    ROOTFS_ISO9660_VFAT_OPTS = --invariant
    ROOTFS_ISO9660_FIX_TIME = touch -d @$(SOURCE_DATE_EPOCH)
    else
    ROOTFS_ISO9660_FIX_TIME = @:

Arg, I made a mistake: the commands fir filesystems are executed as a stadnard Makefile rule; instead, they are stashed into a script and executed by the shell (under fakeroot).

So, we can quiesce a command by prefixing it with '@'.

I've pushed a fix.

Sorry for the mess...

Regards,
Yann E. MORIN.

Signature · 10 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#27 lore raw

PATCH v3 4/7 fs/iso9660: add support for hybrid image using Grub bootloader on BIOS and EFI

In reply to Kory Maincent

Add support for building an hybrid ISO9660 image compatible with legacy and UEFI BIOS.
The option -eltorito-alt-boot need to be used in the xorriso command to generate the hybrid image.

---

Changes in v2:
- Fix typos

Changes in v3:
- Follow changes brougt by modification of the third patch.

 fs/iso9660/Config.in  | 16 +++++++--------
 fs/iso9660/iso9660.mk | 45 ++++++++++++++++++++++++++++---------------
 2 files changed, 36 insertions(+), 25 deletions(-)
fs/iso9660/Config.in +7 -9
index f8f577506d..8c4e7badf8 100644
@@ -27,6 +27,10 @@ choice
 config BR2_TARGET_ROOTFS_ISO9660_GRUB2
 	bool "grub2"
 	depends on BR2_TARGET_GRUB2
+	select BR2_TARGET_ROOTFS_ISO9660_BIOS_BOOTLOADER \
+		if BR2_TARGET_GRUB2_I386_PC
+	select BR2_TARGET_ROOTFS_ISO9660_EFI_BOOTLOADER \
+		if (BR2_TARGET_GRUB2_I386_EFI || BR2_TARGET_GRUB2_X86_64_EFI)
 	help
 	  Use Grub 2 as the bootloader for the ISO9660 image. Make
 	  sure to enable the 'iso9660' module in
@@ -39,21 +43,15 @@ config BR2_TARGET_ROOTFS_ISO9660_GRUB2
 config BR2_TARGET_ROOTFS_ISO9660_ISOLINUX
 	bool "isolinux"
 	depends on BR2_TARGET_SYSLINUX_ISOLINUX
+	select BR2_TARGET_ROOTFS_ISO9660_BIOS_BOOTLOADER
 
 endchoice
 
-choice
-	prompt "Boot payload"
-
 config BR2_TARGET_ROOTFS_ISO9660_BIOS_BOOTLOADER
-	bool "legacy bios"
-	depends on BR2_TARGET_GRUB2_I386_PC || BR2_TARGET_ROOTFS_ISO9660_ISOLINUX
+	bool
 
 config BR2_TARGET_ROOTFS_ISO9660_EFI_BOOTLOADER
-	bool "UEFI"
-	depends on BR2_TARGET_GRUB2_I386_EFI || BR2_TARGET_GRUB2_X86_64_EFI
-
-endchoice
+	bool
 
 config BR2_TARGET_ROOTFS_ISO9660_BOOT_MENU
 	string "Boot menu config file"
fs/iso9660/iso9660.mk +29 -16

90 lines not on this page · raw message

Signature · 6 lines
2.25.1

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#28 lore raw

Re: [Buildroot] [PATCH v3 4/7] fs/iso9660: add support for hybrid image using Grub bootloader on BIOS and EFI

Köry, All,

On 2021-09-23 17:57 +0200, Kory Maincent spake thusly:

Quoted 4 lines

Add support for building an hybrid ISO9660 image compatible with legacy and UEFI BIOS.
The option -eltorito-alt-boot need to be used in the xorriso command to generate the hybrid image.

So I was wondering why you had to rename the variables, and why you split the values with -eltorito-alt-boot right in between...

As I understand it from the xorriso command line, -eltorito-alt-boot basically means "OK, we're done with the previous boot parameters, now we start a new set of boot parameters", so the order of options *is* important.

Right?

Furthermore, -eltorito-alt-boot and -no-emul-boot are not conflicting, because the former is really this separator as discussed above, while the latter specifies the type of the current boot image.

Right?

(For my information: how many such alternate boot parameters can we
define: is it limited to just two, or can we add more?)

Quoted 2 lines

---

[--SNIP--]

fs/iso9660/iso9660.mk

Quoted 4 lines
fs/iso9660/iso9660.mk +0 -0
index 921efa1b02..9c2535d102 100644

[--SNIP--]

fs/iso9660/iso9660.mk endif # ROOTFS_ISO9660_USE_INITRD · line 161

Quoted 11 lines
fs/iso9660/iso9660.mk +2 -4
@@ -151,24 +156,32 @@ endif # ROOTFS_ISO9660_USE_INITRD
 
 ROOTFS_ISO9660_OPTS += \
 	-J \
-	-R \
-	-no-emul-boot
+	-R
 
-ifeq ($(BR2_TARGET_ROOTFS_ISO9660_BIOS_BOOTLOADER),y)
-ROOTFS_ISO9660_OPTS += \
+ROOTFS_ISO9660_BOOTLOADER_OPTS_BIOS = \
    ROOTFS_ISO9660_OPTS_BIOS

fs/iso9660/iso9660.mk endif # ROOTFS_ISO9660_USE_INITRD · line 165

Quoted 7 lines
fs/iso9660/iso9660.mk +3 -3
+	-b $(ROOTFS_ISO9660_BOOT_IMAGE) \
+	-no-emul-boot \
 	-boot-load-size 4 \
-	-boot-info-table \
-	-b $(ROOTFS_ISO9660_BOOT_IMAGE)
-endif
+	-boot-info-table

You are also reordering the remaining options. Is that really necessary? Or is it for symmetry with the EFI case where the "image" option comes first?

fs/iso9660/iso9660.mk endif # ROOTFS_ISO9660_USE_INITRD · line 167

Quoted 4 lines
fs/iso9660/iso9660.mk +1 -3
-ifeq ($(BR2_TARGET_ROOTFS_ISO9660_EFI_BOOTLOADER),y)
-ROOTFS_ISO9660_OPTS += \
-	--efi-boot $(ROOTFS_ISO9660_EFI_PARTITION)
+ROOTFS_ISO9660_BOOTLOADER_OPTS_EFI = \
    ROOTFS_ISO9660_OPTS_EFI

fs/iso9660/iso9660.mk endif # ROOTFS_ISO9660_USE_INITRD · line 174

Quoted 7 lines
fs/iso9660/iso9660.mk +7 -0
+	--efi-boot $(ROOTFS_ISO9660_EFI_PARTITION) \
+	-no-emul-boot
+
+ifeq ($(BR2_TARGET_ROOTFS_ISO9660_BIOS_BOOTLOADER)$(BR2_TARGET_ROOTFS_ISO9660_EFI_BOOTLOADER),yy)
+ROOTFS_ISO9660_BOOTLOADER_OPTS = $(ROOTFS_ISO9660_BOOTLOADER_OPTS_BIOS)
+ROOTFS_ISO9660_BOOTLOADER_OPTS += -eltorito-alt-boot
+ROOTFS_ISO9660_BOOTLOADER_OPTS += $(ROOTFS_ISO9660_BOOTLOADER_OPTS_EFI)

Hmmm This is pretty ugly... What about:

    ROOTFS_ISO9660_OPTS += \
        $(ROOTFS_ISO9660_OPTS_BIOS) \
        -eltorito-alt-boot \
        $(ROOTFS_ISO9660_OPTS_EFI)

Also, does BIOS really has to come before EFI? If so, why?

fs/iso9660/iso9660.mk endif # ROOTFS_ISO9660_USE_INITRD · line 176

Quoted 2 lines
fs/iso9660/iso9660.mk +2 -0
+else ifeq ($(BR2_TARGET_ROOTFS_ISO9660_BIOS_BOOTLOADER),y)
+ROOTFS_ISO9660_BOOTLOADER_OPTS = $(ROOTFS_ISO9660_BOOTLOADER_OPTS_BIOS)
    ROOTFS_ISO9660_OPTS += $(ROOTFS_ISO9660_OPTS_BIOS)

fs/iso9660/iso9660.mk endif # ROOTFS_ISO9660_USE_INITRD · line 178

Quoted 2 lines
fs/iso9660/iso9660.mk +2 -0
+else ifeq ($(BR2_TARGET_ROOTFS_ISO9660_EFI_BOOTLOADER),y)
+ROOTFS_ISO9660_BOOTLOADER_OPTS = $(ROOTFS_ISO9660_BOOTLOADER_OPTS_EFI)
    ROOTFS_ISO9660_OPTS += $(ROOTFS_ISO9660_OPTS_EFI)

fs/iso9660/iso9660.mk endif # ROOTFS_ISO9660_USE_INITRD · line 184

Quoted 6 lines
fs/iso9660/iso9660.mk +1 -0
 endif
 
 define ROOTFS_ISO9660_CMD
 	$(HOST_DIR)/bin/xorriso -as mkisofs \
 		$(ROOTFS_ISO9660_OPTS) \
+		$(ROOTFS_ISO9660_BOOTLOADER_OPTS) \

And thus you don't need the new variable here.

Regards,
Yann E. MORIN.

Quoted 10 lines
fs/iso9660/iso9660.mk +0 -0
 		-o $@ $(ROOTFS_ISO9660_TMP_TARGET_DIR)
 endef
 
Signature · 6 lines
2.25.1

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
Signature · 10 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#29 lore raw

Re: [Buildroot] [PATCH v3 4/7] fs/iso9660: add support for hybrid image using Grub bootloader on BIOS and EFI

Yann,

Quoted 11 lines · On Mon, 27 Sep 2021 23:05:14 +0200 "Yann E. MORIN" <[email protected]>…

Köry, All,

On 2021-09-23 17:57 +0200, Kory Maincent spake thusly:

Quoted 4 lines

Add support for building an hybrid ISO9660 image compatible with legacy and UEFI BIOS.
The option -eltorito-alt-boot need to be used in the xorriso command to generate the hybrid image.

So I was wondering why you had to rename the variables, and why you split the values with -eltorito-alt-boot right in between...

Quoted 6 lines

As I understand it from the xorriso command line, -eltorito-alt-boot basically means "OK, we're done with the previous boot parameters, now we start a new set of boot parameters", so the order of options *is* important.

Right?

Right

Quoted 6 lines

Furthermore, -eltorito-alt-boot and -no-emul-boot are not conflicting, because the former is really this separator as discussed above, while the latter specifies the type of the current boot image.

Right?

Right

Quoted 3 lines

(For my information: how many such alternate boot parameters can we
define: is it limited to just two, or can we add more?)

According to xorriso manual we can add up to 32 boot catalog entries.

Quoted 23 lines
Quoted 3 lines

---

[--SNIP--]

fs/iso9660/iso9660.mk

Quoted 4 lines
fs/iso9660/iso9660.mk +0 -0
index 921efa1b02..9c2535d102 100644

[--SNIP--]

fs/iso9660/iso9660.mk endif # ROOTFS_ISO9660_USE_INITRD · line 161

Quoted 11 lines
fs/iso9660/iso9660.mk +2 -4
@@ -151,24 +156,32 @@ endif # ROOTFS_ISO9660_USE_INITRD
 
 ROOTFS_ISO9660_OPTS += \
 	-J \
-	-R \
-	-no-emul-boot
+	-R
 
-ifeq ($(BR2_TARGET_ROOTFS_ISO9660_BIOS_BOOTLOADER),y)
-ROOTFS_ISO9660_OPTS += \
+ROOTFS_ISO9660_BOOTLOADER_OPTS_BIOS = \  
    ROOTFS_ISO9660_OPTS_BIOS
Quoted 11 lines

fs/iso9660/iso9660.mk endif # ROOTFS_ISO9660_USE_INITRD · line 165

Quoted 7 lines
fs/iso9660/iso9660.mk +3 -3
+	-b $(ROOTFS_ISO9660_BOOT_IMAGE) \
+	-no-emul-boot \
 	-boot-load-size 4 \
-	-boot-info-table \
-	-b $(ROOTFS_ISO9660_BOOT_IMAGE)
-endif
+	-boot-info-table  

You are also reordering the remaining options. Is that really necessary? Or is it for symmetry with the EFI case where the "image" option comes first?

It was for symmetry.

Quoted 19 lines

fs/iso9660/iso9660.mk endif # ROOTFS_ISO9660_USE_INITRD · line 167

Quoted 5 lines
fs/iso9660/iso9660.mk +1 -3
-ifeq ($(BR2_TARGET_ROOTFS_ISO9660_EFI_BOOTLOADER),y)
-ROOTFS_ISO9660_OPTS += \
-	--efi-boot $(ROOTFS_ISO9660_EFI_PARTITION)
+ROOTFS_ISO9660_BOOTLOADER_OPTS_EFI = \  
    ROOTFS_ISO9660_OPTS_EFI

fs/iso9660/iso9660.mk endif # ROOTFS_ISO9660_USE_INITRD · line 174

Quoted 8 lines
fs/iso9660/iso9660.mk +7 -0
+	--efi-boot $(ROOTFS_ISO9660_EFI_PARTITION) \
+	-no-emul-boot
+
+ifeq
 ($(BR2_TARGET_ROOTFS_ISO9660_BIOS_BOOTLOADER)$(BR2_TARGET_ROOTFS_ISO9660_EFI_BOOTLOADER),yy)
+ROOTFS_ISO9660_BOOTLOADER_OPTS = $(ROOTFS_ISO9660_BOOTLOADER_OPTS_BIOS)
+ROOTFS_ISO9660_BOOTLOADER_OPTS += -eltorito-alt-boot
+ROOTFS_ISO9660_BOOTLOADER_OPTS += $(ROOTFS_ISO9660_BOOTLOADER_OPTS_EFI)  

Hmmm This is pretty ugly... What about:

Quoted 6 lines
    ROOTFS_ISO9660_OPTS += \
        $(ROOTFS_ISO9660_OPTS_BIOS) \
        -eltorito-alt-boot \
        $(ROOTFS_ISO9660_OPTS_EFI)

Also, does BIOS really has to come before EFI? If so, why?

It seems BIOS need to be the first boot option. I have not found any information about it but if I invert BIOS and EFI options it doesn't boot.

Regards,
Köry
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

#30 lore raw

Re: [Buildroot] [PATCH v3 4/7] fs/iso9660: add support for hybrid image using Grub bootloader on BIOS and EFI

In reply to Kory Maincent

Köry, All,

On 2021-09-23 17:57 +0200, Kory Maincent spake thusly:

Quoted 6 lines

Add support for building an hybrid ISO9660 image compatible with legacy and UEFI BIOS.
The option -eltorito-alt-boot need to be used in the xorriso command to generate the hybrid image.

Applied to master, thanks, with the folowing changes;

- note about hybrid vs. (iso)hybrid
- explain -eltorito-alt-boot
- explain duplication of -no-emul-boot
- rename the variables
- note about the BIOS-EFI ordering

Since I had done a few changes in the previous patches, of course I had to resolve the conflicts when applying this patch... I've tried to be as carefull as I could, but I may have (again!) let some issues slip theough...

Could you please review/test that it is till works as expected?

I don't think the remaining patches need rebasing, though, so I hope I can handle more than one next evening... :-/

Regards,
Yann E. MORIN.

Quoted 157 lines

---

Changes in v2:
- Fix typos

Changes in v3:
- Follow changes brougt by modification of the third patch.

 fs/iso9660/Config.in  | 16 +++++++--------
 fs/iso9660/iso9660.mk | 45 ++++++++++++++++++++++++++++---------------
 2 files changed, 36 insertions(+), 25 deletions(-)
fs/iso9660/Config.in +7 -9
index f8f577506d..8c4e7badf8 100644
@@ -27,6 +27,10 @@ choice
 config BR2_TARGET_ROOTFS_ISO9660_GRUB2
 	bool "grub2"
 	depends on BR2_TARGET_GRUB2
+	select BR2_TARGET_ROOTFS_ISO9660_BIOS_BOOTLOADER \
+		if BR2_TARGET_GRUB2_I386_PC
+	select BR2_TARGET_ROOTFS_ISO9660_EFI_BOOTLOADER \
+		if (BR2_TARGET_GRUB2_I386_EFI || BR2_TARGET_GRUB2_X86_64_EFI)
 	help
 	  Use Grub 2 as the bootloader for the ISO9660 image. Make
 	  sure to enable the 'iso9660' module in
@@ -39,21 +43,15 @@ config BR2_TARGET_ROOTFS_ISO9660_GRUB2
 config BR2_TARGET_ROOTFS_ISO9660_ISOLINUX
 	bool "isolinux"
 	depends on BR2_TARGET_SYSLINUX_ISOLINUX
+	select BR2_TARGET_ROOTFS_ISO9660_BIOS_BOOTLOADER
 
 endchoice
 
-choice
-	prompt "Boot payload"
-
 config BR2_TARGET_ROOTFS_ISO9660_BIOS_BOOTLOADER
-	bool "legacy bios"
-	depends on BR2_TARGET_GRUB2_I386_PC || BR2_TARGET_ROOTFS_ISO9660_ISOLINUX
+	bool
 
 config BR2_TARGET_ROOTFS_ISO9660_EFI_BOOTLOADER
-	bool "UEFI"
-	depends on BR2_TARGET_GRUB2_I386_EFI || BR2_TARGET_GRUB2_X86_64_EFI
-
-endchoice
+	bool
 
 config BR2_TARGET_ROOTFS_ISO9660_BOOT_MENU
 	string "Boot menu config file"
fs/iso9660/iso9660.mk +29 -16
index 921efa1b02..9c2535d102 100644
@@ -67,18 +67,20 @@ ROOTFS_ISO9660_DEPENDENCIES += grub2
 ROOTFS_ISO9660_BOOTLOADER_CONFIG_PATH = \
 	$(ROOTFS_ISO9660_TMP_TARGET_DIR)/boot/grub/grub.cfg
 ROOTFS_ISO9660_BOOT_IMAGE = boot/grub/grub-eltorito.img
-define ROOTFS_ISO9660_INSTALL_BOOTLOADER
+define ROOTFS_ISO9660_INSTALL_BOOTLOADER_BIOS
 	$(INSTALL) -D -m 0644 $(BINARIES_DIR)/grub-eltorito.img \
 		$(ROOTFS_ISO9660_TMP_TARGET_DIR)/boot/grub/grub-eltorito.img
 endef
-else ifeq ($(BR2_TARGET_ROOTFS_ISO9660_GRUB2)$(BR2_TARGET_ROOTFS_ISO9660_EFI_BOOTLOADER),yy)
+endif
+
+ifeq ($(BR2_TARGET_ROOTFS_ISO9660_GRUB2)$(BR2_TARGET_ROOTFS_ISO9660_EFI_BOOTLOADER),yy)
 ROOTFS_ISO9660_DEPENDENCIES += grub2 host-dosfstools host-mtools
 ROOTFS_ISO9660_EFI_PARTITION = boot/fat.efi
 ROOTFS_ISO9660_EFI_PARTITION_PATH = $(ROOTFS_ISO9660_TMP_TARGET_DIR)/$(ROOTFS_ISO9660_EFI_PARTITION)
 ROOTFS_ISO9660_EFI_PARTITION_CONTENT = $(BINARIES_DIR)/efi-part
 ROOTFS_ISO9660_BOOTLOADER_CONFIG_PATH = \
 	$(ROOTFS_ISO9660_TMP_TARGET_DIR)/boot/grub/grub.cfg
-define ROOTFS_ISO9660_INSTALL_BOOTLOADER
+define ROOTFS_ISO9660_INSTALL_BOOTLOADER_EFI
 	rm -rf $(ROOTFS_ISO9660_EFI_PARTITION_PATH)
 	mkdir -p $(dir $(ROOTFS_ISO9660_EFI_PARTITION_PATH))
 	dd if=/dev/zero of=$(ROOTFS_ISO9660_EFI_PARTITION_PATH) bs=1M count=1
@@ -87,12 +89,14 @@ define ROOTFS_ISO9660_INSTALL_BOOTLOADER
 	$(HOST_DIR)/bin/mcopy -p -m -i $(ROOTFS_ISO9660_EFI_PARTITION_PATH) -s \
 		$(ROOTFS_ISO9660_EFI_PARTITION_CONTENT)/* ::/
 endef
-else ifeq ($(BR2_TARGET_ROOTFS_ISO9660_ISOLINUX),y)
+endif
+
+ifeq ($(BR2_TARGET_ROOTFS_ISO9660_ISOLINUX),y)
 ROOTFS_ISO9660_DEPENDENCIES += syslinux
 ROOTFS_ISO9660_BOOTLOADER_CONFIG_PATH = \
 	$(ROOTFS_ISO9660_TMP_TARGET_DIR)/isolinux/isolinux.cfg
 ROOTFS_ISO9660_BOOT_IMAGE = isolinux/isolinux.bin
-define ROOTFS_ISO9660_INSTALL_BOOTLOADER
+define ROOTFS_ISO9660_INSTALL_BOOTLOADER_BIOS
 	$(INSTALL) -D -m 0644 $(BINARIES_DIR)/syslinux/* \
 		$(ROOTFS_ISO9660_TMP_TARGET_DIR)/isolinux/
 	$(INSTALL) -D -m 0644 $(HOST_DIR)/share/syslinux/ldlinux.c32 \
@@ -105,7 +109,8 @@ define ROOTFS_ISO9660_PREPARATION
 		$(ROOTFS_ISO9660_BOOTLOADER_CONFIG_PATH)
 	$(SED) "s%__KERNEL_PATH__%/boot/$(LINUX_IMAGE_NAME)%" \
 		$(ROOTFS_ISO9660_BOOTLOADER_CONFIG_PATH)
-	$(ROOTFS_ISO9660_INSTALL_BOOTLOADER)
+	$(ROOTFS_ISO9660_INSTALL_BOOTLOADER_BIOS)
+	$(ROOTFS_ISO9660_INSTALL_BOOTLOADER_EFI)
 endef
 
 ROOTFS_ISO9660_PRE_GEN_HOOKS += ROOTFS_ISO9660_PREPARATION
@@ -151,24 +156,32 @@ endif # ROOTFS_ISO9660_USE_INITRD
 
 ROOTFS_ISO9660_OPTS += \
 	-J \
-	-R \
-	-no-emul-boot
+	-R
 
-ifeq ($(BR2_TARGET_ROOTFS_ISO9660_BIOS_BOOTLOADER),y)
-ROOTFS_ISO9660_OPTS += \
+ROOTFS_ISO9660_BOOTLOADER_OPTS_BIOS = \
+	-b $(ROOTFS_ISO9660_BOOT_IMAGE) \
+	-no-emul-boot \
 	-boot-load-size 4 \
-	-boot-info-table \
-	-b $(ROOTFS_ISO9660_BOOT_IMAGE)
-endif
+	-boot-info-table
 
-ifeq ($(BR2_TARGET_ROOTFS_ISO9660_EFI_BOOTLOADER),y)
-ROOTFS_ISO9660_OPTS += \
-	--efi-boot $(ROOTFS_ISO9660_EFI_PARTITION)
+ROOTFS_ISO9660_BOOTLOADER_OPTS_EFI = \
+	--efi-boot $(ROOTFS_ISO9660_EFI_PARTITION) \
+	-no-emul-boot
+
+ifeq ($(BR2_TARGET_ROOTFS_ISO9660_BIOS_BOOTLOADER)$(BR2_TARGET_ROOTFS_ISO9660_EFI_BOOTLOADER),yy)
+ROOTFS_ISO9660_BOOTLOADER_OPTS = $(ROOTFS_ISO9660_BOOTLOADER_OPTS_BIOS)
+ROOTFS_ISO9660_BOOTLOADER_OPTS += -eltorito-alt-boot
+ROOTFS_ISO9660_BOOTLOADER_OPTS += $(ROOTFS_ISO9660_BOOTLOADER_OPTS_EFI)
+else ifeq ($(BR2_TARGET_ROOTFS_ISO9660_BIOS_BOOTLOADER),y)
+ROOTFS_ISO9660_BOOTLOADER_OPTS = $(ROOTFS_ISO9660_BOOTLOADER_OPTS_BIOS)
+else ifeq ($(BR2_TARGET_ROOTFS_ISO9660_EFI_BOOTLOADER),y)
+ROOTFS_ISO9660_BOOTLOADER_OPTS = $(ROOTFS_ISO9660_BOOTLOADER_OPTS_EFI)
 endif
 
 define ROOTFS_ISO9660_CMD
 	$(HOST_DIR)/bin/xorriso -as mkisofs \
 		$(ROOTFS_ISO9660_OPTS) \
+		$(ROOTFS_ISO9660_BOOTLOADER_OPTS) \
 		-o $@ $(ROOTFS_ISO9660_TMP_TARGET_DIR)
 endef
 
Signature · 6 lines
2.25.1

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
Signature · 10 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#31 lore raw

Re: [Buildroot] [PATCH v3 4/7] fs/iso9660: add support for hybrid image using Grub bootloader on BIOS and EFI

Hello Yann,

Thanks for the merge.

Quoted 18 lines · On Wed, 29 Sep 2021 23:26:14 +0200 "Yann E. MORIN" <[email protected]>…

Köry, All,

On 2021-09-23 17:57 +0200, Kory Maincent spake thusly:

Quoted 6 lines

Add support for building an hybrid ISO9660 image compatible with legacy and UEFI BIOS.
The option -eltorito-alt-boot need to be used in the xorriso command to generate the hybrid image.

Applied to master, thanks, with the folowing changes;

- note about hybrid vs. (iso)hybrid
- explain -eltorito-alt-boot
- explain duplication of -no-emul-boot
- rename the variables
- note about the BIOS-EFI ordering

Quoted 6 lines

Since I had done a few changes in the previous patches, of course I had to resolve the conflicts when applying this patch... I've tried to be as carefull as I could, but I may have (again!) let some issues slip theough...

Could you please review/test that it is till works as expected?

I do not see any issue in your change, the bench of tests (last patch) ends well.

Quoted 3 lines

I don't think the remaining patches need rebasing, though, so I hope I can handle more than one next evening... :-/

They indeed don't need rebasing.

Regards,
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

#32 lore raw

PATCH v3 5/7 support/testing/infra/emulator.py: update encoding when calling qemu

In reply to Kory Maincent

With UTF-8 got issue with wrong character returned by Qemu when using EFI BIOS. This breaks the test process with the following error. Update to ISO-8859-1 encoding to avoid it.

emulator.login()
File "/home/kmaincent/Documents/projects/ariane-groupe/buildroot/support/testing/infra/emulator.py", line 89, in login
index = self.qemu.expect(["buildroot login:", pexpect.TIMEOUT],
File "/usr/lib/python3/dist-packages/pexpect/spawnbase.py", line 340, in expect
return self.expect_list(compiled_pattern_list,
File "/usr/lib/python3/dist-packages/pexpect/spawnbase.py", line 369, in expect_list
return exp.expect_loop(timeout)
File "/usr/lib/python3/dist-packages/pexpect/expect.py", line 111, in expect_loop
incoming = spawn.read_nonblocking(spawn.maxread, timeout)
File "/usr/lib/python3/dist-packages/pexpect/pty_spawn.py", line 485, in read_nonblocking
return super(spawn, self).read_nonblocking(size)
File "/usr/lib/python3/dist-packages/pexpect/spawnbase.py", line 178, in read_nonblocking
s = self._decoder.decode(s, final=False)
File "/usr/lib/python3.8/codecs.py", line 322, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xda in position 0: invalid continuation byte

---
 support/testing/infra/emulator.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
support/testing/infra/emulator.py +1 -1
index 0a77eb80fc..1fab9caad8 100644
@@ -76,7 +76,7 @@ class Emulator(object):
         self.logfile.write("> starting qemu with '%s'\n" % " ".join(qemu_cmd))
         self.qemu = pexpect.spawn(qemu_cmd[0], qemu_cmd[1:],
                                   timeout=5 * self.timeout_multiplier,
-                                  encoding='utf-8',
+                                  encoding='ISO-8859-1',
                                   env={"QEMU_AUDIO_DRV": "none"})
         # We want only stdout into the log to avoid double echo
         self.qemu.logfile_read = self.logfile
Signature · 6 lines
2.25.1

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#33 lore raw

Re: [Buildroot] [PATCH v3 5/7] support/testing/infra/emulator.py: update encoding when calling qemu

Köry, All,

+Arnout, +Petr for their python3 expertise

On 2021-09-23 17:57 +0200, Kory Maincent spake thusly:

Quoted 3 lines

With UTF-8 got issue with wrong character returned by Qemu when using EFI BIOS. This breaks the test process with the following error. Update to ISO-8859-1 encoding to avoid it.

I am always skeptical about switching away from UTF-8. Rather, I'd like to undersand why that is the case.

Is it specific to just the edk2 tests you added later in the series? What if you set your locale to 'C' (LC_ALL=C LANG=C) when runing the test-suite? Or to 'C.UTF-8'?

Quoted 8 lines

emulator.login()
File "/home/kmaincent/Documents/projects/ariane-groupe/buildroot/support/testing/infra/emulator.py", line 89, in login
index = self.qemu.expect(["buildroot login:", pexpect.TIMEOUT],
File "/usr/lib/python3/dist-packages/pexpect/spawnbase.py", line 340, in expect
return self.expect_list(compiled_pattern_list,
File "/usr/lib/python3/dist-packages/pexpect/spawnbase.py", line 369, in expect_list
return exp.expect_loop(timeout)
File "/usr/lib/python3/dist-packages/pexpect/expect.py", line 111, in expect_loop

Quoted 8 lines

incoming = spawn.read_nonblocking(spawn.maxread, timeout)
File "/usr/lib/python3/dist-packages/pexpect/pty_spawn.py", line 485, in read_nonblocking
return super(spawn, self).read_nonblocking(size)
File "/usr/lib/python3/dist-packages/pexpect/spawnbase.py", line 178, in read_nonblocking
s = self._decoder.decode(s, final=False)
File "/usr/lib/python3.8/codecs.py", line 322, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xda in position 0: invalid continuation byte

0xda in iso8859-15 is Ú (LATIN CAPITAL LETTER U WITH ACUTE). This is a weird character to see... And his is the first byte... Is edk2 just spitting actual binary?

So I'd really like to understand why that is...

Regards,
Yann E. MORIN.

Quoted 25 lines
---
 support/testing/infra/emulator.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
support/testing/infra/emulator.py +1 -1
index 0a77eb80fc..1fab9caad8 100644
@@ -76,7 +76,7 @@ class Emulator(object):
         self.logfile.write("> starting qemu with '%s'\n" % " ".join(qemu_cmd))
         self.qemu = pexpect.spawn(qemu_cmd[0], qemu_cmd[1:],
                                   timeout=5 * self.timeout_multiplier,
-                                  encoding='utf-8',
+                                  encoding='ISO-8859-1',
                                   env={"QEMU_AUDIO_DRV": "none"})
         # We want only stdout into the log to avoid double echo
         self.qemu.logfile_read = self.logfile
Signature · 6 lines
2.25.1

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
Signature · 10 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#34 lore raw

Re: [Buildroot] [PATCH v3 5/7] support/testing/infra/emulator.py: update encoding when calling qemu

Köry, All,

On 2021-09-30 22:28 +0200, Yann E. MORIN spake thusly:

Quoted 6 lines

On 2021-09-23 17:57 +0200, Kory Maincent spake thusly:

Quoted 3 lines

With UTF-8 got issue with wrong character returned by Qemu when using EFI BIOS. This breaks the test process with the following error. Update to ISO-8859-1 encoding to avoid it.

I am always skeptical about switching away from UTF-8. Rather, I'd like to undersand why that is the case.

[--SNIP--]

Quoted 3 lines

0xda in iso8859-15 is Ú (LATIN CAPITAL LETTER U WITH ACUTE). This is a weird character to see... And his is the first byte... Is edk2 just spitting actual binary?

It's not the first byte; and it's not edk2, but grub2:

    00000150  6d 1b 5b 34 30 6d 1b 5b  30 32 3b 33 30 48 47 4e  |m.[40m.[02;30HGN|
    00000160  55 20 47 52 55 42 20 20  76 65 72 73 69 6f 6e 20  |U GRUB  version |
    00000170  32 2e 30 34 0a 0d 0a 0d  1b 5b 30 34 3b 30 32 48  |2.04.....[04;02H|
    00000180  da c4 c4 c4 c4 c4 c4 c4  c4 c4 c4 c4 c4 c4 c4 c4  |................|
    00000190  c4 c4 c4 c4 c4 c4 c4 c4  c4 c4 c4 c4 c4 c4 c4 c4  |................|
    000001a0  c4 c4 c4 c4 c4 c4 c4 c4  c4 c4 c4 c4 c4 c4 c4 c4  |................|
    000001b0  c4 c4 c4 c4 c4 c4 c4 c4  c4 c4 c4 c4 c4 c4 c4 c4  |................|
    000001c0  c4 c4 c4 c4 c4 c4 c4 c4  c4 c4 c4 c4 c4 bf 1b 5b  |...............[|
    000001d0  30 35 3b 30 32 48 b3 1b  5b 30 35 3b 37 39 48 b3  |05;02H..[05;79H.|

So, what is this "da c4...c4 bf" sequence? It looks a bit more obvious when looking at the folowing lines:

    000001d0  30 35 3b 30 32 48 b3 1b  5b 30 35 3b 37 39 48 b3  |05;02H..[05;79H.|
    000001e0  1b 5b 30 36 3b 30 32 48  b3 1b 5b 30 36 3b 37 39  |.[06;02H..[06;79|
    000001f0  48 b3 1b 5b 30 37 3b 30  32 48 b3 1b 5b 30 37 3b  |H..[07;02H..[07;|
    00000200  37 39 48 b3 1b 5b 30 38  3b 30 32 48 b3 1b 5b 30  |79H..[08;02H..[0|
    00000210  38 3b 37 39 48 b3 1b 5b  30 39 3b 30 32 48 b3 1b  |8;79H..[09;02H..|
    00000220  5b 30 39 3b 37 39 48 b3  1b 5b 31 30 3b 30 32 48  |[09;79H..[10;02H|
    00000230  b3 1b 5b 31 30 3b 37 39  48 b3 1b 5b 31 31 3b 30  |..[10;79H..[11;0|
    00000240  32 48 b3 1b 5b 31 31 3b  37 39 48 b3 1b 5b 31 32  |2H..[11;79H..[12|
    00000250  3b 30 32 48 b3 1b 5b 31  32 3b 37 39 48 b3 1b 5b  |;02H..[12;79H..[|
    00000260  31 33 3b 30 32 48 b3 1b  5b 31 33 3b 37 39 48 b3  |13;02H..[13;79H.|
    00000270  1b 5b 31 34 3b 30 32 48  b3 1b 5b 31 34 3b 37 39  |.[14;02H..[14;79|
    00000280  48 b3 1b 5b 31 35 3b 30  32 48 b3 1b 5b 31 35 3b  |H..[15;02H..[15;|
    00000290  37 39 48 b3 1b 5b 31 36  3b 30 32 48 b3 1b 5b 31  |79H..[16;02H..[1|
    000002a0  36 3b 37 39 48 b3 1b 5b  31 37 3b 30 32 48 b3 1b  |6;79H..[17;02H..|
    000002b0  5b 31 37 3b 37 39 48 b3  1b 5b 31 38 3b 30 32 48  |[17;79H..[18;02H|
    000002c0  c0 c4 c4 c4 c4 c4 c4 c4  c4 c4 c4 c4 c4 c4 c4 c4  |................|
    000002d0  c4 c4 c4 c4 c4 c4 c4 c4  c4 c4 c4 c4 c4 c4 c4 c4  |................|
    000002e0  c4 c4 c4 c4 c4 c4 c4 c4  c4 c4 c4 c4 c4 c4 c4 c4  |................|
    000002f0  c4 c4 c4 c4 c4 c4 c4 c4  c4 c4 c4 c4 c4 c4 c4 c4  |................|
    00000300  c4 c4 c4 c4 c4 c4 c4 c4  c4 c4 c4 c4 c4 d9 1b 5b  |...............[|
    00000310  31 39 3b 30 32 48 1b 5b  32 30 3b 30 32 48 20 20  |19;02H.[20;02H  |

OK, so there are a lot of ANSI Escape Sequences, which are CSI commands:

    1b 5b 30 35 3b 37 39 48     - ESC [ 05 ; 79 H

This is CSI for "Cursor Position", i.e. move sursor to row 5, column 79. OK, it is drawing a box! da is top-left corner, c4 is horizontal line, bf is top-right corner, b3 is vertical line, c0 id lower-left corner, and d9 is lower-right corner.

This is definitely not ISO-8859-15; it is CP437 [0]:

    https://en.wikipedia.org/wiki/Code_page_437

So, the solution to switch over to iso-8859-15 is not the proper solution. We don't want to switch to CP437 either!

What I suggest, instead, is that we tell pexpect.spawn to just fix any non-decodable character, by passing the parameter:
codec_errors='replace'

At least, it makes the two EFI run-test succeed!

If that is OK, I can do that when applying, and fix the commit log accordingly.

Regards,
Yann E. MORIN.

[0] Oh boy, does that bring fond memories... ;-)

Signature · 10 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#35 lore raw

Re: [Buildroot] [PATCH v3 5/7] support/testing/infra/emulator.py: update encoding when calling qemu

Quoted 9 lines · On Sat, 2 Oct 2021 22:28:05 +0200 "Yann E. MORIN" <[email protected]>…

OK, so there are a lot of ANSI Escape Sequences, which are CSI commands:

    1b 5b 30 35 3b 37 39 48     - ESC [ 05 ; 79 H

This is CSI for "Cursor Position", i.e. move sursor to row 5, column 79. OK, it is drawing a box! da is top-left corner, c4 is horizontal line, bf is top-right corner, b3 is vertical line, c0 id lower-left corner, and d9 is lower-right corner.

Quoted 5 lines

This is definitely not ISO-8859-15; it is CP437 [0]:

    https://en.wikipedia.org/wiki/Code_page_437

So, the solution to switch over to iso-8859-15 is not the proper solution. We don't want to switch to CP437 either!

Wow, thanks for the investigation!

Quoted 8 lines

What I suggest, instead, is that we tell pexpect.spawn to just fix any non-decodable character, by passing the parameter:
codec_errors='replace'

At least, it makes the two EFI run-test succeed!

If that is OK, I can do that when applying, and fix the commit log accordingly.

This indeed looks like a better option.

Thanks again!

Thomas

Signature · 7 lines
Thomas Petazzoni, co-owner and CEO, Bootlin
Embedded Linux and Kernel engineering and training
https://bootlin.com
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#36 lore raw

Re: [Buildroot] [PATCH v3 5/7] support/testing/infra/emulator.py: update encoding when calling qemu

In reply to Kory Maincent

Köry, All,

On 2021-09-23 17:57 +0200, Kory Maincent spake thusly:

Quoted 31 lines

With UTF-8 got issue with wrong character returned by Qemu when using EFI BIOS. This breaks the test process with the following error. Update to ISO-8859-1 encoding to avoid it.

emulator.login()
File "/home/kmaincent/Documents/projects/ariane-groupe/buildroot/support/testing/infra/emulator.py", line 89, in login
index = self.qemu.expect(["buildroot login:", pexpect.TIMEOUT],
File "/usr/lib/python3/dist-packages/pexpect/spawnbase.py", line 340, in expect
return self.expect_list(compiled_pattern_list,
File "/usr/lib/python3/dist-packages/pexpect/spawnbase.py", line 369, in expect_list
return exp.expect_loop(timeout)
File "/usr/lib/python3/dist-packages/pexpect/expect.py", line 111, in expect_loop
incoming = spawn.read_nonblocking(spawn.maxread, timeout)
File "/usr/lib/python3/dist-packages/pexpect/pty_spawn.py", line 485, in read_nonblocking
return super(spawn, self).read_nonblocking(size)
File "/usr/lib/python3/dist-packages/pexpect/spawnbase.py", line 178, in read_nonblocking
s = self._decoder.decode(s, final=False)
File "/usr/lib/python3.8/codecs.py", line 322, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xda in position 0: invalid continuation byte

---
 support/testing/infra/emulator.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
support/testing/infra/emulator.py +0 -0
index 0a77eb80fc..1fab9caad8 100644

support/testing/infra/emulator.py class Emulator(object): · line 79

Quoted 5 lines
support/testing/infra/emulator.py +1 -1
@@ -76,7 +76,7 @@ class Emulator(object):
         self.logfile.write("> starting qemu with '%s'\n" % " ".join(qemu_cmd))
         self.qemu = pexpect.spawn(qemu_cmd[0], qemu_cmd[1:],
                                   timeout=5 * self.timeout_multiplier,
-                                  encoding='utf-8',
+                                  encoding='ISO-8859-1',

After review from Thomas, I've indeed reverted to using utf-8, but using codecerrors='replace' to fix invalid utf-8 sequences.

I've also reworded the commit log accordingly.

Applied to master, thanks.

Regards,
Yann E. MORIN.

Quoted 10 lines
support/testing/infra/emulator.py +0 -0
                                   env={"QEMU_AUDIO_DRV": "none"})
         # We want only stdout into the log to avoid double echo
         self.qemu.logfile_read = self.logfile
Signature · 6 lines
2.25.1

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
Signature · 10 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#37 lore raw

Re: [Buildroot] [PATCH v3 5/7] support/testing/infra/emulator.py: update encoding when calling qemu

Hello Yann,

Quoted 6 lines · On Sun, 3 Oct 2021 14:47:19 +0200 "Yann E. MORIN" <[email protected]>…

After review from Thomas, I've indeed reverted to using utf-8, but using codecerrors='replace' to fix invalid utf-8 sequences.

I've also reworded the commit log accordingly.

Applied to master, thanks.

Thanks for this investigation on the utf8 character issue and for completing the merging process :)

Regards,
Köry
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

#38 lore raw

Re: [Buildroot] [PATCH v3 5/7] support/testing/infra/emulator.py: update encoding when calling qemu

In reply to Yann E. MORIN
Quoted 44 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Yann" == Yann E MORIN <[email protected]> writes:

Köry, All,
On 2021-09-23 17:57 +0200, Kory Maincent spake thusly:

Quoted 31 lines

With UTF-8 got issue with wrong character returned by Qemu when using EFI BIOS. This breaks the test process with the following error. Update to ISO-8859-1 encoding to avoid it.

emulator.login()
File "/home/kmaincent/Documents/projects/ariane-groupe/buildroot/support/testing/infra/emulator.py", line 89, in login index = self.qemu.expect(["buildroot login:", pexpect.TIMEOUT],
File "/usr/lib/python3/dist-packages/pexpect/spawnbase.py", line 340, in expect
return self.expect_list(compiled_pattern_list,
File "/usr/lib/python3/dist-packages/pexpect/spawnbase.py", line 369, in expect_list
return exp.expect_loop(timeout)
File "/usr/lib/python3/dist-packages/pexpect/expect.py", line 111, in expect_loop
incoming = spawn.read_nonblocking(spawn.maxread, timeout)
File "/usr/lib/python3/dist-packages/pexpect/pty_spawn.py", line 485, in read_nonblocking
return super(spawn, self).read_nonblocking(size)
File "/usr/lib/python3/dist-packages/pexpect/spawnbase.py", line 178, in read_nonblocking
s = self._decoder.decode(s, final=False)
File "/usr/lib/python3.8/codecs.py", line 322, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xda in position 0: invalid continuation byte

---
support/testing/infra/emulator.py | 2 +-

1 file changed, 1 insertion(+), 1 deletion(-)
support/testing/infra/emulator.py +0 -0
index 0a77eb80fc..1fab9caad8 100644

support/testing/infra/emulator.py class Emulator(object): · line 79

Quoted 5 lines
support/testing/infra/emulator.py +1 -1
@@ -76,7 +76,7 @@ class Emulator(object):
 self.logfile.write("> starting qemu with '%s'\n" % " ".join(qemu_cmd))
 self.qemu = pexpect.spawn(qemu_cmd[0], qemu_cmd[1:],
 timeout=5 * self.timeout_multiplier,
-                                  encoding='utf-8',
+                                  encoding='ISO-8859-1',

After review from Thomas, I've indeed reverted to using utf-8, but using codecerrors='replace' to fix invalid utf-8 sequences.

Quoted 3 lines

I've also reworded the commit log accordingly.

Applied to master, thanks.

Committed to 2021.02.x, 2021.05.x and 2021.08.x, thanks.

Signature · 5 lines
Bye, Peter Korsgaard
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#39 lore raw

PATCH v3 6/7 boot/edk2: add support to i386 architecture

In reply to Kory Maincent

Add support the build the firmware for QEMU i386 pc machine

---
 boot/edk2/Config.in | 12 +++++++++++-
 boot/edk2/edk2.mk   | 12 ++++++++++--
 2 files changed, 21 insertions(+), 3 deletions(-)
boot/edk2/Config.in +11 -1
index 150806899f..9f8988bcca 100644
@@ -1,6 +1,6 @@
 config BR2_TARGET_EDK2
 	bool "EDK2"
-	depends on BR2_x86_64 || BR2_aarch64
+	depends on BR2_x86_64 || BR2_aarch64 || BR2_i386
 	depends on BR2_TOOLCHAIN_GCC_AT_LEAST_5
 	select BR2_PACKAGE_EDK2_PLATFORMS
 	help
@@ -13,9 +13,18 @@ if BR2_TARGET_EDK2
 
 choice
 	prompt "Platform"
+	default BR2_TARGET_EDK2_PLATFORM_OVMF_I386 if BR2_i386
 	default BR2_TARGET_EDK2_PLATFORM_OVMF_X64 if BR2_x86_64
 	default BR2_TARGET_EDK2_PLATFORM_ARM_VIRT_QEMU if BR2_aarch64
 
+config BR2_TARGET_EDK2_PLATFORM_OVMF_I386
+	bool "i386"
+	depends on BR2_i386 || BR2_x86_64
+	help
+	  Platform configuration for a generic i386 target.
+	  This platform will boot from flash address 0x0.
+	  It should therefore be used as the first bootloader.
+
 config BR2_TARGET_EDK2_PLATFORM_OVMF_X64
 	bool "x86-64"
 	depends on BR2_x86_64
@@ -94,6 +103,7 @@ endchoice
 
 config BR2_TARGET_EDK2_FD_NAME
 	string
+	default "OVMF" if BR2_TARGET_EDK2_PLATFORM_OVMF_I386
 	default "OVMF" if BR2_TARGET_EDK2_PLATFORM_OVMF_X64
 	default "QEMU_EFI" if BR2_TARGET_EDK2_PLATFORM_ARM_VIRT_QEMU
 	default "QEMU_EFI" if BR2_TARGET_EDK2_PLATFORM_ARM_VIRT_QEMU_KERNEL
boot/edk2/edk2.mk +10 -2
index fabd0c5b45..ab3cdad464 100644
@@ -14,7 +14,9 @@ EDK2_DEPENDENCIES = edk2-platforms host-python3 host-acpica host-util-linux
 EDK2_INSTALL_TARGET = NO
 EDK2_INSTALL_IMAGES = YES
 
-ifeq ($(BR2_x86_64),y)
+ifeq ($(BR2_i386),y)
+EDK2_ARCH = IA32
+else ifeq ($(BR2_x86_64),y)
 EDK2_ARCH = X64
 else ifeq ($(BR2_aarch64),y)
 EDK2_ARCH = AARCH64
@@ -55,7 +57,13 @@ EDK2_GIT_SUBMODULES = YES
 EDK2_BUILD_PACKAGES = $(@D)/Build/Buildroot
 EDK2_PACKAGES_PATH = $(@D):$(EDK2_BUILD_PACKAGES):$(STAGING_DIR)/usr/share/edk2-platforms
 
-ifeq ($(BR2_TARGET_EDK2_PLATFORM_OVMF_X64),y)
+ifeq ($(BR2_TARGET_EDK2_PLATFORM_OVMF_I386),y)
+EDK2_DEPENDENCIES += host-nasm
+EDK2_PACKAGE_NAME = OvmfPkg
+EDK2_PLATFORM_NAME = OvmfPkgIa32
+EDK2_BUILD_DIR = OvmfIa32
+
+else ifeq ($(BR2_TARGET_EDK2_PLATFORM_OVMF_X64),y)
 EDK2_DEPENDENCIES += host-nasm
 EDK2_PACKAGE_NAME = OvmfPkg
 EDK2_PLATFORM_NAME = OvmfPkgX64
Signature · 6 lines
2.25.1

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#40 lore raw

Re: [Buildroot] [PATCH v3 6/7] boot/edk2: add support to i386 architecture

Köry, All,

On 2021-09-23 17:57 +0200, Kory Maincent spake thusly:

Quoted 13 lines

Add support the build the firmware for QEMU i386 pc machine

---
 boot/edk2/Config.in | 12 +++++++++++-
 boot/edk2/edk2.mk   | 12 ++++++++++--
 2 files changed, 21 insertions(+), 3 deletions(-)
boot/edk2/Config.in +0 -0
index 150806899f..9f8988bcca 100644

boot/edk2/Config.in line 3

Quoted 4 lines
boot/edk2/Config.in +1 -1
@@ -1,6 +1,6 @@
 config BR2_TARGET_EDK2
 	bool "EDK2"
-	depends on BR2_x86_64 || BR2_aarch64
+	depends on BR2_x86_64 || BR2_aarch64 || BR2_i386

With three archicteures, it is now nicer to introduce a symbol dedicated to list the arch support:

config BR2_TARGET_EDK2_ARCH_SUPPORTS
bool
default y if BR2_aarch64
default y if BR2_i386
default y if BR2_x86_64

config BR2_TARGET_EDK2
bool "edk2"
depends on BR2_TARGET_EDK2_ARCH_SUPPORTS
[...]

Quoted 4 lines
boot/edk2/Config.in +0 -0
 	depends on BR2_TOOLCHAIN_GCC_AT_LEAST_5
 	select BR2_PACKAGE_EDK2_PLATFORMS
 	help

boot/edk2/Config.in if BR2_TARGET_EDK2 · line 22

Quoted 10 lines
boot/edk2/Config.in +4 -0
@@ -13,9 +13,18 @@ if BR2_TARGET_EDK2
 
 choice
 	prompt "Platform"
+	default BR2_TARGET_EDK2_PLATFORM_OVMF_I386 if BR2_i386
 	default BR2_TARGET_EDK2_PLATFORM_OVMF_X64 if BR2_x86_64
 	default BR2_TARGET_EDK2_PLATFORM_ARM_VIRT_QEMU if BR2_aarch64
 
+config BR2_TARGET_EDK2_PLATFORM_OVMF_I386
+	bool "i386"
+	depends on BR2_i386 || BR2_x86_64

So, I was wondering on the above could work [...]

Quoted 21 lines
boot/edk2/Config.in +6 -0
+	help
+	  Platform configuration for a generic i386 target.
+	  This platform will boot from flash address 0x0.
+	  It should therefore be used as the first bootloader.
+
 config BR2_TARGET_EDK2_PLATFORM_OVMF_X64
 	bool "x86-64"
 	depends on BR2_x86_64
@@ -94,6 +103,7 @@ endchoice
 
 config BR2_TARGET_EDK2_FD_NAME
 	string
+	default "OVMF" if BR2_TARGET_EDK2_PLATFORM_OVMF_I386
 	default "OVMF" if BR2_TARGET_EDK2_PLATFORM_OVMF_X64
 	default "QEMU_EFI" if BR2_TARGET_EDK2_PLATFORM_ARM_VIRT_QEMU
 	default "QEMU_EFI" if BR2_TARGET_EDK2_PLATFORM_ARM_VIRT_QEMU_KERNEL
boot/edk2/edk2.mk +0 -0
index fabd0c5b45..ab3cdad464 100644

boot/edk2/edk2.mk EDK2_DEPENDENCIES = edk2-platforms host-python3 host-acpica host-util-linux · line 19

Quoted 7 lines
boot/edk2/edk2.mk +3 -1
@@ -14,7 +14,9 @@ EDK2_DEPENDENCIES = edk2-platforms host-python3 host-acpica host-util-linux
 EDK2_INSTALL_TARGET = NO
 EDK2_INSTALL_IMAGES = YES
 
-ifeq ($(BR2_x86_64),y)
+ifeq ($(BR2_i386),y)
+EDK2_ARCH = IA32
+else ifeq ($(BR2_x86_64),y)

[...]
with these settings when the target is actually an x86_64. So I came up with this minimal defconfig:

    BR2_x86_64=y
    BR2_x86_corei7=y
    BR2_TOOLCHAIN_EXTERNAL=y
    BR2_TARGET_EDK2=y
    BR2_TARGET_EDK2_PLATFORM_OVMF_I386=y

And indeed, it does not work:

    $ make edk2
    [...]
Quoted 1 line
Quoted 1 line
Quoted 1 line

edk2 edk2-stable202102 Building

    [...]
    Ran 285 tests in 1.458s

    OK
    Build environment: Linux-5.4.0-84-generic-x86_64-with-glibc2.31
    Build start time: 21:35:13, Sep.30 2021

    WORKSPACE        = /home/ymorin/dev/buildroot/O/build/edk2-edk2-stable202102
    PACKAGES_PATH    = /home/ymorin/dev/buildroot/O/build/edk2-edk2-stable202102:/home/ymorin/dev/buildroot/O>
    EDK_TOOLS_PATH   = /home/ymorin/dev/buildroot/O/build/edk2-edk2-stable202102/BaseTools
    CONF_PATH        = /home/ymorin/dev/buildroot/O/build/edk2-edk2-stable202102/Conf
    PYTHON_COMMAND   = /home/ymorin/dev/buildroot/O/host/bin/python3

    build.py...
     : error 2000: Invalid parameter
        Invalid ARCH specified. [Valid ARCH: IA32]

    - Failed -
    Build end time: 21:35:13, Sep.30 2021
    Build total time: 00:00:00

    make[1]: *** [package/pkg-generic.mk:294: /home/ymorin/dev/buildroot/O/build/edk2-edk2-stable202102/.stamp_built] Error 1
    make: *** [Makefile:23: _all] Error 2

Setting EDK2_ARCH to IA32 fixes the build, but I have no way to know if the build is "correct"...

So, in this case, I think the EDK2_ARCH should depend on one of the BR2_TARGET_EDK2_PLATFORM_OVMF_xxx, not the target architecture.

Alternatively, we could maybe restrict BR2_TARGET_EDK2_PLATFORM_OVMF_I386 to just i386?

Regards,
Yann E. MORIN.

Quoted 25 lines

EDK2_ARCH = X64
else ifeq ($(BR2_aarch64),y)
EDK2_ARCH = AARCH64

boot/edk2/edk2.mk +7 -1
@@ -55,7 +57,13 @@ EDK2_GIT_SUBMODULES = YES
 EDK2_BUILD_PACKAGES = $(@D)/Build/Buildroot
 EDK2_PACKAGES_PATH = $(@D):$(EDK2_BUILD_PACKAGES):$(STAGING_DIR)/usr/share/edk2-platforms
 
-ifeq ($(BR2_TARGET_EDK2_PLATFORM_OVMF_X64),y)
+ifeq ($(BR2_TARGET_EDK2_PLATFORM_OVMF_I386),y)
+EDK2_DEPENDENCIES += host-nasm
+EDK2_PACKAGE_NAME = OvmfPkg
+EDK2_PLATFORM_NAME = OvmfPkgIa32
+EDK2_BUILD_DIR = OvmfIa32
+
+else ifeq ($(BR2_TARGET_EDK2_PLATFORM_OVMF_X64),y)
 EDK2_DEPENDENCIES += host-nasm
 EDK2_PACKAGE_NAME = OvmfPkg
 EDK2_PLATFORM_NAME = OvmfPkgX64
Signature · 6 lines
2.25.1

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
Signature · 10 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#41 lore raw

Re: [Buildroot] [PATCH v3 6/7] boot/edk2: add support to i386 architecture

In reply to Kory Maincent

Köry, All,

On 2021-09-23 17:57 +0200, Kory Maincent spake thusly:

Quoted 21 lines

Add support the build the firmware for QEMU i386 pc machine

---
 boot/edk2/Config.in | 12 +++++++++++-
 boot/edk2/edk2.mk   | 12 ++++++++++--
 2 files changed, 21 insertions(+), 3 deletions(-)
boot/edk2/Config.in +1 -1
index 150806899f..9f8988bcca 100644
@@ -1,6 +1,6 @@
 config BR2_TARGET_EDK2
 	bool "EDK2"
-	depends on BR2_x86_64 || BR2_aarch64
+	depends on BR2_x86_64 || BR2_aarch64 || BR2_i386
 	depends on BR2_TOOLCHAIN_GCC_AT_LEAST_5
 	select BR2_PACKAGE_EDK2_PLATFORMS
 	help

boot/edk2/Config.in if BR2_TARGET_EDK2 · line 22

Quoted 10 lines
boot/edk2/Config.in +4 -0
@@ -13,9 +13,18 @@ if BR2_TARGET_EDK2
 
 choice
 	prompt "Platform"
+	default BR2_TARGET_EDK2_PLATFORM_OVMF_I386 if BR2_i386
 	default BR2_TARGET_EDK2_PLATFORM_OVMF_X64 if BR2_x86_64
 	default BR2_TARGET_EDK2_PLATFORM_ARM_VIRT_QEMU if BR2_aarch64
 
+config BR2_TARGET_EDK2_PLATFORM_OVMF_I386
+	bool "i386"
+	depends on BR2_i386 || BR2_x86_64

As there was a build failure with BR2_x86_64=y, I've just dropped that
possibility: BR2_TARGET_EDK2_PLATFORM_OVMF_I386 is only available for BR2_i386.

If you have the opportunity/need to look into that, a further patch is most welcome!

Applied to master, thanks.

Regards,
Yann E. MORIN.

Quoted 53 lines
boot/edk2/Config.in +6 -0
+	help
+	  Platform configuration for a generic i386 target.
+	  This platform will boot from flash address 0x0.
+	  It should therefore be used as the first bootloader.
+
 config BR2_TARGET_EDK2_PLATFORM_OVMF_X64
 	bool "x86-64"
 	depends on BR2_x86_64
@@ -94,6 +103,7 @@ endchoice
 
 config BR2_TARGET_EDK2_FD_NAME
 	string
+	default "OVMF" if BR2_TARGET_EDK2_PLATFORM_OVMF_I386
 	default "OVMF" if BR2_TARGET_EDK2_PLATFORM_OVMF_X64
 	default "QEMU_EFI" if BR2_TARGET_EDK2_PLATFORM_ARM_VIRT_QEMU
 	default "QEMU_EFI" if BR2_TARGET_EDK2_PLATFORM_ARM_VIRT_QEMU_KERNEL
boot/edk2/edk2.mk +10 -2
index fabd0c5b45..ab3cdad464 100644
@@ -14,7 +14,9 @@ EDK2_DEPENDENCIES = edk2-platforms host-python3 host-acpica host-util-linux
 EDK2_INSTALL_TARGET = NO
 EDK2_INSTALL_IMAGES = YES
 
-ifeq ($(BR2_x86_64),y)
+ifeq ($(BR2_i386),y)
+EDK2_ARCH = IA32
+else ifeq ($(BR2_x86_64),y)
 EDK2_ARCH = X64
 else ifeq ($(BR2_aarch64),y)
 EDK2_ARCH = AARCH64
@@ -55,7 +57,13 @@ EDK2_GIT_SUBMODULES = YES
 EDK2_BUILD_PACKAGES = $(@D)/Build/Buildroot
 EDK2_PACKAGES_PATH = $(@D):$(EDK2_BUILD_PACKAGES):$(STAGING_DIR)/usr/share/edk2-platforms
 
-ifeq ($(BR2_TARGET_EDK2_PLATFORM_OVMF_X64),y)
+ifeq ($(BR2_TARGET_EDK2_PLATFORM_OVMF_I386),y)
+EDK2_DEPENDENCIES += host-nasm
+EDK2_PACKAGE_NAME = OvmfPkg
+EDK2_PLATFORM_NAME = OvmfPkgIa32
+EDK2_BUILD_DIR = OvmfIa32
+
+else ifeq ($(BR2_TARGET_EDK2_PLATFORM_OVMF_X64),y)
 EDK2_DEPENDENCIES += host-nasm
 EDK2_PACKAGE_NAME = OvmfPkg
 EDK2_PLATFORM_NAME = OvmfPkgX64
Signature · 6 lines
2.25.1

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
Signature · 10 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#42 lore raw

Re: [Buildroot] [PATCH v3 6/7] boot/edk2: add support to i386 architecture

Yann,

Quoted 10 lines · On Sun, 3 Oct 2021 14:49:00 +0200 "Yann E. MORIN" <[email protected]>…

boot/edk2/Config.in if BR2_TARGET_EDK2 · line 22

Quoted 3 lines
boot/edk2/Config.in +3 -0
+config BR2_TARGET_EDK2_PLATFORM_OVMF_I386
+	bool "i386"
+	depends on BR2_i386 || BR2_x86_64  

As there was a build failure with BR2_x86_64=y, I've just dropped that
possibility: BR2_TARGET_EDK2_PLATFORM_OVMF_I386 is only available for BR2_i386.

If you have the opportunity/need to look into that, a further patch is most welcome!

Found! The issue is that EDK2_ARCH is configured for X64 even when we select the BR2_TARGET_EDK2_PLATFORM_OVMF_I386 configuration.

Quoted 2 lines
Quoted 2 lines

else ifeq ($(BR2_x86_64),y)
EDK2_ARCH = X64

I will send a patch to fix this.

Regards,

Köry
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

#43 lore raw

PATCH v3 7/7 support/testing/tests/fs/test_iso9660.py: add support to test using EFI BIOS

In reply to Kory Maincent

The ISO9660 tests are only testing BIOS Legacy.
Add support to test an ISO9660 image based on EFI BIOS.
Add support to test an ISO9660 hybrid image based on Legacy and EFI BIOS. Add dedicated Grub2 builtin config for the EFI compatible cases.

---
 support/testing/conf/grub2-efi.cfg       |  2 +
 support/testing/tests/fs/test_iso9660.py | 74 +++++++++++++++++++++++-
 2 files changed, 74 insertions(+), 2 deletions(-)
 create mode 100644 support/testing/conf/grub2-efi.cfg
support/testing/conf/grub2-efi.cfg new file +2 -0
new file mode 100644
index 0000000000..11c32e880e
@@ -0,0 +1,2 @@
+set root=(cd0)
+set prefix=/boot/grub
support/testing/tests/fs/test_iso9660.py +72 -2

105 lines not on this page · raw message

Signature · 6 lines
2.25.1

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#44 lore raw

Re: [Buildroot] [PATCH v3 7/7] support/testing/tests/fs/test_iso9660.py: add support to test using EFI BIOS

Köry, All,

On 2021-09-23 17:57 +0200, Kory Maincent spake thusly:

Quoted 41 lines

The ISO9660 tests are only testing BIOS Legacy.
Add support to test an ISO9660 image based on EFI BIOS.
Add support to test an ISO9660 hybrid image based on Legacy and EFI BIOS. Add dedicated Grub2 builtin config for the EFI compatible cases.

---
 support/testing/conf/grub2-efi.cfg       |  2 +
 support/testing/tests/fs/test_iso9660.py | 74 +++++++++++++++++++++++-
 2 files changed, 74 insertions(+), 2 deletions(-)
 create mode 100644 support/testing/conf/grub2-efi.cfg
support/testing/conf/grub2-efi.cfg new file +2 -0
new file mode 100644
index 0000000000..11c32e880e
@@ -0,0 +1,2 @@
+set root=(cd0)
+set prefix=/boot/grub
support/testing/tests/fs/test_iso9660.py +6 -2
index 1b699e1fef..8315442604 100644
@@ -25,9 +25,13 @@ BASIC_CONFIG = \
     """.format(infra.filepath("conf/minimal-x86-qemu-kernel.config"))
 
 
-def test_mount_internal_external(emulator, builddir, internal=True):
+def test_mount_internal_external(emulator, builddir, internal=True, efi=False):
     img = os.path.join(builddir, "images", "rootfs.iso9660")
-    emulator.boot(arch="i386", options=["-cdrom", img])
+    if efi:
+        efi_img = os.path.join(builddir, "images", "OVMF.fd")
+        emulator.boot(arch="i386", options=["-cdrom", img, "-bios", efi_img])
+    else:
+        emulator.boot(arch="i386", options=["-cdrom", img])
     emulator.login()
 
     if internal:

support/testing/tests/fs/test_iso9660.py class TestIso9660Grub2External(infra.basetest.BRTest): · line 60

Quoted 4 lines
support/testing/tests/fs/test_iso9660.py +1 -0
@@ -53,6 +57,7 @@ class TestIso9660Grub2External(infra.basetest.BRTest):
         BR2_TARGET_ROOTFS_ISO9660=y
         # BR2_TARGET_ROOTFS_ISO9660_INITRD is not set
         BR2_TARGET_GRUB2=y
+        BR2_TARGET_GRUB2_I386_PC=y

This is no longer needed now that we ensure that it is the default setting. Ditto for the following two, of course.

Applied to master, thanks.

Regards,
Yann E. MORIN.

Quoted 96 lines
support/testing/tests/fs/test_iso9660.py +65 -0
         BR2_TARGET_GRUB2_BOOT_PARTITION="cd"
         BR2_TARGET_GRUB2_BUILTIN_MODULES_PC="boot linux ext2 fat part_msdos part_gpt normal biosdisk iso9660"
         BR2_TARGET_ROOTFS_ISO9660_BOOT_MENU="{}"
@@ -74,6 +79,7 @@ class TestIso9660Grub2ExternalCompress(infra.basetest.BRTest):
         # BR2_TARGET_ROOTFS_ISO9660_INITRD is not set
         BR2_TARGET_ROOTFS_ISO9660_TRANSPARENT_COMPRESSION=y
         BR2_TARGET_GRUB2=y
+        BR2_TARGET_GRUB2_I386_PC=y
         BR2_TARGET_GRUB2_BOOT_PARTITION="cd"
         BR2_TARGET_GRUB2_BUILTIN_MODULES_PC="boot linux ext2 fat part_msdos part_gpt normal biosdisk iso9660"
         BR2_TARGET_ROOTFS_ISO9660_BOOT_MENU="{}"
@@ -94,6 +100,7 @@ class TestIso9660Grub2Internal(infra.basetest.BRTest):
         BR2_TARGET_ROOTFS_ISO9660=y
         BR2_TARGET_ROOTFS_ISO9660_INITRD=y
         BR2_TARGET_GRUB2=y
+        BR2_TARGET_GRUB2_I386_PC=y
         BR2_TARGET_GRUB2_BOOT_PARTITION="cd"
         BR2_TARGET_GRUB2_BUILTIN_MODULES_PC="boot linux ext2 fat part_msdos part_gpt normal biosdisk iso9660"
         BR2_TARGET_ROOTFS_ISO9660_BOOT_MENU="{}"
@@ -107,6 +114,69 @@ class TestIso9660Grub2Internal(infra.basetest.BRTest):
         exit_code = test_touch_file(self.emulator)
         self.assertEqual(exit_code, 0)
 
+
+class TestIso9660Grub2EFI(infra.basetest.BRTest):
+    config = BASIC_CONFIG + \
+        """
+        BR2_TARGET_ROOTFS_ISO9660=y
+        BR2_TARGET_ROOTFS_ISO9660_INITRD=y
+        BR2_TARGET_GRUB2=y
+        BR2_TARGET_GRUB2_I386_EFI=y
+        BR2_TARGET_GRUB2_BUILTIN_MODULES_EFI="boot linux ext2 fat part_msdos part_gpt normal iso9660"
+        BR2_TARGET_GRUB2_BUILTIN_CONFIG_EFI="{}"
+        BR2_TARGET_ROOTFS_ISO9660_BOOT_MENU="{}"
+        BR2_TARGET_EDK2=y
+        """.format(infra.filepath("conf/grub2-efi.cfg"),
+                   infra.filepath("conf/grub2.cfg"))
+
+    def test_run(self):
+        exit_code = test_mount_internal_external(self.emulator,
+                                                 self.builddir, internal=True,
+                                                 efi=True)
+        self.assertEqual(exit_code, 0)
+
+        exit_code = test_touch_file(self.emulator)
+        self.assertEqual(exit_code, 0)
+
+
+class TestIso9660Grub2Hybrid(infra.basetest.BRTest):
+    config = BASIC_CONFIG + \
+        """
+        BR2_TARGET_ROOTFS_ISO9660=y
+        BR2_TARGET_ROOTFS_ISO9660_INITRD=y
+        BR2_TARGET_GRUB2=y
+        BR2_TARGET_GRUB2_I386_PC=y
+        BR2_TARGET_GRUB2_I386_EFI=y
+        BR2_TARGET_GRUB2_BOOT_PARTITION="cd"
+        BR2_TARGET_GRUB2_BUILTIN_MODULES_PC="boot linux ext2 fat squash4 part_msdos part_gpt normal iso9660 biosdisk"
+        BR2_TARGET_GRUB2_BUILTIN_CONFIG_PC=""
+        BR2_TARGET_GRUB2_BUILTIN_MODULES_EFI="boot linux ext2 fat squash4 part_msdos part_gpt normal iso9660 efi_gop"
+        BR2_TARGET_GRUB2_BUILTIN_CONFIG_EFI="{}"
+        BR2_TARGET_ROOTFS_ISO9660_BOOT_MENU="{}"
+        BR2_TARGET_EDK2=y
+        """.format(infra.filepath("conf/grub2-efi.cfg"),
+                   infra.filepath("conf/grub2.cfg"))
+
+    def test_run(self):
+        exit_code = test_mount_internal_external(self.emulator,
+                                                 self.builddir, internal=True,
+                                                 efi=False)
+        self.assertEqual(exit_code, 0)
+
+        exit_code = test_touch_file(self.emulator)
+        self.assertEqual(exit_code, 0)
+
+        self.emulator.stop()
+
+        exit_code = test_mount_internal_external(self.emulator,
+                                                 self.builddir, internal=True,
+                                                 efi=True)
+        self.assertEqual(exit_code, 0)
+
+        exit_code = test_touch_file(self.emulator)
+        self.assertEqual(exit_code, 0)
+
+
 #
 # Syslinux
 
Signature · 6 lines
2.25.1

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
Signature · 10 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot