boot/syslinux: fix top-level parallel builds, add lpxelinux

From: "Yann E. MORIN" <[email protected]>

syslinux is... special. It is a target package, but it is installed in HOST_DIR *in the target install commands*: in addition to the boot files that run on the target, syslinux installs a set of host tools that are to be used at build time (e.g. extlinux, to prepare bootable media, like an iso96660 image). Then, from HOST_DIR, the actual boot files are copied into BINARIES_DIR (i.e. images/); we do it that way because the boot files are scattered about everywhere in the build tree, while they are all packed together in a single directory once installed.

However, there is no dependency between the target and image install steps. So, when using top-level parallel builds, there is no guarantee that the target install commands are finished before the image install commands are started.

We fix that by first installing into a temporary location, as part of the build step, and by then copying from there as part of the install step. This ensures that the boot files are easily available, without needing a dependency on the target install step, that we can't express.

Note that we do not change the actual installation into HOST_DIR: it can be set up differently that our temporary location, and we do not want to duplicate that setup here (it's going to diverge over time).

Signed-off-by Yann E. MORIN <[email protected]>

Quoted 5 lines

From: "Yann E. MORIN" <[email protected]>
We fix that by first installing into a temporary location, as part of the build step, and by then copying from there as part of the install step. This ensures that the boot files are easily available, without needing a dependency on the target install step, that we can't express.

Series applied to master, thanks.
For info, I fixed the commit title of patch #2 (by adding the missing "l" to lpxelinux).

Do you think it would be worth extending the syslinux test to cover this new option?
I'm thinking about simply adding BR2_TARGET_SYSLINUX_LPXELINUX=y here:
https://gitlab.com/buildroot.org/buildroot/-/blob/2025.05/support/testing/tests/boot/test_syslinux.py#L23
If yes, could you send a patch doing that please?

Julien, All,

On 2025-06-29 12:16 +0200, Julien Olivain spake thusly:

Quoted 6 lines · On 25/06/2025 12:01, [email protected] wrote:

Quoted 5 lines

From: "Yann E. MORIN" <[email protected]>
We fix that by first installing into a temporary location, as part of the build step, and by then copying from there as part of the install step. This ensures that the boot files are easily available, without needing a dependency on the target install step, that we can't express.

[--SNIP--]

Quoted 3 lines

Series applied to master, thanks.
For info, I fixed the commit title of patch #2 (by adding the missing "l" to lpxelinux).

Woops, thanks! :-)

Quoted 5 lines

Do you think it would be worth extending the syslinux test to cover this new option?
I'm thinking about simply adding BR2_TARGET_SYSLINUX_LPXELINUX=y here:
https://gitlab.com/buildroot.org/buildroot/-/blob/2025.05/support/testing/tests/boot/test_syslinux.py#L23
If yes, could you send a patch doing that please?

I pondered it, but that would only test whether lpxelinux builds on not; checking that it does work would need a much more complex test. And the build test is (suposedly) covered by the autobuilders already.

If you think that is still important to add lpxelinux to the runtime test although it is not actually runtime-tested, then I'll cook up a patch...

Regards,
Yann E. MORIN.

syslinux is... special. It is a target package, but it is installed in HOST_DIR *in the target install commands*: in addition to the boot files that run on the target, syslinux installs a set of host tools that are to be used at build time (e.g. extlinux, to prepare bootable media, like an iso96660 image). Then, from HOST_DIR, the actual boot files are copied into BINARIES_DIR (i.e. images/); we do it that way because the boot files are scattered about everywhere in the build tree, while they are all packed together in a single directory once installed.

However, there is no dependency between the target and image install steps. So, when using top-level parallel builds, there is no guarantee that the target install commands are finished before the image install commands are started.

We fix that by first installing into a temporary location, as part of the build step, and by then copying from there as part of the install step. This ensures that the boot files are easily available, without needing a dependency on the target install step, that we can't express.

Note that we do not change the actual installation into HOST_DIR: it can be set up differently that our temporary location, and we do not want to duplicate that setup here (it's going to diverge over time).

Signed-off-by Yann E. MORIN <[email protected]>

---
 boot/syslinux/syslinux.mk | 17 ++++++++++++++---
 1 file changed, 14 insertions(+), 3 deletions(-)

boot/syslinux/syslinux.mk +14 -3

index cc52c49aee..084acc398e 100644
@@ -56,6 +56,11 @@ SYSLINUX_POST_PATCH_HOOKS += SYSLINUX_CLEANUP
 # syslinux build system has no convenient way to pass CFLAGS,
 # and the internal zlib should take precedence so -I shouldn't
 # be used.
+# Install in a temporary location that eases final install into
+# images/ (see corresponding command, below).
+# Repeat the target, otherwise syslinux will try to build everything
+# Repeat LD (and CC) as it happens that some binaries are linked at
+# install-time.
 define SYSLINUX_BUILD_CMDS
 	$(TARGET_MAKE_ENV) $(MAKE1) \
 		ASCIIDOC_OK=-1 \

@@ -70,6 +75,10 @@ define SYSLINUX_BUILD_CMDS
 		LDFLAGS_FOR_BUILD="$(HOST_LDFLAGS)" \
 		PYTHON=$(HOST_DIR)/bin/python3 \
 		$(SYSLINUX_EFI_ARGS) -C $(@D) $(SYSLINUX_TARGET)
+	$(TARGET_MAKE_ENV) $(MAKE1) $(SYSLINUX_EFI_ARGS) INSTALLROOT=$(@D)/br-root.temp \
+		CC="$(TARGET_CC)" \
+		LD="$(TARGET_LD)" \
+		-C $(@D) $(SYSLINUX_TARGET) install
 endef
 
 # While the actual bootloader is compiled for the target, several

@@ -77,6 +86,8 @@ endef
 # Repeat the target, otherwise syslinux will try to build everything
 # Repeat LD (and CC) as it happens that some binaries are linked at
 # install-time.
+# Don't use the temporarily-installed br-root.temp: HOST_DIR may be setup
+# differently (merged usr, merged bin...)
 define SYSLINUX_INSTALL_TARGET_CMDS
 	$(TARGET_MAKE_ENV) $(MAKE1) $(SYSLINUX_EFI_ARGS) INSTALLROOT=$(HOST_DIR) \
 		CC="$(TARGET_CC)" \

@@ -99,15 +110,15 @@ SYSLINUX_IMAGES-$(BR2_TARGET_SYSLINUX_EFI) += $(SYSLINUX_EFI_BITS)/efi/syslinux.
 
 SYSLINUX_C32 = $(call qstrip,$(BR2_TARGET_SYSLINUX_C32))
 
-# We install the c32 modules from the host-installed tree, where they
-# are all neatly installed in a single location, while they are
+# We install the c32 modules from the temporarily installed tree, where
+# they are all neatly installed in a single location, while they are
 # scattered around everywhere in the build tree.
 define SYSLINUX_INSTALL_IMAGES_CMDS
 	for i in $(SYSLINUX_IMAGES-y); do \
 		$(INSTALL) -D -m 0755 $(@D)/$$i $(BINARIES_DIR)/syslinux/$${i##*/}; \
 	done
 	for i in $(SYSLINUX_C32); do \
-		$(INSTALL) -D -m 0755 $(HOST_DIR)/share/syslinux/$${i} \
+		$(INSTALL) -D -m 0755 $(@D)/br-root.temp/usr/share/syslinux/$${i} \
 			$(BINARIES_DIR)/syslinux/$${i}; \
 	done
 endef

Signature · 18 lines

2.34.1

____________________________________________________________________________________________________________
Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

syslinux is... special. It is a target package, but it is installed in HOST_DIR *in the target install commands*: in addition to the boot files that run on the target, syslinux installs a set of host tools that are to be used at build time (e.g. extlinux, to prepare bootable media, like an iso96660 image). Then, from HOST_DIR, the actual boot files are copied into BINARIES_DIR (i.e. images/); we do it that way because the boot files are scattered about everywhere in the build tree, while they are all packed together in a single directory once installed.

However, there is no dependency between the target and image install steps. So, when using top-level parallel builds, there is no guarantee that the target install commands are finished before the image install commands are started.

We fix that by first installing into a temporary location, as part of the build step, and by then copying from there as part of the install step. This ensures that the boot files are easily available, without needing a dependency on the target install step, that we can't express.

Note that we do not change the actual installation into HOST_DIR: it can be set up differently that our temporary location, and we do not want to duplicate that setup here (it's going to diverge over time).

Signed-off-by Yann E. MORIN <[email protected]>

---
 boot/syslinux/syslinux.mk | 17 ++++++++++++++---
 1 file changed, 14 insertions(+), 3 deletions(-)

boot/syslinux/syslinux.mk +14 -3

index cc52c49aee..084acc398e 100644
@@ -56,6 +56,11 @@ SYSLINUX_POST_PATCH_HOOKS += SYSLINUX_CLEANUP
 # syslinux build system has no convenient way to pass CFLAGS,
 # and the internal zlib should take precedence so -I shouldn't
 # be used.
+# Install in a temporary location that eases final install into
+# images/ (see corresponding command, below).
+# Repeat the target, otherwise syslinux will try to build everything
+# Repeat LD (and CC) as it happens that some binaries are linked at
+# install-time.
 define SYSLINUX_BUILD_CMDS
 	$(TARGET_MAKE_ENV) $(MAKE1) \
 		ASCIIDOC_OK=-1 \

@@ -70,6 +75,10 @@ define SYSLINUX_BUILD_CMDS
 		LDFLAGS_FOR_BUILD="$(HOST_LDFLAGS)" \
 		PYTHON=$(HOST_DIR)/bin/python3 \
 		$(SYSLINUX_EFI_ARGS) -C $(@D) $(SYSLINUX_TARGET)
+	$(TARGET_MAKE_ENV) $(MAKE1) $(SYSLINUX_EFI_ARGS) INSTALLROOT=$(@D)/br-root.temp \
+		CC="$(TARGET_CC)" \
+		LD="$(TARGET_LD)" \
+		-C $(@D) $(SYSLINUX_TARGET) install
 endef
 
 # While the actual bootloader is compiled for the target, several

@@ -77,6 +86,8 @@ endef
 # Repeat the target, otherwise syslinux will try to build everything
 # Repeat LD (and CC) as it happens that some binaries are linked at
 # install-time.
+# Don't use the temporarily-installed br-root.temp: HOST_DIR may be setup
+# differently (merged usr, merged bin...)
 define SYSLINUX_INSTALL_TARGET_CMDS
 	$(TARGET_MAKE_ENV) $(MAKE1) $(SYSLINUX_EFI_ARGS) INSTALLROOT=$(HOST_DIR) \
 		CC="$(TARGET_CC)" \

@@ -99,15 +110,15 @@ SYSLINUX_IMAGES-$(BR2_TARGET_SYSLINUX_EFI) += $(SYSLINUX_EFI_BITS)/efi/syslinux.
 
 SYSLINUX_C32 = $(call qstrip,$(BR2_TARGET_SYSLINUX_C32))
 
-# We install the c32 modules from the host-installed tree, where they
-# are all neatly installed in a single location, while they are
+# We install the c32 modules from the temporarily installed tree, where
+# they are all neatly installed in a single location, while they are
 # scattered around everywhere in the build tree.
 define SYSLINUX_INSTALL_IMAGES_CMDS
 	for i in $(SYSLINUX_IMAGES-y); do \
 		$(INSTALL) -D -m 0755 $(@D)/$$i $(BINARIES_DIR)/syslinux/$${i##*/}; \
 	done
 	for i in $(SYSLINUX_C32); do \
-		$(INSTALL) -D -m 0755 $(HOST_DIR)/share/syslinux/$${i} \
+		$(INSTALL) -D -m 0755 $(@D)/br-root.temp/usr/share/syslinux/$${i} \
 			$(BINARIES_DIR)/syslinux/$${i}; \
 	done
 endef

Signature · 18 lines

2.34.1

____________________________________________________________________________________________________________
Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot