Skip to contentSkip to conversation

package/linux-firmware: install into images for early loading support

PATCH Superseded Peter Korsgaard 1 participant 6 messages Feb 15, 2021 – Feb 23, 2021

lore
Share & tools
lore
#0 lore raw

v2 cover package/linux-firmware: install into images for early loading support

Some drivers request their firmware very early when built into the kernel, even before the initramfs is mounted - So the only way to provide firmware for those drivers is to include them directly in the kernel with the CONFIG_EXTRA_FIRMWARE option.

To support this, let linux-firmware install its files into BINARIES_DIR in addition to TARGET_DIR, similar to how we do it for intel-microcode.

Peter Korsgaard (3):
package/linux-firmware: make install logic macro accept a destination
parameter
package/linux-firmware: also install into images for early loading
support
linux: build after linux-firmware if enabled for early loading support

Yann E. MORIN (1):
package/linux-firmware: rationalise install step

 linux/linux.mk                           |  3 ++-
 package/linux-firmware/linux-firmware.mk | 34 +++++++++---------------
 2 files changed, 15 insertions(+), 22 deletions(-)
Signature · 1 line
2.20.1
#1 lore raw

PATCH v2 1/4 package/linux-firmware: rationalise install step

From: "Yann E. MORIN" <[email protected]>

The logic we have for the installation of the firmware files is, to say the least, non conventional. It is split in two parts:

- one that copies files via an intermediate tarball: the tarball creation is used to detect if firmware files are missing (i.e. on a version bump) and fail the build if so, while the tarball extraction is the actual firmware installation;

- one that copies directories one by one in a loop, removing the destination before the copy, to maintain a proper layout.

Needless to say, this is not very clean. First, there is no reason why the directories can not be copied with the same mechanism as the files themselves; not sure what I had in mind with b55bd5a9e25e...

Second, we're soon going to need the same installation step to copy the firmware files in the images/ directory, to ease embedding in the kernel image.

Rationalise this installation procedure.

Cherry-picking files and directories with cp, while still maintaining the directory layout, is not trivial; rsync is not one of our pre-requisites. So we're left with tar, which makes it easy. So we keep using an intermediate tarball, but we use it for both files and directories, and we generate it at build time, not install time.

That archive is then extracted during the installation.

Now the installation complexity is mostly located in the creation of the symlinks, so we merge all of that directly into the _INSTALL_TARGET_CMDS and drop the intermediate macros that have no longer any reason to exist.

This will also make it pretty simple to later install in the images/ directory.

---
Changes since v1:
- New patch

package/linux-firmware/linux-firmware.mk | 29 +++++-------------------

 1 file changed, 6 insertions(+), 23 deletions(-)
package/linux-firmware/linux-firmware.mk +6 -23
index f23da171c6..9bc59435ef 100644
@@ -638,22 +638,10 @@ LINUX_FIRMWARE_FILES += ti_3410.fw ti_5052.fw \
 LINUX_FIRMWARE_ALL_LICENSE_FILES += LICENCE.moxa
 endif
 
-ifneq ($(LINUX_FIRMWARE_FILES),)
-define LINUX_FIRMWARE_INSTALL_FILES
+ifneq ($(LINUX_FIRMWARE_FILES)$(LINUX_FIRMWARE_DIRS),)
+define LINUX_FIRMWARE_BUILD_CMDS
 	cd $(@D) && \
-		$(TAR) cf install.tar $(sort $(LINUX_FIRMWARE_FILES)) && \
-		$(TAR) xf install.tar -C $(TARGET_DIR)/lib/firmware
-endef
-endif
-
-ifneq ($(LINUX_FIRMWARE_DIRS),)
-# We need to rm-rf the destination directory to avoid copying
-# into it in itself, should we re-install the package.
-define LINUX_FIRMWARE_INSTALL_DIRS
-	$(foreach d,$(LINUX_FIRMWARE_DIRS), \
-		rm -rf $(TARGET_DIR)/lib/firmware/$(d); \
-		mkdir -p $(dir $(TARGET_DIR)/lib/firmware/$(d)); \
-		cp -a $(@D)/$(d) $(TARGET_DIR)/lib/firmware/$(d)$(sep))
+	$(TAR) cf br-firmware.tar $(sort $(LINUX_FIRMWARE_FILES) $(LINUX_FIRMWARE_DIRS))
 endef
 endif
 
@@ -686,7 +674,9 @@ endif
 # sure we canonicalize the pointed-to file, to cover the symlinks of the form
 # a/foo -> ../b/foo  where a/ (the directory where to put the symlink) does
 # not yet exist.
-define LINUX_FIRMWARE_CREATE_SYMLINKS
+define LINUX_FIRMWARE_INSTALL_TARGET_CMDS
+	mkdir -p $(TARGET_DIR)/lib/firmware
+	$(TAR) xf $(@D)/br-firmware.tar -C $(TARGET_DIR)/lib/firmware/
 	cd $(TARGET_DIR)/lib/firmware/ ; \
 	sed -r -e '/^Link: (.+) -> (.+)$$/!d; s//\1 \2/' $(@D)/WHENCE | \
 	while read f d; do \
@@ -697,11 +687,4 @@ define LINUX_FIRMWARE_CREATE_SYMLINKS
 	done
 endef
 
-define LINUX_FIRMWARE_INSTALL_TARGET_CMDS
-	mkdir -p $(TARGET_DIR)/lib/firmware
-	$(LINUX_FIRMWARE_INSTALL_FILES)
-	$(LINUX_FIRMWARE_INSTALL_DIRS)
-	$(LINUX_FIRMWARE_CREATE_SYMLINKS)
-endef
-
 $(eval $(generic-package))
Signature · 1 line
2.20.1
#2 lore raw

PATCH v2 2/4 package/linux-firmware: make install logic macro accept a destination parameter

In reply to Peter Korsgaard

So it can be reused for also installing into the images directory.

---
Changes since v1:
- Reworked after patch 1 changed install logic

package/linux-firmware/linux-firmware.mk | 12 ++++++++----

 1 file changed, 8 insertions(+), 4 deletions(-)
package/linux-firmware/linux-firmware.mk +8 -4
index 9bc59435ef..8defedd309 100644
@@ -674,10 +674,10 @@ endif
 # sure we canonicalize the pointed-to file, to cover the symlinks of the form
 # a/foo -> ../b/foo  where a/ (the directory where to put the symlink) does
 # not yet exist.
-define LINUX_FIRMWARE_INSTALL_TARGET_CMDS
-	mkdir -p $(TARGET_DIR)/lib/firmware
-	$(TAR) xf $(@D)/br-firmware.tar -C $(TARGET_DIR)/lib/firmware/
-	cd $(TARGET_DIR)/lib/firmware/ ; \
+define LINUX_FIRMWARE_INSTALL_FW
+	mkdir -p $(1)
+	$(TAR) xf $(@D)/br-firmware.tar -C $(1)
+	cd $(1) ; \
 	sed -r -e '/^Link: (.+) -> (.+)$$/!d; s//\1 \2/' $(@D)/WHENCE | \
 	while read f d; do \
 		if test -f $$(readlink -m $$(dirname $$f)/$$d); then \
@@ -687,4 +687,8 @@ define LINUX_FIRMWARE_INSTALL_TARGET_CMDS
 	done
 endef
 
+define LINUX_FIRMWARE_INSTALL_TARGET_CMDS
+	$(call LINUX_FIRMWARE_INSTALL_FW, $(TARGET_DIR)/lib/firmware)
+endef
+
 $(eval $(generic-package))
Signature · 1 line
2.20.1
#3 lore raw

PATCH v2 3/4 package/linux-firmware: also install into images for early loading support

In reply to Peter Korsgaard

Some drivers request their firmware very early when built into the kernel, even before the initramfs is mounted - So the only way to provide firmware for those drivers is to include them directly in the kernel with the CONFIG_EXTRA_FIRMWARE option.

An example of this is the uC firmware for modern Intel GPUs.

Conceptually you can point CONFIG_EXTRA_FIRMWARE to ${TARGET_DIR}/lib/firmware, but then you cannot remove the firmware from the initramfs and pay the size cost twice (inside the kernel + in initramfs), so instead also install linux-firmware to the images dir, similar to how we do it for intel-microcode.

---
Changes since v1:
- Reworked after patch 1 changed install logic

package/linux-firmware/linux-firmware.mk | 5 +++++

 1 file changed, 5 insertions(+)
package/linux-firmware/linux-firmware.mk +5 -0
index 8defedd309..5e92b0446e 100644
@@ -7,6 +7,7 @@
 LINUX_FIRMWARE_VERSION = 20201022
 LINUX_FIRMWARE_SITE = http://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git
 LINUX_FIRMWARE_SITE_METHOD = git
+LINUX_FIRMWARE_INSTALL_IMAGES = YES
 
 LINUX_FIRMWARE_CPE_ID_VENDOR = kernel
 
@@ -691,4 +692,8 @@ define LINUX_FIRMWARE_INSTALL_TARGET_CMDS
 	$(call LINUX_FIRMWARE_INSTALL_FW, $(TARGET_DIR)/lib/firmware)
 endef
 
+define LINUX_FIRMWARE_INSTALL_IMAGES_CMDS
+	$(call LINUX_FIRMWARE_INSTALL_FW, $(BINARIES_DIR))
+endef
+
 $(eval $(generic-package))
Signature · 1 line
2.20.1
#4 lore raw

PATCH v2 4/4 linux: build after linux-firmware if enabled for early loading support

In reply to Peter Korsgaard

To support building in (a subset of) the linux-firmware files into the kernel using the CONFIG_EXTRA_FIRMWARE option, we need to ensure that the firmware files are installed before the Linux kernel is built, similar to how it is done for intel-microcode.

(cherry picked from commit 248f47cdd407ba52a851d50ca29bd7fb2d0a9353)

---
Changes since v1:
- No changes

 linux/linux.mk | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
linux/linux.mk +2 -1
index a212f42c28..5e4b319cf1 100644
@@ -78,7 +78,8 @@ LINUX_MAKE_ENV = \
 
 LINUX_INSTALL_IMAGES = YES
 LINUX_DEPENDENCIES = host-kmod \
-	$(if $(BR2_PACKAGE_INTEL_MICROCODE),intel-microcode)
+	$(if $(BR2_PACKAGE_INTEL_MICROCODE),intel-microcode) \
+	$(if $(BR2_PACKAGE_LINUX_FIRMWARE),linux-firmware)
 
 # Starting with 4.16, the generated kconfig paser code is no longer
 # shipped with the kernel sources, so we need flex and bison, but
Signature · 1 line
2.20.1
#5 lore raw

v2 cover package/linux-firmware: install into images for early loading support

In reply to Peter Korsgaard
Quoted 18 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Peter" == Peter Korsgaard <[email protected]> writes:

Some drivers request their firmware very early when built into the kernel, even before the initramfs is mounted - So the only way to provide firmware for those drivers is to include them directly in the kernel with the CONFIG_EXTRA_FIRMWARE option.

To support this, let linux-firmware install its files into BINARIES_DIR in addition to TARGET_DIR, similar to how we do it for intel-microcode.

Peter Korsgaard (3):
package/linux-firmware: make install logic macro accept a destination
parameter
package/linux-firmware: also install into images for early loading
support
linux: build after linux-firmware if enabled for early loading support

Quoted 2 lines

Yann E. MORIN (1):
package/linux-firmware: rationalise install step

Committed to next, thanks.

Signature · 1 line
Bye, Peter Korsgaard