firmware: Automatically pull missing FW files
PATCH 1/1 firmware: Automatically pull missing FW files
Hi,
When a binary FW file is not present in the firmware dir then the build will fail. This patch enables auto-download any missing FW file for both shipped and external targets.
Generated against linux-next.
--- firmware/Makefile | 25 +++++++++++++++++++++++-- 1 file changed, 23 insertions(+), 2 deletions(-)
firmware/Makefile +23 -2
@@ -178,10 +178,31 @@ wordsize_deps := $(wildcard include/config/64bit.h include/config/32bit.h \ include/config/superh32.h include/config/superh64.h \ include/config/x86_32.h include/config/x86_64.h) -$(patsubst %,$(obj)/%.gen.S, $(fw-shipped-y)): %: $(wordsize_deps) +FW_URL := git.kernel.org/cgit/linux/kernel/git/firmware/linux-firmware.git/plain + +.NOTPARALLEL: get_missing_external $(patsubst %,$(obj)/%.gen.S, $(fw-external-y)) +.NOTPARALLEL: get_missing_shipped $(patsubst %,$(obj)/%.gen.S, $(fw-shipped-y)) + +get_missing_shipped: + @for file in $(fw-shipped-y) $(fw-shipped-m); do \ + if [ ! -f $(obj)/$$file ]; then \ + wget $(FW_URL)/$$file -o /dev/null -O $(obj)/$$file; \ + fi; \ + done; + +get_missing_external: + @for file in $(fw-external-y); do \ + if [ ! -f $(obj)/$$file ]; then \ + wget $(FW_URL)/$$file -o /dev/null -O $(obj)/$$file; \ + fi; \ + done; + +$(patsubst %,$(obj)/%.gen.S, $(fw-shipped-y)): %: $(wordsize_deps) \ + get_missing_shipped $(call cmd,fwbin,$(patsubst %.gen.S,%,$@)) + $(patsubst %,$(obj)/%.gen.S, $(fw-external-y)): %: $(wordsize_deps) \ - include/config/extra/firmware/dir.h + include/config/extra/firmware/dir.h get_missing_external $(call cmd,fwbin,$(fwabs)/$(patsubst $(obj)/%.gen.S,%,$@)) # The .o files depend on the binaries directly; the .S files don't.
Quoted 3 lines · Tadeusz Struk
Hi,
When a binary FW file is not present in the firmware dir then the build will fail.
I'm not sure I understand. Precisely what fails?
I don't like this patch very much. We should be removing the legacy firmware/ directory entirely, not patching it up.
Userspace is responsible for providing the firmware, and it should generally come from an entirely separate checkout of the linux-firmware repository.
Signature · 1 line
dwmw2
Hi David,
Quoted 1 line · David Woodhouse
I'm not sure I understand. Precisely what fails?
I clone a subsystem, configure it to use CONFIG_EXTRA_FIRMWARE="qat_895xcc.bin", type make && make install and get:
MK_FW firmware/qat_895xxc.bin.gen.S
make[1]: *** No rule to make target firmware/qat_895xxc.bin', needed by firmware/qat_895xxc.bin.gen.o'. Stop.
I thought it might be useful if it would pull whatever FW it needs and not just give up. It also might be useful if one wants to "refresh" the FW binaries. In this case one can do rm firmware/*.[bin|ihex] && make && make install
Quoted 7 lines
I don't like this patch very much. We should be removing the legacy
firmware/directory entirely, not patching it up.Userspace is responsible for providing the firmware, and it should generally come from an entirely separate checkout of the linux-firmware repository.
Yes, if you use udev helper. When you want to compile in the blobs to your kernel it is needed in build time, right?
In both cases you need the binary anyway so you can copy it manually from linux-firmware or use this nice feature to do it for you. If you want to remove firmware/ directory entirely then this makefile will be gone as well so what's the problem?
Quoted 10 lines · Tadeusz Struk
Hi David,
Quoted 1 line · David Woodhouse
I'm not sure I understand. Precisely what fails?
I clone a subsystem, configure it to use
CONFIG_EXTRA_FIRMWARE="qat_895xcc.bin", type make && make install and get:
MK_FWfirmware/qat_895xxc.bin.gen.S
make[1]: *** No rule to make targetfirmware/qat_895xxc.bin', needed byfirmware/qat_895xxc.bin.gen.o'. Stop.
Is this what you were looking for?
config EXTRA_FIRMWARE_DIR
string "Firmware blobs root directory"
depends on EXTRA_FIRMWARE != ""
default "firmware"
help
This option controls the directory in which the kernel build system looks for the firmware files listed in the EXTRA_FIRMWARE option. The default is firmware/ in the kernel source tree, but by changing this option you can point it elsewhere, such as /lib/firmware/ or some other directory containing the firmware files.
Quoted 10 lines · Tadeusz Struk
Hi David,
Quoted 1 line · David Woodhouse
I'm not sure I understand. Precisely what fails?
I clone a subsystem, configure it to use
CONFIG_EXTRA_FIRMWARE="qat_895xcc.bin", type make && make install and get:
MK_FWfirmware/qat_895xxc.bin.gen.S
make[1]: *** No rule to make targetfirmware/qat_895xxc.bin', needed byfirmware/qat_895xxc.bin.gen.o'. Stop.
Can't you already just use CONFIG_EXTRA_FIRMWARE_DIR ?
Quoted 2 lines
Yes, if you use udev helper. When you want to compile in the blobs to your kernel it is needed in build time, right?
Yes. But seriously: don't do that. Let firmware get loaded from userspace the normal way. Don't build kernel images that you can't distribute because they include non-GPL parts.
Signature · 1 line
dwmw2
Quoted 5 lines · David Woodhouse
Quoted 4 lines · Tadeusz Struk
Hi David,
Quoted 1 line · David Woodhouse
I'm not sure I understand. Precisely what fails?
Quoted 8 lines
Quoted 6 lines
I clone a subsystem, configure it to use
CONFIG_EXTRA_FIRMWARE="qat_895xcc.bin", type make && make install and get:
MK_FWfirmware/qat_895xxc.bin.gen.S
make[1]: *** No rule to make targetfirmware/qat_895xxc.bin', needed byfirmware/qat_895xxc.bin.gen.o'. Stop.Can't you already just use
CONFIG_EXTRA_FIRMWARE_DIR?
What for? The point is that you don't have the FW binary file and you have to download it, which can happen automatically and this is what this patch does. The director where it will be stored doesn't really matter here.
Quoted 8 lines
Quoted 3 lines
Yes, if you use udev helper. When you want to compile in the blobs to your kernel it is needed in build time, right?
Yes. But seriously: don't do that. Let firmware get loaded from userspace the normal way. Don't build kernel images that you can't distribute because they include non-GPL parts.
Quoted 4 lines · Tadeusz Struk
What for? The point is that you don't have the FW binary file and you have to download it, which can happen automatically and this is what this patch does. The director where it will be stored doesn't really matter here.
"git pull" will at least tell you something is seriously wrong (if you know the firmware tree should never require a forced update, that is).
I'd really appreciate if the firmware tree had signed tags or signed commits, though. Those at least provide decent end-to-end security (both at rest, and while in transit) for data once it has been "git push"'d.
Signature · 4 lines
"One disk to rule them all, One disk to find them. One disk to bring them all and in the darkness grind them. In the Land of Redmond where the shadows lie." -- The Silicon Valley Tarot Henrique Holschuh