Skip to contentSkip to conversation

firmware: Automatically pull missing FW files

PATCH In review Tadeusz Struk 4 participants 7 messages Aug 20, 2014 – Aug 21, 2014

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 firmware: Automatically pull missing FW files

Hi,
When a binary FW file is not present in the firmware dir then the build will fail. This patch enables auto-download any missing FW file for both shipped and external targets.
Generated against linux-next.

---
 firmware/Makefile |   25 +++++++++++++++++++++++--
 1 file changed, 23 insertions(+), 2 deletions(-)
firmware/Makefile +23 -2
index 5747417..3ad12f6 100644
@@ -178,10 +178,31 @@ wordsize_deps := $(wildcard include/config/64bit.h include/config/32bit.h \
 		include/config/superh32.h include/config/superh64.h \
 		include/config/x86_32.h include/config/x86_64.h)
 
-$(patsubst %,$(obj)/%.gen.S, $(fw-shipped-y)): %: $(wordsize_deps)
+FW_URL := git.kernel.org/cgit/linux/kernel/git/firmware/linux-firmware.git/plain
+
+.NOTPARALLEL: get_missing_external $(patsubst %,$(obj)/%.gen.S, $(fw-external-y))
+.NOTPARALLEL: get_missing_shipped $(patsubst %,$(obj)/%.gen.S, $(fw-shipped-y))
+
+get_missing_shipped:
+	@for file in $(fw-shipped-y) $(fw-shipped-m); do \
+		if [ ! -f $(obj)/$$file ]; then \
+			wget $(FW_URL)/$$file -o /dev/null -O $(obj)/$$file; \
+		fi; \
+	done;
+
+get_missing_external:
+	@for file in $(fw-external-y); do \
+		if [ ! -f $(obj)/$$file ]; then \
+			wget $(FW_URL)/$$file -o /dev/null -O $(obj)/$$file; \
+		fi; \
+	done;
+
+$(patsubst %,$(obj)/%.gen.S, $(fw-shipped-y)): %: $(wordsize_deps) \
+		get_missing_shipped
 	$(call cmd,fwbin,$(patsubst %.gen.S,%,$@))
+
 $(patsubst %,$(obj)/%.gen.S, $(fw-external-y)): %: $(wordsize_deps) \
-		include/config/extra/firmware/dir.h
+		include/config/extra/firmware/dir.h get_missing_external
 	$(call cmd,fwbin,$(fwabs)/$(patsubst $(obj)/%.gen.S,%,$@))
 
 # The .o files depend on the binaries directly; the .S files don't.
#1 lore raw
Quoted 3 lines · Tadeusz Struk

Hi,
When a binary FW file is not present in the firmware dir then the build will fail.

I'm not sure I understand. Precisely what fails?

I don't like this patch very much. We should be removing the legacy firmware/ directory entirely, not patching it up.

Userspace is responsible for providing the firmware, and it should generally come from an entirely separate checkout of the linux-firmware repository.

Signature · 1 line
dwmw2
#2 lore raw

Hi David,

Quoted 1 line · David Woodhouse

I'm not sure I understand. Precisely what fails?

I clone a subsystem, configure it to use CONFIG_EXTRA_FIRMWARE="qat_895xcc.bin", type make && make install and get:

MK_FW firmware/qat_895xxc.bin.gen.S
make[1]: *** No rule to make target firmware/qat_895xxc.bin', needed by firmware/qat_895xxc.bin.gen.o'. Stop.

I thought it might be useful if it would pull whatever FW it needs and not just give up. It also might be useful if one wants to "refresh" the FW binaries. In this case one can do rm firmware/*.[bin|ihex] && make && make install

Quoted 7 lines

I don't like this patch very much. We should be removing the legacy firmware/ directory entirely, not patching it up.

Userspace is responsible for providing the firmware, and it should generally come from an entirely separate checkout of the linux-firmware repository.

Yes, if you use udev helper. When you want to compile in the blobs to your kernel it is needed in build time, right?
In both cases you need the binary anyway so you can copy it manually from linux-firmware or use this nice feature to do it for you. If you want to remove firmware/ directory entirely then this makefile will be gone as well so what's the problem?

#3 lore raw
Quoted 10 lines · Tadeusz Struk

Hi David,

Quoted 1 line · David Woodhouse

I'm not sure I understand. Precisely what fails?

I clone a subsystem, configure it to use CONFIG_EXTRA_FIRMWARE="qat_895xcc.bin", type make && make install and get:

MK_FW firmware/qat_895xxc.bin.gen.S
make[1]: *** No rule to make target firmware/qat_895xxc.bin', needed by firmware/qat_895xxc.bin.gen.o'. Stop.

Is this what you were looking for?

config EXTRA_FIRMWARE_DIR
string "Firmware blobs root directory"
depends on EXTRA_FIRMWARE != ""
default "firmware"
help
This option controls the directory in which the kernel build system looks for the firmware files listed in the EXTRA_FIRMWARE option. The default is firmware/ in the kernel source tree, but by changing this option you can point it elsewhere, such as /lib/firmware/ or some other directory containing the firmware files.

#4 lore raw
In reply to Tadeusz Struk
Quoted 10 lines · Tadeusz Struk

Hi David,

Quoted 1 line · David Woodhouse

I'm not sure I understand. Precisely what fails?

I clone a subsystem, configure it to use CONFIG_EXTRA_FIRMWARE="qat_895xcc.bin", type make && make install and get:

MK_FW firmware/qat_895xxc.bin.gen.S
make[1]: *** No rule to make target firmware/qat_895xxc.bin', needed by firmware/qat_895xxc.bin.gen.o'. Stop.

Can't you already just use CONFIG_EXTRA_FIRMWARE_DIR ?

Quoted 2 lines

Yes, if you use udev helper. When you want to compile in the blobs to your kernel it is needed in build time, right?

Yes. But seriously: don't do that. Let firmware get loaded from userspace the normal way. Don't build kernel images that you can't distribute because they include non-GPL parts.

Signature · 1 line
dwmw2
#5 lore raw
Quoted 5 lines · David Woodhouse
Quoted 4 lines · Tadeusz Struk

Hi David,

Quoted 1 line · David Woodhouse

I'm not sure I understand. Precisely what fails?

Quoted 8 lines
Quoted 6 lines

I clone a subsystem, configure it to use CONFIG_EXTRA_FIRMWARE="qat_895xcc.bin", type make && make install and get:

MK_FW firmware/qat_895xxc.bin.gen.S
make[1]: *** No rule to make target firmware/qat_895xxc.bin', needed by firmware/qat_895xxc.bin.gen.o'. Stop.

Can't you already just use CONFIG_EXTRA_FIRMWARE_DIR ?

What for? The point is that you don't have the FW binary file and you have to download it, which can happen automatically and this is what this patch does. The director where it will be stored doesn't really matter here.

Quoted 8 lines
Quoted 3 lines

Yes, if you use udev helper. When you want to compile in the blobs to your kernel it is needed in build time, right?

Yes. But seriously: don't do that. Let firmware get loaded from userspace the normal way. Don't build kernel images that you can't distribute because they include non-GPL parts.

#6 lore raw
Quoted 4 lines · Tadeusz Struk

What for? The point is that you don't have the FW binary file and you have to download it, which can happen automatically and this is what this patch does. The director where it will be stored doesn't really matter here.

"git pull" will at least tell you something is seriously wrong (if you know the firmware tree should never require a forced update, that is).

I'd really appreciate if the firmware tree had signed tags or signed commits, though. Those at least provide decent end-to-end security (both at rest, and while in transit) for data once it has been "git push"'d.

Signature · 4 lines
  "One disk to rule them all, One disk to find them. One disk to bring
  them all and in the darkness grind them. In the Land of Redmond
  where the shadows lie." -- The Silicon Valley Tarot
  Henrique Holschuh