package/linux-firmware: fix symlink support
PATCH 1/1 package/linux-firmware: fix symlink support
Since Linux-firmware's commit 9cfefbd7fbda ("Remove duplicate symlinks") symlinks aren't distributed anymore. They are rather created at installation time by a script provided in the project, copy-firmware.sh. The description of the symlinks is done in the WHENCE file. Since the bump to version 20200122, in commit 48cc1a89ae04, installation for many firmwares was broken as Buildroot tried to install missing symlinks from Linux-firmware.
The fix is not only to remove now missing symlinks, but to add logic to create those symlinks as kernel modules will depend on them. The solution taken by this patch is to create dynamically symlinks based on their description in the WHENCE file *and* only if the file they'll point to was installed in the target directory.
--- package/linux-firmware/linux-firmware.mk | 33 ++++++++++++++---------- 1 file changed, 19 insertions(+), 14 deletions(-)
package/linux-firmware/linux-firmware.mk +19 -14
@@ -71,9 +71,7 @@ endif # rt2xx ifeq ($(BR2_PACKAGE_LINUX_FIRMWARE_RALINK_RT2XX),y) -# rt3090.bin is a symlink to rt2860.bin -# rt3070.bin is a symlink to rt2870.bin -LINUX_FIRMWARE_FILES += rt2860.bin rt2870.bin rt3070.bin rt3071.bin rt3090.bin +LINUX_FIRMWARE_FILES += rt2860.bin rt2870.bin rt3071.bin LINUX_FIRMWARE_ALL_LICENSE_FILES += LICENCE.ralink-firmware.txt endif
@@ -214,8 +212,6 @@ endif # sd8688 ifeq ($(BR2_PACKAGE_LINUX_FIRMWARE_LIBERTAS_SD8688),y) -LINUX_FIRMWARE_FILES += libertas/sd8688.bin libertas/sd8688_helper.bin -# The two files above are but symlinks to those two ones: LINUX_FIRMWARE_FILES += mrvl/sd8688.bin mrvl/sd8688_helper.bin LINUX_FIRMWARE_ALL_LICENSE_FILES += LICENCE.Marvell endif
@@ -320,12 +316,10 @@ endif # wl127x ifeq ($(BR2_PACKAGE_LINUX_FIRMWARE_TI_WL127X),y) -# wl1271-nvs.bin is a symlink to wl127x-nvs.bin LINUX_FIRMWARE_FILES += \ ti-connectivity/wl1271-fw-2.bin \ ti-connectivity/wl1271-fw-ap.bin \ ti-connectivity/wl1271-fw.bin \ - ti-connectivity/wl1271-nvs.bin \ ti-connectivity/wl127x-fw-3.bin \ ti-connectivity/wl127x-fw-plt-3.bin \ ti-connectivity/wl127x-nvs.bin \
@@ -341,15 +335,12 @@ endif # wl128x ifeq ($(BR2_PACKAGE_LINUX_FIRMWARE_TI_WL128X),y) -# wl1271-nvs.bin and wl12xx-nvs.bin are symlinks to wl127x-nvs.bin LINUX_FIRMWARE_FILES += \ ti-connectivity/wl128x-fw-3.bin \ ti-connectivity/wl128x-fw-ap.bin \ ti-connectivity/wl128x-fw-plt-3.bin \ ti-connectivity/wl128x-fw.bin \ - ti-connectivity/wl1271-nvs.bin \ ti-connectivity/wl128x-nvs.bin \ - ti-connectivity/wl12xx-nvs.bin \ ti-connectivity/wl127x-nvs.bin \ ti-connectivity/wl128x-fw-4-mr.bin \ ti-connectivity/wl128x-fw-4-plt.bin \
@@ -363,13 +354,11 @@ endif # wl18xx ifeq ($(BR2_PACKAGE_LINUX_FIRMWARE_TI_WL18XX),y) -# wl1271-nvs.bin is a symlink to wl127x-nvs.bin LINUX_FIRMWARE_FILES += \ ti-connectivity/wl18xx-fw.bin \ ti-connectivity/wl18xx-fw-2.bin \ ti-connectivity/wl18xx-fw-3.bin \ ti-connectivity/wl18xx-fw-4.bin \ - ti-connectivity/wl1271-nvs.bin \ ti-connectivity/wl127x-nvs.bin \ ti-connectivity/TIInit_7.2.31.bts LINUX_FIRMWARE_ALL_LICENSE_FILES += LICENCE.ti-connectivity
@@ -568,8 +557,7 @@ LINUX_FIRMWARE_FILES += \ endif ifeq ($(BR2_PACKAGE_LINUX_FIRMWARE_QAT_DH895XCC),y) -# qat_mmp.bin is a symlink to qat_895xcc_mmp.bin -LINUX_FIRMWARE_FILES += qat_895xcc.bin qat_895xcc_mmp.bin qat_mmp.bin +LINUX_FIRMWARE_FILES += qat_895xcc.bin qat_895xcc_mmp.bin LINUX_FIRMWARE_ALL_LICENSE_FILES += LICENCE.qat_firmware endif
@@ -626,4 +614,21 @@ define LINUX_FIRMWARE_INSTALL_TARGET_CMDS $(LINUX_FIRMWARE_INSTALL_DIRS) endef +# Some firmware are distributed as a symlink, for drivers to load them using a +# defined name other than the real one. Since 9cfefbd7fbda ("Remove duplicate +# symlinks") those symlink aren't distributed in linux-firmware but are created +# automatically by its copy-firmware.sh script during the installation, which +# parses the WHENCE file where symlinks are described. We follow the same logic +# here, adding symlink only for firmwares installed in the target directory. +# The grep/sed parsing is taken from the script mentioned before. +define LINUX_FIRMWARE_CREATE_SYMLINKS + grep -E '^Link:' $(@D)/WHENCE | sed -e's/^Link: *//g' -e's/-> //g' | while read f d; do \ + if test -f $(TARGET_DIR)/lib/firmware/$$d; then \ + ln -sf $$d $(TARGET_DIR)/lib/firmware/$$f ; \ + fi ; \ + done +endef + +LINUX_FIRMWARE_POST_INSTALL_TARGET_HOOKS += LINUX_FIRMWARE_CREATE_SYMLINKS + $(eval $(generic-package))
Signature · 1 line
2.24.1
PATCH 1/1 package/linux-firmware: fix symlink support
Hi Antoine,
Quoted 14 lines · Antoine Tenart
Since Linux-firmware's commit
9cfefbd7fbda("Remove duplicate symlinks") symlinks aren't distributed anymore. They are rather created at installation time by a script provided in the project,copy-firmware.sh. The description of the symlinks is done in the WHENCE file. Since the bump to version 20200122, in commit48cc1a89ae04, installation for many firmwares was broken as Buildroot tried to install missing symlinks from Linux-firmware.The fix is not only to remove now missing symlinks, but to add logic to create those symlinks as kernel modules will depend on them. The solution taken by this patch is to create dynamically symlinks based on their description in the WHENCE file *and* only if the file they'll point to was installed in the target directory.
Quoted 3 lines
Fixes 48cc1a89ae04 ("package/linux-firmware: bump to version 20200122")Cc james.hilliard1 at gmail.comSigned-off-by Antoine Tenart <[email protected]>
See commit 23d12793d5 ("package/linux-firmware: add missing symlinks") in the master branch.
baruch
Quoted 103 lines
--- package/linux-firmware/linux-firmware.mk | 33 ++++++++++++++---------- 1 file changed, 19 insertions(+), 14 deletions(-)package/linux-firmware/linux-firmware.mk +19
-14@@ -71,9 +71,7 @@ endif # rt2xx ifeq ($(BR2_PACKAGE_LINUX_FIRMWARE_RALINK_RT2XX),y) -# rt3090.bin is a symlink to rt2860.bin -# rt3070.bin is a symlink to rt2870.bin -LINUX_FIRMWARE_FILES += rt2860.bin rt2870.bin rt3070.bin rt3071.bin rt3090.bin +LINUX_FIRMWARE_FILES += rt2860.bin rt2870.bin rt3071.bin LINUX_FIRMWARE_ALL_LICENSE_FILES += LICENCE.ralink-firmware.txt endif@@ -214,8 +212,6 @@ endif # sd8688 ifeq ($(BR2_PACKAGE_LINUX_FIRMWARE_LIBERTAS_SD8688),y) -LINUX_FIRMWARE_FILES += libertas/sd8688.bin libertas/sd8688_helper.bin -# The two files above are but symlinks to those two ones: LINUX_FIRMWARE_FILES += mrvl/sd8688.bin mrvl/sd8688_helper.bin LINUX_FIRMWARE_ALL_LICENSE_FILES += LICENCE.Marvell endif@@ -320,12 +316,10 @@ endif # wl127x ifeq ($(BR2_PACKAGE_LINUX_FIRMWARE_TI_WL127X),y) -# wl1271-nvs.bin is a symlink to wl127x-nvs.bin LINUX_FIRMWARE_FILES += \ ti-connectivity/wl1271-fw-2.bin \ ti-connectivity/wl1271-fw-ap.bin \ ti-connectivity/wl1271-fw.bin \ - ti-connectivity/wl1271-nvs.bin \ ti-connectivity/wl127x-fw-3.bin \ ti-connectivity/wl127x-fw-plt-3.bin \ ti-connectivity/wl127x-nvs.bin \@@ -341,15 +335,12 @@ endif # wl128x ifeq ($(BR2_PACKAGE_LINUX_FIRMWARE_TI_WL128X),y) -# wl1271-nvs.bin and wl12xx-nvs.bin are symlinks to wl127x-nvs.bin LINUX_FIRMWARE_FILES += \ ti-connectivity/wl128x-fw-3.bin \ ti-connectivity/wl128x-fw-ap.bin \ ti-connectivity/wl128x-fw-plt-3.bin \ ti-connectivity/wl128x-fw.bin \ - ti-connectivity/wl1271-nvs.bin \ ti-connectivity/wl128x-nvs.bin \ - ti-connectivity/wl12xx-nvs.bin \ ti-connectivity/wl127x-nvs.bin \ ti-connectivity/wl128x-fw-4-mr.bin \ ti-connectivity/wl128x-fw-4-plt.bin \@@ -363,13 +354,11 @@ endif # wl18xx ifeq ($(BR2_PACKAGE_LINUX_FIRMWARE_TI_WL18XX),y) -# wl1271-nvs.bin is a symlink to wl127x-nvs.bin LINUX_FIRMWARE_FILES += \ ti-connectivity/wl18xx-fw.bin \ ti-connectivity/wl18xx-fw-2.bin \ ti-connectivity/wl18xx-fw-3.bin \ ti-connectivity/wl18xx-fw-4.bin \ - ti-connectivity/wl1271-nvs.bin \ ti-connectivity/wl127x-nvs.bin \ ti-connectivity/TIInit_7.2.31.bts LINUX_FIRMWARE_ALL_LICENSE_FILES += LICENCE.ti-connectivity@@ -568,8 +557,7 @@ LINUX_FIRMWARE_FILES += \ endif ifeq ($(BR2_PACKAGE_LINUX_FIRMWARE_QAT_DH895XCC),y) -# qat_mmp.bin is a symlink to qat_895xcc_mmp.bin -LINUX_FIRMWARE_FILES += qat_895xcc.bin qat_895xcc_mmp.bin qat_mmp.bin +LINUX_FIRMWARE_FILES += qat_895xcc.bin qat_895xcc_mmp.bin LINUX_FIRMWARE_ALL_LICENSE_FILES += LICENCE.qat_firmware endif@@ -626,4 +614,21 @@ define LINUX_FIRMWARE_INSTALL_TARGET_CMDS $(LINUX_FIRMWARE_INSTALL_DIRS) endef +# Some firmware are distributed as a symlink, for drivers to load them using a +# defined name other than the real one. Since 9cfefbd7fbda ("Remove duplicate +# symlinks") those symlink aren't distributed in linux-firmware but are created +# automatically by its copy-firmware.sh script during the installation, which +# parses the WHENCE file where symlinks are described. We follow the same logic +# here, adding symlink only for firmwares installed in the target directory. +# The grep/sed parsing is taken from the script mentioned before. +define LINUX_FIRMWARE_CREATE_SYMLINKS + grep -E '^Link:' $(@D)/WHENCE | sed -e's/^Link: *//g' -e's/-> //g' | while read f d; do \ + if test -f $(TARGET_DIR)/lib/firmware/$$d; then \ + ln -sf $$d $(TARGET_DIR)/lib/firmware/$$f ; \ + fi ; \ + done +endef + +LINUX_FIRMWARE_POST_INSTALL_TARGET_HOOKS += LINUX_FIRMWARE_CREATE_SYMLINKS + $(eval $(generic-package))
Signature · 3 lines
http://baruch.siach.name/blog/ ~. .~ Tk Open Systems
=}------------------------------------------------ooO--U--Ooo------------{=
- baruch at tkos.co.il - tel: +972.2.679.5364, http://www.tkos.co.il -PATCH 1/1 package/linux-firmware: fix symlink support
Baruch, All,
On 2020-03-03 15:41 +0200, Baruch Siach spake thusly:
Quoted 17 lines
Hi Antoine,
Quoted 14 lines · Antoine Tenart
Since Linux-firmware's commit
9cfefbd7fbda("Remove duplicate symlinks") symlinks aren't distributed anymore. They are rather created at installation time by a script provided in the project,copy-firmware.sh. The description of the symlinks is done in the WHENCE file. Since the bump to version 20200122, in commit48cc1a89ae04, installation for many firmwares was broken as Buildroot tried to install missing symlinks from Linux-firmware.The fix is not only to remove now missing symlinks, but to add logic to create those symlinks as kernel modules will depend on them. The solution taken by this patch is to create dynamically symlinks based on their description in the WHENCE file *and* only if the file they'll point to was installed in the target directory.
Quoted 6 lines
Quoted 3 lines
Fixes 48cc1a89ae04 ("package/linux-firmware: bump to version 20200122")Cc james.hilliard1 at gmail.comSigned-off-by Antoine Tenart <[email protected]>See commit
23d12793d5("package/linux-firmware: add missing symlinks") in the master branch.
Indeed, but that's a screw up on my part: 23d12793d5 was supposed to be applied on next, not master, and Antoine's patch is for next.
Besides, Antoine's patch implements exactly what I suggested earlier as part of my reply to James:
http://lists.busybox.net/pipermail/buildroot/2020-March/275504.html
So, as discussed on IRC with Antoine and Thomas, I'll cleanup my mess on master (and handle the symlinks issue on next).
Regards,
Yann E. MORIN.
Quoted 114 lines
baruch
Quoted 103 lines
--- package/linux-firmware/linux-firmware.mk | 33 ++++++++++++++---------- 1 file changed, 19 insertions(+), 14 deletions(-)package/linux-firmware/linux-firmware.mk +19
-14@@ -71,9 +71,7 @@ endif # rt2xx ifeq ($(BR2_PACKAGE_LINUX_FIRMWARE_RALINK_RT2XX),y) -# rt3090.bin is a symlink to rt2860.bin -# rt3070.bin is a symlink to rt2870.bin -LINUX_FIRMWARE_FILES += rt2860.bin rt2870.bin rt3070.bin rt3071.bin rt3090.bin +LINUX_FIRMWARE_FILES += rt2860.bin rt2870.bin rt3071.bin LINUX_FIRMWARE_ALL_LICENSE_FILES += LICENCE.ralink-firmware.txt endif@@ -214,8 +212,6 @@ endif # sd8688 ifeq ($(BR2_PACKAGE_LINUX_FIRMWARE_LIBERTAS_SD8688),y) -LINUX_FIRMWARE_FILES += libertas/sd8688.bin libertas/sd8688_helper.bin -# The two files above are but symlinks to those two ones: LINUX_FIRMWARE_FILES += mrvl/sd8688.bin mrvl/sd8688_helper.bin LINUX_FIRMWARE_ALL_LICENSE_FILES += LICENCE.Marvell endif@@ -320,12 +316,10 @@ endif # wl127x ifeq ($(BR2_PACKAGE_LINUX_FIRMWARE_TI_WL127X),y) -# wl1271-nvs.bin is a symlink to wl127x-nvs.bin LINUX_FIRMWARE_FILES += \ ti-connectivity/wl1271-fw-2.bin \ ti-connectivity/wl1271-fw-ap.bin \ ti-connectivity/wl1271-fw.bin \ - ti-connectivity/wl1271-nvs.bin \ ti-connectivity/wl127x-fw-3.bin \ ti-connectivity/wl127x-fw-plt-3.bin \ ti-connectivity/wl127x-nvs.bin \@@ -341,15 +335,12 @@ endif # wl128x ifeq ($(BR2_PACKAGE_LINUX_FIRMWARE_TI_WL128X),y) -# wl1271-nvs.bin and wl12xx-nvs.bin are symlinks to wl127x-nvs.bin LINUX_FIRMWARE_FILES += \ ti-connectivity/wl128x-fw-3.bin \ ti-connectivity/wl128x-fw-ap.bin \ ti-connectivity/wl128x-fw-plt-3.bin \ ti-connectivity/wl128x-fw.bin \ - ti-connectivity/wl1271-nvs.bin \ ti-connectivity/wl128x-nvs.bin \ - ti-connectivity/wl12xx-nvs.bin \ ti-connectivity/wl127x-nvs.bin \ ti-connectivity/wl128x-fw-4-mr.bin \ ti-connectivity/wl128x-fw-4-plt.bin \@@ -363,13 +354,11 @@ endif # wl18xx ifeq ($(BR2_PACKAGE_LINUX_FIRMWARE_TI_WL18XX),y) -# wl1271-nvs.bin is a symlink to wl127x-nvs.bin LINUX_FIRMWARE_FILES += \ ti-connectivity/wl18xx-fw.bin \ ti-connectivity/wl18xx-fw-2.bin \ ti-connectivity/wl18xx-fw-3.bin \ ti-connectivity/wl18xx-fw-4.bin \ - ti-connectivity/wl1271-nvs.bin \ ti-connectivity/wl127x-nvs.bin \ ti-connectivity/TIInit_7.2.31.bts LINUX_FIRMWARE_ALL_LICENSE_FILES += LICENCE.ti-connectivity@@ -568,8 +557,7 @@ LINUX_FIRMWARE_FILES += \ endif ifeq ($(BR2_PACKAGE_LINUX_FIRMWARE_QAT_DH895XCC),y) -# qat_mmp.bin is a symlink to qat_895xcc_mmp.bin -LINUX_FIRMWARE_FILES += qat_895xcc.bin qat_895xcc_mmp.bin qat_mmp.bin +LINUX_FIRMWARE_FILES += qat_895xcc.bin qat_895xcc_mmp.bin LINUX_FIRMWARE_ALL_LICENSE_FILES += LICENCE.qat_firmware endif@@ -626,4 +614,21 @@ define LINUX_FIRMWARE_INSTALL_TARGET_CMDS $(LINUX_FIRMWARE_INSTALL_DIRS) endef +# Some firmware are distributed as a symlink, for drivers to load them using a +# defined name other than the real one. Since 9cfefbd7fbda ("Remove duplicate +# symlinks") those symlink aren't distributed in linux-firmware but are created +# automatically by its copy-firmware.sh script during the installation, which +# parses the WHENCE file where symlinks are described. We follow the same logic +# here, adding symlink only for firmwares installed in the target directory. +# The grep/sed parsing is taken from the script mentioned before. +define LINUX_FIRMWARE_CREATE_SYMLINKS + grep -E '^Link:' $(@D)/WHENCE | sed -e's/^Link: *//g' -e's/-> //g' | while read f d; do \ + if test -f $(TARGET_DIR)/lib/firmware/$$d; then \ + ln -sf $$d $(TARGET_DIR)/lib/firmware/$$f ; \ + fi ; \ + done +endef + +LINUX_FIRMWARE_POST_INSTALL_TARGET_HOOKS += LINUX_FIRMWARE_CREATE_SYMLINKS + $(eval $(generic-package))Signature · 7 lines
http://baruch.siach.name/blog/ ~. .~ Tk Open Systems =}------------------------------------------------ooO--U--Ooo------------{= - baruch at tkos.co.il - tel: +972.2.679.5364, http://www.tkos.co.il - _______________________________________________ buildroot mailing list buildroot at busybox.net http://lists.busybox.net/mailman/listinfo/buildroot
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 561 099 427 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'
PATCH 1/1 package/linux-firmware: fix symlink support
In reply to Antoine TenartAntoine, All,
On 2020-03-03 14:33 +0100, Antoine Tenart spake thusly:
Quoted 8 lines
Since Linux-firmware's commit
9cfefbd7fbda("Remove duplicate symlinks") symlinks aren't distributed anymore. They are rather created at installation time by a script provided in the project,copy-firmware.sh. The description of the symlinks is done in the WHENCE file. Since the bump to version 20200122, in commit48cc1a89ae04, installation for many firmwares was broken as Buildroot tried to install missing symlinks from Linux-firmware.
Quoted 5 lines
The fix is not only to remove now missing symlinks, but to add logic to create those symlinks as kernel modules will depend on them. The solution taken by this patch is to create dynamically symlinks based on their description in the WHENCE file *and* only if the file they'll point to was installed in the target directory.
Thanks for this patch, which basically does what I called for in a previous review:
http://lists.busybox.net/pipermail/buildroot/2020-March/275504.html
However, to avoid replicating my previous mistakes, I did a bit more in-depth analysis of this symlink issue, and it is a bit more involved than just this patch.
The problem is that we have bunch of LINUX_FIRMWARE_FILES that are globs, for example:
37 ifeq ($(BR2_PACKAGE_LINUX_FIRMWARE_QCOM_ADRENO),y)
38 LINUX_FIRMWARE_FILES += qcom/a*
39 LINUX_FIRMWARE_ALL_LICENSE_FILES += LICENSE.qcom qcom/NOTICE.txt
40 endifSo, because we have a glob here, the following symlinks will not be created:
Link: a300_pfp.fw -> qcom/a300_pfp.fw
Link: a300_pm4.fw -> qcom/a300_pm4.fwSo, we need to get rid of the globs and only ever use full names.
See another comment below...
package/linux-firmware/linux-firmware.mk
Quoted 8 lines
Fixes 48cc1a89ae04 ("package/linux-firmware: bump to version 20200122")Cc james.hilliard1 at gmail.comSigned-off-by Antoine Tenart <[email protected]>---
package/linux-firmware/linux-firmware.mk +0
-0
[--SNIP--]
package/linux-firmware/linux-firmware.mk define LINUX_FIRMWARE_INSTALL_TARGET_CMDS · line 625
Quoted 13 lines
package/linux-firmware/linux-firmware.mk +9
-0@@ -626,4 +614,21 @@ define LINUX_FIRMWARE_INSTALL_TARGET_CMDS $(LINUX_FIRMWARE_INSTALL_DIRS) endef +# Some firmware are distributed as a symlink, for drivers to load them using a +# defined name other than the real one. Since 9cfefbd7fbda ("Remove duplicate +# symlinks") those symlink aren't distributed in linux-firmware but are created +# automatically by its copy-firmware.sh script during the installation, which +# parses the WHENCE file where symlinks are described. We follow the same logic +# here, adding symlink only for firmwares installed in the target directory. +# The grep/sed parsing is taken from the script mentioned before. +define LINUX_FIRMWARE_CREATE_SYMLINKS + grep -E '^Link:' $(@D)/WHENCE | sed -e's/^Link: *//g' -e's/-> //g' | while read f d; do \
You can combine the grep and sed into a single sed call:
sed -r -e '/^Link: (.+) -> (.+)$$/!d; s//\1 \2/' $(@D)/WHENCE
package/linux-firmware/linux-firmware.mk define LINUX_FIRMWARE_INSTALL_TARGET_CMDS · line 632
Quoted 7 lines
package/linux-firmware/linux-firmware.mk +7
-0+ if test -f $(TARGET_DIR)/lib/firmware/$$d; then \ + ln -sf $$d $(TARGET_DIR)/lib/firmware/$$f ; \ + fi ; \ + done +endef + +LINUX_FIRMWARE_POST_INSTALL_TARGET_HOOKS += LINUX_FIRMWARE_CREATE_SYMLINKS
Use of a hook is not really useful here: we are using the generic infra, so we can write whatever we want in the _INSTALL_TARGET_CMDS anyway, so just define the macro before _INSTALL_TARGET_CMDS and call it from there, like the other parts (notice the rename of the macro to match):
621 define LINUX_FIRMWARE_INSTALL_TARGET_CMDS
622 mkdir -p $(TARGET_DIR)/lib/firmware
623 $(LINUX_FIRMWARE_INSTALL_FILES)
624 $(LINUX_FIRMWARE_INSTALL_DIRS)
625 $(LINUX_FIRMWARE_INSTALL_SYMLINKS)
626 endefRegards,
Yann E. MORIN.
Quoted 8 lines
package/linux-firmware/linux-firmware.mk +0
-0$(eval $(generic-package))Signature · 6 lines
2.24.1 _______________________________________________ buildroot mailing list buildroot at busybox.net http://lists.busybox.net/mailman/listinfo/buildroot
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 561 099 427 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'
PATCH 1/1 package/linux-firmware: fix symlink support
Hi Yann,
Quoted 10 lines · Yann E. MORIN
On 2020-03-03 14:33 +0100, Antoine Tenart spake thusly:
The problem is that we have bunch of
LINUX_FIRMWARE_FILESthat are globs, for example:37 ifeq ($(BR2_PACKAGE_LINUX_FIRMWARE_QCOM_ADRENO),y) 38 LINUX_FIRMWARE_FILES += qcom/a* 39 LINUX_FIRMWARE_ALL_LICENSE_FILES += LICENSE.qcom qcom/NOTICE.txt 40 endif
Quoted 5 lines
So, because we have a glob here, the following symlinks will not be created:
Link: a300_pfp.fw -> qcom/a300_pfp.fw Link: a300_pm4.fw -> qcom/a300_pm4.fw
I don't get why they wouldn't be created. The symlink creation is done after the files have been installed to the target directory, and is done based on what has been installed. So the definition used to get those firmware installed (being full names or not) shouldn't matter.
With this patch, and BR2_PACKAGE_LINUX_FIRMWARE_QCOM_ADRENO enabled, I get the following files in $(TARGET_DIR)/lib/firmware:
./output/target/lib/firmware/
??? a300_pfp.fw -> qcom/a300_pfp.fw
??? a300_pm4.fw -> qcom/a300_pm4.fw
??? qcom
??? a300_pfp.fw
??? a300_pm4.fw
??? a530_pfp.fw
??? a530_pm4.fw
??? a530v3_gpmu.fw2
??? a530_zap.b00
??? a530_zap.b01
??? a530_zap.b02
??? a530_zap.mdt
??? a630_gmu.bin
??? a630_sqe.fwQuoted 12 lines
package/linux-firmware/linux-firmware.mk define LINUX_FIRMWARE_INSTALL_TARGET_CMDS · line 625
Quoted 9 lines
package/linux-firmware/linux-firmware.mk +9
-0+# Some firmware are distributed as a symlink, for drivers to load them using a +# defined name other than the real one. Since 9cfefbd7fbda ("Remove duplicate +# symlinks") those symlink aren't distributed in linux-firmware but are created +# automatically by its copy-firmware.sh script during the installation, which +# parses the WHENCE file where symlinks are described. We follow the same logic +# here, adding symlink only for firmwares installed in the target directory. +# The grep/sed parsing is taken from the script mentioned before. +define LINUX_FIRMWARE_CREATE_SYMLINKS + grep -E '^Link:' $(@D)/WHENCE | sed -e's/^Link: *//g' -e's/-> //g' | while read f d; do \You can combine the grep and sed into a single sed call:
Quoted 1 line
sed -r -e '/^Link: (.+) -> (.+)$$/!d; s//\1 \2/' $(@D)/WHENCE
The reasoning here was to reuse with as little change as possible the code used in Linux-firmware's copy-firmware.sh script, to be easily maintainable should they decide to update the link definition format.
I'm not pushing for either solution, I can change this for v2 if you believe the two command should still be combined.
Quoted 13 lines
package/linux-firmware/linux-firmware.mk define LINUX_FIRMWARE_INSTALL_TARGET_CMDS · line 632
Quoted 7 lines
package/linux-firmware/linux-firmware.mk +7
-0+ if test -f $(TARGET_DIR)/lib/firmware/$$d; then \ + ln -sf $$d $(TARGET_DIR)/lib/firmware/$$f ; \ + fi ; \ + done +endef + +LINUX_FIRMWARE_POST_INSTALL_TARGET_HOOKS += LINUX_FIRMWARE_CREATE_SYMLINKSUse of a hook is not really useful here: we are using the generic infra, so we can write whatever we want in the
_INSTALL_TARGET_CMDSanyway, so just define the macro before_INSTALL_TARGET_CMDSand call it from there, like the other parts (notice the rename of the macro to match):
Quoted 6 lines
621 define LINUX_FIRMWARE_INSTALL_TARGET_CMDS
622 mkdir -p $(TARGET_DIR)/lib/firmware
623 $(LINUX_FIRMWARE_INSTALL_FILES)
624 $(LINUX_FIRMWARE_INSTALL_DIRS)
625 $(LINUX_FIRMWARE_INSTALL_SYMLINKS)
626 endefOK, will do!
Thanks!
Antoine
Signature · 3 lines
Antoine T?nart, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
PATCH 1/1 package/linux-firmware: fix symlink support
Antoine, All,
On 2020-03-03 19:47 +0100, Antoine Tenart spake thusly:
Quoted 29 lines · Yann E. MORIN
Quoted 10 lines
On 2020-03-03 14:33 +0100, Antoine Tenart spake thusly:
The problem is that we have bunch of
LINUX_FIRMWARE_FILESthat are globs, for example:37 ifeq ($(BR2_PACKAGE_LINUX_FIRMWARE_QCOM_ADRENO),y) 38 LINUX_FIRMWARE_FILES += qcom/a* 39 LINUX_FIRMWARE_ALL_LICENSE_FILES += LICENSE.qcom qcom/NOTICE.txt 40 endifQuoted 5 lines
So, because we have a glob here, the following symlinks will not be created:
Link: a300_pfp.fw -> qcom/a300_pfp.fw Link: a300_pm4.fw -> qcom/a300_pm4.fwI don't get why they wouldn't be created. The symlink creation is done after the files have been installed to the target directory, and is done based on what has been installed. So the definition used to get those firmware installed (being full names or not) shouldn't matter.
With this patch, and
BR2_PACKAGE_LINUX_FIRMWARE_QCOM_ADRENOenabled, I get the following files in $(TARGET_DIR)/lib/firmware:./output/target/lib/firmware/
??? a300_pfp.fw -> qcom/a300_pfp.fw
??? a300_pm4.fw -> qcom/a300_pm4.fw
??? qcom
Quoted 11 lines
??? a300_pfp.fw
??? a300_pm4.fw
??? a530_pfp.fw
??? a530_pm4.fw
??? a530v3_gpmu.fw2
??? a530_zap.b00
??? a530_zap.b01
??? a530_zap.b02
??? a530_zap.mdt
??? a630_gmu.bin
??? a630_sqe.fwIndeed, and so do I. I just got confused between the 'Link:' tags and the usual order of arguments to ln: they are reversed, and I looke at file and links in the wrong location...
Quoted 12 lines
Quoted 12 lines
package/linux-firmware/linux-firmware.mk define LINUX_FIRMWARE_INSTALL_TARGET_CMDS · line 625
Quoted 9 lines
package/linux-firmware/linux-firmware.mk +9
-0+# Some firmware are distributed as a symlink, for drivers to load them using a +# defined name other than the real one. Since 9cfefbd7fbda ("Remove duplicate +# symlinks") those symlink aren't distributed in linux-firmware but are created +# automatically by its copy-firmware.sh script during the installation, which +# parses the WHENCE file where symlinks are described. We follow the same logic +# here, adding symlink only for firmwares installed in the target directory. +# The grep/sed parsing is taken from the script mentioned before. +define LINUX_FIRMWARE_CREATE_SYMLINKS + grep -E '^Link:' $(@D)/WHENCE | sed -e's/^Link: *//g' -e's/-> //g' | while read f d; do \You can combine the grep and sed into a single sed call:
Quoted 5 lines
Quoted 1 line
sed -r -e '/^Link: (.+) -> (.+)$$/!d; s//\1 \2/' $(@D)/WHENCEThe reasoning here was to reuse with as little change as possible the code used in Linux-firmware's
copy-firmware.shscript, to be easily maintainable should they decide to update the link definition format.
I see, and that's a good reason. Yet, I've looked at the copy-firmware script, and I am not impressed... I find the above to be more readable.
If the upstream script was callable with a list of firmware files to install, then that would wonderfull, as we could re-use it instead of reimplementing the logic ourselves (hint, hint ;-) ).
Quoted 2 lines
I'm not pushing for either solution, I can change this for v2 if you believe the two command should still be combined.
no need for a v2: the major road-block (the globs above) is not. I can fix the sed and hook stuff locally (I already did it).
Thanks! :-)
Regards,
Yann E. MORIN.
Quoted 29 lines
Quoted 13 lines
package/linux-firmware/linux-firmware.mk define LINUX_FIRMWARE_INSTALL_TARGET_CMDS · line 632
Quoted 7 lines
package/linux-firmware/linux-firmware.mk +7
-0+ if test -f $(TARGET_DIR)/lib/firmware/$$d; then \ + ln -sf $$d $(TARGET_DIR)/lib/firmware/$$f ; \ + fi ; \ + done +endef + +LINUX_FIRMWARE_POST_INSTALL_TARGET_HOOKS += LINUX_FIRMWARE_CREATE_SYMLINKSUse of a hook is not really useful here: we are using the generic infra, so we can write whatever we want in the
_INSTALL_TARGET_CMDSanyway, so just define the macro before_INSTALL_TARGET_CMDSand call it from there, like the other parts (notice the rename of the macro to match):Quoted 6 lines
621 define LINUX_FIRMWARE_INSTALL_TARGET_CMDS 622 mkdir -p $(TARGET_DIR)/lib/firmware 623 $(LINUX_FIRMWARE_INSTALL_FILES) 624 $(LINUX_FIRMWARE_INSTALL_DIRS) 625 $(LINUX_FIRMWARE_INSTALL_SYMLINKS) 626 endefOK, will do!
Thanks!
AntoineSignature · 3 lines
Antoine T?nart, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 561 099 427 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'
PATCH 1/1 package/linux-firmware: fix symlink support
Yann,
Quoted 15 lines · Yann E. MORIN
On 2020-03-03 19:47 +0100, Antoine Tenart spake thusly:
Quoted 14 lines · Yann E. MORIN
Quoted 13 lines
On 2020-03-03 14:33 +0100, Antoine Tenart spake thusly:
package/linux-firmware/linux-firmware.mk define LINUX_FIRMWARE_INSTALL_TARGET_CMDS · line 625
Quoted 9 lines
package/linux-firmware/linux-firmware.mk +9
-0+# Some firmware are distributed as a symlink, for drivers to load them using a +# defined name other than the real one. Since 9cfefbd7fbda ("Remove duplicate +# symlinks") those symlink aren't distributed in linux-firmware but are created +# automatically by its copy-firmware.sh script during the installation, which +# parses the WHENCE file where symlinks are described. We follow the same logic +# here, adding symlink only for firmwares installed in the target directory. +# The grep/sed parsing is taken from the script mentioned before. +define LINUX_FIRMWARE_CREATE_SYMLINKS + grep -E '^Link:' $(@D)/WHENCE | sed -e's/^Link: *//g' -e's/-> //g' | while read f d; do \You can combine the grep and sed into a single sed call:
Quoted 8 lines
Quoted 5 lines
Quoted 1 line
sed -r -e '/^Link: (.+) -> (.+)$$/!d; s//\1 \2/' $(@D)/WHENCEThe reasoning here was to reuse with as little change as possible the code used in Linux-firmware's
copy-firmware.shscript, to be easily maintainable should they decide to update the link definition format.I see, and that's a good reason. Yet, I've looked at the copy-firmware script, and I am not impressed... I find the above to be more readable.
That is the least we can say, not impressed :-)
Quoted 5 lines
Quoted 2 lines
I'm not pushing for either solution, I can change this for v2 if you believe the two command should still be combined.
no need for a v2: the major road-block (the globs above) is not. I can fix the sed and hook stuff locally (I already did it).
Thanks!
Antoine
Signature · 3 lines
Antoine T?nart, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
PATCH 1/1 package/linux-firmware: fix symlink support
In reply to Antoine TenartAntoine, All,
On 2020-03-03 14:33 +0100, Antoine Tenart spake thusly:
Quoted 14 lines
Since Linux-firmware's commit
9cfefbd7fbda("Remove duplicate symlinks") symlinks aren't distributed anymore. They are rather created at installation time by a script provided in the project,copy-firmware.sh. The description of the symlinks is done in the WHENCE file. Since the bump to version 20200122, in commit48cc1a89ae04, installation for many firmwares was broken as Buildroot tried to install missing symlinks from Linux-firmware.The fix is not only to remove now missing symlinks, but to add logic to create those symlinks as kernel modules will depend on them. The solution taken by this patch is to create dynamically symlinks based on their description in the WHENCE file *and* only if the file they'll point to was installed in the target directory.
Quoted 3 lines
Fixes 48cc1a89ae04 ("package/linux-firmware: bump to version 20200122")Cc james.hilliard1 at gmail.comSigned-off-by Antoine Tenart <[email protected]>
I did find a problem with some symlinks that is not created.
Link: cxgb4/t4fw.bin -> t4fw-1.24.11.0.bin
The target is itself in the cxgb4/ directory, so your code does not catch it as an isntalled blob, and thus does not create the symlink.
I anyway applied, because it at least fixes the build by dropping the entries tht no longer exist as they are symlinks.
So this patch is just a partial fix.
Care to send an update to also account for the kind of symlinks above?
Thanks!
Regards,
Yann E. MORIN.
Quoted 110 lines
--- package/linux-firmware/linux-firmware.mk | 33 ++++++++++++++---------- 1 file changed, 19 insertions(+), 14 deletions(-)package/linux-firmware/linux-firmware.mk +19
-14@@ -71,9 +71,7 @@ endif # rt2xx ifeq ($(BR2_PACKAGE_LINUX_FIRMWARE_RALINK_RT2XX),y) -# rt3090.bin is a symlink to rt2860.bin -# rt3070.bin is a symlink to rt2870.bin -LINUX_FIRMWARE_FILES += rt2860.bin rt2870.bin rt3070.bin rt3071.bin rt3090.bin +LINUX_FIRMWARE_FILES += rt2860.bin rt2870.bin rt3071.bin LINUX_FIRMWARE_ALL_LICENSE_FILES += LICENCE.ralink-firmware.txt endif@@ -214,8 +212,6 @@ endif # sd8688 ifeq ($(BR2_PACKAGE_LINUX_FIRMWARE_LIBERTAS_SD8688),y) -LINUX_FIRMWARE_FILES += libertas/sd8688.bin libertas/sd8688_helper.bin -# The two files above are but symlinks to those two ones: LINUX_FIRMWARE_FILES += mrvl/sd8688.bin mrvl/sd8688_helper.bin LINUX_FIRMWARE_ALL_LICENSE_FILES += LICENCE.Marvell endif@@ -320,12 +316,10 @@ endif # wl127x ifeq ($(BR2_PACKAGE_LINUX_FIRMWARE_TI_WL127X),y) -# wl1271-nvs.bin is a symlink to wl127x-nvs.bin LINUX_FIRMWARE_FILES += \ ti-connectivity/wl1271-fw-2.bin \ ti-connectivity/wl1271-fw-ap.bin \ ti-connectivity/wl1271-fw.bin \ - ti-connectivity/wl1271-nvs.bin \ ti-connectivity/wl127x-fw-3.bin \ ti-connectivity/wl127x-fw-plt-3.bin \ ti-connectivity/wl127x-nvs.bin \@@ -341,15 +335,12 @@ endif # wl128x ifeq ($(BR2_PACKAGE_LINUX_FIRMWARE_TI_WL128X),y) -# wl1271-nvs.bin and wl12xx-nvs.bin are symlinks to wl127x-nvs.bin LINUX_FIRMWARE_FILES += \ ti-connectivity/wl128x-fw-3.bin \ ti-connectivity/wl128x-fw-ap.bin \ ti-connectivity/wl128x-fw-plt-3.bin \ ti-connectivity/wl128x-fw.bin \ - ti-connectivity/wl1271-nvs.bin \ ti-connectivity/wl128x-nvs.bin \ - ti-connectivity/wl12xx-nvs.bin \ ti-connectivity/wl127x-nvs.bin \ ti-connectivity/wl128x-fw-4-mr.bin \ ti-connectivity/wl128x-fw-4-plt.bin \@@ -363,13 +354,11 @@ endif # wl18xx ifeq ($(BR2_PACKAGE_LINUX_FIRMWARE_TI_WL18XX),y) -# wl1271-nvs.bin is a symlink to wl127x-nvs.bin LINUX_FIRMWARE_FILES += \ ti-connectivity/wl18xx-fw.bin \ ti-connectivity/wl18xx-fw-2.bin \ ti-connectivity/wl18xx-fw-3.bin \ ti-connectivity/wl18xx-fw-4.bin \ - ti-connectivity/wl1271-nvs.bin \ ti-connectivity/wl127x-nvs.bin \ ti-connectivity/TIInit_7.2.31.bts LINUX_FIRMWARE_ALL_LICENSE_FILES += LICENCE.ti-connectivity@@ -568,8 +557,7 @@ LINUX_FIRMWARE_FILES += \ endif ifeq ($(BR2_PACKAGE_LINUX_FIRMWARE_QAT_DH895XCC),y) -# qat_mmp.bin is a symlink to qat_895xcc_mmp.bin -LINUX_FIRMWARE_FILES += qat_895xcc.bin qat_895xcc_mmp.bin qat_mmp.bin +LINUX_FIRMWARE_FILES += qat_895xcc.bin qat_895xcc_mmp.bin LINUX_FIRMWARE_ALL_LICENSE_FILES += LICENCE.qat_firmware endif@@ -626,4 +614,21 @@ define LINUX_FIRMWARE_INSTALL_TARGET_CMDS $(LINUX_FIRMWARE_INSTALL_DIRS) endef +# Some firmware are distributed as a symlink, for drivers to load them using a +# defined name other than the real one. Since 9cfefbd7fbda ("Remove duplicate +# symlinks") those symlink aren't distributed in linux-firmware but are created +# automatically by its copy-firmware.sh script during the installation, which +# parses the WHENCE file where symlinks are described. We follow the same logic +# here, adding symlink only for firmwares installed in the target directory. +# The grep/sed parsing is taken from the script mentioned before. +define LINUX_FIRMWARE_CREATE_SYMLINKS + grep -E '^Link:' $(@D)/WHENCE | sed -e's/^Link: *//g' -e's/-> //g' | while read f d; do \ + if test -f $(TARGET_DIR)/lib/firmware/$$d; then \ + ln -sf $$d $(TARGET_DIR)/lib/firmware/$$f ; \ + fi ; \ + done +endef + +LINUX_FIRMWARE_POST_INSTALL_TARGET_HOOKS += LINUX_FIRMWARE_CREATE_SYMLINKS + $(eval $(generic-package))Signature · 6 lines
2.24.1 _______________________________________________ buildroot mailing list buildroot at busybox.net http://lists.busybox.net/mailman/listinfo/buildroot
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 561 099 427 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'
PATCH 1/1 package/linux-firmware: fix symlink support
Yann,
Quoted 26 lines · Yann E. MORIN
On 2020-03-03 14:33 +0100, Antoine Tenart spake thusly:
Quoted 14 lines
Since Linux-firmware's commit
9cfefbd7fbda("Remove duplicate symlinks") symlinks aren't distributed anymore. They are rather created at installation time by a script provided in the project,copy-firmware.sh. The description of the symlinks is done in the WHENCE file. Since the bump to version 20200122, in commit48cc1a89ae04, installation for many firmwares was broken as Buildroot tried to install missing symlinks from Linux-firmware.The fix is not only to remove now missing symlinks, but to add logic to create those symlinks as kernel modules will depend on them. The solution taken by this patch is to create dynamically symlinks based on their description in the WHENCE file *and* only if the file they'll point to was installed in the target directory.
Quoted 3 lines
Fixes 48cc1a89ae04 ("package/linux-firmware: bump to version 20200122")Cc james.hilliard1 at gmail.comSigned-off-by Antoine Tenart <[email protected]>I did find a problem with some symlinks that is not created.
Link: cxgb4/t4fw.bin -> t4fw-1.24.11.0.binThe target is itself in the cxgb4/ directory, so your code does not catch it as an isntalled blob, and thus does not create the symlink.
Quoted 6 lines
I anyway applied, because it at least fixes the build by dropping the entries tht no longer exist as they are symlinks.
So this patch is just a partial fix.
Care to send an update to also account for the kind of symlinks above?
Good catch! I'll have a look at this.
Thanks,
Antoine
Signature · 3 lines
Antoine T?nart, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
PATCH 1/1 package/linux-firmware: fix symlink support
Quoted 27 lines · Antoine Tenart
Quoted 26 lines · Yann E. MORIN
On 2020-03-03 14:33 +0100, Antoine Tenart spake thusly:
Quoted 14 lines
Since Linux-firmware's commit
9cfefbd7fbda("Remove duplicate symlinks") symlinks aren't distributed anymore. They are rather created at installation time by a script provided in the project,copy-firmware.sh. The description of the symlinks is done in the WHENCE file. Since the bump to version 20200122, in commit48cc1a89ae04, installation for many firmwares was broken as Buildroot tried to install missing symlinks from Linux-firmware.The fix is not only to remove now missing symlinks, but to add logic to create those symlinks as kernel modules will depend on them. The solution taken by this patch is to create dynamically symlinks based on their description in the WHENCE file *and* only if the file they'll point to was installed in the target directory.
Quoted 3 lines
Fixes 48cc1a89ae04 ("package/linux-firmware: bump to version 20200122")Cc james.hilliard1 at gmail.comSigned-off-by Antoine Tenart <[email protected]>I did find a problem with some symlinks that is not created.
Link: cxgb4/t4fw.bin -> t4fw-1.24.11.0.binThe target is itself in the cxgb4/ directory, so your code does not catch it as an isntalled blob, and thus does not create the symlink.
Quoted 8 lines
Quoted 6 lines
I anyway applied, because it at least fixes the build by dropping the entries tht no longer exist as they are symlinks.
So this patch is just a partial fix.
Care to send an update to also account for the kind of symlinks above?
Good catch! I'll have a look at this.
It seems to be fixed by:
- if test -f $(TARGET_DIR)/lib/firmware/$$d; then \
+ if test -f $(TARGET_DIR)/lib/firmware/$$(dirname $$f)/$$d; then \
I'll make some tests tomorrow.
Antoine
Signature · 3 lines
Antoine T?nart, Bootlin Embedded Linux and Kernel engineering https://bootlin.com