bitbake.conf: require lz4 instead of lz4c
PATCH 1/1 bitbake.conf: require lz4 instead of lz4c
With bitbake commit 'bb: compress: use lz4 instead of lz4c' we require lz4, not lz4c [1]. Going as far back as Ubuntu 16.04/Fedora 25, they are packaged together so it should be safe to simply adjust the requirement from lz4c to lz4.
Note that the two are largely the same, but upstream has considered lz4c deprecated since at least 2018 [2].
1. https://git.openembedded.org/bitbake/commit/?id=907472034b344e4eb73cfd43059a413469f52e1c
2. https://github.com/lz4/lz4/pull/553
--- meta/conf/bitbake.conf | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
meta/conf/bitbake.conf +1 -1
@@ -517,7 +517,7 @@ HOSTTOOLS += " \ [ ar as awk basename bash bunzip2 bzip2 cat chgrp chmod chown chrpath cmp comm cp cpio \ cpp cut date dd diff diffstat dirname du echo egrep env expand expr false \ fgrep file find flock g++ gawk gcc getconf getopt git grep gunzip gzip \ - head hostname iconv id install ld ldd ln ls lz4c make md5sum mkdir mkfifo mknod \ + head hostname iconv id install ld ldd ln ls lz4 make md5sum mkdir mkfifo mknod \ mktemp mv nm objcopy objdump od patch perl pr printf pwd \ python3 pzstd ranlib readelf readlink realpath rm rmdir rpcgen sed seq sh \ sha1sum sha224sum sha256sum sha384sum sha512sum \
Signature · 1 line
2.46.1
Quoted 22 lines · Justin Bronder via lists.openembedded.org
With bitbake commit 'bb: compress: use lz4 instead of lz4c' we require lz4, not lz4c [1]. Going as far back as Ubuntu 16.04/Fedora 25, they are packaged together so it should be safe to simply adjust the requirement from lz4c to lz4.
Note that the two are largely the same, but upstream has considered lz4c deprecated since at least 2018 [2].
1. https://git.openembedded.org/bitbake/commit/?id=907472034b344e4eb73cfd43059a413469f52e1c
2. https://github.com/lz4/lz4/pull/553Signed-off-by Justin Bronder <[email protected]>---
meta/conf/bitbake.conf | 2 +-1 file changed, 1 insertion(+), 1 deletion(-)meta/conf/bitbake.conf +0
-0
meta/conf/bitbake.conf HOSTTOOLS += " \ · line 523
Quoted 8 lines
meta/conf/bitbake.conf +1
-1@@ -517,7 +517,7 @@ HOSTTOOLS += " \ [ ar as awk basename bash bunzip2 bzip2 cat chgrp chmod chown chrpath cmp comm cp cpio \ cpp cut date dd diff diffstat dirname du echo egrep env expand expr false \ fgrep file find flock g++ gawk gcc getconf getopt git grep gunzip gzip \ - head hostname iconv id install ld ldd ln ls lz4c make md5sum mkdir mkfifo mknod \ + head hostname iconv id install ld ldd ln ls lz4 make md5sum mkdir mkfifo mknod \ mktemp mv nm objcopy objdump od patch perl pr printf pwd \ python3 pzstd ranlib readelf readlink realpath rm rmdir rpcgen sed seq sh \ sha1sum sha224sum sha256sum sha384sum sha512sum \
This caused a failure in barebox:
https://valkyrie.yoctoproject.org/#/builders/23/builds/409/steps/14/logs/stdio
Cheers,
Richard
Quoted 34 lines · Richard Purdie
Quoted 22 lines · Justin Bronder via lists.openembedded.org
With bitbake commit 'bb: compress: use lz4 instead of lz4c' we require lz4, not lz4c [1].� Going as far back as Ubuntu 16.04/Fedora 25, they are packaged together so it should be safe to simply adjust the requirement from lz4c to lz4.
Note that the two are largely the same, but upstream has considered lz4c deprecated since at least 2018 [2].
1. https://git.openembedded.org/bitbake/commit/?id=907472034b344e4eb73cfd43059a413469f52e1c
2. https://github.com/lz4/lz4/pull/553Signed-off-by Justin Bronder <[email protected]>---
�meta/conf/bitbake.conf | 2 +-
�1 file changed, 1 insertion(+), 1 deletion(-)meta/conf/bitbake.conf +0
-0meta/conf/bitbake.conf HOSTTOOLS += " \ · line 523
Quoted 8 lines
meta/conf/bitbake.conf +1
-1@@ -517,7 +517,7 @@ HOSTTOOLS += " \ ���� [ ar as awk basename bash bunzip2 bzip2 cat chgrp chmod chown chrpath cmp comm cp cpio \ ���� cpp cut date dd diff diffstat dirname du echo egrep env expand expr false \ ���� fgrep file find flock g++ gawk gcc getconf getopt git grep gunzip gzip \ -��� head hostname iconv id install ld ldd ln ls lz4c make md5sum mkdir mkfifo mknod \ +��� head hostname iconv id install ld ldd ln ls lz4 make md5sum mkdir mkfifo mknod \ ���� mktemp mv nm objcopy objdump od patch perl pr printf pwd \ ���� python3 pzstd ranlib readelf readlink realpath rm rmdir rpcgen sed seq sh \ ���� sha1sum sha224sum sha256sum sha384sum sha512sum \This caused a failure in barebox:
I see, sorry about that.
What would be the recommended path forward?
- Add lz4c to HOSTTOOLS_NONFATAL so it's still symlinked. This would fix barebox and any other recipes that still depend on lz4c on distributions where lz4c is still packaged but leave others without an early warning system.
- Submit patches to barebox and others to use lz4 instead of lz4c. This is probably more correct but will leave Tumbleweed unsupported by default with other distros to presumably follow.
- Convince Tumbleweed (and others) to continue packaging lz4c.
Signature · 1 line
Justin Bronder
Quoted 53 lines · Justin Bronder
Quoted 45 lines · Richard Purdie
Quoted 31 lines · Justin Bronder via lists.openembedded.org
With bitbake commit 'bb: compress: use lz4 instead of lz4c' we require
lz4, not lz4c [1]. Going as far back as Ubuntu 16.04/Fedora 25, they
are packaged together so it should be safe to simply adjust the requirement from lz4c to lz4.Note that the two are largely the same, but upstream has considered lz4c
deprecated since at least 2018 [2].1.
https://git.openembedded.org/bitbake/commit/?id=907472034b344e4eb73cfd43059a413469f52e1c
2. https://github.com/lz4/lz4/pull/553Signed-off-by Justin Bronder <[email protected]>---
meta/conf/bitbake.conf | 2 +-1 file changed, 1 insertion(+), 1 deletion(-)meta/conf/bitbake.conf +0
-0@@ -517,7 +517,7 @@ HOSTTOOLS += " \[ ar as awk basename bash bunzip2 bzip2 cat chgrp chmod chown chrpath cmp comm cp cpio \
cpp cut date dd diff diffstat dirname du echo egrep env expand expr false \
fgrep file find flock g++ gawk gcc getconf getopt git grep gunzip gzip \Quoted 9 lines
- head hostname iconv id install ld ldd ln ls lz4c make md5sum mkdir mkfifo mknod \
+ head hostname iconv id install ld ldd ln ls lz4 make md5sum mkdir mkfifo mknod \
mktemp mv nm objcopy objdump od patch perl pr printf pwd \
python3 pzstd ranlib readelf readlink realpath rm rmdir rpcgen sed seq sh \
sha1sum sha224sum sha256sum sha384sum sha512sum \This caused a failure in barebox:
I see, sorry about that.
What would be the recommended path forward?
Quoted 7 lines
- Add lz4c to
HOSTTOOLS_NONFATALso it's still symlinked. This would fix barebox and any other recipes that still depend on lz4c on distributions where lz4c is still packaged but leave others without an early warning system.
- Submit patches to barebox and others to use lz4 instead of lz4c. This is probably more correct but will leave Tumbleweed unsupported by default with other distros to presumably follow.
barebox only recently merged so I think it should be fine to see if we can switch barebox to use lz4. How would that leave Tumbleweed supported?
Cheers,
Richard
Quoted 62 lines · Richard Purdie
Quoted 53 lines · Justin Bronder
Quoted 45 lines · Richard Purdie
Quoted 31 lines · Justin Bronder via lists.openembedded.org
With bitbake commit 'bb: compress: use lz4 instead of lz4c' we require
lz4, not lz4c [1].� Going as far back as Ubuntu 16.04/Fedora 25, they
are packaged together so it should be safe to simply adjust the requirement from lz4c to lz4.Note that the two are largely the same, but upstream has considered lz4c
deprecated since at least 2018 [2].1.
https://git.openembedded.org/bitbake/commit/?id=907472034b344e4eb73cfd43059a413469f52e1c
2. https://github.com/lz4/lz4/pull/553Signed-off-by Justin Bronder <[email protected]>---
�meta/conf/bitbake.conf | 2 +-
�1 file changed, 1 insertion(+), 1 deletion(-)meta/conf/bitbake.conf +0
-0@@ -517,7 +517,7 @@ HOSTTOOLS += " \���� [ ar as awk basename bash bunzip2 bzip2 cat chgrp chmod chown chrpath cmp comm cp cpio \
���� cpp cut date dd diff diffstat dirname du echo egrep env expand expr false \
���� fgrep file find flock g++ gawk gcc getconf getopt git grep gunzip gzip \Quoted 9 lines
-��� head hostname iconv id install ld ldd ln ls lz4c make md5sum mkdir mkfifo mknod \
+��� head hostname iconv id install ld ldd ln ls lz4 make md5sum mkdir mkfifo mknod \
���� mktemp mv nm objcopy objdump od patch perl pr printf pwd \
���� python3 pzstd ranlib readelf readlink realpath rm rmdir rpcgen sed seq sh \
���� sha1sum sha224sum sha256sum sha384sum sha512sum \This caused a failure in barebox:
I see, sorry about that.
What would be the recommended path forward?
Quoted 7 lines
- Add lz4c to
HOSTTOOLS_NONFATALso it's still symlinked.� This would fix barebox and any other recipes that still depend on lz4c on distributions where lz4c is still packaged but leave others without an early warning system.
- Submit patches to barebox and others to use lz4 instead of lz4c.� This is probably more correct but will leave Tumbleweed unsupported by default with other distros to presumably follow.
Quoted 3 lines
barebox only recently merged so I think it should be fine to see if we can switch barebox to use lz4. How would that leave Tumbleweed supported?
Ok, I've submitted a quite simple patch to barebox upstream and I'll resubmit this along with that patch. Pending other recipes requiring lz4c, that will fix things for Tumbleweed.
Signature · 1 line
Justin Bronder
Can we rather make the affected recipe depend on the native tool rather than the host one? And pls file a ticket with barebox upstream or maybe the maintainer can respond right here.
Alex
Quoted 76 lines · Justin Bronder via lists.openembedded.org
Quoted 1 line · Richard Purdie
On Tue, 2024-11-05 at 10:37 -0500, Justin Bronder via
Quoted 6 lines · lists.openembedded.org
Quoted 6 lines
With bitbake commit 'bb: compress: use lz4 instead of lz4c' we require lz4, not lz4c [1]. Going as far back as Ubuntu 16.04/Fedora 25, they are packaged together so it should be safe to simply adjust the requirement from lz4c to lz4.
Note that the two are largely the same, but upstream has considered
lz4c
Quoted 3 lines
Quoted 3 lines
deprecated since at least 2018 [2].
1.
https://git.openembedded.org/bitbake/commit/?id=907472034b344e4eb73cfd43059a413469f52e1c
Quoted 7 lines
Quoted 7 lines
2. https://github.com/lz4/lz4/pull/553
Signed-off-by Justin Bronder <[email protected]>--- meta/conf/bitbake.conf | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)Quoted 6 lines
meta/conf/bitbake.conf HOSTTOOLS += " \ · line 517
Quoted 6 lines
meta/conf/bitbake.conf +0
-0@@ -517,7 +517,7 @@ HOSTTOOLS += " \ [ ar as awk basename bash bunzip2 bzip2 cat chgrp chmod chownchrpath cmp comm cp cpio \
Quoted 1 line
meta/conf/bitbake.conf HOSTTOOLS += " \ · line 518
Quoted 1 line
meta/conf/bitbake.conf +0
-0cpp cut date dd diff diffstat dirname du echo egrep env expandexpr false \
Quoted 1 line
meta/conf/bitbake.conf HOSTTOOLS += " \ · line 519
Quoted 1 line
meta/conf/bitbake.conf +0
-0fgrep file find flock g++ gawk gcc getconf getopt git grep gunzipgzip \
Quoted 1 line
meta/conf/bitbake.conf HOSTTOOLS += " \ · line 520
Quoted 1 line
meta/conf/bitbake.conf +0
-1- head hostname iconv id install ld ldd ln ls lz4c make md5summkdir mkfifo mknod \
Quoted 1 line
meta/conf/bitbake.conf HOSTTOOLS += " \ · line 520
Quoted 1 line
meta/conf/bitbake.conf +1
-0+ head hostname iconv id install ld ldd ln ls lz4 make md5sum mkdirmkfifo mknod \
Quoted 2 lines
meta/conf/bitbake.conf HOSTTOOLS += " \ · line 522
Quoted 2 lines
meta/conf/bitbake.conf +0
-0mktemp mv nm objcopy objdump od patch perl pr printf pwd \ python3 pzstd ranlib readelf readlink realpath rm rmdir rpcgensed seq sh \
Quoted 6 lines
meta/conf/bitbake.conf HOSTTOOLS += " \ · line 523
Quoted 2 lines
meta/conf/bitbake.conf +0
-0sha1sum sha224sum sha256sum sha384sum sha512sum \This caused a failure in barebox:
https://valkyrie.yoctoproject.org/#/builders/23/builds/409/steps/14/logs/stdio
Quoted 1 line
I see, sorry about that.
What would be the recommended path forward?
- Add lz4c to
HOSTTOOLS_NONFATALso it's still symlinked. This would fix barebox and any other recipes that still depend on lz4c on distributions where lz4c is still packaged but leave others without an early warning system.
- Submit patches to barebox and others to use lz4 instead of lz4c. This is probably more correct but will leave Tumbleweed unsupported by default with other distros to presumably follow.
- Convince Tumbleweed (and others) to continue packaging lz4c.--
Justin Bronder-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#206790):
https://lists.openembedded.org/g/openembedded-core/message/206790
Mute This Topic: https://lists.openembedded.org/mt/109427109/1686489
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ [email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Hi Alex,
Am Mittwoch, dem 06.11.2024 um 17:03 +0100 schrieb Alexander Kanavin:
Quoted 1 line
Can we rather make the affected recipe depend on the native tool rather than the host one?
we could and I agree this would be the more correct way.
The only concern I have is that this would require adding dependencies for all potential compression methods currently supported by barebox (lz4, lzo, gzip, xz).
Or I have to add some more pseudo-PACKAGECONFIG options and enable only lz4 by default. However, I fear that this would quickly result in accidentally using the host tools again. Not sure if there is local way of filtering out allowed host tools?..
Quoted 1 line
And pls file a ticket with barebox upstream or maybe the maintainer can respond right here.
Thankfully, Justin already contributed the fix to barebox upstream in the meantime and as far as I can see, it was quickly merged and should thus be part of the November release.
Regards, Enrico
Quoted 56 lines
Alex
Quoted 52 lines · Justin Bronder via lists.openembedded.org
Quoted 35 lines · Richard Purdie
Quoted 22 lines · Justin Bronder via lists.openembedded.org
With bitbake commit 'bb: compress: use lz4 instead of lz4c' we require lz4, not lz4c [1]. Going as far back as Ubuntu 16.04/Fedora 25, they are packaged together so it should be safe to simply adjust the requirement from lz4c to lz4.
Note that the two are largely the same, but upstream has considered lz4c deprecated since at least 2018 [2].
1. https://git.openembedded.org/bitbake/commit/?id=907472034b344e4eb73cfd43059a413469f52e1c
2. https://github.com/lz4/lz4/pull/553Signed-off-by Justin Bronder <[email protected]>---
meta/conf/bitbake.conf | 2 +-1 file changed, 1 insertion(+), 1 deletion(-)meta/conf/bitbake.conf +0
-0@@ -517,7 +517,7 @@ HOSTTOOLS += " \[ ar as awk basename bash bunzip2 bzip2 cat chgrp chmod chown chrpath cmp comm cp cpio
Quoted 9 lines
\
cpp cut date dd diff diffstat dirname du echo egrep env expand expr false \
fgrep file find flock g++ gawk gcc getconf getopt git grep gunzip gzip \
- head hostname iconv id install ld ldd ln ls lz4c make md5sum mkdir mkfifo mknod \
+ head hostname iconv id install ld ldd ln ls lz4 make md5sum mkdir mkfifo mknod \
mktemp mv nm objcopy objdump od patch perl pr printf pwd \
python3 pzstd ranlib readelf readlink realpath rm rmdir rpcgen sed seq sh \
sha1sum sha224sum sha256sum sha384sum sha512sum \This caused a failure in barebox:
I see, sorry about that.
What would be the recommended path forward?
- Add lz4c to
HOSTTOOLS_NONFATALso it's still symlinked. This would fix barebox and any other recipes that still depend on lz4c on distributions where lz4c is still packaged but leave others without an early warning system.
- Submit patches to barebox and others to use lz4 instead of lz4c. This is probably more correct but will leave Tumbleweed unsupported by default with other distros to presumably follow.
- Convince Tumbleweed (and others) to continue packaging lz4c.
Signature · 4 lines
Pengutronix e.K. | Enrico Jörns | Embedded Linux Consulting & Support | https://www.pengutronix.de/ | Steuerwalder Str. 21 | Phone: +49-5121-206917-180 | 31137 Hildesheim, Germany | Fax: +49-5121-206917-9 |
Quoted 2 lines · Enrico Jörns
Thankfully, Justin already contributed the fix to barebox upstream in the meantime and as far as I can see, it was quickly merged and should thus be part of the November release.
I'd say you could resend the above patch with a barebox backport.
Alex
Am Dienstag, dem 12.11.2024 um 14:34 +0100 schrieb Alexander Kanavin:
Quoted 6 lines · Enrico Jörns
Quoted 3 lines
Thankfully, Justin already contributed the fix to barebox upstream in the meantime and as far as I
can see, it was quickly merged and should thus be part of the November release.I'd say you could resend the above patch with a barebox backport.
To clarify: Justin already did this in a v2 and the patch is in oe-core upstream.
I read this as meaning that you no longer consider a native lz4 dependency necessary.
Thanks and regards, Enrico
Quoted 2 lines
Alex
Signature · 4 lines
Pengutronix e.K. | Enrico Jörns | Embedded Linux Consulting & Support | https://www.pengutronix.de/ | Steuerwalder Str. 21 | Phone: +49-5121-206917-180 | 31137 Hildesheim, Germany | Fax: +49-5121-206917-9 |
Hi I'm seeing a build error on latest master when building Linux 6.10 due to missing lz4c tool.
I see it's still used even in Linux 6.12: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/Makefile?h=v6.12.1#n511
On my build host, lz4c is symlinked to lz4. But I suspect the issue is lz4c was removed from HOSTTOOLS.
Anyone else seeing this problem?
Quoted 7 lines · Gu�ni M�r Gilbert via lists.openembedded.org
Hi I'm seeing a build error on latest master when building Linux 6.10 due to missing lz4c tool.
I see it's still used even in Linux 6.12: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/Makefile?h=v6.12.1#n511
On my build host, lz4c is symlinked to lz4. But I suspect the issue is lz4c was removed from HOSTTOOLS.
Anyone else seeing this problem?
Yes, I can reproduce it when setting CONFIG_KERNEL_LZ4. I've sent a patch upstream. If they are receptive I'll submit it for linux-yocto. But perhaps in the meantime we should add lz4c to HOSTTOOLS_NONFATAL?
https://lore.kernel.org/linux-kbuild/[email protected]/T/#u
Signature · 1 line
Justin Bronder
Hi Justin, thanks for submitting a patch upstream. I can confirm I have this in my defconfig: “CONFIG_KERNEL_LZ4=y”
Quoted 11 lines · Justin Bronder
Quoted 7 lines · Guðni Már Gilbert via lists.openembedded.org
Hi I'm seeing a build error on latest master when building Linux 6.10 due to missing lz4c tool.
I see it's still used even in Linux 6.12: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/Makefile?h=v6.12.1#n511
On my build host, lz4c is symlinked to lz4. But I suspect the issue is lz4c was removed from HOSTTOOLS.
Anyone else seeing this problem?
Yes, I can reproduce it when setting
CONFIG_KERNEL_LZ4. I've sent a patch upstream. If they are receptive I'll submit it for linux-yocto.
Ok, this got merged to 6.13-rc kernel series, but it's now broken for all the downstream BSPs and older kernel versions that use LZ4, besides linux-yocto. Sigh...
Quoted 1 line
But perhaps in the meantime we should add lz4c to
HOSTTOOLS_NONFATAL?
Was there a patch submitted to have both lz4 and lz4c listed in HOSTTOOLS* (at least temporatily until this gets propagated everywhere)?
Signature · 1 line
Denys
Quoted 18 lines · Denys Dmytriyenko
Quoted 11 lines · Justin Bronder
Quoted 7 lines · Gu�ni M�r Gilbert via lists.openembedded.org
Hi I'm seeing a build error on latest master when building Linux 6.10 due to missing lz4c tool.
I see it's still used even in Linux 6.12: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/Makefile?h=v6.12.1#n511
On my build host, lz4c is symlinked to lz4. But I suspect the issue is lz4c was removed from HOSTTOOLS.
Anyone else seeing this problem?
Yes, I can reproduce it when setting
CONFIG_KERNEL_LZ4. I've sent a patch upstream. If they are receptive I'll submit it for linux-yocto.Ok, this got merged to 6.13-rc kernel series, but it's now broken for all the downstream BSPs and older kernel versions that use LZ4, besides linux-yocto. Sigh...
Quoted 4 lines
Quoted 1 line
But perhaps in the meantime we should add lz4c to
HOSTTOOLS_NONFATAL?Was there a patch submitted to have both lz4 and lz4c listed in HOSTTOOLS* (at least temporatily until this gets propagated everywhere)?
I sent one just now. Apologies for the annoyances this has caused. No good deed goes unpunished ;)
https://lore.kernel.org/openembedded-core/[email protected]/T/#u
Signature · 1 line
Justin Bronder