Skip to contentSkip to conversation

pkg-generic: Depend on host-{xz, lzip} only for fitting archives

PATCH New Norbert Lange 5 participants 9 messages Mar 17, 2019 – Apr 5, 2019

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/2 pkg-generic: Depend on host-{xz, lzip} only for fitting archives

results in cleaner dependency graphs, and usually omits host-lzip from being built.

---
 package/pkg-generic.mk | 4 ++++
 1 file changed, 4 insertions(+)
package/pkg-generic.mk +4 -0
index 4353bd3868..89da43d5e5 100644
@@ -607,12 +607,16 @@ $(2)_EXTRACT_DEPENDENCIES += $$(BR2_TAR_HOST_DEPENDENCY)
 endif
 
 ifeq ($$(filter host-tar host-skeleton host-xz host-lzip host-fakedate,$(1)),)
+ifneq ($$(filter .xz .lzma,$$(suffix $$($(2)_SOURCE))),)
 $(2)_EXTRACT_DEPENDENCIES += $$(BR2_XZCAT_HOST_DEPENDENCY)
 endif
+endif
 
 ifeq ($$(filter host-tar host-skeleton host-xz host-lzip host-fakedate,$(1)),)
+ifeq ($$(suffix $$($(2)_SOURCE)),.lz)
 $(2)_EXTRACT_DEPENDENCIES += $$(BR2_LZIP_HOST_DEPENDENCY)
 endif
+endif
 
 ifeq ($$(BR2_CCACHE),y)
 ifeq ($$(filter host-tar host-skeleton host-xz host-lzip host-fakedate host-ccache,$(1)),)
Signature · 1 line
2.20.1
#1 lore raw

PATCH v2 2/2 pkg-generic: Cleanup some redundant checks

With the last change, packages will depend only on host-{xz,lzip} if the source archives have filenames requesting the corresponding compressor.

This allows using a single guard.

---
 package/pkg-generic.mk | 5 +----
 1 file changed, 1 insertion(+), 4 deletions(-)
package/pkg-generic.mk +1 -4
index 89da43d5e5..11f16cab18 100644
@@ -604,18 +604,15 @@ endif
 
 ifeq ($$(filter host-tar host-skeleton host-fakedate,$(1)),)
 $(2)_EXTRACT_DEPENDENCIES += $$(BR2_TAR_HOST_DEPENDENCY)
-endif
 
-ifeq ($$(filter host-tar host-skeleton host-xz host-lzip host-fakedate,$(1)),)
 ifneq ($$(filter .xz .lzma,$$(suffix $$($(2)_SOURCE))),)
 $(2)_EXTRACT_DEPENDENCIES += $$(BR2_XZCAT_HOST_DEPENDENCY)
 endif
-endif
 
-ifeq ($$(filter host-tar host-skeleton host-xz host-lzip host-fakedate,$(1)),)
 ifeq ($$(suffix $$($(2)_SOURCE)),.lz)
 $(2)_EXTRACT_DEPENDENCIES += $$(BR2_LZIP_HOST_DEPENDENCY)
 endif
+
 endif
 
 ifeq ($$(BR2_CCACHE),y)
Signature · 1 line
2.20.1
#2 lore raw

PATCH v2 2/2 pkg-generic: Cleanup some redundant checks

Norbert, All,

On 2019-03-17 22:20 +0100, Norbert Lange spake thusly:

Quoted 5 lines

With the last change, packages will depend only on host-{xz,lzip} if the source archives have filenames requesting the corresponding compressor.

This allows using a single guard.

This commit log is an improvement agaisnt the previous one, yet I still had to think a bit too hard to understand the reason that works.

And I think there is an issue with that. Not today, but that opens up a case where we can introduce a subtil bug in the future.

If, say, xz changes its distribution archive from .bz2 to .lz, iand lzip changes theirs from .gz to .xz, then we'd introduce a circular dependency at the make level, and make silently and arbitrarily drops one of the dependencies.

However, now that I think about it, that is of not big consequence: in either case, the build woulld break on the first one we try to extract, and we would notice quite early and quite easily.

Still, I'd like we think a bit harder about those special cases.

Regards,
Yann E. MORIN.

Quoted 36 lines
---
 package/pkg-generic.mk | 5 +----
 1 file changed, 1 insertion(+), 4 deletions(-)
package/pkg-generic.mk +1 -4
index 89da43d5e5..11f16cab18 100644
@@ -604,18 +604,15 @@ endif
 
 ifeq ($$(filter host-tar host-skeleton host-fakedate,$(1)),)
 $(2)_EXTRACT_DEPENDENCIES += $$(BR2_TAR_HOST_DEPENDENCY)
-endif
 
-ifeq ($$(filter host-tar host-skeleton host-xz host-lzip host-fakedate,$(1)),)
 ifneq ($$(filter .xz .lzma,$$(suffix $$($(2)_SOURCE))),)
 $(2)_EXTRACT_DEPENDENCIES += $$(BR2_XZCAT_HOST_DEPENDENCY)
 endif
-endif
 
-ifeq ($$(filter host-tar host-skeleton host-xz host-lzip host-fakedate,$(1)),)
 ifeq ($$(suffix $$($(2)_SOURCE)),.lz)
 $(2)_EXTRACT_DEPENDENCIES += $$(BR2_LZIP_HOST_DEPENDENCY)
 endif
+
 endif
 
 ifeq ($$(BR2_CCACHE),y)
Signature · 6 lines
2.20.1

_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
#3 lore raw

PATCH v2 2/2 pkg-generic: Cleanup some redundant checks

Am Di., 19. M?rz 2019 um 23:03 Uhr schrieb Yann E. MORIN <[email protected]>:

Quoted 13 lines

Norbert, All,

On 2019-03-17 22:20 +0100, Norbert Lange spake thusly:

Quoted 5 lines

With the last change, packages will depend only on host-{xz,lzip} if the source archives have filenames requesting the corresponding compressor.

This allows using a single guard.

This commit log is an improvement agaisnt the previous one, yet I still had to think a bit too hard to understand the reason that works.

Quoted 7 lines

And I think there is an issue with that. Not today, but that opens up a case where we can introduce a subtil bug in the future.

If, say, xz changes its distribution archive from .bz2 to .lz, iand lzip changes theirs from .gz to .xz, then we'd introduce a circular dependency at the make level, and make silently and arbitrarily drops one of the dependencies.

What would you do in that case anyway?
(besides I dont think lzip is getting any traction, or that a sane compressor package should use anything but gzip for "boostrapping").

Quoted 5 lines

However, now that I think about it, that is of not big consequence: in either case, the build woulld break on the first one we try to extract, and we would notice quite early and quite easily.

Still, I'd like we think a bit harder about those special cases.

I really don't think the patch makes things worse.

Norbert

#4 lore raw

PATCH v2 2/2 pkg-generic: Cleanup some redundant checks

In reply to Norbert Lange

Hello Norbert,

Quoted 10 lines · On Sun, 17 Mar 2019 22:20:14 +0100 Norbert Lange <[email protected]> wrote:

With the last change, packages will depend only on host-{xz,lzip} if the source archives have filenames requesting the corresponding compressor.

This allows using a single guard.

---
 package/pkg-generic.mk | 5 +----
 1 file changed, 1 insertion(+), 4 deletions(-)

I hesitated a bit on this. Originally, I was quite in favor of the change and found Yann's opinion a bit too strict. However, after applying the patch, and looking at the code, I ended up feeling a bit like Yann: even though there are less lines of code, the conditions become a bit tricky, while currently, they are perfectly clear.

So I'll mark this patch as Rejected. Of course, if other BR maintainers object, we can revisit, my choice is not super strong.

Best regards,

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
#5 lore raw

PATCH v2 2/2 pkg-generic: Cleanup some redundant checks

In reply to Norbert Lange
Quoted 7 lines · Norbert Lange

With the last change, packages will depend only on host-{xz,lzip} if the source archives have filenames requesting the corresponding compressor.

This allows using a single guard.

I disagree with Yann and Thomas that this makes it less clear. Except, I don't
think you should have even the single guard...

Regarding the potential circular dependency that Yann mentioned: yes, that's
true. But conversely, by adding the filter like we have now, you remove a potentially required dependency. So in the end it's the same thing. And for circular dependencies we have graph-depends that detects them.

package/pkg-generic.mk endif · line 605

Quoted 11 lines
---
 package/pkg-generic.mk | 5 +----
 1 file changed, 1 insertion(+), 4 deletions(-)
package/pkg-generic.mk +0 -0
index 89da43d5e5..11f16cab18 100644
@@ -604,18 +604,15 @@ endif
 
 ifeq ($$(filter host-tar host-skeleton host-fakedate,$(1)),)

This condition is needed because skeleton and fakedate are dependencies of tar,
but host-skeleton and host-fakedate don't actually need tar. However, since there's nothing that detects that host-skeleton and host-fakedate don't have a tarball to extract, we have to remove those dependencies by hand.

package/pkg-generic.mk endif · line 608

Quoted 5 lines
package/pkg-generic.mk +0 -2
 $(2)_EXTRACT_DEPENDENCIES += $$(BR2_TAR_HOST_DEPENDENCY)
-endif
 
-ifeq ($$(filter host-tar host-skeleton host-xz host-lzip host-fakedate,$(1)),)
 ifneq ($$(filter .xz .lzma,$$(suffix $$($(2)_SOURCE))),)

Here, however, there is an explicit dependency. So the condition on tar,
skeleton, fakedate is not needed.

Bottom line: I don't think the endif above should be removed.

Regards,
Arnout

Quoted 13 lines
package/pkg-generic.mk +1 -2
 $(2)_EXTRACT_DEPENDENCIES += $$(BR2_XZCAT_HOST_DEPENDENCY)
 endif
-endif
 
-ifeq ($$(filter host-tar host-skeleton host-xz host-lzip host-fakedate,$(1)),)
 ifeq ($$(suffix $$($(2)_SOURCE)),.lz)
 $(2)_EXTRACT_DEPENDENCIES += $$(BR2_LZIP_HOST_DEPENDENCY)
 endif
+
 endif
 
 ifeq ($$(BR2_CCACHE),y)
#6 lore raw

PATCH v2 1/2 pkg-generic: Depend on host-{xz, lzip} only for fitting archives

In reply to Norbert Lange

Norbert, All,

On 2019-03-17 22:20 +0100, Norbert Lange spake thusly:

Quoted 2 lines

results in cleaner dependency graphs, and usually omits host-lzip from being built.

As for dependency graphs, there will still be a lot of package that may depend on host-xz, and that would still clutter it a bit more than necessary (we have much less packages needing host-lzip, 4 exactly).

But that sure is still a nice improvement.

Quoted 10 lines
---
 package/pkg-generic.mk | 4 ++++
 1 file changed, 4 insertions(+)
package/pkg-generic.mk +0 -0
index 4353bd3868..89da43d5e5 100644

package/pkg-generic.mk $(2)_EXTRACT_DEPENDENCIES += $$(BR2_TAR_HOST_DEPENDENCY) · line 616

Quoted 10 lines
package/pkg-generic.mk +3 -0
@@ -607,12 +607,16 @@ $(2)_EXTRACT_DEPENDENCIES += $$(BR2_TAR_HOST_DEPENDENCY)
 endif
 
 ifeq ($$(filter host-tar host-skeleton host-xz host-lzip host-fakedate,$(1)),)
+ifneq ($$(filter .xz .lzma,$$(suffix $$($(2)_SOURCE))),)
 $(2)_EXTRACT_DEPENDENCIES += $$(BR2_XZCAT_HOST_DEPENDENCY)
 endif
+endif
 
 ifeq ($$(filter host-tar host-skeleton host-xz host-lzip host-fakedate,$(1)),)
+ifeq ($$(suffix $$($(2)_SOURCE)),.lz)

Please use the same pattern here as for the other one. It helps with readability and maintainability.

Regards,
Yann E. MORIN.

Quoted 13 lines
package/pkg-generic.mk +1 -0
 $(2)_EXTRACT_DEPENDENCIES += $$(BR2_LZIP_HOST_DEPENDENCY)
 endif
+endif
 
 ifeq ($$(BR2_CCACHE),y)
 ifeq ($$(filter host-tar host-skeleton host-xz host-lzip host-fakedate host-ccache,$(1)),)
Signature · 6 lines
2.20.1

_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
#7 lore raw

PATCH v2 1/2 pkg-generic: Depend on host-{xz, lzip} only for fitting archives

In reply to Norbert Lange
Quoted 7 lines · On Sun, 17 Mar 2019 22:20:13 +0100 Norbert Lange <[email protected]> wrote:

results in cleaner dependency graphs, and usually omits host-lzip from being built.

---
 package/pkg-generic.mk | 4 ++++
 1 file changed, 4 insertions(+)

I improved the commit log, modified the code as suggested by Yann, and applied. Thanks, this is a very useful change!

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
#8 lore raw

PATCH v2 1/2 pkg-generic: Depend on host-{xz, lzip} only for fitting archives

Quoted 8 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Thomas" == Thomas Petazzoni <[email protected]> writes:

Quoted 3 lines · On Sun, 17 Mar 2019 22:20:13 +0100 Norbert Lange <[email protected]> wrote:

results in cleaner dependency graphs, and usually omits host-lzip from being built.

Quoted 7 lines
Quoted 4 lines

---
package/pkg-generic.mk | 4 ++++

1 file changed, 4 insertions(+)

I improved the commit log, modified the code as suggested by Yann, and applied. Thanks, this is a very useful change!

Committed to 2019.02.x, thanks.

Signature · 1 line
Bye, Peter Korsgaard