Skip to contentSkip to conversation

core/pkg-generic: fix instrumentation for packages that install nowhere

PATCH New Yann E. MORIN 3 participants 4 messages Jun 13, 2021 – Jun 21, 2021

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 core/pkg-generic: fix instrumentation for packages that install nowhere

A (target [0]) package can independently declare installing in various
locations: target, staging, or images. The default is to only install in target.

When a package opts out from installing to target, but does not opts in to install in any other location, the package is not downloaded, extracted, patched, configured, nor built at all. As a consequence, none of the per-step instrumentation is not executed, specifically the listing of files before/after the package sequence.

Down the line, the package infra does not cope well with that situation, because the gathering-install step, the one that synchronises all the optional target, staging, or images install steps, still gets run.

And as #13836 shows, this does not go well:

    /bin/sh: /home/tbuild/myboard/build/foo/.files-list.after: No such file or directory
    make[1]: *** [/home/tbuild/myboard/build/foo/.stamp_installed] Error 1
    make: *** [_all] Error 2

So, we should have ensured that the gathering-install step itself depends on the build step, which would have solved the issue.

However, this bug really illustrates a more fundamental issue: does it even make sense to have a package that installs nothing in any location? Indeed, why even bother with that package to begin with if it will not provide anything at all?

It turns out that yes, this makes sense. We have some packages, that do not install anything at all, and do not even build anything; they are there just to ensure that we can download something that will ultimately be used by another package. This is the case for example for packages that provide linux extensions, like aufs [1].

Additionally, some ugly out-of-tree packages could conceivably install things during the build (or even configure!) steps. That's not unheard of... [2]

So, the solution is to ensure that the gathering-install step does depend on the build step, to trigger the proper dependency chain and have the instrumentation hooks properly run even in that degenerate case.

Fixes #13836

[0] a host package can't opt out of installing anything.
[1] that one is actually missing AUFS_INSTALL_TARGET = NO, so this hides the issue.
[2] even us are not 100% clean on that topic: gcc will install files in staging and target as part of the same step (not the build, granted, but still...)

---
 package/pkg-generic.mk | 1 +
 1 file changed, 1 insertion(+)
package/pkg-generic.mk +1 -0
index 9fbc63d19e..45589bcbb4 100644
@@ -826,6 +826,7 @@ endif
 # human-friendly targets and target sequencing
 $(1):			$(1)-install
 $(1)-install:		$$($(2)_TARGET_INSTALL)
+$$($(2)_TARGET_INSTALL): $$($(2)_TARGET_BUILD)
 
 ifeq ($$($(2)_TYPE),host)
 $$($(2)_TARGET_INSTALL): $$($(2)_TARGET_INSTALL_HOST)
Signature · 1 line
2.25.1
#1 lore raw

PATCH 1/1 core/pkg-generic: fix instrumentation for packages that install nowhere

Yann,

Quoted 10 lines

From: Yann E. MORIN <[email protected]> on behalf of Yann E. MORIN <[email protected]>
Sent: Sunday, June 13, 2021 9:32 AM
To: buildroot at buildroot.org <[email protected]>
Cc: Yann E. MORIN <[email protected]>; Weber, Matthew L Collins <[email protected]>; Thomas Petazzoni <[email protected]>
Subject: [External] [PATCH] core/pkg-generic: fix instrumentation for packages that install nowhere
?
A (target [0]) package can independently declare installing in various
locations: target, staging, or images. The default is to only install
in target.

[snip]

Quoted 10 lines
Fixes #13836

[0] a host package can't opt out of installing anything.
[1] that one is actually missing AUFS_INSTALL_TARGET = NO, so this ??? hides the issue.
[2] even us are not 100% clean on that topic: gcc will install files in ??? staging and target as part of the same step (not the build, granted, ??? but still...)

Quoted 3 lines
author Applied
#2 lore raw

PATCH 1/1 core/pkg-generic: fix instrumentation for packages that install nowhere

Matthew, All,

On 2021-06-15 19:40 +0000, Weber, Matthew L Collins spake thusly:

Quoted 9 lines
Quoted 9 lines

From: Yann E. MORIN <[email protected]> on behalf of Yann E. MORIN <[email protected]>
Sent: Sunday, June 13, 2021 9:32 AM
To: buildroot at buildroot.org <[email protected]>
Cc: Yann E. MORIN <[email protected]>; Weber, Matthew L Collins <[email protected]>; Thomas Petazzoni <[email protected]>
Subject: [External] [PATCH] core/pkg-generic: fix instrumentation for packages that install nowhere
?
A (target [0]) package can independently declare installing in various
locations: target, staging, or images. The default is to only install
in target.

[--SNIP--]

Quoted 9 lines
Quoted 9 lines
Fixes #13836

[0] a host package can't opt out of installing anything.
[1] that one is actually missing AUFS_INSTALL_TARGET = NO, so this ??? hides the issue.
[2] even us are not 100% clean on that topic: gcc will install files in ??? staging and target as part of the same step (not the build, granted, ??? but still...)

Quoted 4 lines
Quoted 3 lines

Applied to master, thanks.

Regards,
Yann E. MORIN.

Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
#3 lore raw

PATCH 1/1 core/pkg-generic: fix instrumentation for packages that install nowhere

In reply to Yann E. MORIN
Quoted 54 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Yann" == Yann E MORIN <[email protected]> writes:

A (target [0]) package can independently declare installing in various
locations: target, staging, or images. The default is to only install in target.

When a package opts out from installing to target, but does not opts in to install in any other location, the package is not downloaded, extracted, patched, configured, nor built at all. As a consequence, none of the per-step instrumentation is not executed, specifically the listing of files before/after the package sequence.

Down the line, the package infra does not cope well with that situation, because the gathering-install step, the one that synchronises all the optional target, staging, or images install steps, still gets run.

And as #13836 shows, this does not go well:

    /bin/sh: /home/tbuild/myboard/build/foo/.files-list.after: No such file or directory
    make[1]: *** [/home/tbuild/myboard/build/foo/.stamp_installed] Error 1
    make: *** [_all] Error 2

So, we should have ensured that the gathering-install step itself depends on the build step, which would have solved the issue.

However, this bug really illustrates a more fundamental issue: does it even make sense to have a package that installs nothing in any location? Indeed, why even bother with that package to begin with if it will not provide anything at all?

It turns out that yes, this makes sense. We have some packages, that do not install anything at all, and do not even build anything; they are there just to ensure that we can download something that will ultimately be used by another package. This is the case for example for packages that provide linux extensions, like aufs [1].

Additionally, some ugly out-of-tree packages could conceivably install things during the build (or even configure!) steps. That's not unheard of... [2]

So, the solution is to ensure that the gathering-install step does depend on the build step, to trigger the proper dependency chain and have the instrumentation hooks properly run even in that degenerate case.

Fixes #13836

[0] a host package can't opt out of installing anything.
[1] that one is actually missing AUFS_INSTALL_TARGET = NO, so this hides the issue.
[2] even us are not 100% clean on that topic: gcc will install files in staging and target as part of the same step (not the build, granted, but still...)

Quoted 3 lines

Committed to 2021.02.x and 2021.05.x, thanks.

Signature · 1 line
Bye, Peter Korsgaard