sstate: allow specifying indirect dependencies to exclude from sysroot

RFC PATCH In review André Draszik 1 participant 2 messages Apr 25, 2018

lore

Share & tools

lore

André Draszik

author

2018-04-25 14:12 UTC #0 lore raw

RFC PATCH 1/1 sstate: allow specifying indirect dependencies to exclude from sysroot

From: André Draszik <[email protected]>

Currently, a dependency on any -native recipe will pull in all dependencies of that -native recipe in the recipe sysroot. This behaviour might not always be wanted, e.g. when that -native recipe depends on build-tools that are not relevant for the current recipe.

This change adds a SSTATE_EXCLUDEDEPS_SYSROOT variable, which will be evaluated for such recursive dependencies to be excluded. The idea is similar to
http://lists.openembedded.org/pipermail/openembedded-core/2018-January/146324.html
except that the list is not hard-coded anymore.

SSTATE_EXCLUDEDEPS_SYSROOT is evaluated as two regular expressions of recipe and dependency to ignore, e.g. in the above flex-native / bison-native use-case, one would specify

    SSTATE_EXCLUDEDEPS_SYSROOT = ".*->(flex|bison)-native"

in layer.conf.

The existing special handling of "-initial" as well as "base-passwd" and "shadow-sysroot" could also be streamlined:

    SSTATE_EXCLUDEDEPS_SYSROOT += "\
        .*->.*-initial.* \
        .*(base-passwd|shadow-sysroot)->.* \
    "

Another anticipated user is meta-java, where certain newer JDKs can only be bootstrapped (built) using older JDKs, but it doesn't make much sense to copy all those older JDKs and their own build tools (ant, etc.) into the sysroot of recipes wanting to be built using the newer JDK (only), e.g.:

    SSTATE_EXCLUDEDEPS_SYSROOT += "\
        openjdk-8-native->(ant-native|attr-native|coreutils-native|icedtea7-native|libxslt-native|make-native|openssl-native|zip-native|unzip-native) \
    "

Signed-off-by André Draszik <[email protected]>

---
 meta/classes/sstate.bbclass | 13 +++++++++++++
 1 file changed, 13 insertions(+)

meta/classes/sstate.bbclass +13 -0

index 68089421f5..62e784e497 100644
@@ -901,6 +901,7 @@ def setscene_depvalid(task, taskdependees, notneeded, d, log=None):
     # task is included in taskdependees too
     # Return - False - We need this dependency
     #        - True - We can skip this dependency
+    import re
 
     def logit(msg, log):
         if log is not None:

@@ -961,6 +962,18 @@ def setscene_depvalid(task, taskdependees, notneeded, d, log=None):
             # Nothing need depend on libc-initial/gcc-cross-initial
             if "-initial" in taskdependees[task][0]:
                 continue
+            # Allow excluding certain recursive dependencies. If a recipe needs it should add a
+            # specific dependency itself, rather than relying on one of its dependees to pull
+            # them in.
+            # See also http://lists.openembedded.org/pipermail/openembedded-core/2018-January/146324.html
+            not_needed = False
+            for excl in (d.getVar('SSTATE_EXCLUDEDEPS_SYSROOT') or "").split():
+                if re.match(excl.split('->', 1)[0], taskdependees[dep][0]):
+                    if re.match(excl.split('->', 1)[1], taskdependees[task][0]):
+                        not_needed = True
+                        break
+            if not_needed:
+                continue
             # For meta-extsdk-toolchain we want all sysroot dependencies
             if taskdependees[dep][0] == 'meta-extsdk-toolchain':
                 return False

Signature · 1 line

2.16.2

André Draszik

author

2018-04-25 14:35 UTC #1 lore raw

I realise that this is basically working around the fact that -native recipes don't get any RDEPENDS treatment AFAICS, except that RDEPENDS == DEPENDS, which is what I want to avoid, at least in certain cases...

I guess if computing proper RDEPENDS for -native recipes was easy, it would have been done already?

Cheers,
Andre'

Quoted 90 lines · André Draszik

From: André Draszik <[email protected]>
Currently, a dependency on any -native recipe will pull in all dependencies of that -native recipe in the recipe sysroot. This behaviour might not always be wanted, e.g. when that -native recipe depends on build-tools that are not relevant for the current recipe.
This change adds a SSTATE_EXCLUDEDEPS_SYSROOT variable, which will be evaluated for such recursive dependencies to be excluded. The idea is similar to
http://lists.openembedded.org/pipermail/openembedded-core/2018-January/
146324.html
except that the list is not hard-coded anymore.
SSTATE_EXCLUDEDEPS_SYSROOT is evaluated as two regular expressions of recipe and dependency to ignore, e.g. in the above flex-native / bison-native use-case, one would specify
    SSTATE_EXCLUDEDEPS_SYSROOT = ".*->(flex|bison)-native"
in layer.conf.
The existing special handling of "-initial" as well as "base-passwd" and "shadow-sysroot" could also be streamlined:
    SSTATE_EXCLUDEDEPS_SYSROOT += "\
        .*->.*-initial.* \
        .*(base-passwd|shadow-sysroot)->.* \
    "
Another anticipated user is meta-java, where certain newer JDKs can only be bootstrapped (built) using older JDKs, but it doesn't make much sense to copy all those older JDKs and their own build tools (ant, etc.) into the sysroot of recipes wanting to be built using the newer JDK (only), e.g.:
    SSTATE_EXCLUDEDEPS_SYSROOT += "\
        openjdk-8-native->(ant-native|attr-native|coreutils-
native|icedtea7-native|libxslt-native|make-native|openssl-native|zip-
native|unzip-native) \
    "
Signed-off-by André Draszik <[email protected]>
---
 meta/classes/sstate.bbclass | 13 +++++++++++++
 1 file changed, 13 insertions(+)
meta/classes/sstate.bbclass +0 -0
index 68089421f5..62e784e497 100644
@@ -901,6 +901,7 @@ def setscene_depvalid(task, taskdependees, notneeded,
d, log=None):
     # task is included in taskdependees too
     # Return - False - We need this dependency
     #        - True - We can skip this dependency
meta/classes/sstate.bbclass +1 -0
+    import re
 
     def logit(msg, log):
         if log is not None:
@@ -961,6 +962,18 @@ def setscene_depvalid(task, taskdependees, notneeded,
d, log=None):
# Nothing need depend on libc-initial/gcc-cross-initial if "-initial" in taskdependees[task][0]:
continue
+ # Allow excluding certain recursive dependencies. If a recipe
needs it should add a
+ # specific dependency itself, rather than relying on one of
its dependees to pull
meta/classes/sstate.bbclass +2 -0
+            # them in.
+            # See also http://lists.openembedded.org/pipermail/openembedd
ed-core/2018-January/146324.html
meta/classes/sstate.bbclass +2 -0
+            not_needed = False
+            for excl in (d.getVar('SSTATE_EXCLUDEDEPS_SYSROOT') or
"").split():
+ if re.match(excl.split('->', 1)[0],
taskdependees[dep][0]):
+ if re.match(excl.split('->', 1)[1],
taskdependees[task][0]):
meta/classes/sstate.bbclass +4 -0
+                        not_needed = True
+                        break
+            if not_needed:
+                continue
             # For meta-extsdk-toolchain we want all sysroot dependencies
             if taskdependees[dep][0] == 'meta-extsdk-toolchain':
                 return False