Wacky dependencies
I wanted to run an experiment with X11 vs GTK-directfb on my platform. I built a complete image with X11 in DISTRO_FEATURES.
When I changed DISTRO_FEATURES to have gtk-directfb, nearly everything was rebuilt! Why on earth do GCC & EGLIBC have to be rebuilt?? This doesn't seem logical at all to me.
Signature · 4 lines
------------------------------------------------------------ Gary Thomas | Consulting for the MLB Associates | Embedded world ------------------------------------------------------------
Quoted 5 lines · On Friday 16 November 2012 07:12:38 Gary Thomas wrote:
I wanted to run an experiment with X11 vs GTK-directfb on my platform. I built a complete image with X11 in
DISTRO_FEATURES.
When I changedDISTRO_FEATURESto have gtk-directfb, nearly everything was rebuilt! Why on earth do GCC & EGLIBC have to be rebuilt?? This doesn't seem logical at all to me.
The build system is not able to determine whether a variable within a recipe depends on a single value within DISTRO_FEATURES, it only knows that it looks at the variable; thus any change to DISTRO_FEATURES will change the hash of all recipes that read DISTRO_FEATURES.
I'd like to fix this but given that we check if DISTRO_FEATURES and other list variables like it contain values using inline python code i.e. ${@base_contains(...)} this seems like it would be tricky to implement in a reliable way.
Cheers,
Paul
Signature · 2 lines
Paul Eggleton Intel Open Source Technology Centre
Quoted 12 lines · Paul Eggleton
Quoted 5 lines · On Friday 16 November 2012 07:12:38 Gary Thomas wrote:
I wanted to run an experiment with X11 vs GTK-directfb on my platform. I built a complete image with X11 in
DISTRO_FEATURES.
When I changedDISTRO_FEATURESto have gtk-directfb, nearly everything was rebuilt! Why on earth do GCC & EGLIBC have to be rebuilt?? This doesn't seem logical at all to me.The build system is not able to determine whether a variable within a recipe depends on a single value within
DISTRO_FEATURES, it only knows that it looks at the variable; thus any change toDISTRO_FEATURESwill change the hash of all recipes that readDISTRO_FEATURES.
Quoted 4 lines
I'd like to fix this but given that we check if
DISTRO_FEATURESand other list variables like it contain values using inline python code i.e. ${@base_contains(...)} this seems like it would be tricky to implement in a reliable way.
In this particular case, I think the problem is that DISTRO_FEATURES contains too many controls. In particular, all of the LIBC settings which *are* important to GCC & EGLIBC. Maybe that part should be kept separate and only have DISTRO_FEATURES contain non-compiler items?
Signature · 4 lines
------------------------------------------------------------ Gary Thomas | Consulting for the MLB Associates | Embedded world ------------------------------------------------------------
Quoted 5 lines · Gary Thomas
In this particular case, I think the problem is that
DISTRO_FEATUREScontains too many controls. In particular, all of the LIBC settings which *are* important to GCC & EGLIBC. Maybe that part should be kept separate and only haveDISTRO_FEATUREScontain non-compiler items?
Changing distro features is not something you do on daily basis, but when you do, the breakage potential is significant, so this really needs to be robust more than efficient. If the check can be made smarter the way Paul suggested, that would have some benefit, but multiplying the variables that make up distro config will make maintaining distros, and packages that depend on distro features, harder and more error prone, I think.
Tomas
Signature · 1 line
http://sleepfive.com
Quoted 8 lines · Gary Thomas
I'd like to fix this but given that we check if
DISTRO_FEATURESand otherQuoted 5 lines
list
variables like it contain values using inline python code i.e. ${@base_contains(...)} this seems like it would be tricky to implement in a
reliable way.
I think you'd probably have to add the appropriate vardepsexclude on a case-by-case basis, unless you wanted to play with the ast module and try to determine what's being passed to base_contains :)
Signature · 1 line
Christopher Larson
Quoted 9 lines · Paul Eggleton
The build system is not able to determine whether a variable within a recipe depends on a single value within
DISTRO_FEATURES, it only knows that it looks at the variable; thus any change toDISTRO_FEATURESwill change the hash of all recipes that readDISTRO_FEATURES.I'd like to fix this but given that we check if
DISTRO_FEATURESand other list variables like it contain values using inline python code i.e. ${@base_contains(...)} this seems like it would be tricky to implement in a reliable way.
I've been playing with variable dependency tracking recently and saw issues like this too. I don't think it can be done compatibly (in the sense of preserving the existing list-in-a-string representation) at all. The problem is that dependencies are only tracked at the variable level and the "logical dependency" is on substructure in the data that isn't visible to the dependency system.
You could do this with the existing data model though, but splitting things out into distinct variables. Test for the presence of "DISTRO_FEATURES_directfb" for example. Or use flags:
"DISTRO_FEATURES[directfb]". Flags would require bitbake modification though, because they aren't part of the taskdeps field in the siguatures.
DISTRO_FEATURES is interpreted outside bitbake, of course. But in a more limited sense this could be done for OVERRIDES, as it's handled internally and could be special-cased in the dependency handling.
Andy
Quoted 25 lines · Andy Ross
Quoted 9 lines · Paul Eggleton
The build system is not able to determine whether a variable within a recipe depends on a single value within
DISTRO_FEATURES, it only knows that it looks at the variable; thus any change toDISTRO_FEATURESwill change the hash of all recipes that readDISTRO_FEATURES.I'd like to fix this but given that we check if
DISTRO_FEATURESand other list variables like it contain values using inline python code i.e. ${@base_contains(...)} this seems like it would be tricky to implement in a reliable way.I've been playing with variable dependency tracking recently and saw issues like this too. I don't think it can be done compatibly (in the sense of preserving the existing list-in-a-string representation) at all. The problem is that dependencies are only tracked at the variable level and the "logical dependency" is on substructure in the data that isn't visible to the dependency system.
You could do this with the existing data model though, but splitting things out into distinct variables. Test for the presence of "
DISTRO_FEATURES_directfb" for example. Or use flags:
"DISTRO_FEATURES[directfb]". Flags would require bitbake modification though, because they aren't part of the taskdeps field in the siguatures.
Quoted 3 lines
DISTRO_FEATURESis interpreted outside bitbake, of course. But in a more limited sense this could be done for OVERRIDES, as it's handled internally and could be special-cased in the dependency handling.
Thanks, I can see that this is a truly "sticky widget".
Sadly, it probably doesn't matter as gtk-directfb isn't really ready for prime time (it does not play nice with my touch screen device), so I'll be switching back to X11 and fighting my battles there.
... off to rebuild the world [again]
Signature · 4 lines
------------------------------------------------------------ Gary Thomas | Consulting for the MLB Associates | Embedded world ------------------------------------------------------------