Native vs not
As pointed out in another thread, I'm trying to build a native package which sends "I need native" ripples throughout much of the Poky infrastructure.
Does having a native version available for any given package incur a cost? If not, would patches for [all of] the packages I need be acceptable?
So far, nearly all of the affected packages built fine, just adding native to BBCLASSEXTEND. Many already build nativesdk versions already.
Signature · 4 lines
------------------------------------------------------------ Gary Thomas | Consulting for the MLB Associates | Embedded world ------------------------------------------------------------
Quoted 9 lines · Gary Thomas
As pointed out in another thread, I'm trying to build a native package which sends "I need native" ripples throughout much of the Poky infrastructure.
Does having a native version available for any given package incur a cost? If not, would patches for [all of] the packages I need be acceptable?
So far, nearly all of the affected packages built fine, just adding native to BBCLASSEXTEND. Many already build nativesdk versions already.
There is a cost incurred by doing this since it does increase parse time and this is something user exposed which we do try and keep under control.
Having said that, the BBCLASSEXTEND technology has a lot less overhead than some of the older approaches to native/sdk recipes.
The main reason I've been against native everywhere is that having native versions available makes it far too easy for people to add native dependencies which encourage feature creep without thinking through the huge additional dependency chains, the extra build time and other implications. Often there are slightly more difficult but worthwhile ways we can avoid the native dependency.
Cheers,
Richard
Quoted 11 lines · Richard Purdie
Quoted 9 lines · Gary Thomas
As pointed out in another thread, I'm trying to build a native package which sends "I need native" ripples throughout much of the Poky infrastructure.
Does having a native version available for any given package incur a cost? If not, would patches for [all of] the packages I need be acceptable?
So far, nearly all of the affected packages built fine, just adding native to BBCLASSEXTEND. Many already build nativesdk versions already.
Quoted 6 lines
There is a cost incurred by doing this since it does increase parse time and this is something user exposed which we do try and keep under control.
Having said that, the BBCLASSEXTEND technology has a lot less overhead than some of the older approaches to native/sdk recipes.
As is, I created a separate layer with a bunch of bbappend files that are only
BBCLASSEXTEND += " native "
I suppose if I never needed them, I could just not enable that layer.
Quoted 7 lines
The main reason I've been against native everywhere is that having native versions available makes it far too easy for people to add native dependencies which encourage feature creep without thinking through the huge additional dependency chains, the extra build time and other implications. Often there are slightly more difficult but worthwhile ways we can avoid the native dependency.
Such as? I started down this path needing a native tool (which admittedly came from an OE recipe librsvg) which then cascaded into cairo-native and beyond, totally 22 packages!. If I knew of a short circuit for this, I'd certainly entertain it.
Signature · 4 lines
------------------------------------------------------------ Gary Thomas | Consulting for the MLB Associates | Embedded world ------------------------------------------------------------
Quoted 34 lines · Gary Thomas
Quoted 13 lines · Richard Purdie
Quoted 11 lines · Gary Thomas
As pointed out in another thread, I'm trying to build a native package which sends "I need native" ripples throughout much of the Poky infrastructure.
Does having a native version available for any given package incur a cost?
If not, would patches for [all of] the packages I need be acceptable?So far, nearly all of the affected packages built fine, just adding native
to BBCLASSEXTEND. Many already build nativesdk versions already.Quoted 6 lines
There is a cost incurred by doing this since it does increase parse time and this is something user exposed which we do try and keep under control.
Having said that, the BBCLASSEXTEND technology has a lot less overhead than some of the older approaches to native/sdk recipes.
As is, I created a separate layer with a bunch of bbappend files that are only
BBCLASSEXTEND += " native "
I suppose if I never needed them, I could just not enable that layer.Quoted 7 lines
The main reason I've been against native everywhere is that having native versions available makes it far too easy for people to add native dependencies which encourage feature creep without thinking through the huge additional dependency chains, the extra build time and other implications. Often there are slightly more difficult but worthwhile ways we can avoid the native dependency.
Quoted 4 lines
Such as? I started down this path needing a native tool (which admittedly came from an OE recipe librsvg) which then cascaded into cairo-native and beyond, totally 22 packages!. If I knew of a short circuit for this, I'd certainly entertain it.
Having just skimmed the OE recipes, yeah, there we just build a ton of stuff in librsvg-native. But I would start by looking at what we can pass to configure to disable things as part of the host tool build.
Doing some git grep'ing I see there's just a few things in OE that need librsvg-native so perhaps we can build a much more limited native recipe instead and still generate what we need to.
Signature · 2 lines
Tom Rini Mentor Graphics Corporation
Quoted 40 lines · Tom Rini
Quoted 34 lines · Gary Thomas
Quoted 13 lines · Richard Purdie
Quoted 11 lines · Gary Thomas
As pointed out in another thread, I'm trying to build a native package which sends "I need native" ripples throughout much of the Poky infrastructure.
Does having a native version available for any given package incur a cost?
If not, would patches for [all of] the packages I need be acceptable?So far, nearly all of the affected packages built fine, just adding native
to BBCLASSEXTEND. Many already build nativesdk versions already.Quoted 6 lines
There is a cost incurred by doing this since it does increase parse time and this is something user exposed which we do try and keep under control.
Having said that, the BBCLASSEXTEND technology has a lot less overhead than some of the older approaches to native/sdk recipes.
As is, I created a separate layer with a bunch of bbappend files that are only
BBCLASSEXTEND += " native "
I suppose if I never needed them, I could just not enable that layer.Quoted 7 lines
The main reason I've been against native everywhere is that having native versions available makes it far too easy for people to add native dependencies which encourage feature creep without thinking through the huge additional dependency chains, the extra build time and other implications. Often there are slightly more difficult but worthwhile ways we can avoid the native dependency.
Quoted 4 lines
Such as? I started down this path needing a native tool (which admittedly came from an OE recipe librsvg) which then cascaded into cairo-native and beyond, totally 22 packages!. If I knew of a short circuit for this, I'd certainly entertain it.
Quoted 5 lines
Having just skimmed the OE recipes, yeah, there we just build a ton of stuff in librsvg-native. But I would start by looking at what we can pass to configure to disable things as part of the host tool build.
Doing some git grep'ing I see there's just a few things in OE that need librsvg-native so perhaps we can build a much more limited native recipe instead and still generate what we need to.
What I really needed was rsvg-convert which is used during the build of midori. In Poky/Yocto, that tool has to come from librsvg-native. OE seems to be happy leaning on the host version.
Signature · 4 lines
------------------------------------------------------------ Gary Thomas | Consulting for the MLB Associates | Embedded world ------------------------------------------------------------
Quoted 44 lines · Gary Thomas
Quoted 43 lines · Tom Rini
Quoted 34 lines · Gary Thomas
Quoted 13 lines · Richard Purdie
Quoted 11 lines · Gary Thomas
As pointed out in another thread, I'm trying to build a native package which sends "I need native" ripples throughout much of the Poky infrastructure.
Does having a native version available for any given package incur a cost?
If not, would patches for [all of] the packages I need be acceptable?So far, nearly all of the affected packages built fine, just adding native
to BBCLASSEXTEND. Many already build nativesdk versions already.Quoted 6 lines
There is a cost incurred by doing this since it does increase parse time and this is something user exposed which we do try and keep under control.
Having said that, the BBCLASSEXTEND technology has a lot less overhead than some of the older approaches to native/sdk recipes.
As is, I created a separate layer with a bunch of bbappend files that are only
BBCLASSEXTEND += " native "
I suppose if I never needed them, I could just not enable that layer.Quoted 7 lines
The main reason I've been against native everywhere is that having native versions available makes it far too easy for people to add native dependencies which encourage feature creep without thinking through the huge additional dependency chains, the extra build time and other implications. Often there are slightly more difficult but worthwhile ways we can avoid the native dependency.
Quoted 4 lines
Such as? I started down this path needing a native tool (which admittedly came from an OE recipe librsvg) which then cascaded into cairo-native and beyond, totally 22 packages!. If I knew of a short circuit for this, I'd certainly entertain it.
Having just skimmed the OE recipes, yeah, there we just build a ton of stuff in librsvg-native. But I would start by looking at what we can pass to configure to disable things as part of the host tool build.
Quoted 6 lines
Quoted 2 lines
Doing some git grep'ing I see there's just a few things in OE that need librsvg-native so perhaps we can build a much more limited native recipe instead and still generate what we need to.
What I really needed was rsvg-convert which is used during the build of midori. In Poky/Yocto, that tool has to come from librsvg-native. OE seems to be happy leaning on the host version.
I think having some buildhost/native packages (based on distro/version) trusted and assume provided or even installed using the build host package management could be one approach
in modern distros I think we could use lot of host packages as is. or
may be trust all and keep a list of blacklisted packages for a given
distro might be something useful
Quoted 10 lines
Signature · 9 lines
-- ------------------------------------------------------------ Gary Thomas | Consulting for the MLB Associates | Embedded world ------------------------------------------------------------ _______________________________________________ poky mailing list [email protected] https://lists.yoctoproject.org/listinfo/poky
Signature · 1 line
-Khem
Quoted 5 lines · Khem Raj
I think having some buildhost/native packages (based on distro/version) trusted and assume provided or even installed using the build host package management could be one approach
in modern distros I think we could use lot of host packages as is. or
may be trust all and keep a list of blacklisted packages for a given
distro might be something useful
I've been thinking about this a bit and at least at the high level there is an argument for making it easy to opt out of a big class of host tools, on a modern distro and trade reprodicibility / reliability for faster initial build times. And on the flip side, opt out of a bit more of our host tool dependencies.
Signature · 2 lines
Tom Rini Mentor Graphics Corporation
Quoted 8 lines · Tom Rini
Quoted 5 lines · Khem Raj
I think having some buildhost/native packages (based on distro/version) trusted and assume provided or even installed using the build host package management could be one approach
in modern distros I think we could use lot of host packages as is. or
may be trust all and keep a list of blacklisted packages for a given
distro might be something useful
Quoted 5 lines
I've been thinking about this a bit and at least at the high level there is an argument for making it easy to opt out of a big class of host tools, on a modern distro and trade reprodicibility / reliability for faster initial build times. And on the flip side, opt out of a bit more of our host tool dependencies.
This is what ASSUME_PROVIDED is there for :)
Cheers,
Richard
Quoted 9 lines · Richard Purdie
Quoted 8 lines · Tom Rini
Quoted 5 lines · Khem Raj
I think having some buildhost/native packages (based on distro/version) trusted and assume provided or even installed using the build host package management could be one approach
in modern distros I think we could use lot of host packages as is. or
may be trust all and keep a list of blacklisted packages for a given
distro might be something useful
Quoted 7 lines
Quoted 5 lines
I've been thinking about this a bit and at least at the high level there is an argument for making it easy to opt out of a big class of host tools, on a modern distro and trade reprodicibility / reliability for faster initial build times. And on the flip side, opt out of a bit more of our host tool dependencies.
This is what
ASSUME_PROVIDEDis there for :)
Well yes. But it'd be nice if we had a few helpers so you could say:ASSUME_PROVIDED += "${SCM_FETCH_TOOLS} ${DOCUMENTATION_GENERATION_TOOLS} ..." and so forth.
Signature · 2 lines
Tom Rini Mentor Graphics Corporation
Quoted 16 lines · Tom Rini
Quoted 15 lines · Richard Purdie
Quoted 13 lines · Tom Rini
Quoted 5 lines · Khem Raj
I think having some buildhost/native packages (based on distro/version) trusted and assume provided or even installed using the build host package management could be one approach
in modern distros I think we could use lot of host packages as is. or
may be trust all and keep a list of blacklisted packages for a given
distro might be something usefulI've been thinking about this a bit and at least at the high level there is an argument for making it easy to opt out of a big class of host tools, on a modern distro and trade reprodicibility / reliability for faster initial build times. And on the flip side, opt out of a bit more of our host tool dependencies.
Quoted 6 lines
Quoted 1 line
This is what
ASSUME_PROVIDEDis there for :)Well yes. But it'd be nice if we had a few helpers so you could say:
ASSUME_PROVIDED+= "${SCM_FETCH_TOOLS} ${DOCUMENTATION_GENERATION_TOOLS} ..."
and so forth.
We need to user to be very clearly aware of what they're adding IMO so I'd prefer maybe just some commented out lines such as:
# Uncomment this if you want to use the system provided scm utilities
# instead of having the system build them #ASSUME_PROVIDED += "git-native svn-native"
I'm actually thinking about a local.conf.sample.advanced file where we list a load of things like this. This means we can keep local.conf.sample simple (the things they probably *need* to look at) yet still show users some of the other more advanced options.
Cheers,
Richard
Quoted 27 lines · Richard Purdie
Quoted 16 lines · Tom Rini
Quoted 15 lines · Richard Purdie
Quoted 13 lines · Tom Rini
Quoted 5 lines · Khem Raj
I think having some buildhost/native packages (based on distro/version) trusted and assume provided or even installed using the build host package management could be one approach
in modern distros I think we could use lot of host packages as is. or
may be trust all and keep a list of blacklisted packages for a given
distro might be something usefulI've been thinking about this a bit and at least at the high level there is an argument for making it easy to opt out of a big class of host tools, on a modern distro and trade reprodicibility / reliability for faster initial build times. And on the flip side, opt out of a bit more of our host tool dependencies.
Quoted 6 lines
Quoted 1 line
This is what
ASSUME_PROVIDEDis there for :)Well yes. But it'd be nice if we had a few helpers so you could say:
ASSUME_PROVIDED+= "${SCM_FETCH_TOOLS} ${DOCUMENTATION_GENERATION_TOOLS} ..."
and so forth.We need to user to be very clearly aware of what they're adding IMO so I'd prefer maybe just some commented out lines such as:
Quoted 8 lines
# Uncomment this if you want to use the system provided scm utilities
# instead of having the system build them #ASSUME_PROVIDED += "git-native svn-native"I'm actually thinking about a local.conf.sample.advanced file where we list a load of things like this. This means we can keep local.conf.sample simple (the things they probably *need* to look at) yet still show users some of the other more advanced options.
I agree. There's very real risk involved in not assuming things. But there's also very real cases where people are fine taking the risk (and in some cases it's not really that risky, eg scm fetching tools, but others really are).
Signature · 2 lines
Tom Rini Mentor Graphics Corporation
Quoted 32 lines · Tom Rini
Quoted 31 lines · Richard Purdie
Quoted 16 lines · Tom Rini
Quoted 15 lines · Richard Purdie
Quoted 13 lines · Tom Rini
Quoted 5 lines · Khem Raj
I think having some buildhost/native packages (based on distro/version) trusted and assume provided or even installed using the build host package management could be one approach
in modern distros I think we could use lot of host packages as is. or
may be trust all and keep a list of blacklisted packages for a given
distro might be something usefulI've been thinking about this a bit and at least at the high level there is an argument for making it easy to opt out of a big class of host tools, on a modern distro and trade reprodicibility / reliability for faster initial build times. And on the flip side, opt out of a bit more of our host tool dependencies.
Quoted 6 lines
Quoted 1 line
This is what
ASSUME_PROVIDEDis there for :)Well yes. But it'd be nice if we had a few helpers so you could say:
ASSUME_PROVIDED+= "${SCM_FETCH_TOOLS} ${DOCUMENTATION_GENERATION_TOOLS} ..."
and so forth.We need to user to be very clearly aware of what they're adding IMO so I'd prefer maybe just some commented out lines such as:
# Uncomment this if you want to use the system provided scm utilities
# instead of having the system build them #ASSUME_PROVIDED += "git-native svn-native"
Quoted 7 lines
Quoted 4 lines
I'm actually thinking about a local.conf.sample.advanced file where we list a load of things like this. This means we can keep local.conf.sample simple (the things they probably *need* to look at) yet still show users some of the other more advanced options.
I agree. There's very real risk involved in not assuming things. But there's also very real cases where people are fine taking the risk (and in some cases it's not really that risky, eg scm fetching tools, but others really are).
I fully agree with the basic idea, but I'm not sure it solves everything. I ended up down this road when a configure step tried to use a host tool, in this case /usr/bin/rsvg-convert. I think there still would have been some grousing even if I had ASSUME_PROVIDED="librsvg-native" which is the package I ended up needing to build.
Signature · 4 lines
------------------------------------------------------------ Gary Thomas | Consulting for the MLB Associates | Embedded world ------------------------------------------------------------