base: Improve handling of switching virtual/x providers
PATCH 1/1 base: Improve handling of switching virtual/x providers
If you build virtual/kernel, then change PREFERRED_PROVIDER_virtual/kernel from say "linux-yocto" to "linux-yocto-dev", you see errors from the sysroot about overlapping files. The automatic uninstall logic doesn't trigger since the other recipes is still technically parsed/buildable.
What we can do is look at the value of PREFERRED_PROVIDER_virtual/X and raise SkipRecipe (skip parsing) if it provides this thing and its not selected. We skip cases no preferred provider is set, or the value is in MULTI_PROVIDER_WHITELIST.We also inform the user if they try to build something which conflicts with the configuration:
$ bitbake linux-yocto-tiny ERROR: Nothing PROVIDES 'linux-yocto-tiny' ERROR: linux-yocto-tiny was skipped: PREFERRED_PROVIDER_virtual/kernel set to linux-yocto, not linux-yocto-tiny
[YOCTO #4102]
meta/classes/base.bbclass +20 -1
@@ -204,7 +204,7 @@ def buildcfg_neededvars(d): bb.fatal('The following variable(s) were not set: %s\nPlease set them directly, or choose a MACHINE or DISTRO that sets them.' % ', '.join(pesteruser)) addhandler base_eventhandler -base_eventhandler[eventmask] = "bb.event.ConfigParsed bb.event.BuildStarted bb.event.RecipePreFinalise bb.runqueue.sceneQueueComplete" +base_eventhandler[eventmask] = "bb.event.ConfigParsed bb.event.BuildStarted bb.event.RecipePreFinalise bb.runqueue.sceneQueueComplete bb.event.RecipeParsed" python base_eventhandler() { import bb.runqueue
@@ -257,6 +257,25 @@ python base_eventhandler() { bb.debug(1, "Executing SceneQueue Completion commands: %s" % "\n".join(cmds)) bb.build.exec_func("completion_function", e.data) os.remove(completions) + + if isinstance(e, bb.event.RecipeParsed): + # + # If we have multiple providers of virtual/X and a PREFERRED_PROVIDER_virtual/X is set + # skip parsing for all the other providers which will mean they get uninstalled from the + # sysroot since they're now "unreachable". This makes switching virtual/kernel work in + # particular. + # + pn = d.getVar('PN', True) + source_mirror_fetch = d.getVar('SOURCE_MIRROR_FETCH', False) + if not source_mirror_fetch: + provs = (d.getVar("PROVIDES", True) or "").split() + multiwhitelist = (d.getVar("MULTI_PROVIDER_WHITELIST", True) or "").split() + for p in provs: + if p.startswith("virtual/") and p not in multiwhitelist: + profprov = d.getVar("PREFERRED_PROVIDER_" + p, True) + if profprov and pn != profprov: + bb.warn("PREFERRED_PROVIDER_%s set to %s, not %s" % (p, profprov, pn)) + raise bb.parse.SkipPackage("PREFERRED_PROVIDER_%s set to %s, not %s" % (p, profprov, pn)) } CONFIGURESTAMPFILE = "${WORKDIR}/configure.sstate"
Quoted 43 lines · Richard Purdie
If you build virtual/kernel, then change PREFERRED_PROVIDER_virtual/kernel from say "linux-yocto" to "linux-yocto-dev", you see errors from the sysroot about overlapping files. The automatic uninstall logic doesn't trigger since the other recipes is still technically parsed/buildable.
What we can do is look at the value of PREFERRED_PROVIDER_virtual/X and raise SkipRecipe (skip parsing) if it provides this thing and its not selected. We skip cases no preferred provider is set, or the value is in MULTI_PROVIDER_WHITELIST.We also inform the user if they try to build something which conflicts with the configuration:
$ bitbake linux-yocto-tiny ERROR: Nothing PROVIDES 'linux-yocto-tiny' ERROR: linux-yocto-tiny was skipped: PREFERRED_PROVIDER_virtual/kernel set to linux-yocto, not linux-yocto-tiny[YOCTO #4102]
Signed-off-by Richard Purdie <[email protected]>meta/classes/base.bbclass +9
-1@@ -204,7 +204,7 @@ def buildcfg_neededvars(d): bb.fatal('The following variable(s) were not set: %s\nPlease set them directly, or choose a MACHINE or DISTRO that sets them.' % ', '.join(pesteruser)) addhandler base_eventhandler -base_eventhandler[eventmask] = "bb.event.ConfigParsed bb.event.BuildStarted bb.event.RecipePreFinalise bb.runqueue.sceneQueueComplete" +base_eventhandler[eventmask] = "bb.event.ConfigParsed bb.event.BuildStarted bb.event.RecipePreFinalise bb.runqueue.sceneQueueComplete bb.event.RecipeParsed" python base_eventhandler() { import bb.runqueue@@ -257,6 +257,25 @@ python base_eventhandler() { bb.debug(1, "Executing SceneQueue Completion commands: %s" % "\n".join(cmds)) bb.build.exec_func("completion_function", e.data) os.remove(completions) + + if isinstance(e, bb.event.RecipeParsed): + # + # If we have multiple providers of virtual/X and a PREFERRED_PROVIDER_virtual/X is set + # skip parsing for all the other providers which will mean they get uninstalled from the + # sysroot since they're now "unreachable". This makes switching virtual/kernel work in + # particular. + #
meta/classes/base.bbclass python base_eventhandler() { · line 277
Quoted 10 lines
meta/classes/base.bbclass +10
-0+ pn = d.getVar('PN', True) + source_mirror_fetch = d.getVar('SOURCE_MIRROR_FETCH', False) + if not source_mirror_fetch: + provs = (d.getVar("PROVIDES", True) or "").split() + multiwhitelist = (d.getVar("MULTI_PROVIDER_WHITELIST", True) or "").split() + for p in provs: + if p.startswith("virtual/") and p not in multiwhitelist: + profprov = d.getVar("PREFERRED_PROVIDER_" + p, True) + if profprov and pn != profprov: + bb.warn("PREFERRED_PROVIDER_%s set to %s, not %s" % (p, profprov, pn))
Obviously minus the above line. I'll remove that in any version I add to -next.
Cheers,
Richard
Quoted 6 lines
meta/classes/base.bbclass +1
-0+ raise bb.parse.SkipPackage("PREFERRED_PROVIDER_%s set to %s, not %s" % (p, profprov, pn)) } CONFIGURESTAMPFILE = "${WORKDIR}/configure.sstate"
Quoted 5 lines · Richard Purdie
If you build virtual/kernel, then change PREFERRED_PROVIDER_virtual/kernel from say "linux-yocto" to "linux-yocto-dev", you see errors from the sysroot about overlapping files. The automatic uninstall logic doesn't trigger since the other recipes is still technically parsed/buildable.
Quoted 8 lines
What we can do is look at the value of PREFERRED_PROVIDER_virtual/X and raise SkipRecipe (skip parsing) if it provides this thing and its not selected. We skip cases no preferred provider is set, or the value is in MULTI_PROVIDER_WHITELIST.We also inform the user if they try to build something which conflicts with the configuration:
$ bitbake linux-yocto-tiny ERROR: Nothing PROVIDES 'linux-yocto-tiny' ERROR: linux-yocto-tiny was skipped: PREFERRED_PROVIDER_virtual/kernel set to linux-yocto, not linux-yocto-tiny
This change seems to cause problems for musl libc builds (I'm using today's oe-core master and Khem's meta-musl layer).
ERROR: Nothing RPROVIDES 'nativesdk-glibc' (but/home/andre/rdk/rdk-master-musl/openembedded-core/meta/recipes-core/meta/uninative-tarball.bb RDEPENDS on or otherwise requires it)
ERROR: nativesdk-glibc was skipped:
PREFERRED_PROVIDER_virtual/nativesdk-x86_64-rdksdk-linux-libc-for-gcc
set to nativesdk-musl, not nativesdk-glibc
ERROR: Required build target 'rdk-image-wpe' has no buildable providers. Missing or unbuildable dependency chain was: ['rdk-image-wpe',
'uninative-tarball', 'nativesdk-glibc']
Reverting this one commit (oe-core 9a3b992) gets the builds working again.
Quoted 50 lines
[YOCTO #4102]
Signed-off-by Richard Purdie <[email protected]>meta/classes/base.bbclass +20
-2@@ -204,7 +204,7 @@ def buildcfg_neededvars(d): bb.fatal('The following variable(s) were not set: %s\nPlease set them directly, or choose a MACHINE or DISTRO that sets them.' % ', '.join(pesteruser)) addhandler base_eventhandler -base_eventhandler[eventmask] = "bb.event.ConfigParsed bb.event.BuildStarted bb.event.RecipePreFinalise bb.runqueue.sceneQueueComplete" +base_eventhandler[eventmask] = "bb.event.ConfigParsed bb.event.BuildStarted bb.event.RecipePreFinalise bb.runqueue.sceneQueueComplete bb.event.RecipeParsed" python base_eventhandler() { import bb.runqueue@@ -257,6 +257,25 @@ python base_eventhandler() { bb.debug(1, "Executing SceneQueue Completion commands: %s" % "\n".join(cmds)) bb.build.exec_func("completion_function", e.data) os.remove(completions) + + if isinstance(e, bb.event.RecipeParsed): + # + # If we have multiple providers of virtual/X and a PREFERRED_PROVIDER_virtual/X is set + # skip parsing for all the other providers which will mean they get uninstalled from the + # sysroot since they're now "unreachable". This makes switching virtual/kernel work in + # particular. + # + pn = d.getVar('PN', True) + source_mirror_fetch = d.getVar('SOURCE_MIRROR_FETCH', False) + if not source_mirror_fetch: + provs = (d.getVar("PROVIDES", True) or "").split() + multiwhitelist = (d.getVar("MULTI_PROVIDER_WHITELIST", True) or "").split() + for p in provs: + if p.startswith("virtual/") and p not in multiwhitelist: + profprov = d.getVar("PREFERRED_PROVIDER_" + p, True) + if profprov and pn != profprov: + bb.warn("PREFERRED_PROVIDER_%s set to %s, not %s" % (p, profprov, pn)) + raise bb.parse.SkipPackage("PREFERRED_PROVIDER_%s set to %s, not %s" % (p, profprov, pn)) } CONFIGURESTAMPFILE = "${WORKDIR}/configure.sstate" --_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Quoted 14 lines · Andre McCurdy
This change seems to cause problems for musl libc builds (I'm using today's oe-core master and Khem's meta-musl layer).
ERROR: Nothing RPROVIDES 'nativesdk-glibc' (but
/home/andre/rdk/rdk-master-musl/openembedded-core/meta/recipes-core/meta/uninative-tarball.bb
RDEPENDS on or otherwise requires it)
ERROR: nativesdk-glibc was skipped:
PREFERRED_PROVIDER_virtual/nativesdk-x86_64-rdksdk-linux-libc-for-gcc set to nativesdk-musl, not nativesdk-glibc
ERROR: Required build target 'rdk-image-wpe' has no buildable providers. Missing or unbuildable dependency chain was: ['rdk-image-wpe',
'uninative-tarball', 'nativesdk-glibc']
Quoted 2 lines
Reverting this one commit (oe-core 9a3b992) gets the builds working again.
It's breaking universe fetch too:
ERROR: Nothing PROVIDES 'linux-yocto-rt' (but /home/pokybuild/yocto-autobuilder/yocto-worker/build-appliance/build/meta/recipes-rt/images/ core-image-rt-sdk.bb, /home/pokybuild/yocto-autobuilder/yocto-worker/build-appliance/build/meta/recipes-rt/images/ core-image-rt.bb DEPENDS on or otherwise requires it)
ERROR: linux-yocto-rt was skipped: PREFERRED_PROVIDER_virtual/kernel set to linux-yocto, not linux-yocto-rt
ERROR: linux-yocto-rt was skipped: PREFERRED_PROVIDER_virtual/kernel set to linux-yocto, not linux-yocto-rt
ERROR: Nothing RPROVIDES 'core-image-rt-sdk'
ERROR: No eligible RPROVIDERs exist for 'core-image-rt-sdk'
Ross