Using devtool without an absolute path to the workspace in bblayers.conf
I'm trying to work out how we can make use of devtool to make our lives easier during development. In general it seems to work very well, but the way that it modifies bblayers.conf to add an absolute path to the workspace directory to BBLAYERS is incompatible with that file being held in a Git repository and shared by multiple users in multiple trees. There's a high risk that the file will accidentally be committed containing a path that's only meaningful for a single user in a single source tree. All of our other paths in bblayers.conf are relative to a variable that contains the path to the top of the source tree.
I can manually add ${TOPDIR}/workspace to BBLAYERS in bblayers.conf and commit that, but unfortunately devtool insists on continuing to add the absolute path since it doesn't know that's the same as ${TOPDIR}/workspace.
I can set "workspace_path = workspace" in devtool.conf to use a relative path, and that superficially works until externalsrc.bbclass gets upset that the EXTERNALSRC is not an absolute path.
I've tried teaching devtool to prepend ${TOPDIR} if the workspace directory is not absolute when writing bblayers.conf, but it looks like I'd also need to do so in many other places.
My current workaround is just to add ${TOPDIR}/workspace to the committed bblayers.conf myself and nobble devtool's _enable_workspace_layer with an early return. This is ugly and we'd have to carry that change around forever.
Since I'm clearly swimming against the tide I'm left wondering whether I've missed something. Is there a way to use devtool without having an absolute path to the workspace in bblayers.conf?
(At the moment I'm still using dunfell, but I looked at current master and didn't spot anything that looked like it changed this behaviour.)
Thanks.
Mike.
Quoted 10 lines · Mike Crowe via lists.openembedded.org
I'm trying to work out how we can make use of devtool to make our lives easier during development. In general it seems to work very well, but the way that it modifies bblayers.conf to add an absolute path to the workspace directory to BBLAYERS is incompatible with that file being held in a Git repository and shared by multiple users in multiple trees. There's a high risk that the file will accidentally be committed containing a path that's only meaningful for a single user in a single source tree. All of our other paths in bblayers.conf are relative to a variable that contains the path to the top of the source tree.
Hi Mike,
I think it is a bad idea to version files under the build directory.
Maybe you have a specific use case that you can address in a different way? You need to share bblayers.conf, I can see 2 options here:
- either you want your co-workers to fetch it only once, when setting up their source tree
- or you need to frequently modify bblayer.conf because you are adding or removing layers and want your coworkers to be up-to date
In the first case, TEMPLATECONF is the way to go! More information here; https://docs.yoctoproject.org/ref-manual/variables.html#term-TEMPLATECONF
In the second scenario, if this is something you *really* need, maybe you can switch to some other tools such as 'kas'
(https://kas.readthedocs.io/en/latest/) that should be useful here.
Maybe someone else will have another idea?
If you have a particular use case that is not described here, please add more details :)
Cheers
Julien
Quoted 37 lines
I can manually add ${TOPDIR}/workspace to BBLAYERS in bblayers.conf and commit that, but unfortunately devtool insists on continuing to add the absolute path since it doesn't know that's the same as ${TOPDIR}/workspace.
I can set "
workspace_path= workspace" in devtool.conf to use a relative path, and that superficially works until externalsrc.bbclass gets upset that the EXTERNALSRC is not an absolute path.I've tried teaching devtool to prepend ${TOPDIR} if the workspace directory is not absolute when writing bblayers.conf, but it looks like I'd also need to do so in many other places.
My current workaround is just to add ${TOPDIR}/workspace to the committed bblayers.conf myself and nobble devtool's
_enable_workspace_layerwith an early return. This is ugly and we'd have to carry that change around forever.Since I'm clearly swimming against the tide I'm left wondering whether I've missed something. Is there a way to use devtool without having an absolute path to the workspace in bblayers.conf?
(At the moment I'm still using dunfell, but I looked at current master and didn't spot anything that looked like it changed this behaviour.)
Thanks.
Mike.
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#189396): https://lists.openembedded.org/g/openembedded-core/message/189396
Mute This Topic: https://lists.openembedded.org/mt/102039990/4356723
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Quoted 13 lines · Julien Stephan
Le mer. 18 oct. 2023 � 16:29, Mike Crowe via lists.openembedded.org <[email protected]> a �crit :
Quoted 10 lines
I'm trying to work out how we can make use of devtool to make our lives easier during development. In general it seems to work very well, but the way that it modifies bblayers.conf to add an absolute path to the workspace directory to BBLAYERS is incompatible with that file being held in a Git repository and shared by multiple users in multiple trees. There's a high risk that the file will accidentally be committed containing a path that's only meaningful for a single user in a single source tree. All of our other paths in bblayers.conf are relative to a variable that contains the path to the top of the source tree.
Quoted 3 lines
Hi Mike,
I think it is a bad idea to version files under the build directory.
Hi Julien,
We don't version all files under the build directory, just ones that control the configuration that we want to share. We've been using OE that way for about twelve years now and this is the first time we've run into a problem. That might at least partly be because we haven't taken advantage of newer features like devtool though! We mostly avoided devtool by putting the sources that we modify heavily in submodules and used externalsrc.bbclass (with some local progressively-less-hacky modifications that I should try to upstream sometime.)
Quoted 10 lines
Maybe you have a specific use case that you can address in a different way? You need to share bblayers.conf, I can see 2 options here:
- either you want your co-workers to fetch it only once, when setting up their source tree
- or you need to frequently modify bblayer.conf because you are adding or removing layers and want your coworkers to be up-to dateIn the first case, TEMPLATECONF is the way to go! More information here;
https://docs.yoctoproject.org/ref-manual/variables.html#term-TEMPLATECONF
We do need to modify bblayers.conf from time to time to add and remove layers.
Using templates might be possible, but it would appear that this would force developers to manually incorporate changes (or just wipe their bblayers.conf file) when the template changes. Just keeping bblayers.conf under version control doesn't suffer from that problem.
Quoted 3 lines
In the second scenario, if this is something you *really* need, maybe you can switch to some other tools such as 'kas'
(https://kas.readthedocs.io/en/latest/) that should be useful here.
I have looked at kas in the past (and listened to podcasts about it), but it seemed to solve a different set of problems. I will look at it again.
Thanks for the suggestions.
Mike
Quoted 7 lines · Mike Crowe via lists.openembedded.org
We do need to modify bblayers.conf from time to time to add and remove layers.
Using templates might be possible, but it would appear that this would force developers to manually incorporate changes (or just wipe their bblayers.conf file) when the template changes. Just keeping bblayers.conf under version control doesn't suffer from that problem.
Generally content of build/conf/, or anything else in build/ is not designed for being under version control. You may be able to get away with it by coincidence, but it's neither documented nor tested, and so any issues are your own.
I don't have a quick solution to ensure that templates and configurations coming from them are strictly in sync, but welcome to suggestions. People modify local.conf and bblayers.conf all the time as well for local experiments, you're simply not supposed to lock them down.
Alex
Alex
Quoted 15 lines · Alexander Kanavin
Quoted 7 lines · Mike Crowe via lists.openembedded.org
We do need to modify bblayers.conf from time to time to add and remove layers.
Using templates might be possible, but it would appear that this would force developers to manually incorporate changes (or just wipe their bblayers.conf file) when the template changes. Just keeping bblayers.conf under version control doesn't suffer from that problem.
Generally content of build/conf/, or anything else in build/ is not designed for being under version control. You may be able to get away with it by coincidence, but it's neither documented nor tested, and so any issues are your own.
Quoted 5 lines
I don't have a quick solution to ensure that templates and configurations coming from them are strictly in sync, but welcome to suggestions. People modify local.conf and bblayers.conf all the time as well for local experiments, you're simply not supposed to lock them down.
There's no problem with modifying them for local experiments, even if they are under version control, but it's clear when those experiments need to be committed that those files need to be committed too.
However, it's clear that I'm pushing against the tide. I'll stick with the early-return hack in devtool for the short term to see if we come across any other reasons to make changes or any other breakage. Once everything is working I'll investigate our options for not keeping bblayers.conf under version control.
Thanks for you help.
Mike.
Quoted 28 lines · Mike Crowe via lists.openembedded.org
I'm trying to work out how we can make use of devtool to make our lives easier during development. In general it seems to work very well, but the way that it modifies bblayers.conf to add an absolute path to the workspace directory to BBLAYERS is incompatible with that file being held in a Git repository and shared by multiple users in multiple trees. There's a high risk that the file will accidentally be committed containing a path that's only meaningful for a single user in a single source tree. All of our other paths in bblayers.conf are relative to a variable that contains the path to the top of the source tree.
I can manually add ${TOPDIR}/workspace to BBLAYERS in bblayers.conf and commit that, but unfortunately devtool insists on continuing to add the absolute path since it doesn't know that's the same as ${TOPDIR}/workspace.
I can set "
workspace_path= workspace" in devtool.conf to use a relative path, and that superficially works until externalsrc.bbclass gets upset that the EXTERNALSRC is not an absolute path.I've tried teaching devtool to prepend ${TOPDIR} if the workspace directory is not absolute when writing bblayers.conf, but it looks like I'd also need to do so in many other places.
My current workaround is just to add ${TOPDIR}/workspace to the committed bblayers.conf myself and nobble devtool's
_enable_workspace_layerwith an early return. This is ugly and we'd have to carry that change around forever.
Quoted 6 lines
Since I'm clearly swimming against the tide I'm left wondering whether I've missed something. Is there a way to use devtool without having an absolute path to the workspace in bblayers.conf?
(At the moment I'm still using dunfell, but I looked at current master and didn't spot anything that looked like it changed this behaviour.)
Perhaps an enhancement to not add the path if its already part of BBLAYERS might do it. You might also print a diagnostics about its pre-existence to make sure it's not accidental.
Quoted 13 lines
Thanks.
Mike.
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#189396): https://lists.openembedded.org/g/openembedded-core/message/189396
Mute This Topic: https://lists.openembedded.org/mt/102039990/1997914
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Quoted 34 lines · Khem Raj
Quoted 32 lines · Mike Crowe via lists.openembedded.org
I'm trying to work out how we can make use of devtool to make our lives easier during development. In general it seems to work very well, but the way that it modifies bblayers.conf to add an absolute path to the workspace directory to BBLAYERS is incompatible with that file being held in a Git repository and shared by multiple users in multiple trees. There's a high risk that the file will accidentally be committed containing a path that's only meaningful for a single user in a single source tree. All of our other paths in bblayers.conf are relative to a variable that contains the path to the top of the source tree.
I can manually add ${TOPDIR}/workspace to BBLAYERS in bblayers.conf and commit that, but unfortunately devtool insists on continuing to add the absolute path since it doesn't know that's the same as ${TOPDIR}/workspace.
I can set "
workspace_path= workspace" in devtool.conf to use a relative path, and that superficially works until externalsrc.bbclass gets upset that the EXTERNALSRC is not an absolute path.I've tried teaching devtool to prepend ${TOPDIR} if the workspace directory is not absolute when writing bblayers.conf, but it looks like I'd also need to do so in many other places.
My current workaround is just to add ${TOPDIR}/workspace to the committed bblayers.conf myself and nobble devtool's
_enable_workspace_layerwith an early return. This is ugly and we'd have to carry that change around forever.Since I'm clearly swimming against the tide I'm left wondering whether I've missed something. Is there a way to use devtool without having an absolute path to the workspace in bblayers.conf?
Quoted 6 lines
Quoted 2 lines
(At the moment I'm still using dunfell, but I looked at current master and didn't spot anything that looked like it changed this behaviour.)
Perhaps an enhancement to not add the path if its already part of BBLAYERS might do it. You might also print a diagnostics about its pre-existence to make sure it's not accidental.
devtool already tries to do that. It doesn't work in this case since the code compares the absolute path against the paths in bblayers.conf and isn't in a position to expand variables at that point.
Thanks.
Mike.
Quoted 39 lines · Mike Crowe
Quoted 37 lines · Khem Raj
Quoted 34 lines · Mike Crowe via lists.openembedded.org
I'm trying to work out how we can make use of devtool to make our lives easier during development. In general it seems to work very well, but the way that it modifies bblayers.conf to add an absolute path to the workspace directory to BBLAYERS is incompatible with that file being held in a Git repository and shared by multiple users in multiple trees. There's a high risk that the file will accidentally be committed containing a path that's only meaningful for a single user in a single source tree. All of our other paths in bblayers.conf are relative to a variable that contains the path to the top of the source tree.
I can manually add ${TOPDIR}/workspace to BBLAYERS in bblayers.conf and commit that, but unfortunately devtool insists on continuing to add the absolute path since it doesn't know that's the same as ${TOPDIR}/workspace.
I can set "
workspace_path= workspace" in devtool.conf to use a relative path, and that superficially works until externalsrc.bbclass gets upset that the EXTERNALSRC is not an absolute path.I've tried teaching devtool to prepend ${TOPDIR} if the workspace directory is not absolute when writing bblayers.conf, but it looks like I'd also need to do so in many other places.
My current workaround is just to add ${TOPDIR}/workspace to the committed bblayers.conf myself and nobble devtool's
_enable_workspace_layerwith an early return. This is ugly and we'd have to carry that change around forever.Since I'm clearly swimming against the tide I'm left wondering whether I've missed something. Is there a way to use devtool without having an absolute path to the workspace in bblayers.conf?
(At the moment I'm still using dunfell, but I looked at current master and didn't spot anything that looked like it changed this behaviour.)
Quoted 7 lines
Quoted 3 lines
Perhaps an enhancement to not add the path if its already part of BBLAYERS might do it. You might also print a diagnostics about its pre-existence to make sure it's not accidental.
devtool already tries to do that. It doesn't work in this case since the code compares the absolute path against the paths in bblayers.conf and isn't in a position to expand variables at that point.
yeah it perhaps has to compare the evaluated paths
Quoted 4 lines
Thanks.
Mike.