Extensible SDK improvements
cover Extensible SDK improvements
"add option to append lines into local.conf":
- What about dropping "sdk_extraconf"?
Andrej Valek (2):
populate_sdk_ext.bbclass: enable custom templateconf.cfg populate_sdk_ext.bbclass: add option to append lines into local.conf
meta/classes/buildhistory.bbclass | 4 ++-- meta/classes/populate_sdk_ext.bbclass | 22 ++++++++++++++++++---- 2 files changed, 20 insertions(+), 6 deletions(-)
Signature · 1 line
2.11.0
PATCH 1/2 populate_sdk_ext: enable custom templateconf.cfg
Do not always override templateconf.cfg content. Add option to use already existing file.
--- meta/classes/populate_sdk_ext.bbclass | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-)
meta/classes/populate_sdk_ext.bbclass +7 -3
@@ -388,9 +388,13 @@ python copy_buildsystem () { bb.utils.mkdirhier(os.path.join(baseoutpath, 'cache')) shutil.copyfile(builddir + '/cache/bb_unihashes.dat', baseoutpath + '/cache/bb_unihashes.dat') - # Write a templateconf.cfg - with open(baseoutpath + '/conf/templateconf.cfg', 'w') as f: - f.write('meta/conf\n') + # Use templateconf.cfg file from builddir if exists + if os.path.exists(builddir + '/conf/templateconf.cfg'): + shutil.copyfile(builddir + '/conf/templateconf.cfg', baseoutpath + '/conf/templateconf.cfg') + else: + # Write a templateconf.cfg + with open(baseoutpath + '/conf/templateconf.cfg', 'w') as f: + f.write('meta/conf\n') # Ensure any variables set from the external environment (by way of # BB_ENV_EXTRAWHITE) are set in the SDK's configuration
Signature · 1 line
2.11.0
PATCH 2/2 populate_sdk_ext: add option to append lines into local.conf
In reply to Andrej ValekAdd option to add new lines into local.conf content in ext_sdk case via SDK_LOCAL_CONF_EXTRALIST variable.
--- meta/classes/buildhistory.bbclass | 4 ++-- meta/classes/populate_sdk_ext.bbclass | 12 +++++++++++- 2 files changed, 13 insertions(+), 3 deletions(-)
meta/classes/buildhistory.bbclass +2 -2
@@ -757,8 +757,8 @@ def buildhistory_get_sdkvars(d): sdkvars = "DISTRO DISTRO_VERSION SDK_NAME SDK_VERSION SDKMACHINE SDKIMAGE_FEATURES BAD_RECOMMENDATIONS NO_RECOMMENDATIONS PACKAGE_EXCLUDE" if d.getVar('BB_CURRENTTASK') == 'populate_sdk_ext': # Extensible SDK uses some additional variables - sdkvars += " SDK_LOCAL_CONF_WHITELIST SDK_LOCAL_CONF_BLACKLIST SDK_INHERIT_BLACKLIST SDK_UPDATE_URL SDK_EXT_TYPE SDK_RECRDEP_TASKS SDK_INCLUDE_PKGDATA SDK_INCLUDE_TOOLCHAIN" - listvars = "SDKIMAGE_FEATURES BAD_RECOMMENDATIONS PACKAGE_EXCLUDE SDK_LOCAL_CONF_WHITELIST SDK_LOCAL_CONF_BLACKLIST SDK_INHERIT_BLACKLIST" + sdkvars += " SDK_LOCAL_CONF_EXTRALIST SDK_LOCAL_CONF_WHITELIST SDK_LOCAL_CONF_BLACKLIST SDK_INHERIT_BLACKLIST SDK_UPDATE_URL SDK_EXT_TYPE SDK_RECRDEP_TASKS SDK_INCLUDE_PKGDATA SDK_INCLUDE_TOOLCHAIN" + listvars = "SDKIMAGE_FEATURES BAD_RECOMMENDATIONS PACKAGE_EXCLUDE SDK_LOCAL_CONF_EXTRALIST SDK_LOCAL_CONF_WHITELIST SDK_LOCAL_CONF_BLACKLIST SDK_INHERIT_BLACKLIST" return outputvars(sdkvars, listvars, d)
meta/classes/populate_sdk_ext.bbclass +11 -1
@@ -25,6 +25,7 @@ SDK_INCLUDE_BUILDTOOLS ?= '1' SDK_RECRDEP_TASKS ?= "" +SDK_LOCAL_CONF_EXTRALIST ?= "" SDK_LOCAL_CONF_WHITELIST ?= "" SDK_LOCAL_CONF_BLACKLIST ?= "CONF_VERSION \ BB_NUMBER_THREADS \
@@ -372,7 +373,16 @@ python copy_buildsystem () { for line in xf: f.write(line) - # If you define a sdk_extraconf() function then it can contain additional config + # Allow additional config through SDK_LOCAL_CONF_EXTRALIST + local_conf_extralist = (d.getVar('SDK_LOCAL_CONF_EXTRALIST') or '').split() + if local_conf_extralist: + for var in local_conf_extralist: + value = d.getVar(var) or "" + if value: + f.write('%s = "%s"\n' % (var, value)) + f.write('\n') + + # If you define a sdk_extraconf variable then it can contain additional config # (Though this is awkward; sdk-extra.conf should probably be used instead) extraconf = (d.getVar('sdk_extraconf') or '').strip() if extraconf:
Signature · 1 line
2.11.0
Quoted 2 lines · Andrej Valek
"add option to append lines into local.conf":
- What about dropping "sdk_extraconf"?
I'm curious what you didn't find useful about the other format and why this version is better?
Cheers,
Richard
Hello Richard,
I can try to explain it in some examples.
I would like to add some extra (dynamic) variables into local.conf in eSDK mode. I have found, that there are 2 possibilities (in populate_sdk_ext.bbclass) sdk-extra.conf and sdk_extraconf.
- sdk-extra.conf
- You have to have handle own function to write into sdk-extra.conf. I
think, this is not the right way for me.
- sdk_extraconf
- This variable is fine, but there are some restrictions. There is no
easy way to add multiple variables including values.
I would like to have:
aaa = "valueA"
bbb = "valueB"
in local.conf file.
1. sdk_extraconf = 'aaa = "valueA"\nbbb = "valueB"'
- this will write aaa = "valueA"\nbbb = "valueB"
2. python copy_buildsystem_prepend() { d.setVar('sdk_extraconf','aaa = "valueA"\nbbb = "valueB"')
}
- This will write exactly, what you want, but you have to have ancopy_buildsystem_prepend and mentioned all variables with values defined in your class/recipe.
3. sdk_extraconf = "# My notes about local.conf"
- Yes, you can write some string into local.conf with this way.
So I have founded an way, how to add multiple variables without explicitly specifying their values.
# format variables for sdk_extraconf
def write_sdk_vars(d):
return '\n' + '\n'.join([var + ' = "' + d.getVar(var) + '"' for var in
d.getVar('SDK_EXTRACONF_VARS').split()])SDK_EXTRACONF_VARS = "aaa bbb"sdk_extraconf_append = " ${@write_sdk_vars(d)}"
It will do exactly what I want, but this way is little bit nasty. So I have created a more complex solution for community usage.
I think, this patch adds more functionality also for other projects. They can add only variables into list and the class will handle all the rest.
Is it enough for the explanation?
Regards,
Andrej
Quoted 11 lines · Richard Purdie
Quoted 2 lines · Andrej Valek
"add option to append lines into local.conf":
- What about dropping "sdk_extraconf"?I'm curious what you didn't find useful about the other format and why this version is better?
Cheers,
Richard
Hello Richard,
Did you have some time to take a look on it?
It would be really helpful to have this feature in.
As I said I am trying to use this functionality for write some extra variables into local.conf only in eSDK case. I am using a custom oe-init-buildenv script, means that not all cases are covered by the general one.
Thank you,
Andrej
Quoted 65 lines · Andrej Valek
Hello Richard,
I can try to explain it in some examples.
I would like to add some extra (dynamic) variables into local.conf in eSDK mode. I have found, that there are 2 possibilities (in populate_sdk_ext.bbclass) sdk-extra.conf and
sdk_extraconf.- sdk-extra.conf
- You have to have handle own function to write into sdk-extra.conf. I
think, this is not the right way for me.
-sdk_extraconf
- This variable is fine, but there are some restrictions. There is no
easy way to add multiple variables including values.I would like to have:
aaa = "valueA"
bbb = "valueB"
in local.conf file.1.
sdk_extraconf= 'aaa = "valueA"\nbbb = "valueB"'
- this will write aaa = "valueA"\nbbb = "valueB"
2. pythoncopy_buildsystem_prepend(){d.setVar('sdk_extraconf','aaa = "valueA"\nbbb = "valueB"')
}
- This will write exactly, what you want, but you have to have ancopy_buildsystem_prependand mentioned all variables with values defined in your class/recipe.
3.sdk_extraconf= "# My notes about local.conf"
- Yes, you can write some string into local.conf with this way.So I have founded an way, how to add multiple variables without explicitly specifying their values.
# format variables for sdk_extraconf def write_sdk_vars(d): return '\n' + '\n'.join([var + ' = "' + d.getVar(var) + '"' for var in d.getVar('SDK_EXTRACONF_VARS').split()])
SDK_EXTRACONF_VARS= "aaa bbb"sdk_extraconf_append= " ${@write_sdk_vars(d)}"It will do exactly what I want, but this way is little bit nasty. So I have created a more complex solution for community usage.
I think, this patch adds more functionality also for other projects. They can add only variables into list and the class will handle all the rest.Is it enough for the explanation?
Regards,
AndrejQuoted 11 lines · Richard Purdie
Quoted 2 lines · Andrej Valek
"add option to append lines into local.conf":
- What about dropping "sdk_extraconf"?I'm curious what you didn't find useful about the other format and why this version is better?
Cheers,
Richard
Hi Andrej,
Quoted 20 lines · Andrej Valek
I can try to explain it in some examples.
I would like to add some extra (dynamic) variables into local.conf in eSDK mode. I have found, that there are 2 possibilities (in populate_sdk_ext.bbclass) sdk-extra.conf and
sdk_extraconf.- sdk-extra.conf
- You have to have handle own function to write into sdk-extra.conf.
I
think, this is not the right way for me.
-sdk_extraconf
- This variable is fine, but there are some restrictions. There is
no
easy way to add multiple variables including values.I would like to have:
aaa = "valueA"
bbb = "valueB"
in local.conf file.
Quoted 8 lines
1.
sdk_extraconf= 'aaa = "valueA"\nbbb = "valueB"'
- this will write aaa = "valueA"\nbbb = "valueB"
2. pythoncopy_buildsystem_prepend(){d.setVar('sdk_extraconf','aaa = "valueA"\nbbb = "valueB"')
}
- This will write exactly, what you want, but you have to have ancopy_buildsystem_prependand mentioned all variables with values defined in your class/recipe.
Why do you have to set this in a copy_buildsystem_prepend ? Shouldn't setting the variable in your distro config or class or somewhere else work?
Quoted 2 lines
3.
sdk_extraconf= "# My notes about local.conf"
- Yes, you can write some string into local.conf with this way.
I suspect you can also do:
sdk_extraconf () {
# My notes about local.conf
aaa = "valueA"
bbb = "valueB"
}
or something like that, I admit I've not tested it though.
Quoted 12 lines
So I have founded an way, how to add multiple variables without explicitly specifying their values.
# format variables for sdk_extraconf def write_sdk_vars(d): return '\n' + '\n'.join([var + ' = "' + d.getVar(var) + '"' for var in d.getVar('SDK_EXTRACONF_VARS').split()])
SDK_EXTRACONF_VARS= "aaa bbb"sdk_extraconf_append= " ${@write_sdk_vars(d)}"
Quoted 6 lines
It will do exactly what I want, but this way is little bit nasty. So I have created a more complex solution for community usage. I think, this patch adds more functionality also for other projects. They can add only variables into list and the class will handle all the
rest.
I do see the problem with expansion. I suspect the code in populate_sdk_ext should be:
extraconf = (d.getVar('sdk_extraconf', False) or '').strip() instead of:
extraconf = (d.getVar('sdk_extraconf') or '').strip()
so that variables can be used as part of the expressions. It may be our change to the defaults for getVar messed this up, or it may be we just didn't correctly predict this problem.
Quoted 1 line
Is it enough for the explanation?
It helps, yes, thanks. What I don't want to do is add multiple confusing mechanisms which do approximately the same thing slightly differently. I'd much prefer one mechanism most people can use.
As such the original proposal still concerns me. I'd prefer to tweak one of the existing ones (even merge them!) rather than add a third slightly different one.
I do also still think some (but not all) of what you're describing should be in your distro/classes rather than local.conf.
Is there a way we can improve the current code rather than adding more options. Would the expansion change be enough?
Cheers,
Richard
Hello Richard,
Thank you for taking care of it. I didn't know, that is possible to use "functions" in d.getVar case. Yes, I have tested it. And yes, what you have mentioned is working, as we were expecting.
I will rather use my mechanism to just fill a list with variables. Then I don't need to take a care, if there are fully expanded it or not.
So we can keep like it is.
But, yes, a small improvement for someone else could be, that we will put a "False" there, but not for me.
Regards,
Andrej
Quoted 100 lines · Richard Purdie
Hi Andrej,
Quoted 20 lines · Andrej Valek
I can try to explain it in some examples.
I would like to add some extra (dynamic) variables into local.conf in eSDK mode. I have found, that there are 2 possibilities (in populate_sdk_ext.bbclass) sdk-extra.conf and
sdk_extraconf.- sdk-extra.conf
- You have to have handle own function to write into sdk-extra.conf.
I
think, this is not the right way for me.
-sdk_extraconf
- This variable is fine, but there are some restrictions. There is
no
easy way to add multiple variables including values.I would like to have:
aaa = "valueA"
bbb = "valueB"
in local.conf file.Quoted 8 lines
1.
sdk_extraconf= 'aaa = "valueA"\nbbb = "valueB"'
- this will write aaa = "valueA"\nbbb = "valueB"
2. pythoncopy_buildsystem_prepend(){d.setVar('sdk_extraconf','aaa = "valueA"\nbbb = "valueB"')
}
- This will write exactly, what you want, but you have to have ancopy_buildsystem_prependand mentioned all variables with values defined in your class/recipe.Why do you have to set this in a
copy_buildsystem_prepend? Shouldn't setting the variable in your distro config or class or somewhere else work?Quoted 2 lines
3.
sdk_extraconf= "# My notes about local.conf"
- Yes, you can write some string into local.conf with this way.I suspect you can also do:
sdk_extraconf() {
# My notes about local.conf
aaa = "valueA"
bbb = "valueB"
}or something like that, I admit I've not tested it though.
Quoted 12 lines
So I have founded an way, how to add multiple variables without explicitly specifying their values.
# format variables for sdk_extraconf def write_sdk_vars(d): return '\n' + '\n'.join([var + ' = "' + d.getVar(var) + '"' for var in d.getVar('SDK_EXTRACONF_VARS').split()])
SDK_EXTRACONF_VARS= "aaa bbb"sdk_extraconf_append= " ${@write_sdk_vars(d)}"Quoted 6 lines
It will do exactly what I want, but this way is little bit nasty. So I have created a more complex solution for community usage. I think, this patch adds more functionality also for other projects. They can add only variables into list and the class will handle all the
rest.I do see the problem with expansion. I suspect the code in
populate_sdk_extshould be:extraconf = (
d.getVar('sdk_extraconf', False)or '').strip() instead of:
extraconf = (d.getVar('sdk_extraconf')or '').strip()so that variables can be used as part of the expressions. It may be our change to the defaults for getVar messed this up, or it may be we just didn't correctly predict this problem.
Quoted 1 line
Is it enough for the explanation?
It helps, yes, thanks. What I don't want to do is add multiple confusing mechanisms which do approximately the same thing slightly differently. I'd much prefer one mechanism most people can use.
As such the original proposal still concerns me. I'd prefer to tweak one of the existing ones (even merge them!) rather than add a third slightly different one.
I do also still think some (but not all) of what you're describing should be in your distro/classes rather than local.conf.
Is there a way we can improve the current code rather than adding more options. Would the expansion change be enough?
Cheers,
Richard