command.py: cleanup bb.cache.parse_recipe
PATCH 1/1 command.py: cleanup bb.cache.parse_recipe
The parse_recipe has been removed from bb.cache, replace it with the databuilder._parse_recipe.
--- lib/bb/command.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
lib/bb/command.py +1 -1
@@ -564,7 +564,7 @@ class CommandsSync: if config_data: # We have to use a different function here if we're passing in a datastore # NOTE: we took a copy above, so we don't do it here again - envdata = bb.cache.parse_recipe(config_data, fn, appendfiles, mc)[''] + envdata = command.cooker.databuilder._parse_recipe(config_data, fn, appendfiles, mc)[''] else: # Use the standard path envdata = command.cooker.databuilder.parseRecipe(fn, appendfiles)
Signature · 1 line
2.37.1
Quoted 13 lines · Chen Qi
The
parse_recipehas been removed from bb.cache, replace it with the databuilder._parse_recipe.Signed-off-by Chen Qi <[email protected]>--- lib/bb/command.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)lib/bb/command.py +0
-0
lib/bb/command.py class CommandsSync: · line 570
Quoted 8 lines
lib/bb/command.py +1
-1@@ -564,7 +564,7 @@ class CommandsSync: if config_data: # We have to use a different function here if we're passing in a datastore # NOTE: we took a copy above, so we don't do it here again - envdata = bb.cache.parse_recipe(config_data, fn, appendfiles, mc)[''] + envdata = command.cooker.databuilder._parse_recipe(config_data, fn, appendfiles, mc)[''] else: # Use the standard path envdata = command.cooker.databuilder.parseRecipe(fn, appendfiles)
How did you run into that? I'm a bit worried we're missing some test coverage here?
Cheers,
Richard
Hi Richard,
I met the error on our layerindex.
Error message is as below.
NOTE: Starting bitbake server...
ERROR: Unable to read /home/oelayer/layerindex/git___lxgit_wrs_com_layers_meta-arm/meta-arm/recipes-bsp/trusted -firmware-m/trusted-firmware-m_1.6.0.bb: Traceback (most recent call last):
File "/home/oelayer/layerindex/git___lxgit_wrs_com_bitbake/lib/bb/command.py", line 90, in runCommand
result = command_method(self, commandline)
File "/home/oelayer/layerindex/git___lxgit_wrs_com_bitbake/lib/bb/command.py", line 567, in parseRecipeFile
envdata = bb.cache.parse_recipe(config_data, fn, appendfiles, mc)['']
AttributeError: module 'bb.cache' has no attribute 'parse_recipe'
Regards,
Qi
-----Original Message-----
From: Richard Purdie <[email protected]>
Sent: Thursday, November 24, 2022 8:02 PM
To: Chen, Qi <[email protected]>; [email protected]
Subject: Re: [bitbake-devel] [PATCH] command.py: cleanup bb.cache.parse_recipe
Quoted 23 lines · Chen Qi
The
parse_recipehas been removed from bb.cache, replace it with the databuilder._parse_recipe.Signed-off-by Chen Qi <[email protected]>--- lib/bb/command.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)lib/bb/command.py index renamed from lib/bb/command.py +0
-0
fa9fd054c..c9571f63d100644lib/bb/command.py +3
-1@@ -564,7 +564,7 @@ class CommandsSync: if config_data: # We have to use a different function here if we're passing in a datastore # NOTE: we took a copy above, so we don't do it here again - envdata = bb.cache.parse_recipe(config_data, fn, appendfiles, mc)[''] + envdata = + command.cooker.databuilder._parse_recipe(config_data, fn, + appendfiles, mc)[''] else: # Use the standard path envdata = command.cooker.databuilder.parseRecipe(fn,
Quoted 1 line
appendfiles)
How did you run into that? I'm a bit worried we're missing some test coverage here?
Cheers,
Richard
Hi Qi,
Quoted 11 lines · Qi
I met the error on our layerindex.
Error message is as below.
NOTE: Starting bitbake server...
ERROR: Unable to read/home/oelayer/layerindex/git___lxgit_wrs_com_layers_meta-arm/meta-arm/recipes-bsp/trusted-firmware-m/trusted-firmware-m_1.6.0.bb: Traceback (most recent call last):
File "/home/oelayer/layerindex/git___lxgit_wrs_com_bitbake/lib/bb/command.py", line 90, in runCommand
result =command_method(self, commandline)
File "/home/oelayer/layerindex/git___lxgit_wrs_com_bitbake/lib/bb/command.py", line 567, in parseRecipeFile
envdata =bb.cache.parse_recipe(config_data, fn, appendfiles, mc)['']
AttributeError: module 'bb.cache' has no attribute 'parse_recipe'
Thanks.
I had a look at what could call parseRecipeFile(). I can see some commented out code in layerindex but it doesn't pass a config_data. The only other calls are from tinfoil's parse_recipe_file() function and none of those I can see pass config_data either.
Can you see where the config_data is coming from?
It looks like we've broken this at least once before:
so we really need to figure out what is using it and add a test to ensure coverage. Is this some local customisation you have to layerindex?
Cheers,
Richard
Quoted 4 lines · Chen Qi
Hi Richard,
I met the error on our layerindex.
"our" is
https://layers.openembedded.org
or some other instance in windriver?
I'm asking because parsing on https://layers.openembedded.org seems to be broken for a while. I've checked randomly selected 3 layers and all show the same issue as "Updates" tab in:
https://layers.openembedded.org/layerindex/branch/master/layer/openembedded-core/
ERROR: fatal: Unable to create '/opt/workdir/git___git_openembedded_org_bitbake/.git/index.lock': File exists.
Another git process seems to be running in this repository, e.g. an editor opened by 'git commit'. Please make sure all processes are terminated then try again. If it still fails, a git process may have crashed in this repository earlier:
remove the file manually to continue.
ERROR: Traceback (most recent call last):
File "update_layer.py", line 384, in main
(tinfoil, tempdir) = recipeparse.init_parser(settings, branch,
bitbakepath, nocheckout=options.nocheckout, logger=logger)
File "/opt/layerindex/layerindex/recipeparse.py", line 36, in init_parser
utils.checkout_repo(bitbakepath, bitbake_ref, logger=logger)
File "/opt/layerindex/layerindex/utils.py", line 282, in checkout_repo
runcmd(['git', 'reset', '--hard'], repodir, logger=logger)
File "/opt/layerindex/layerindex/utils.py", line 369, in runcmd
raise e
subprocess.CalledProcessError: Command '['git', 'reset', '--hard']' returned non-zero exit status 128.
Quoted 72 lines
Error message is as below.
NOTE: Starting bitbake server...
ERROR: Unable to read/home/oelayer/layerindex/git___lxgit_wrs_com_layers_meta-arm/meta-arm/recipes-bsp/trusted-firmware-m/trusted-firmware-m_1.6.0.bb: Traceback (most recent call
last):
File
"/home/oelayer/layerindex/git___lxgit_wrs_com_bitbake/lib/bb/command.py", line 90, in runCommand
result =command_method(self, commandline)
File
"/home/oelayer/layerindex/git___lxgit_wrs_com_bitbake/lib/bb/command.py", line 567, in parseRecipeFile
envdata =bb.cache.parse_recipe(config_data, fn, appendfiles, mc)['']
AttributeError: module 'bb.cache' has no attribute 'parse_recipe'Regards,
Qi-----Original Message-----
From: Richard Purdie <[email protected]>
Sent: Thursday, November 24, 2022 8:02 PM
To: Chen, Qi <[email protected]>; [email protected]
Subject: Re: [bitbake-devel] [PATCH]command.py: cleanup bb.cache.parse_recipeQuoted 13 lines · Chen Qi
The
parse_recipehas been removed from bb.cache, replace it with the databuilder._parse_recipe.Signed-off-by Chen Qi <[email protected]>--- lib/bb/command.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)lib/bb/command.py index renamed from lib/bb/command.py +0
-0
fa9fd054c..c9571f63d100644lib/bb/command.py class CommandsSync: · line 565
Quoted 2 lines
lib/bb/command.py +0
-0@@ -564,7 +564,7 @@ class CommandsSync: if config_data: # We have to use a different function here if we're passingin a datastore
lib/bb/command.py class CommandsSync: · line 567
Quoted 2 lines
lib/bb/command.py +0
-1# NOTE: we took a copy above, so we don't do it here again - envdata = bb.cache.parse_recipe(config_data, fn,appendfiles, mc)['']
Quoted 7 lines
lib/bb/command.py +3
-0+ envdata = + command.cooker.databuilder._parse_recipe(config_data, fn, + appendfiles, mc)[''] else: # Use the standard path envdata = command.cooker.databuilder.parseRecipe(fn,appendfiles)
How did you run into that? I'm a bit worried we're missing some test coverage here?
Cheers,
Richard
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#14107):
https://lists.openembedded.org/g/bitbake-devel/message/14107
Mute This Topic: https://lists.openembedded.org/mt/95234638/3617156
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [ [email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Hi Richard and Martin,
I've setup a pure oe-core layerindex (No Wind River specific settings) locally, it works well.
For Richard's question, where is the config_data from, it is from layerindex-web/layerindex/update_layer.py:
envdata = tinfoil.parse_recipe_file(fn, appends=False, config_data=data)
The error will be gone if remove "config_date=data", so we can confirm that it is from the line.
For Martin, maybe a git process had crashed so that index.lock was not cleaned up, you can try to remove the lock file (Or ask layerindex maintainer to do that):
$ rm -f /opt/workdir/git___git_openembedded_org_bitbake/.git/index.lock
// Robert
Quoted 117 lines · Martin Jansa
On Thu, Nov 24, 2022 at 1:17 PM Chen Qi <[email protected] <mailto:[email protected]>> wrote:
Hi Richard,
I met the error on our layerindex.
"our" is
https://layers.openembedded.org <https://urldefense.com/v3/__https://layers.openembedded.org__;!!AjveYdw8EvQ!dAyNXqU5oP_MdnjNgg7iOIT0fNgYfCx6IJx_5jvGByabqoJrG8NnQky1xP1KOAGSW5K8aoKC-Im1stueEvYcmIy7OIU$>or some other instance in windriver?
I'm asking because parsing on https://layers.openembedded.org <https://urldefense.com/v3/__https://layers.openembedded.org__;!!AjveYdw8EvQ!dAyNXqU5oP_MdnjNgg7iOIT0fNgYfCx6IJx_5jvGByabqoJrG8NnQky1xP1KOAGSW5K8aoKC-Im1stueEvYcmIy7OIU$> seems to be broken for a while. I've checked randomly selected 3 layers and all show the same issue as "Updates" tab in:
https://layers.openembedded.org/layerindex/branch/master/layer/openembedded-core/ <https://urldefense.com/v3/__https://layers.openembedded.org/layerindex/branch/master/layer/openembedded-core/__;!!AjveYdw8EvQ!dAyNXqU5oP_MdnjNgg7iOIT0fNgYfCx6IJx_5jvGByabqoJrG8NnQky1xP1KOAGSW5K8aoKC-Im1stueEvYcM7oxI8A$>ERROR: fatal: Unable to create '
/opt/workdir/git___git_openembedded_org_bitbake/.git/index.lock': File exists.Another git process seems to be running in this repository, e.g.
an editor opened by 'git commit'. Please make sure all processes
are terminated then try again. If it still fails, a git process
may have crashed in this repository earlier:
remove the file manually to continue.
ERROR: Traceback (most recent call last):
File "update_layer.py", line 384, in main
(tinfoil, tempdir) = recipeparse.init_parser(settings, branch, bitbakepath,
nocheckout=options.nocheckout, logger=logger)
File "/opt/layerindex/layerindex/recipeparse.py", line 36, ininit_parserutils.checkout_repo(bitbakepath, bitbake_ref, logger=logger)
File "/opt/layerindex/layerindex/utils.py", line 282, incheckout_reporuncmd(['git', 'reset', '--hard'], repodir, logger=logger)
File "/opt/layerindex/layerindex/utils.py", line 369, in runcmd
raise e
subprocess.CalledProcessError: Command '['git', 'reset', '--hard']' returned non-zero exit status 128.Error message is as below.
NOTE: Starting bitbake server...
ERROR: Unable to read
/home/oelayer/layerindex/git___lxgit_wrs_com_layers_meta-arm/meta-arm/recipes-bsp/trusted
-firmware-m/trusted-firmware-m_1.6.0.bb
<https://urldefense.com/v3/__http://trusted-firmware-m_1.6.0.bb__;!!AjveYdw8EvQ!dAyNXqU5oP_MdnjNgg7iOIT0fNgYfCx6IJx_5jvGByabqoJrG8NnQky1xP1KOAGSW5K8aoKC-Im1stueEvYcR8tYvdQ$>: Traceback (most recent call last):
File
"/home/oelayer/layerindex/git___lxgit_wrs_com_bitbake/lib/bb/command.py",
line 90, in runCommand
result =command_method(self, commandline)
File
"/home/oelayer/layerindex/git___lxgit_wrs_com_bitbake/lib/bb/command.py",
line 567, in parseRecipeFile
envdata =bb.cache.parse_recipe(config_data, fn, appendfiles, mc)['']
AttributeError: module 'bb.cache' has no attribute 'parse_recipe'Regards,
Qi-----Original Message-----
From: Richard Purdie <[email protected]
<mailto:[email protected]>>
Sent: Thursday, November 24, 2022 8:02 PM
To: Chen, Qi <[email protected] <mailto:[email protected]>>;
[email protected]
<mailto:[email protected]>
Subject: Re: [bitbake-devel] [PATCH]command.py
<https://urldefense.com/v3/__http://command.py__;!!AjveYdw8EvQ!dAyNXqU5oP_MdnjNgg7iOIT0fNgYfCx6IJx_5jvGByabqoJrG8NnQky1xP1KOAGSW5K8aoKC-Im1stueEvYcJ_BcC5o$>: cleanup bb.cache.parse_recipeOn Thu, 2022-11-24 at 01:10 -0800, Chen Qi wrote:
> Theparse_recipehas been removed from bb.cache, replace it with the
> databuilder._parse_recipe.
>
> Signed-off-by: Chen Qi <[email protected] <mailto:[email protected]>>
> ---
>lib/bb/command.py
<https://urldefense.com/v3/__http://command.py__;!!AjveYdw8EvQ!dAyNXqU5oP_MdnjNgg7iOIT0fNgYfCx6IJx_5jvGByabqoJrG8NnQky1xP1KOAGSW5K8aoKC-Im1stueEvYcJ_BcC5o$> | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --gita/lib/bb/command.py
<https://urldefense.com/v3/__http://command.py__;!!AjveYdw8EvQ!dAyNXqU5oP_MdnjNgg7iOIT0fNgYfCx6IJx_5jvGByabqoJrG8NnQky1xP1KOAGSW5K8aoKC-Im1stueEvYcJ_BcC5o$>b/lib/bb/command.py<https://urldefense.com/v3/__http://command.py__;!!AjveYdw8EvQ!dAyNXqU5oP_MdnjNgg7iOIT0fNgYfCx6IJx_5jvGByabqoJrG8NnQky1xP1KOAGSW5K8aoKC-Im1stueEvYcJ_BcC5o$> index
>fa9fd054c..c9571f63d100644
> ---a/lib/bb/command.py
<https://urldefense.com/v3/__http://command.py__;!!AjveYdw8EvQ!dAyNXqU5oP_MdnjNgg7iOIT0fNgYfCx6IJx_5jvGByabqoJrG8NnQky1xP1KOAGSW5K8aoKC-Im1stueEvYcJ_BcC5o$>
> +++b/lib/bb/command.py
<https://urldefense.com/v3/__http://command.py__;!!AjveYdw8EvQ!dAyNXqU5oP_MdnjNgg7iOIT0fNgYfCx6IJx_5jvGByabqoJrG8NnQky1xP1KOAGSW5K8aoKC-Im1stueEvYcJ_BcC5o$>
> @@ -564,7 +564,7 @@ class CommandsSync:
> ifconfig_data:
> # We have to use a different function here if we're passing
in a datastore
> # NOTE: we took a copy above, so we don't do it here again
> - envdata = bb.cache.parse_recipe(config_data, fn,
appendfiles, mc)['']
> + envdata =
> + command.cooker.databuilder._parse_recipe(config_data, fn,
> + appendfiles, mc)['']
> else:
> # Use the standard path
> envdata = command.cooker.databuilder.parseRecipe(fn,
> appendfiles)How did you run into that? I'm a bit worried we're missing some test coverage here?
Cheers,
Richard
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#14107):
https://lists.openembedded.org/g/bitbake-devel/message/14107
<https://urldefense.com/v3/__https://lists.openembedded.org/g/bitbake-devel/message/14107__;!!AjveYdw8EvQ!dAyNXqU5oP_MdnjNgg7iOIT0fNgYfCx6IJx_5jvGByabqoJrG8NnQky1xP1KOAGSW5K8aoKC-Im1stueEvYcFi8osuw$>
Mute This Topic: https://lists.openembedded.org/mt/95234638/3617156
<https://urldefense.com/v3/__https://lists.openembedded.org/mt/95234638/3617156__;!!AjveYdw8EvQ!dAyNXqU5oP_MdnjNgg7iOIT0fNgYfCx6IJx_5jvGByabqoJrG8NnQky1xP1KOAGSW5K8aoKC-Im1stueEvYcb2VAskM$>
Group Owner: [email protected]
<mailto:bitbake-devel%[email protected]>
Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub
<https://urldefense.com/v3/__https://lists.openembedded.org/g/bitbake-devel/unsub__;!!AjveYdw8EvQ!dAyNXqU5oP_MdnjNgg7iOIT0fNgYfCx6IJx_5jvGByabqoJrG8NnQky1xP1KOAGSW5K8aoKC-Im1stueEvYc339AjuA$> [[email protected] <mailto:[email protected]>]
-=-=-=-=-=-=-=-=-=-=-=-
Quoted 5 lines · Robert Yang
Hi Richard and Martin,
I've setup a pure oe-core layerindex (No Wind River specific settings) locally, it works well.
Quoted 8 lines
For Richard's question, where is the
config_datafrom, it is fromlayerindex-web/layerindex/update_layer.py:envdata =
tinfoil.parse_recipe_file(fn, appends=False, config_data=data)The error will be gone if remove "config_date=data", so we can confirm that it is from the line.
Thanks, that helps. I've sent a patch to the tinfoil selftests in OE- Core so we will test this piece of API in future and try and prevent regressions.
Cheers,
Richard