scripts/lib/compatlayer: detect_layers always use realpath's
PATCH 1/2 scripts/lib/compatlayer: detect_layers always use realpath's
If you are using relative paths and change to other folder for execution it will fail, so use realpaths always.
[YOCTO #11164]
--- scripts/lib/compatlayer/__init__.py | 1 + 1 file changed, 1 insertion(+)
scripts/lib/compatlayer/__init__.py +1 -0
@@ -112,6 +112,7 @@ def detect_layers(layer_directories, no_auto): layers = [] for directory in layer_directories: + directory = os.path.realpath(directory) if directory[-1] == '/': directory = directory[0:-1]
Signature · 1 line
2.1.4
PATCH 2/2 scripts: Add yocto-compat-layer-wrapper
This script will be used to create it's own environment to make runs of yocto-compat-layer.py againts layers isolated.
It clones the oe-core and bitbake repository in a temporary folder and then calls yocto-compat-layer.py with specified LAYERS_DIRs to test.
Example:
$ ./scripts/yocto-compat-layer-wrapper LAYER_DIR LAYER_DIR_N
[YOCTO #11164]
--- scripts/yocto-compat-layer-wrapper | 68 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 68 insertions(+) create mode 100755 scripts/yocto-compat-layer-wrapper
scripts/yocto-compat-layer-wrapper new file +68 -0
@@ -0,0 +1,68 @@ +#!/usr/bin/env bash + +# Yocto Project compatibility layer tool wrapper +# +# Copyright (C) 2017 Intel Corporation +# Released under the MIT license (see COPYING.MIT) + +oe_core_repo="git://git.openembedded.org/openembedded-core" +bitbake_repo="git://git.openembedded.org/bitbake" + +show_help() { + printf "Usage: %s [-o output_log] [-h] LAYER_DIR ...\n" $0 +} + +output_log='' +layer_dirs="" +while getopts o:h arg +do + case $arg in + o) + output_log=$(realpath "$OPTARG");; + h) + show_help + exit 0;; + ?) + show_help + exit 2;; + esac +done +shift $(($OPTIND - 1)) +if [[ -z $* ]]; then + show_help + exit 1 +else + for layer_dir in $(realpath $*); do + layer_dirs="$layer_dirs $layer_dir" + done +fi + +env_dir=$(mktemp -d -t yocto-compat-XXXX) +echo "The environment will be setup at $env_dir" +echo "" + +echo "Cloning oe-core..." +git clone $oe_core_repo $env_dir +if [ $? -ne 0 ]; then + echo "Failed to clone oe-core repository" + exit 1 +fi + +echo "Cloning bitbake..." +git clone $bitbake_repo $env_dir/bitbake +if [ $? -ne 0 ]; then + echo "Failed to clone bitbake repository" + exit 1 +fi + +echo "" +echo "Changing directory to $env_dir" +cd $env_dir +source oe-init-build-env +if [[ -z $output_log ]]; then + echo "Running yocto-compat-layer.py $layer_dirs" + yocto-compat-layer.py $layer_dirs +else + echo "Running yocto-compat-layer.py -o $output_log $layer_dirs" + yocto-compat-layer.py -o $output_log $layer_dirs +fi
Signature · 1 line
2.1.4
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): [email protected]
Re: [PATCH 2/2] scripts: Add yocto-compat-layer-wrapper
Sending to oe-core ml, i used wrong oe-core mail address, :/,
Anibal
Quoted 65 lines · Aníbal Limón
Quoted 7 lines · Patrick Ohly
On Wed, 2017-03-29 at 15:44 -0600, Aníbal Limón wrote:
...Quoted 4 lines
scripts/yocto-compat-layer-wrapper new file +4
-0+show_help() { + printf "Usage: %s [-o output_log] [-h] LAYER_DIR ...\n" $0 +} +...
Quoted 5 lines
scripts/yocto-compat-layer-wrapper line 42
Quoted 3 lines
scripts/yocto-compat-layer-wrapper new file +3
-0+env_dir=$(mktemp -d -t yocto-compat-XXXX) +echo "The environment will be setup at $env_dir" +echo ""The directory gets created, but not removed.
I didn't remove the temp directory because may be the user wants to access the dir after the check.
Quoted 19 lines
Quoted 4 lines
scripts/yocto-compat-layer-wrapper new file +3
-0+echo "Cloning oe-core..." +git clone $oe_core_repo $env_dir +if [ $? -ne 0 ]; thenscripts/yocto-compat-layer-wrapper line 56
Quoted 10 lines
scripts/yocto-compat-layer-wrapper new file +10
-0+ echo "Failed to clone oe-core repository" + exit 1 +fi + +echo "Cloning bitbake..." +git clone $bitbake_repo $env_dir/bitbake +if [ $? -ne 0 ]; then + echo "Failed to clone bitbake repository" + exit 1 +fiCloning bitbake and OE-core each time the script runs will be fairly slow. There's also a chicken-and-egg problem: if you don't have bitbake, where's the script?
Quoted 2 lines
I'd prefer to use an existing checkout of both, just as for the layers which are to be tested.
I choose to clone the oe-core/bitbake to ensure there are a clean environment, without any previous layer added. I could add an option to specify a oe-core/bitbake dir if isn't set then clone.
Quoted 14 lines
scripts/yocto-compat-layer-wrapper line 68
Quoted 10 lines
scripts/yocto-compat-layer-wrapper new file +9
-0+cd $env_dir +source oe-init-build-env +if [[ -z $output_log ]]; then + echo "Running yocto-compat-layer.py $layer_dirs" + yocto-compat-layer.py $layer_dirs +else + echo "Running yocto-compat-layer.py -o $output_log $layer_dirs" + yocto-compat-layer.py -o $output_log $layer_dirs +fiIt would be useful to accept and pass through all
yocto-compat-layer.pyparameters, ideally without having to repeat a full list of them in the argument parsing of the wrapper.Agree,
Anibal
Quoted 1 line
Re: [PATCH 2/2] scripts: Add yocto-compat-layer-wrapper
In reply to a message not in the archiveQuoted 9 lines · Aníbal Limón
Quoted 7 lines · Patrick Ohly
On Wed, 2017-03-29 at 15:44 -0600, Aníbal Limón wrote:
...Quoted 4 lines
scripts/yocto-compat-layer-wrapper new file +4
-0+show_help() { + printf "Usage: %s [-o output_log] [-h] LAYER_DIR ...\n" $0 +} +...
Quoted 8 lines
Quoted 5 lines
scripts/yocto-compat-layer-wrapper line 42
Quoted 3 lines
scripts/yocto-compat-layer-wrapper new file +3
-0+env_dir=$(mktemp -d -t yocto-compat-XXXX) +echo "The environment will be setup at $env_dir" +echo ""The directory gets created, but not removed.
I didn't remove the temp directory because may be the user wants to access the dir after the check.
I think that this should be something that the user explicitly needs to request. As it is now, accumulating directories in /tmp when invoking the script is just unexpected and not the normal behavior of tools creating something in /tmp.
Quoted 18 lines
Quoted 18 lines
scripts/yocto-compat-layer-wrapper line 56
Quoted 13 lines
scripts/yocto-compat-layer-wrapper new file +13
-0+echo "Cloning oe-core..." +git clone $oe_core_repo $env_dir +if [ $? -ne 0 ]; then + echo "Failed to clone oe-core repository" + exit 1 +fi + +echo "Cloning bitbake..." +git clone $bitbake_repo $env_dir/bitbake +if [ $? -ne 0 ]; then + echo "Failed to clone bitbake repository" + exit 1 +fiCloning bitbake and OE-core each time the script runs will be fairly slow. There's also a chicken-and-egg problem: if you don't have bitbake, where's the script?
Quoted 5 lines
Quoted 2 lines
I'd prefer to use an existing checkout of both, just as for the layers which are to be tested.
I choose to clone the oe-core/bitbake to ensure there are a clean environment, without any previous layer added.
I don't quite get that argument. When using existing bitbake and OE-core directories, how can they have layers added to them? I understand that the existing checkout might contain additional modifications and/or might not match current master, but I consider that a feature. As it stands now, the caller of the script cannot test against Yocto 2.3 once it is released, because the script will always check out master.
Control over that can of course also be added to the script, but I just don't see the need for all this additional complexity. Just my 2 cents.
Quoted 2 lines
I could add an option to
specify a oe-core/bitbake dir if isn't set then clone.
At the very least please do that.
Signature · 6 lines
Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter.
Re: [PATCH 2/2] scripts: Add yocto-compat-layer-wrapper
Quoted 16 lines · Patrick Ohly
Quoted 15 lines · Aníbal Limón
Quoted 12 lines · Patrick Ohly
On Wed, 2017-03-29 at 15:44 -0600, Aníbal Limón wrote:
...Quoted 4 lines
scripts/yocto-compat-layer-wrapper new file +4
-0+show_help() { + printf "Usage: %s [-o output_log] [-h] LAYER_DIR ...\n" $0 +} +...
scripts/yocto-compat-layer-wrapper line 42
Quoted 3 lines
scripts/yocto-compat-layer-wrapper new file +3
-0+env_dir=$(mktemp -d -t yocto-compat-XXXX) +echo "The environment will be setup at $env_dir" +echo ""The directory gets created, but not removed.
Quoted 7 lines
Quoted 2 lines
I didn't remove the temp directory because may be the user wants to access the dir after the check.
I think that this should be something that the user explicitly needs to request. As it is now, accumulating directories in /tmp when invoking the script is just unexpected and not the normal behavior of tools creating something in /tmp.
Ok, i will add an option to don't delete and change the default behavior to delete after check.
Quoted 32 lines
Quoted 19 lines
Quoted 19 lines
Quoted 4 lines
scripts/yocto-compat-layer-wrapper new file +3
-0+echo "Cloning oe-core..." +git clone $oe_core_repo $env_dir +if [ $? -ne 0 ]; thenscripts/yocto-compat-layer-wrapper line 56
Quoted 10 lines
scripts/yocto-compat-layer-wrapper new file +10
-0+ echo "Failed to clone oe-core repository" + exit 1 +fi + +echo "Cloning bitbake..." +git clone $bitbake_repo $env_dir/bitbake +if [ $? -ne 0 ]; then + echo "Failed to clone bitbake repository" + exit 1 +fiCloning bitbake and OE-core each time the script runs will be fairly slow. There's also a chicken-and-egg problem: if you don't have bitbake, where's the script?
Quoted 5 lines
Quoted 2 lines
I'd prefer to use an existing checkout of both, just as for the layers which are to be tested.
I choose to clone the oe-core/bitbake to ensure there are a clean environment, without any previous layer added.
I don't quite get that argument. When using existing bitbake and OE-core directories, how can they have layers added to them? I understand that the existing checkout might contain additional modifications and/or might not match current master, but I consider that a feature. As it stands now, the caller of the script cannot test against Yocto 2.3 once it is released, because the script will always check out master.
Quoted 2 lines
Control over that can of course also be added to the script, but I just don't see the need for all this additional complexity. Just my 2 cents.
That's the original issue to create this script, if a user tries to validate a layer and has modifications in configuration (bblayers, local, auto), the check will be contaminated in some how.
If you want to run in some previous cloned repository you can use directly the yocto-compat-layer.py without the wrapper.
Quoted 5 lines
Quoted 3 lines
I could add an option to
specify a oe-core/bitbake dir if isn't set then clone.At the very least please do that.
Yes i will do.
Anibal
Quoted 1 line
Re: [PATCH 2/2] scripts: Add yocto-compat-layer-wrapper
Quoted 75 lines · Aníbal Limón
Quoted 22 lines · Patrick Ohly
Quoted 17 lines · Aníbal Limón
Quoted 9 lines · Patrick Ohly
On Wed, 2017-03-29 at 15:44 -0600, Aníbal Limón wrote:
...Quoted 5 lines
(quoted diff) +2
-0+show_help() { + printf "Usage: %s [-o output_log] [-h] LAYER_DIR...\n" $0
(quoted diff) +2
-0+} +...
Quoted 5 lines
scripts/yocto-compat-layer-wrapper line 42
Quoted 3 lines
scripts/yocto-compat-layer-wrapper new file +3
-0+env_dir=$(mktemp -d -t yocto-compat-XXXX) +echo "The environment will be setup at $env_dir" +echo ""The directory gets created, but not removed.
Quoted 3 lines
I didn't remove the temp directory because may be the user wants to
access the dir after the check.Quoted 6 lines
I think that this should be something that the user explicitly needs to
request. As it is now, accumulating directories in /tmp when invoking
the script is just unexpected and not the normal behavior of tools creating something in /tmp.Ok, i will add an option to don't delete and change the default behavior
to delete after check.Quoted 39 lines
Quoted 21 lines
Quoted 21 lines
Quoted 4 lines
scripts/yocto-compat-layer-wrapper new file +3
-0+echo "Cloning oe-core..." +git clone $oe_core_repo $env_dir +if [ $? -ne 0 ]; thenscripts/yocto-compat-layer-wrapper line 56
Quoted 10 lines
scripts/yocto-compat-layer-wrapper new file +10
-0+ echo "Failed to clone oe-core repository" + exit 1 +fi + +echo "Cloning bitbake..." +git clone $bitbake_repo $env_dir/bitbake +if [ $? -ne 0 ]; then + echo "Failed to clone bitbake repository" + exit 1 +fiCloning bitbake and OE-core each time the script runs will be fairly
slow. There's also a chicken-and-egg problem: if you don't have bitbake,
where's the script?Quoted 6 lines
Quoted 3 lines
I'd prefer to use an existing checkout of both, just as for the layers
which are to be tested.I choose to clone the oe-core/bitbake to ensure there are a clean environment, without any previous layer added.
I don't quite get that argument. When using existing bitbake and OE-core
directories, how can they have layers added to them? I understand that
the existing checkout might contain additional modifications and/or might not match current master, but I consider that a feature. As it
stands now, the caller of the script cannot test against Yocto 2.3 once
it is released, because the script will always check out master.
Quoted 8 lines
Quoted 4 lines
Control over that can of course also be added to the script, but I just
don't see the need for all this additional complexity. Just my 2 cents.That's the original issue to create this script, if a user tries to validate a layer and has modifications in configuration (bblayers, local, auto), the check will be contaminated in some how.
Those are all build-directory state, not items which are tracked in the git repository.
Rather than cloning repositories to ensure clean state your script could ensure a unique/new build directory by sourcing oe-init-build-env with a non-standard build directory, perhaps checking whether the directory exists first and adding random characters until you get something new?
Joshua
Re: [PATCH 2/2] scripts: Add yocto-compat-layer-wrapper
Quoted 85 lines · Joshua Lock
Quoted 75 lines · Aníbal Limón
Quoted 22 lines · Patrick Ohly
Quoted 17 lines · Aníbal Limón
Quoted 9 lines · Patrick Ohly
On Wed, 2017-03-29 at 15:44 -0600, Aníbal Limón wrote:
...Quoted 5 lines
(quoted diff) +2
-0+show_help() { + printf "Usage: %s [-o output_log] [-h] LAYER_DIR...\n" $0
(quoted diff) +2
-0+} +...
Quoted 5 lines
scripts/yocto-compat-layer-wrapper line 42
Quoted 3 lines
scripts/yocto-compat-layer-wrapper new file +3
-0+env_dir=$(mktemp -d -t yocto-compat-XXXX) +echo "The environment will be setup at $env_dir" +echo ""The directory gets created, but not removed.
Quoted 3 lines
I didn't remove the temp directory because may be the user wants to
access the dir after the check.Quoted 6 lines
I think that this should be something that the user explicitly needs to
request. As it is now, accumulating directories in /tmp when invoking
the script is just unexpected and not the normal behavior of tools creating something in /tmp.Ok, i will add an option to don't delete and change the default behavior
to delete after check.Quoted 39 lines
Quoted 21 lines
Quoted 21 lines
Quoted 4 lines
scripts/yocto-compat-layer-wrapper new file +3
-0+echo "Cloning oe-core..." +git clone $oe_core_repo $env_dir +if [ $? -ne 0 ]; thenscripts/yocto-compat-layer-wrapper line 56
Quoted 10 lines
scripts/yocto-compat-layer-wrapper new file +10
-0+ echo "Failed to clone oe-core repository" + exit 1 +fi + +echo "Cloning bitbake..." +git clone $bitbake_repo $env_dir/bitbake +if [ $? -ne 0 ]; then + echo "Failed to clone bitbake repository" + exit 1 +fiCloning bitbake and OE-core each time the script runs will be fairly
slow. There's also a chicken-and-egg problem: if you don't have bitbake,
where's the script?Quoted 6 lines
Quoted 3 lines
I'd prefer to use an existing checkout of both, just as for the layers
which are to be tested.I choose to clone the oe-core/bitbake to ensure there are a clean environment, without any previous layer added.
I don't quite get that argument. When using existing bitbake and OE-core
directories, how can they have layers added to them? I understand that
the existing checkout might contain additional modifications and/or might not match current master, but I consider that a feature. As it
stands now, the caller of the script cannot test against Yocto 2.3 once
it is released, because the script will always check out master.Quoted 8 lines
Quoted 4 lines
Control over that can of course also be added to the script, but I just
don't see the need for all this additional complexity. Just my 2 cents.That's the original issue to create this script, if a user tries to validate a layer and has modifications in configuration (bblayers, local, auto), the check will be contaminated in some how.
Quoted 8 lines
Those are all build-directory state, not items which are tracked in the git repository.
Rather than cloning repositories to ensure clean state your script could ensure a unique/new build directory by sourcing oe-init-build-env with a non-standard build directory, perhaps checking whether the directory exists first and adding random characters until you get something new?
That's a very good idea, i can generate tmp build directory and in this way we can delete the logic for clone a new repository, so if anyone reject, i'll do that.
Anibal
Quoted 3 lines
Joshua