Skip to contentSkip to conversation

devtools: control location of test builds

PATCH New Bruce Richardson 4 participants 6 messages Oct 9, 2019 – Nov 28, 2019 Arm/Amlogic Meson SoC support

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 devtools: control location of test builds

By default, both test-build.sh and test-meson-builds.sh scripts create the builds they generate in the current working directory, leading to a large number of build directories being present when testing patches. This patchset modifies both scripts to use a DPDK_TEST_BUILD_DIR environment variable to control where the build outputs are put.

For example, doing:

    export DPDK_TEST_BUILD_DIR=__builds
    ./devtools/test-meson-builds.sh && ./devtools/test-build.sh \
            x86_64-native-linux-clang+shared i686-native-linux-gcc

gives a "__builds" directory with 14 meson and 2 make builds (with the meson build count depending on compiler availability)

---
 devtools/test-build.sh              | 3 ++-
 devtools/test-meson-builds.sh       | 7 ++++---
 doc/guides/contributing/patches.rst | 6 ++++++
 3 files changed, 12 insertions(+), 4 deletions(-)
devtools/test-build.sh +2 -1
index 2bedbdb98..b64c963d7 100755
@@ -63,6 +63,7 @@ print_help () {
 [ -z $MAKE ] && echo "Cannot find make or gmake" && exit 1
 
 J=$DPDK_MAKE_JOBS
+builds_dir=${DPDK_TEST_BUILD_DIR:-.}
 short=false
 unset verbose
 maxerr=-Wfatal-errors
@@ -234,7 +235,7 @@ for conf in $configs ; do
 	. $(dirname $(readlink -f $0))/load-devel-config
 
 	options=$(echo $conf | sed 's,[^~+]*,,')
-	dir=$conf
+	dir=$builds_dir/$conf
 	config $dir $target $options
 
 	echo "================== Build $dir"
devtools/test-meson-builds.sh +4 -3
index 08e83eb5c..f61709761 100755
@@ -14,6 +14,7 @@ set -o | grep -q pipefail && set -o pipefail && PIPEFAIL=1
 srcdir=$(dirname $(readlink -f $0))/..
 MESON=${MESON:-meson}
 use_shared="--default-library=shared"
+builds_dir=${DPDK_TEST_BUILD_DIR:-.}
 
 if command -v gmake >/dev/null 2>&1 ; then
 	MAKE=gmake
@@ -50,7 +51,7 @@ load_env () # <target compiler>
 
 build () # <directory> <target compiler> <meson options>
 {
-	builddir=$1
+	builddir=$builds_dir/$1
 	shift
 	targetcc=$1
 	shift
@@ -125,8 +126,8 @@ done
 
 # Test installation of the x86-default target, to be used for checking
 # the sample apps build using the pkg-config file for cflags and libs
-build_path=build-x86-default
-export DESTDIR=$(pwd)/$build_path/install-root
+build_path=$(readlink -f $builds_dir/build-x86-default)
+export DESTDIR=$build_path/install-root
 $ninja_cmd -C $build_path install
 
 load_env cc
doc/guides/contributing/patches.rst +6 -0
index 9e1013b70..b2d94fb28 100644
@@ -495,6 +495,12 @@ Compilation of patches is to be tested with ``devtools/test-meson-builds.sh`` sc
 
 The script internally checks for dependencies, then builds for several
 combinations of compilation configuration.
+By default, each build will be put in a subfolder of the current working directory.
+However, if it is preferred to place the builds in a different location,
+the environment variable ``DPDK_TEST_BUILD_DIR`` can be set to that desired location.
+For example, setting ``DPDK_TEST_BUILD_DIR=__builds`` will put all builds
+in a single subfolder called "__builds" created in the current directory.
+Setting ``DPDK_TEST_BUILD_DIR`` to an absolute directory path e.g. ``/tmp`` is also supported.
 
 
 Sending Patches
Signature · 1 line
2.21.0
#1 lore raw

PATCH v2 1/1 devtools: control location of test builds

From: Bruce Richardson <[email protected]>

By default, both test-build.sh and test-meson-builds.sh scripts create the builds they generate in the current working directory, leading to a large number of build directories being present when testing patches. This patchset modifies both scripts to use a DPDK_BUILD_TEST_DIR environment variable to control where the build outputs are put.

For example, doing:

    export DPDK_BUILD_TEST_DIR=__builds
    ./devtools/test-meson-builds.sh && ./devtools/test-build.sh \
            x86_64-native-linux-clang+shared i686-native-linux-gcc

gives a "__builds" directory with 14 meson and 2 make builds (with the meson build count depending on compiler availability)

---
v2:
- rename DPDK_TEST_BUILD_DIR to DPDK_BUILD_TEST_DIR
- add comment in devtools/test-build.sh and its guide section
- load from config file in test-meson-builds.sh

---
 devtools/test-build.sh              | 4 +++-
 devtools/test-meson-builds.sh       | 9 ++++++---
 doc/guides/contributing/patches.rst | 7 +++++++
 3 files changed, 16 insertions(+), 4 deletions(-)
devtools/test-build.sh +3 -1
index 2bedbdb980..f55d3120aa 100755
@@ -7,6 +7,7 @@ default_path=$PATH
 # Load config options:
 # - ARMV8_CRYPTO_LIB_PATH
 # - DPDK_BUILD_TEST_CONFIGS (defconfig1+option1+option2 defconfig2)
+# - DPDK_BUILD_TEST_DIR
 # - DPDK_DEP_ARCHIVE
 # - DPDK_DEP_BPF (y/[n])
 # - DPDK_DEP_CFLAGS
@@ -63,6 +64,7 @@ print_help () {
 [ -z $MAKE ] && echo "Cannot find make or gmake" && exit 1
 
 J=$DPDK_MAKE_JOBS
+builds_dir=${DPDK_BUILD_TEST_DIR:-.}
 short=false
 unset verbose
 maxerr=-Wfatal-errors
@@ -234,7 +236,7 @@ for conf in $configs ; do
 	. $(dirname $(readlink -f $0))/load-devel-config
 
 	options=$(echo $conf | sed 's,[^~+]*,,')
-	dir=$conf
+	dir=$builds_dir/$conf
 	config $dir $target $options
 
 	echo "================== Build $dir"
devtools/test-meson-builds.sh +6 -3
index 07d42985e6..571deec257 100755
@@ -12,8 +12,11 @@ PIPEFAIL=""
 set -o | grep -q pipefail && set -o pipefail && PIPEFAIL=1
 
 srcdir=$(dirname $(readlink -f $0))/..
+. $srcdir/devtools/load-devel-config
+
 MESON=${MESON:-meson}
 use_shared="--default-library=shared"
+builds_dir=${DPDK_BUILD_TEST_DIR:-.}
 
 if command -v gmake >/dev/null 2>&1 ; then
 	MAKE=gmake
@@ -50,7 +53,7 @@ load_env () # <target compiler>
 
 build () # <directory> <target compiler> <meson options>
 {
-	builddir=$1
+	builddir=$builds_dir/$1
 	shift
 	targetcc=$1
 	shift
@@ -125,8 +128,8 @@ done
 
 # Test installation of the x86-default target, to be used for checking
 # the sample apps build using the pkg-config file for cflags and libs
-build_path=build-x86-default
-export DESTDIR=$(pwd)/$build_path/install-root
+build_path=$(readlink -f $builds_dir/build-x86-default)
+export DESTDIR=$build_path/install-root
 $ninja_cmd -C $build_path install
 
 load_env cc
doc/guides/contributing/patches.rst +7 -0
index 2140303464..e3246aca5e 100644
@@ -471,6 +471,7 @@ Examples of configs are::
 The builds can be modified via the following environmental variables:
 
 * ``DPDK_BUILD_TEST_CONFIGS`` (target1+option1+option2 target2)
+* ``DPDK_BUILD_TEST_DIR``
 * ``DPDK_DEP_CFLAGS``
 * ``DPDK_DEP_LDFLAGS``
 * ``DPDK_DEP_PCAP`` (y/[n])
@@ -495,6 +496,12 @@ Compilation of patches is to be tested with ``devtools/test-meson-builds.sh`` sc
 
 The script internally checks for dependencies, then builds for several
 combinations of compilation configuration.
+By default, each build will be put in a subfolder of the current working directory.
+However, if it is preferred to place the builds in a different location,
+the environment variable ``DPDK_BUILD_TEST_DIR`` can be set to that desired location.
+For example, setting ``DPDK_BUILD_TEST_DIR=__builds`` will put all builds
+in a single subfolder called "__builds" created in the current directory.
+Setting ``DPDK_BUILD_TEST_DIR`` to an absolute directory path e.g. ``/tmp`` is also supported.
 
 
 Sending Patches
Signature · 1 line
2.23.0
#2 lore raw
Quoted 16 lines · Thomas Monjalon

From: Bruce Richardson <[email protected]>

By default, both test-build.sh and test-meson-builds.sh scripts create the builds they generate in the current working directory, leading to a large number of build directories being present when testing patches. This patchset modifies both scripts to use a DPDK_BUILD_TEST_DIR environment variable to control where the build outputs are put.

For example, doing:

    export DPDK_BUILD_TEST_DIR=__builds
    ./devtools/test-meson-builds.sh && ./devtools/test-build.sh \
            x86_64-native-linux-clang+shared i686-native-linux-gcc

gives a "__builds" directory with 14 meson and 2 make builds (with the meson build count depending on compiler availability)

Quoted 8 lines

---
v2:
- rename DPDK_TEST_BUILD_DIR to DPDK_BUILD_TEST_DIR
- add comment in devtools/test-build.sh and its guide section
- load from config file in test-meson-builds.sh
---

Thanks for the update, those seem ok changes to me. The doc one I should have caught in the V1, so it's especially welcome.

/Bruce

#3 lore raw
In reply to Thomas Monjalon

HI Bruce, Thomas,
Doesn't the RTE_OUTPUT environment var already resolve this issue? Fiona

Quoted 129 lines

-----Original Message-----
From: dev <[email protected]> On Behalf Of Thomas Monjalon
Sent: Wednesday, November 27, 2019 11:01 PM
To: Mcnamara, John <[email protected]>; Kovacevic, Marko <[email protected]>
Cc: [email protected]; Yigit, Ferruh <[email protected]>; [email protected]; Richardson, Bruce <[email protected]>
Subject: [dpdk-dev] [PATCH v2] devtools: control location of test builds

From: Bruce Richardson <[email protected]>

By default, both test-build.sh and test-meson-builds.sh scripts create the builds they generate in the current working directory, leading to a large number of build directories being present when testing patches. This patchset modifies both scripts to use a DPDK_BUILD_TEST_DIR environment variable to control where the build outputs are put.

For example, doing:

    export DPDK_BUILD_TEST_DIR=__builds
    ./devtools/test-meson-builds.sh && ./devtools/test-build.sh \
            x86_64-native-linux-clang+shared i686-native-linux-gcc

gives a "__builds" directory with 14 meson and 2 make builds (with the meson build count depending on compiler availability)

---
v2:
- rename DPDK_TEST_BUILD_DIR to DPDK_BUILD_TEST_DIR
- add comment in devtools/test-build.sh and its guide section
- load from config file in test-meson-builds.sh

---
 devtools/test-build.sh              | 4 +++-
 devtools/test-meson-builds.sh       | 9 ++++++---
 doc/guides/contributing/patches.rst | 7 +++++++
 3 files changed, 16 insertions(+), 4 deletions(-)
devtools/test-build.sh +3 -1
index 2bedbdb980..f55d3120aa 100755
@@ -7,6 +7,7 @@ default_path=$PATH
 # Load config options:
 # - ARMV8_CRYPTO_LIB_PATH
 # - DPDK_BUILD_TEST_CONFIGS (defconfig1+option1+option2 defconfig2)
+# - DPDK_BUILD_TEST_DIR
 # - DPDK_DEP_ARCHIVE
 # - DPDK_DEP_BPF (y/[n])
 # - DPDK_DEP_CFLAGS
@@ -63,6 +64,7 @@ print_help () {
 [ -z $MAKE ] && echo "Cannot find make or gmake" && exit 1
 
 J=$DPDK_MAKE_JOBS
+builds_dir=${DPDK_BUILD_TEST_DIR:-.}
 short=false
 unset verbose
 maxerr=-Wfatal-errors
@@ -234,7 +236,7 @@ for conf in $configs ; do
 	. $(dirname $(readlink -f $0))/load-devel-config
 
 	options=$(echo $conf | sed 's,[^~+]*,,')
-	dir=$conf
+	dir=$builds_dir/$conf
 	config $dir $target $options
 
 	echo "================== Build $dir"
devtools/test-meson-builds.sh +6 -3
index 07d42985e6..571deec257 100755
@@ -12,8 +12,11 @@ PIPEFAIL=""
 set -o | grep -q pipefail && set -o pipefail && PIPEFAIL=1
 
 srcdir=$(dirname $(readlink -f $0))/..
+. $srcdir/devtools/load-devel-config
+
 MESON=${MESON:-meson}
 use_shared="--default-library=shared"
+builds_dir=${DPDK_BUILD_TEST_DIR:-.}
 
 if command -v gmake >/dev/null 2>&1 ; then
 	MAKE=gmake
@@ -50,7 +53,7 @@ load_env () # <target compiler>
 
 build () # <directory> <target compiler> <meson options>
 {
-	builddir=$1
+	builddir=$builds_dir/$1
 	shift
 	targetcc=$1
 	shift
@@ -125,8 +128,8 @@ done
 
 # Test installation of the x86-default target, to be used for checking
 # the sample apps build using the pkg-config file for cflags and libs
-build_path=build-x86-default
-export DESTDIR=$(pwd)/$build_path/install-root
+build_path=$(readlink -f $builds_dir/build-x86-default)
+export DESTDIR=$build_path/install-root
 $ninja_cmd -C $build_path install
 
 load_env cc
doc/guides/contributing/patches.rst +7 -1
index 2140303464..e3246aca5e 100644
@@ -471,6 +471,7 @@ Examples of configs are::
 The builds can be modified via the following environmental variables:
 
 * ``DPDK_BUILD_TEST_CONFIGS`` (target1+option1+option2 target2)
+* ``DPDK_BUILD_TEST_DIR``
 * ``DPDK_DEP_CFLAGS``
 * ``DPDK_DEP_LDFLAGS``
 * ``DPDK_DEP_PCAP`` (y/[n])
@@ -495,6 +496,12 @@ Compilation of patches is to be tested with ``devtools/test-meson-builds.sh`` sc
 
 The script internally checks for dependencies, then builds for several
 combinations of compilation configuration.
+By default, each build will be put in a subfolder of the current working directory.
+However, if it is preferred to place the builds in a different location,
+the environment variable ``DPDK_BUILD_TEST_DIR`` can be set to that desired location.
+For example, setting ``DPDK_BUILD_TEST_DIR=__builds`` will put all builds
+in a single subfolder called "__builds" created in the current directory.
+Setting ``DPDK_BUILD_TEST_DIR`` to an absolute directory path e.g. ``/tmp`` is also supported.
 
 
 Sending Patches
--

2.23.0

#4 lore raw

28/11/2019 12:47, Trahe, Fiona:

Quoted 2 lines

HI Bruce, Thomas,
Doesn't the RTE_OUTPUT environment var already resolve this issue?

RTE_OUTPUT is for the build directory, yes.

This patch is about having a top-level directory for all builds done by the build test scripts.

Acked-by on v2 1/1 Applied
#5 lore raw
In reply to Thomas Monjalon
Quoted 14 lines · Thomas Monjalon

From: Bruce Richardson <[email protected]>

By default, both test-build.sh and test-meson-builds.sh scripts create the builds they generate in the current working directory, leading to a large number of build directories being present when testing patches. This patchset modifies both scripts to use a DPDK_BUILD_TEST_DIR environment variable to control where the build outputs are put.

For example, doing:

    export DPDK_BUILD_TEST_DIR=__builds
    ./devtools/test-meson-builds.sh && ./devtools/test-build.sh \
            x86_64-native-linux-clang+shared i686-native-linux-gcc
Quoted 5 lines

gives a "__builds" directory with 14 meson and 2 make builds (with the meson build count depending on compiler availability)

Applied, thanks.

Signature · 1 line
David Marchand