Skip to contentSkip to conversation

add testing of libdpdk pkg-config file

PATCH Superseded Bruce Richardson 4 participants 44 messages Apr 23, 2019 – May 2, 2019 Arm/Amlogic Meson SoC support

lore
Share & tools
lore
#0 lore raw

cover add testing of libdpdk pkg-config file

As part of the meson build, a pkg-config file for libdpdk is created, which allows apps to be compiled and linked against DPDK by taking the cflags and lib parameter from pkgconfig. The example app makefiles have been reworked to take account of this support, but the build of them against the .pc file was not regularly tested.

To rectify this, and give us greater confidence in the correctness of the .pc file, this set adds in the sample apps to the installation set for "ninja install" and then builds a subset of those apps against the pkg-config file to test it. In the process a small error when compiling the cmdline sample app using the .pc file was fixed.

Bruce Richardson (4):
examples: install examples as part of ninja install
examples: simplify getting list of all examples devtools/test-meson-builds: add testing of pkg-config file
build: add libbsd to pkg-config file if enabled

 config/meson.build            | 10 ++++------
 devtools/test-meson-builds.sh | 17 +++++++++++++++++
 examples/meson.build          | 17 +++++++++++++----
 meson.build                   |  2 ++
 4 files changed, 36 insertions(+), 10 deletions(-)
Signature · 1 line
2.20.1
#1 lore raw

PATCH 1/4 examples: install examples as part of ninja install

When we install dpdk onto a system, we want to put the examples into the /usr/share/dpdk (or /usr/local/share/dpdk) directory for reference.

---
 examples/meson.build | 16 ++++++++++++----
 1 file changed, 12 insertions(+), 4 deletions(-)
examples/meson.build +12 -4
index e4babf6bf..1a6134f12 100644
@@ -8,12 +8,20 @@ endif
 
 execinfo = cc.find_library('execinfo', required: false)
 
-allow_skips = true # don't flag an error if we can't build an app
+all_examples = run_command('sh', '-c',
+	'cd $MESON_SOURCE_ROOT/$MESON_SUBDIR && for d in * ; do if [ -d $d ] ; then echo $d ; fi ; done'
+	).stdout().split()
+# install all example code on install - irrespective of whether the example in
+# question is to be built as part of this build or not.
+foreach ex:all_examples
+	install_subdir(ex,
+			install_dir: get_option('datadir') + '/dpdk/examples',
+			exclude_files: 'meson.build')
+endforeach
 
 if get_option('examples').to_lower() == 'all'
-	dirs = run_command('sh', '-c',
-		'cd $MESON_SOURCE_ROOT/$MESON_SUBDIR && for d in * ; do if [ -d $d ] ; then echo $d ; fi ; done')
-	examples = dirs.stdout().split()
+	examples = all_examples
+	allow_skips = true # don't flag an error if we can't build an app
 else
 	examples = get_option('examples').split(',')
 	allow_skips = false # error out if we can't build a requested app
Signature · 1 line
2.20.1
#2 lore raw

PATCH 2/4 examples: simplify getting list of all examples

In reply to Bruce Richardson

Using find rather than shelling out to a one-liner script is easier to read and understand, as well as allowing shorter lines in the code.

---
 examples/meson.build | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
examples/meson.build +3 -2
index 1a6134f12..19f2686b2 100644
@@ -8,8 +8,9 @@ endif
 
 execinfo = cc.find_library('execinfo', required: false)
 
-all_examples = run_command('sh', '-c',
-	'cd $MESON_SOURCE_ROOT/$MESON_SUBDIR && for d in * ; do if [ -d $d ] ; then echo $d ; fi ; done'
+all_examples = run_command('find', meson.current_source_dir(),
+	'-maxdepth', '1', '-mindepth', '1',
+	'-type', 'd', '-printf', '%f '
 	).stdout().split()
 # install all example code on install - irrespective of whether the example in
 # question is to be built as part of this build or not.
Signature · 1 line
2.20.1
#3 lore raw

PATCH 3/4 devtools/test-meson-builds: add testing of pkg-config file

In reply to Bruce Richardson

The pkg-config file generated as part of the build of DPDK should allow applications to be built with an installed DPDK. We can test this as part of the build by doing an install of DPDK to a temporary directory within the build folder, and by then compiling up a few sample apps using make working off that directory.

---
 devtools/test-meson-builds.sh | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)
devtools/test-meson-builds.sh +17 -0
index 630a1a6fe..dfba2a782 100755
@@ -90,3 +90,20 @@ if command -v $c >/dev/null 2>&1 ; then
 			$use_shared --cross-file $f
 	done
 fi
+
+##############
+# Test installation of the x86-default target, to be used for checking
+# the sample apps build using the pkg-config file for cflags and libs
+###############
+build_path=build-x86-default
+DESTDIR=`pwd`/$build_path/install-root ; export DESTDIR
+PKG_CONFIG_PATH=$DESTDIR/usr/local/lib64/pkgconfig ; export PKG_CONFIG_PATH
+$ninja_cmd -C $build_path install
+
+# rather than hacking our environment, just edit the .pc file prefix value
+sed -i "s|prefix=|prefix=$DESTDIR|" $PKG_CONFIG_PATH/libdpdk.pc
+
+for example in helloworld l2fwd l3fwd skeleton timer; do
+	echo "## Building $example"
+	make -C $DESTDIR/usr/local/share/dpdk/examples/$example
+done
Signature · 1 line
2.20.1
#4 lore raw

Re: [dpdk-dev] [PATCH 3/4] devtools/test-meson-builds: add testing of pkg-config file

Quoted 38 lines · Bruce Richardson

The pkg-config file generated as part of the build of DPDK should allow
applications to be built with an installed DPDK. We can test this as part of the build by doing an install of DPDK to a temporary directory
within the build folder, and by then compiling up a few sample apps using make working off that directory.

Signed-off-by: Bruce Richardson <
[email protected]

Quoted 1 line
---
 devtools/test-meson-builds.sh | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)
devtools/test-meson- renamed from devtools/test-meson-builds.sh +0 -0

builds.sh
index 630a1a6fe..dfba2a782 100755

devtools/test-meson-builds.sh +3 -0
@@ -90,3 +90,20 @@ if command -v $c >/dev/null 2>&1 ; then
 			$use_shared --cross-file $f
 	done
 fi
+
+##############
+# Test installation of the x86-default target, to be used for

checking
+# the sample apps build using the pkg-config file for cflags and libs

devtools/test-meson-builds.sh +4 -0
+###############
+build_path=build-x86-default
+DESTDIR=`pwd`/$build_path/install-root ; export DESTDIR
+PKG_CONFIG_PATH=$DESTDIR/usr/local/lib64/pkgconfig ; export

PKG_CONFIG_PATH
+$ninja_cmd -C $build_path install

devtools/test-meson-builds.sh +2 -0
+
+# rather than hacking our environment, just edit the .pc file prefix
Quoted 2 lines

value
+sed -i "s|prefix=|prefix=$DESTDIR|" $PKG_CONFIG_PATH/libdpdk.pc

What about just using meson's prefix option instead? Which is how it would be used in a real use case

Quoted 5 lines
devtools/test-meson-builds.sh +4 -0
+for example in helloworld l2fwd l3fwd skeleton timer; do
+	echo "## Building $example"
+	make -C $DESTDIR/usr/local/share/dpdk/examples/$example
+done
Signature · 2 lines
Kind regards,
Luca Boccassi
#5 lore raw

Re: [dpdk-dev] [PATCH 3/4] devtools/test-meson-builds: add testing of pkg-config file

Quoted 12 lines · Luca Boccassi
Quoted 11 lines · Bruce Richardson

The pkg-config file generated as part of the build of DPDK should allow applications to be built with an installed DPDK. We can test this as part of the build by doing an install of DPDK to a temporary directory within the build folder, and by then compiling up a few sample apps using make working off that directory.

Quoted 1 line

--- devtools/test-meson-builds.sh | 17 +++++++++++++++++ 1 file changed, 17 insertions(+)

Quoted 16 lines
Quoted 4 lines
devtools/test-meson- renamed from devtools/test-meson-builds.sh +0 -0

builds.sh index 630a1a6fe..dfba2a782 100755 --- a/devtools/test-meson-builds.sh +++ b/devtools/test-meson-builds.sh @@ -90,3 +90,20 @@ if command -v $c >/dev/null 2>&1 ; then $use_shared

Quoted 8 lines

--cross-file $f done fi + +############## +# Test installation of the x86-default target, to be used for checking +# the sample apps build using the pkg-config file for cflags and libs +############### +build_path=build-x86-default +DESTDIR=pwd/$build_path/install-root ; export DESTDIR +PKG_CONFIG_PATH=$DESTDIR/usr/local/lib64/pkgconfig ; export PKG_CONFIG_PATH +$ninja_cmd -C $build_path install + +# rather than hacking our environment, just edit the .pc file prefix value +sed -i "s|prefix=|prefix=$DESTDIR|" $PKG_CONFIG_PATH/libdpdk.pc

What about just using meson's prefix option instead? Which is how it would be used in a real use case

I don't think that would fully work, as my understanding is that the prefix option would apply only to the /usr/local parts, but not to the kernel modules which would still try and install in /lib/.

/Bruce

#6 lore raw

Re: [dpdk-dev] [PATCH 3/4] devtools/test-meson-builds: add testing of pkg-config file

Quoted 45 lines · Bruce Richardson
Quoted 34 lines · Luca Boccassi
Quoted 33 lines · Bruce Richardson

The pkg-config file generated as part of the build of DPDK should allow
applications to be built with an installed DPDK. We can test this as
part of the build by doing an install of DPDK to a temporary directory
within the build folder, and by then compiling up a few sample apps
using make working off that directory.

Signed-off-by: Bruce Richardson <
[email protected]

--- devtools/test-meson-builds.sh | 17 +++++++++++++++++ 1 file changed, 17 insertions(+)

devtools/test-meson- renamed from devtools/test-meson-builds.sh +0 -0

builds.sh index 630a1a6fe..dfba2a782 100755 --- a/devtools/test-meson-builds.sh +++ b/devtools/test-meson- builds.sh @@
-90,3 +90,20 @@ if command -v $c >/dev/null 2>&1 ; then $use_shared
--cross-file $f done fi + +############## +# Test installation of the
x86-default target, to be used for checking +# the sample apps build
using the pkg-config file for cflags and libs +############### +build_path=build-x86-default +DESTDIR=pwd/$build_path/install- root ;
export DESTDIR +PKG_CONFIG_PATH=$DESTDIR/usr/local/lib64/pkgconfig ; export PKG_CONFIG_PATH +$ninja_cmd -C $build_path install + +#

Quoted 9 lines

devtools/test-meson-

Quoted 4 lines

rather
than hacking our environment, just edit the .pc file prefix value

devtools/test-meson- renamed from devtools/test-meson-builds.sh +1 -1
+sed
-i "s|prefix=|prefix=$DESTDIR|" $PKG_CONFIG_PATH/libdpdk.pc

What about just using meson's prefix option instead? Which is how it
would be used in a real use case

Quoted 7 lines

I don't think that would fully work, as my understanding is that the prefix
option would apply only to the /usr/local parts, but not to the kernel
modules which would still try and install in /lib/.

/Bruce

What about doing a meson configure -Denable_kmods=false before the ninja install? The modules are not needed for that test anyway, right? Alternatively, the kernel src dir could be symlinked in the build path, and the kernel_dir option could be used

I'm just worried that the test should be as "realistic" as possible, to avoid missing something

Signature · 2 lines
Kind regards,
Luca Boccassi
#7 lore raw

Re: [dpdk-dev] [PATCH 3/4] devtools/test-meson-builds: add testing of pkg-config file

Quoted 54 lines · Luca Boccassi
Quoted 45 lines · Bruce Richardson
Quoted 34 lines · Luca Boccassi
Quoted 33 lines · Bruce Richardson

The pkg-config file generated as part of the build of DPDK should allow
applications to be built with an installed DPDK. We can test this as
part of the build by doing an install of DPDK to a temporary directory
within the build folder, and by then compiling up a few sample apps
using make working off that directory.

Signed-off-by: Bruce Richardson <
[email protected]

--- devtools/test-meson-builds.sh | 17 +++++++++++++++++ 1 file changed, 17 insertions(+)

devtools/test-meson- renamed from devtools/test-meson-builds.sh +0 -0

builds.sh index 630a1a6fe..dfba2a782 100755 --- a/devtools/test-meson-builds.sh +++ b/devtools/test-meson- builds.sh @@
-90,3 +90,20 @@ if command -v $c >/dev/null 2>&1 ; then $use_shared
--cross-file $f done fi + +############## +# Test installation of the
x86-default target, to be used for checking +# the sample apps build
using the pkg-config file for cflags and libs +############### +build_path=build-x86-default +DESTDIR=pwd/$build_path/install- root ;
export DESTDIR +PKG_CONFIG_PATH=$DESTDIR/usr/local/lib64/pkgconfig ; export PKG_CONFIG_PATH +$ninja_cmd -C $build_path install + +#

Quoted 9 lines

devtools/test-meson-

Quoted 4 lines

rather
than hacking our environment, just edit the .pc file prefix value

devtools/test-meson- renamed from devtools/test-meson-builds.sh +1 -1
+sed
-i "s|prefix=|prefix=$DESTDIR|" $PKG_CONFIG_PATH/libdpdk.pc

What about just using meson's prefix option instead? Which is how it
would be used in a real use case

Quoted 7 lines

I don't think that would fully work, as my understanding is that the prefix
option would apply only to the /usr/local parts, but not to the kernel
modules which would still try and install in /lib/.

/Bruce

Quoted 8 lines

What about doing a meson configure -Denable_kmods=false before the ninja install? The modules are not needed for that test anyway, right? Alternatively, the kernel src dir could be symlinked in the build path, and the kernel_dir option could be used

I'm just worried that the test should be as "realistic" as possible, to avoid missing something

Yes, I did think of that too, but it does mean doing another configuration run in meson, and possibly a rebuild too if the rte_build_config.h file changes. Therefore I decided to use DESTDIR for the sake of speed here. I assumed there would be a pkg-config variable to adjust the output paths based on a sysroot, but couldn't find a suitable one.

In any case, I'll see about changing things as you suggest in V2 - correctness is more important that speed here.

/Bruce

#8 lore raw

Re: [dpdk-dev] [PATCH 3/4] devtools/test-meson-builds: add testing of pkg-config file

Quoted 91 lines · Bruce Richardson
Quoted 75 lines · Luca Boccassi
Quoted 58 lines · Bruce Richardson
Quoted 47 lines · Luca Boccassi
Quoted 46 lines · Bruce Richardson

The pkg-config file generated as part of the build of DPDK should
allow
applications to be built with an installed DPDK. We can test this
as
part of the build by doing an install of DPDK to a temporary directory
within the build folder, and by then compiling up a few sample
apps
using make working off that directory.

Signed-off-by: Bruce Richardson <
[email protected]

--- devtools/test-meson-builds.sh | 17 +++++++++++++++++ 1 file
changed, 17 insertions(+)

devtools/test- renamed from devtools/test-meson-builds.sh +0 -0

meson-
builds.sh index 630a1a6fe..dfba2a782 100755 --- a/devtools/test-meson-builds.sh +++ b/devtools/test-meson- builds.sh @@
-90,3 +90,20 @@ if command -v $c >/dev/null 2>&1 ; then $use_shared
--cross-file $f done fi + +############## +# Test installation of
the
x86-default target, to be used for checking +# the sample apps
build
using the pkg-config file for cflags and libs

devtools/test- renamed from devtools/test-meson-builds.sh +3 -0
+###############
+build_path=build-x86-default
+DESTDIR=`pwd`/$build_path/install-

root ;
export DESTDIR +PKG_CONFIG_PATH=$DESTDIR/usr/local/lib64/pkgconfig ; export PKG_CONFIG_PATH +$ninja_cmd -C $build_path install + +#
rather
than hacking our environment, just edit the .pc file prefix

Quoted 9 lines

devtools/test-

Quoted 3 lines

value

devtools/test- renamed from devtools/test-meson-builds.sh +1 -1
+sed
-i "s|prefix=|prefix=$DESTDIR|" $PKG_CONFIG_PATH/libdpdk.pc

What about just using meson's prefix option instead? Which is how
it
would be used in a real use case

Quoted 8 lines

I don't think that would fully work, as my understanding is that the
prefix
option would apply only to the /usr/local parts, but not to the kernel
modules which would still try and install in /lib/.

/Bruce

What about doing a meson configure -Denable_kmods=false before the ninja install? The modules are not needed for that test anyway, right?
Alternatively, the kernel src dir could be symlinked in the build path,
and the kernel_dir option could be used

Quoted 4 lines

I'm just worried that the test should be as "realistic" as possible, to
avoid missing something

Yes, I did think of that too, but it does mean doing another configuration
run in meson, and possibly a rebuild too if the rte_build_config.h file
changes. Therefore I decided to use DESTDIR for the sake of speed here. I
assumed there would be a pkg-config variable to adjust the output paths
based on a sysroot, but couldn't find a suitable one.

Quoted 4 lines

In any case, I'll see about changing things as you suggest in V2 - correctness is more important that speed here.

/Bruce

There actually is a pkg-config binary option, I just tried and it works (it seems to be disabled by default on Debian and derivatives): -- define-prefix

Signature · 2 lines
Kind regards,
Luca Boccassi
#9 lore raw

Re: [dpdk-dev] [PATCH 3/4] devtools/test-meson-builds: add testing of pkg-config file

Quoted 92 lines · Luca Boccassi
Quoted 91 lines · Bruce Richardson
Quoted 75 lines · Luca Boccassi
Quoted 58 lines · Bruce Richardson
Quoted 47 lines · Luca Boccassi
Quoted 46 lines · Bruce Richardson

The pkg-config file generated as part of the build of DPDK should
allow
applications to be built with an installed DPDK. We can test this
as
part of the build by doing an install of DPDK to a temporary directory
within the build folder, and by then compiling up a few sample
apps
using make working off that directory.

Signed-off-by: Bruce Richardson <
[email protected]

--- devtools/test-meson-builds.sh | 17 +++++++++++++++++ 1 file
changed, 17 insertions(+)

devtools/test- renamed from devtools/test-meson-builds.sh +0 -0

meson-
builds.sh index 630a1a6fe..dfba2a782 100755 --- a/devtools/test-meson-builds.sh +++ b/devtools/test-meson- builds.sh @@
-90,3 +90,20 @@ if command -v $c >/dev/null 2>&1 ; then $use_shared
--cross-file $f done fi + +############## +# Test installation of
the
x86-default target, to be used for checking +# the sample apps
build
using the pkg-config file for cflags and libs

devtools/test- renamed from devtools/test-meson-builds.sh +3 -0
+###############
+build_path=build-x86-default
+DESTDIR=`pwd`/$build_path/install-

root ;
export DESTDIR +PKG_CONFIG_PATH=$DESTDIR/usr/local/lib64/pkgconfig ; export PKG_CONFIG_PATH +$ninja_cmd -C $build_path install + +#
rather
than hacking our environment, just edit the .pc file prefix

Quoted 9 lines

devtools/test-

Quoted 3 lines

value

devtools/test- renamed from devtools/test-meson-builds.sh +1 -1
+sed
-i "s|prefix=|prefix=$DESTDIR|" $PKG_CONFIG_PATH/libdpdk.pc

What about just using meson's prefix option instead? Which is how
it
would be used in a real use case

Quoted 8 lines

I don't think that would fully work, as my understanding is that the
prefix
option would apply only to the /usr/local parts, but not to the kernel
modules which would still try and install in /lib/.

/Bruce

What about doing a meson configure -Denable_kmods=false before the ninja install? The modules are not needed for that test anyway, right?
Alternatively, the kernel src dir could be symlinked in the build path,
and the kernel_dir option could be used

Quoted 4 lines

I'm just worried that the test should be as "realistic" as possible, to
avoid missing something

Yes, I did think of that too, but it does mean doing another configuration
run in meson, and possibly a rebuild too if the rte_build_config.h file
changes. Therefore I decided to use DESTDIR for the sake of speed here. I
assumed there would be a pkg-config variable to adjust the output paths
based on a sysroot, but couldn't find a suitable one.

Quoted 9 lines
Quoted 4 lines

In any case, I'll see about changing things as you suggest in V2 - correctness is more important that speed here.

/Bruce

There actually is a pkg-config binary option, I just tried and it works (it seems to be disabled by default on Debian and derivatives): -- define-prefix

Any cmdline options to pkg-config don't solve the problem here as it means that the makefiles for any app need to be modified to use all those.

Also, I've been looking at the option you suggest of disabling the kernel modules for the install step - the problem that this brings is that it either:
* disables them permanently for the default build, meaning subsequent runs may fail to catch errors
* causes us to constantly reconfigure the build directory with/without the kmod setting, causing unnecessary work and slowdown in the script.

A third solution is to use a separate build folder for the pkg-config test builds, but I think we have enough builds already in the setup without adding another one.

All-in-all, I feel at this point that the original solution of making a small change to the pkg-config file manually is the best solution for now. I don't see it as being terribly fragile, and it should catch 95% of problems with the pkg-config files. I suggest that any rework be looked at in a later set to improve things.

Regards,
/Bruce

#10 lore raw

Re: [dpdk-dev] [PATCH 3/4] devtools/test-meson-builds: add testing of pkg-config file

Quoted 154 lines · Bruce Richardson
Quoted 113 lines · Luca Boccassi
Quoted 112 lines · Bruce Richardson
Quoted 91 lines · Luca Boccassi
Quoted 79 lines · Bruce Richardson
Quoted 58 lines · Luca Boccassi
Quoted 57 lines · Bruce Richardson

The pkg-config file generated as part of the build of DPDK
should
allow
applications to be built with an installed DPDK. We can test
this
as
part of the build by doing an install of DPDK to a temporary
directory
within the build folder, and by then compiling up a few sample
apps
using make working off that directory.

Signed-off-by: Bruce Richardson <
[email protected]

--- devtools/test-meson-builds.sh | 17 +++++++++++++++++ 1
file
changed, 17 insertions(+)

devtools/test- renamed from devtools/test-meson-builds.sh +0 -0

meson-
builds.sh index 630a1a6fe..dfba2a782 100755 --- a/devtools/test-meson-builds.sh +++ b/devtools/test- meson-
builds.sh @@
-90,3 +90,20 @@ if command -v $c >/dev/null 2>&1 ; then $use_shared
--cross-file $f done fi + +############## +# Test installation of
the
x86-default target, to be used for checking +# the sample apps
build
using the pkg-config file for cflags and libs

devtools/test- renamed from devtools/test-meson-builds.sh +3 -0
+###############
+build_path=build-x86-default
+DESTDIR=`pwd`/$build_path/install-

root ;
export DESTDIR +PKG_CONFIG_PATH=$DESTDIR/usr/local/lib64/pkgconfig ; export PKG_CONFIG_PATH +$ninja_cmd -C $build_path install

devtools/test- renamed from devtools/test-meson-builds.sh +2 -0
+
+#

rather
than hacking our environment, just edit the .pc file prefix
value
+sed

Quoted 9 lines
Quoted 2 lines

-i "s|prefix=|prefix=$DESTDIR|"
$PKG_CONFIG_PATH/libdpdk.pc

What about just using meson's prefix option instead? Which is
how
it
would be used in a real use case

I don't think that would fully work, as my understanding is that
the
prefix
option would apply only to the /usr/local parts, but not to the
kernel
modules which would still try and install in /lib/.

Quoted 1 line

/Bruce

What about doing a meson configure -Denable_kmods=false before the
ninja install? The modules are not needed for that test anyway, right?
Alternatively, the kernel src dir could be symlinked in the build
path,
and the kernel_dir option could be used

Quoted 4 lines

I'm just worried that the test should be as "realistic" as possible, to
avoid missing something

Yes, I did think of that too, but it does mean doing another configuration
run in meson, and possibly a rebuild too if the rte_build_config.h
file
changes. Therefore I decided to use DESTDIR for the sake of speed here. I
assumed there would be a pkg-config variable to adjust the output paths
based on a sysroot, but couldn't find a suitable one.

In any case, I'll see about changing things as you suggest in V2 -
correctness is more important that speed here.

Quoted 7 lines
Quoted 1 line

/Bruce

There actually is a pkg-config binary option, I just tried and it works
(it seems to be disabled by default on Debian and derivatives): -- define-prefix

Any cmdline options to pkg-config don't solve the problem here as it means
that the makefiles for any app need to be modified to use all those.

Also, I've been looking at the option you suggest of disabling the kernel
modules for the install step - the problem that this brings is that it either:
* disables them permanently for the default build, meaning subsequent runs
may fail to catch errors
* causes us to constantly reconfigure the build directory with/without
the kmod setting, causing unnecessary work and slowdown in the
script.

A third solution is to use a separate build folder for the pkg-config test
builds, but I think we have enough builds already in the setup without adding
another one.

All-in-all, I feel at this point that the original solution of making a small
change to the pkg-config file manually is the best solution for now. I don't
see it as being terribly fragile, and it should catch 95% of problems with
the pkg-config files. I suggest that any rework be looked at in a later set
to improve things.

Quoted 2 lines

Regards,
/Bruce

Makes sense, I had hoped it would be easier - thanks for giving it a shot!

Signature · 2 lines
Kind regards,
Luca Boccassi
#11 lore raw

Re: [dpdk-dev] [PATCH 3/4] devtools/test-meson-builds: add testing of pkg-config file

In reply to Bruce Richardson

26/04/2019 16:56, Bruce Richardson:

Quoted 45 lines · Luca Boccassi
Quoted 44 lines · Bruce Richardson
Quoted 43 lines · Luca Boccassi
Quoted 29 lines · Bruce Richardson
Quoted 14 lines · Luca Boccassi
Quoted 12 lines · Bruce Richardson

Review

Quoted 6 lines

+#
rather
than hacking our environment, just edit the .pc file prefix value

(quoted diff) +1 -1
+sed
-i "s|prefix=|prefix=$DESTDIR|" $PKG_CONFIG_PATH/libdpdk.pc

What about just using meson's prefix option instead? Which is how
it
would be used in a real use case

Quoted 6 lines

I don't think that would fully work, as my understanding is that the
prefix
option would apply only to the /usr/local parts, but not to the kernel
modules which would still try and install in /lib/.

What about doing a meson configure -Denable_kmods=false before the ninja install? The modules are not needed for that test anyway, right?
Alternatively, the kernel src dir could be symlinked in the build path,
and the kernel_dir option could be used

Quoted 3 lines

I'm just worried that the test should be as "realistic" as possible, to
avoid missing something

Yes, I did think of that too, but it does mean doing another configuration
run in meson, and possibly a rebuild too if the rte_build_config.h file
changes. Therefore I decided to use DESTDIR for the sake of speed here. I
assumed there would be a pkg-config variable to adjust the output paths
based on a sysroot, but couldn't find a suitable one.

[...]

Quoted 7 lines
Quoted 5 lines

There actually is a pkg-config binary option, I just tried and it works (it seems to be disabled by default on Debian and derivatives): -- define-prefix

Any cmdline options to pkg-config don't solve the problem here as it means that the makefiles for any app need to be modified to use all those.

It looks to be a good solution.
Being able to update the DPDK install directory when building an application should be a mandatory requirement.
I suggest to wrap the call to pkg-config so we can have this ability.

#12 lore raw

Re: [dpdk-dev] [PATCH 3/4] devtools/test-meson-builds: add testing of pkg-config file

Quoted 52 lines · Thomas Monjalon

26/04/2019 16:56, Bruce Richardson:

Quoted 45 lines · Luca Boccassi
Quoted 44 lines · Bruce Richardson
Quoted 43 lines · Luca Boccassi
Quoted 29 lines · Bruce Richardson
Quoted 14 lines · Luca Boccassi
Quoted 12 lines · Bruce Richardson

Review

Quoted 6 lines

+#
rather
than hacking our environment, just edit the .pc file prefix value

(quoted diff) +1 -1
+sed
-i "s|prefix=|prefix=$DESTDIR|" $PKG_CONFIG_PATH/libdpdk.pc

What about just using meson's prefix option instead? Which is how
it
would be used in a real use case

Quoted 6 lines

I don't think that would fully work, as my understanding is that the
prefix
option would apply only to the /usr/local parts, but not to the kernel
modules which would still try and install in /lib/.

What about doing a meson configure -Denable_kmods=false before the ninja install? The modules are not needed for that test anyway, right?
Alternatively, the kernel src dir could be symlinked in the build path,
and the kernel_dir option could be used

Quoted 3 lines

I'm just worried that the test should be as "realistic" as possible, to
avoid missing something

Yes, I did think of that too, but it does mean doing another configuration
run in meson, and possibly a rebuild too if the rte_build_config.h file
changes. Therefore I decided to use DESTDIR for the sake of speed here. I
assumed there would be a pkg-config variable to adjust the output paths
based on a sysroot, but couldn't find a suitable one.

[...]

Quoted 5 lines
Quoted 5 lines

There actually is a pkg-config binary option, I just tried and it works (it seems to be disabled by default on Debian and derivatives): -- define-prefix

Quoted 8 lines
Quoted 2 lines

Any cmdline options to pkg-config don't solve the problem here as it means that the makefiles for any app need to be modified to use all those.

It looks to be a good solution.
Being able to update the DPDK install directory when building an application should be a mandatory requirement.
I suggest to wrap the call to pkg-config so we can have this ability.

I would have expected that this issue has already been solved for packagers. I was surprised that I couldn't find an easy way to do so.

#13 lore raw

Re: [dpdk-dev] [PATCH 3/4] devtools/test-meson-builds: add testing of pkg-config file

Quoted 82 lines · Bruce Richardson
Quoted 74 lines · Thomas Monjalon

26/04/2019 16:56, Bruce Richardson:

Quoted 59 lines · Luca Boccassi
Quoted 58 lines · Bruce Richardson
Quoted 57 lines
Quoted 39 lines · Luca Boccassi
Quoted 19 lines · Bruce Richardson
Quoted 16 lines · Luca Boccassi
Quoted 8 lines · Bruce Richardson

+#
rather
than hacking our environment, just edit the .pc file prefix
value

(quoted diff) +1 -1
+sed
-i "s|prefix=|prefix=$DESTDIR|"

$PKG_CONFIG_PATH/libdpdk.pc

What about just using meson's prefix option instead? Which is
how
it
would be used in a real use case

Quoted 8 lines

I don't think that would fully work, as my understanding is that
the
prefix
option would apply only to the /usr/local parts, but not to the
kernel
modules which would still try and install in /lib/.

What about doing a meson configure -Denable_kmods=false before the
ninja install? The modules are not needed for that test anyway,
right?
Alternatively, the kernel src dir could be symlinked in the build
path,
and the kernel_dir option could be used

Quoted 3 lines

I'm just worried that the test should be as "realistic" as possible, to
avoid missing something

Yes, I did think of that too, but it does mean doing another configuration
run in meson, and possibly a rebuild too if the rte_build_config.h
file
changes. Therefore I decided to use DESTDIR for the sake of speed
here. I
assumed there would be a pkg-config variable to adjust the output
paths
based on a sysroot, but couldn't find a suitable one.

[...]

Quoted 11 lines
Quoted 6 lines

There actually is a pkg-config binary option, I just tried and it works
(it seems to be disabled by default on Debian and derivatives):
--
define-prefix

Any cmdline options to pkg-config don't solve the problem here as it means
that the makefiles for any app need to be modified to use all those.

Quoted 6 lines

It looks to be a good solution.
Being able to update the DPDK install directory when building an application should be a mandatory requirement.
I suggest to wrap the call to pkg-config so we can have this ability.

Quoted 2 lines

I would have expected that this issue has already been solved for packagers. I was surprised that I couldn't find an easy way to do so.

Packagers use standard paths - so it never becomes a problem.

If editing the file on the fly is not acceptable for the test script, then I'd suggest to fall back to the pkg-config option, and just document the need to use it in the docs for this scenarios.

Signature · 2 lines
Kind regards,
Luca Boccassi
#14 lore raw

Re: [dpdk-dev] [PATCH 3/4] devtools/test-meson-builds: add testing of pkg-config file

02/05/2019 16:08, Luca Boccassi:

Quoted 83 lines · Bruce Richardson
Quoted 82 lines · Thomas Monjalon
Quoted 80 lines

26/04/2019 16:56, Bruce Richardson:

Quoted 59 lines · Luca Boccassi
Quoted 58 lines · Bruce Richardson
Quoted 57 lines
Quoted 39 lines · Luca Boccassi
Quoted 19 lines · Bruce Richardson
Quoted 16 lines · Luca Boccassi
Quoted 8 lines · Bruce Richardson

+#
rather
than hacking our environment, just edit the .pc file prefix
value

(quoted diff) +1 -1
+sed
-i "s|prefix=|prefix=$DESTDIR|"

$PKG_CONFIG_PATH/libdpdk.pc

What about just using meson's prefix option instead? Which is
how
it
would be used in a real use case

Quoted 8 lines

I don't think that would fully work, as my understanding is that
the
prefix
option would apply only to the /usr/local parts, but not to the
kernel
modules which would still try and install in /lib/.

What about doing a meson configure -Denable_kmods=false before the
ninja install? The modules are not needed for that test anyway,
right?
Alternatively, the kernel src dir could be symlinked in the build
path,
and the kernel_dir option could be used

Quoted 3 lines

I'm just worried that the test should be as "realistic" as possible, to
avoid missing something

Yes, I did think of that too, but it does mean doing another configuration
run in meson, and possibly a rebuild too if the rte_build_config.h
file
changes. Therefore I decided to use DESTDIR for the sake of speed
here. I
assumed there would be a pkg-config variable to adjust the output
paths
based on a sysroot, but couldn't find a suitable one.

[...]

Quoted 11 lines
Quoted 6 lines

There actually is a pkg-config binary option, I just tried and it works
(it seems to be disabled by default on Debian and derivatives):
--
define-prefix

Any cmdline options to pkg-config don't solve the problem here as it means
that the makefiles for any app need to be modified to use all those.

It looks to be a good solution.
Being able to update the DPDK install directory when building an application should be a mandatory requirement.
I suggest to wrap the call to pkg-config so we can have this ability.

Quoted 8 lines
Quoted 2 lines

I would have expected that this issue has already been solved for packagers. I was surprised that I couldn't find an easy way to do so.

Packagers use standard paths - so it never becomes a problem.

If editing the file on the fly is not acceptable for the test script, then I'd suggest to fall back to the pkg-config option, and just document the need to use it in the docs for this scenarios.

What I mean is that we should be able to compile our apps after using DESTDIR, not related to the test script. Can we use an environment variable like RTE_TARGET? DPDK_DIR?

#15 lore raw

Re: [dpdk-dev] [PATCH 3/4] devtools/test-meson-builds: add testing of pkg-config file

Quoted 89 lines · Thomas Monjalon

02/05/2019 16:08, Luca Boccassi:

Quoted 88 lines · Bruce Richardson
Quoted 82 lines · Thomas Monjalon
Quoted 74 lines

26/04/2019 16:56, Bruce Richardson:

Quoted 59 lines · Luca Boccassi
Quoted 58 lines · Bruce Richardson
Quoted 57 lines
Quoted 39 lines · Luca Boccassi
Quoted 19 lines · Bruce Richardson
Quoted 16 lines · Luca Boccassi
Quoted 8 lines · Bruce Richardson

+#
rather
than hacking our environment, just edit the .pc file prefix
value

(quoted diff) +1 -1
+sed
-i "s|prefix=|prefix=$DESTDIR|"

$PKG_CONFIG_PATH/libdpdk.pc

What about just using meson's prefix option instead? Which is
how
it
would be used in a real use case

Quoted 8 lines

I don't think that would fully work, as my understanding is that
the
prefix
option would apply only to the /usr/local parts, but not to the
kernel
modules which would still try and install in /lib/.

What about doing a meson configure -Denable_kmods=false before the
ninja install? The modules are not needed for that test anyway,
right?
Alternatively, the kernel src dir could be symlinked in the build
path,
and the kernel_dir option could be used

Quoted 3 lines

I'm just worried that the test should be as "realistic" as possible, to
avoid missing something

Yes, I did think of that too, but it does mean doing another configuration
run in meson, and possibly a rebuild too if the rte_build_config.h
file
changes. Therefore I decided to use DESTDIR for the sake of speed
here. I
assumed there would be a pkg-config variable to adjust the output
paths
based on a sysroot, but couldn't find a suitable one.

[...]

Quoted 11 lines
Quoted 6 lines

There actually is a pkg-config binary option, I just tried and it works
(it seems to be disabled by default on Debian and derivatives):
--
define-prefix

Any cmdline options to pkg-config don't solve the problem here as it means
that the makefiles for any app need to be modified to use all those.

Quoted 6 lines

It looks to be a good solution.
Being able to update the DPDK install directory when building an application should be a mandatory requirement.
I suggest to wrap the call to pkg-config so we can have this ability.

Quoted 2 lines

I would have expected that this issue has already been solved for packagers. I was surprised that I couldn't find an easy way to do so.

Packagers use standard paths - so it never becomes a problem.

Quoted 8 lines
Quoted 3 lines

If editing the file on the fly is not acceptable for the test script, then I'd suggest to fall back to the pkg-config option, and just document the need to use it in the docs for this scenarios.

What I mean is that we should be able to compile our apps after using DESTDIR, not related to the test script. Can we use an environment variable like RTE_TARGET? DPDK_DIR?

We can certainly add one to our example Makefiles, it's just a couple of lines change to each one to add a prefix to the return value from pkg-config. I can attempt to do so in a later version of this patch, though I'd prefer to take more time over it than we have in 19.05.

Alternatively, we can defer the last couple of patches in this set to 19.08, though again I'd prefer to have even this level of minimal testing of pkg-config into 19.05.

Let me know what you think is best?

/Bruce

#16 lore raw

Re: [dpdk-dev] [PATCH 3/4] devtools/test-meson-builds: add testing of pkg-config file

02/05/2019 17:30, Bruce Richardson:

Quoted 103 lines · Thomas Monjalon
Quoted 89 lines

02/05/2019 16:08, Luca Boccassi:

Quoted 88 lines · Bruce Richardson
Quoted 82 lines · Thomas Monjalon
Quoted 74 lines

26/04/2019 16:56, Bruce Richardson:

Quoted 59 lines · Luca Boccassi
Quoted 58 lines · Bruce Richardson
Quoted 57 lines
Quoted 39 lines · Luca Boccassi
Quoted 19 lines · Bruce Richardson
Quoted 16 lines · Luca Boccassi
Quoted 8 lines · Bruce Richardson

+#
rather
than hacking our environment, just edit the .pc file prefix
value

(quoted diff) +1 -1
+sed
-i "s|prefix=|prefix=$DESTDIR|"

$PKG_CONFIG_PATH/libdpdk.pc

What about just using meson's prefix option instead? Which is
how
it
would be used in a real use case

Quoted 8 lines

I don't think that would fully work, as my understanding is that
the
prefix
option would apply only to the /usr/local parts, but not to the
kernel
modules which would still try and install in /lib/.

What about doing a meson configure -Denable_kmods=false before the
ninja install? The modules are not needed for that test anyway,
right?
Alternatively, the kernel src dir could be symlinked in the build
path,
and the kernel_dir option could be used

Quoted 3 lines

I'm just worried that the test should be as "realistic" as possible, to
avoid missing something

Yes, I did think of that too, but it does mean doing another configuration
run in meson, and possibly a rebuild too if the rte_build_config.h
file
changes. Therefore I decided to use DESTDIR for the sake of speed
here. I
assumed there would be a pkg-config variable to adjust the output
paths
based on a sysroot, but couldn't find a suitable one.

[...]

Quoted 11 lines
Quoted 6 lines

There actually is a pkg-config binary option, I just tried and it works
(it seems to be disabled by default on Debian and derivatives):
--
define-prefix

Any cmdline options to pkg-config don't solve the problem here as it means
that the makefiles for any app need to be modified to use all those.

Quoted 6 lines

It looks to be a good solution.
Being able to update the DPDK install directory when building an application should be a mandatory requirement.
I suggest to wrap the call to pkg-config so we can have this ability.

Quoted 2 lines

I would have expected that this issue has already been solved for packagers. I was surprised that I couldn't find an easy way to do so.

Packagers use standard paths - so it never becomes a problem.

Quoted 8 lines
Quoted 3 lines

If editing the file on the fly is not acceptable for the test script, then I'd suggest to fall back to the pkg-config option, and just document the need to use it in the docs for this scenarios.

What I mean is that we should be able to compile our apps after using DESTDIR, not related to the test script. Can we use an environment variable like RTE_TARGET? DPDK_DIR?

We can certainly add one to our example Makefiles, it's just a couple of lines change to each one to add a prefix to the return value from pkg-config. I can attempt to do so in a later version of this patch, though I'd prefer to take more time over it than we have in 19.05.

Quoted 5 lines

Alternatively, we can defer the last couple of patches in this set to 19.08, though again I'd prefer to have even this level of minimal testing of pkg-config into 19.05.

Let me know what you think is best?

I think it's best to merge only the fixes at this stage.

#17 lore raw

Re: [dpdk-dev] [PATCH 3/4] devtools/test-meson-builds: add testing of pkg-config file

Quoted 104 lines · Thomas Monjalon

02/05/2019 17:30, Bruce Richardson:

Quoted 103 lines · Thomas Monjalon
Quoted 89 lines

02/05/2019 16:08, Luca Boccassi:

Quoted 88 lines · Bruce Richardson
Quoted 82 lines · Thomas Monjalon
Quoted 74 lines

26/04/2019 16:56, Bruce Richardson:

Quoted 59 lines · Luca Boccassi
Quoted 58 lines · Bruce Richardson
Quoted 57 lines
Quoted 39 lines · Luca Boccassi
Quoted 19 lines · Bruce Richardson
Quoted 16 lines · Luca Boccassi
Quoted 8 lines · Bruce Richardson

+#
rather
than hacking our environment, just edit the .pc file
prefix
value
+sed
-i "s|prefix=|prefix=$DESTDIR|"
$PKG_CONFIG_PATH/libdpdk.pc

What about just using meson's prefix option instead? Which is
how
it
would be used in a real use case

Quoted 8 lines

I don't think that would fully work, as my understanding is that
the
prefix
option would apply only to the /usr/local parts, but not to the
kernel
modules which would still try and install in /lib/.

What about doing a meson configure -Denable_kmods=false before the
ninja install? The modules are not needed for that test anyway,
right?
Alternatively, the kernel src dir could be symlinked in the build
path,
and the kernel_dir option could be used

Quoted 3 lines

I'm just worried that the test should be as "realistic" as possible, to
avoid missing something

Yes, I did think of that too, but it does mean doing another configuration
run in meson, and possibly a rebuild too if the rte_build_config.h
file
changes. Therefore I decided to use DESTDIR for the sake of speed
here. I
assumed there would be a pkg-config variable to adjust the output
paths
based on a sysroot, but couldn't find a suitable one.

[...]

Quoted 11 lines
Quoted 6 lines

There actually is a pkg-config binary option, I just tried and it works
(it seems to be disabled by default on Debian and derivatives):
--
define-prefix

Any cmdline options to pkg-config don't solve the problem here as it means
that the makefiles for any app need to be modified to use all those.

Quoted 6 lines

It looks to be a good solution.
Being able to update the DPDK install directory when building an application should be a mandatory requirement.
I suggest to wrap the call to pkg-config so we can have this ability.

Quoted 2 lines

I would have expected that this issue has already been solved for packagers. I was surprised that I couldn't find an easy way to do so.

Packagers use standard paths - so it never becomes a problem.

Quoted 8 lines
Quoted 3 lines

If editing the file on the fly is not acceptable for the test script, then I'd suggest to fall back to the pkg-config option, and just document the need to use it in the docs for this scenarios.

What I mean is that we should be able to compile our apps after using DESTDIR, not related to the test script. Can we use an environment variable like RTE_TARGET? DPDK_DIR?

We can certainly add one to our example Makefiles, it's just a couple of lines change to each one to add a prefix to the return value from pkg-config. I can attempt to do so in a later version of this patch, though I'd prefer to take more time over it than we have in 19.05.

Quoted 9 lines
Quoted 5 lines

Alternatively, we can defer the last couple of patches in this set to 19.08, though again I'd prefer to have even this level of minimal testing of pkg-config into 19.05.

Let me know what you think is best?

I think it's best to merge only the fixes at this stage.

Ok, can patches 1-3 be merged, and I'll do a new patch for the -lbsd fix in patch 5.

Thanks,
/Bruce

#18 lore raw

PATCH 4/4 build: add libbsd to pkg-config file if enabled

In reply to Bruce Richardson

If libbsd is enabled in DPDK, the strlcpy and strlcat functions in rte_string_fns.h redirect to the varients in libbsd, only using the fallbacks if it is not enabled. Therefore, if libbsd is enabled, it needs to be called out as a DPDK dependency in the pkgconfig file.

To ensure that we don't have undefined variables on non-Linux platforms, we can remove the linux condition around the libbsd check - no harm comes in looking for it on other OS, since it's an optional dependency.

To verify this builds correctly, the cmdline sample app is added to the list of examples compiled by test-meson-builds. Without this change compilation fails if libbsd is present on the system

---
 config/meson.build            | 10 ++++------
 devtools/test-meson-builds.sh |  2 +-
 meson.build                   |  2 ++
 3 files changed, 7 insertions(+), 7 deletions(-)
config/meson.build +4 -6
index f8aded6ed..64e2315dd 100644
@@ -125,12 +125,10 @@ if numa_dep.found() and cc.has_header('numaif.h')
 	dpdk_extra_ldflags += '-lnuma'
 endif
 
-# check for strlcpy
-if is_linux
-	libbsd = dependency('libbsd', required: false)
-	if libbsd.found()
-		dpdk_conf.set('RTE_USE_LIBBSD', 1)
-	endif
+# check for libbsd
+libbsd = dependency('libbsd', required: false)
+if libbsd.found()
+	dpdk_conf.set('RTE_USE_LIBBSD', 1)
 endif
 
 # add -include rte_config to cflags
devtools/test-meson-builds.sh +1 -1
index dfba2a782..41080353b 100755
@@ -103,7 +103,7 @@ $ninja_cmd -C $build_path install
 # rather than hacking our environment, just edit the .pc file prefix value
 sed -i "s|prefix=|prefix=$DESTDIR|" $PKG_CONFIG_PATH/libdpdk.pc
 
-for example in helloworld l2fwd l3fwd skeleton timer; do
+for example in cmdline helloworld l2fwd l3fwd skeleton timer; do
 	echo "## Building $example"
 	make -C $DESTDIR/usr/local/share/dpdk/examples/$example
 done
meson.build +2 -0
index a96486597..3c173a587 100644
@@ -77,6 +77,8 @@ pkg.generate(name: meson.project_name(),
 	libraries: dpdk_libraries,
 	libraries_private: dpdk_drivers + dpdk_static_libraries +
 			['-Wl,-Bdynamic'] + dpdk_extra_ldflags,
+	requires: libbsd, # apps using rte_string_fns.h may need this if enabled
+	                  # if libbsd is not enabled, then this is blank
 	description: '''The Data Plane Development Kit (DPDK).
 Note that CFLAGS might contain an -march flag higher than typical baseline.
 This is required for a number of static inline functions in the public headers.''',
Signature · 1 line
2.20.1
#19 lore raw
In reply to Bruce Richardson
Quoted 18 lines · On Tue, 23 Apr 2019 23:06:40 +0100 Bruce Richardson…

As part of the meson build, a pkg-config file for libdpdk is created, which allows apps to be compiled and linked against DPDK by taking the cflags and lib parameter from pkgconfig. The example app makefiles have been reworked to take account of this support, but the build of them against the .pc file was not regularly tested.

To rectify this, and give us greater confidence in the correctness of the .pc file, this set adds in the sample apps to the installation set for "ninja install" and then builds a subset of those apps against the pkg-config file to test it. In the process a small error when compiling the cmdline sample app using the .pc file was fixed.

Bruce Richardson (4):
examples: install examples as part of ninja install
examples: simplify getting list of all examples devtools/test-meson-builds: add testing of pkg-config file
build: add libbsd to pkg-config file if enabled

Quoted 6 lines
 config/meson.build            | 10 ++++------
 devtools/test-meson-builds.sh | 17 +++++++++++++++++
 examples/meson.build          | 17 +++++++++++++----
 meson.build                   |  2 ++
 4 files changed, 36 insertions(+), 10 deletions(-)

My experiments with Ubuntu 18.04 showed the default version of meson was too old (broken) and generated bad cflags.

Getting later one for Debian stable-backports worked.

#20 lore raw
Quoted 28 lines · Stephen Hemminger
Quoted 18 lines · On Tue, 23 Apr 2019 23:06:40 +0100 Bruce Richardson…

As part of the meson build, a pkg-config file for libdpdk is created, which allows apps to be compiled and linked against DPDK by taking the cflags and lib parameter from pkgconfig. The example app makefiles have been reworked to take account of this support, but the build of them against the .pc file was not regularly tested.

To rectify this, and give us greater confidence in the correctness of the .pc file, this set adds in the sample apps to the installation set for "ninja install" and then builds a subset of those apps against the pkg-config file to test it. In the process a small error when compiling the cmdline sample app using the .pc file was fixed.

Bruce Richardson (4):
examples: install examples as part of ninja install
examples: simplify getting list of all examples devtools/test-meson-builds: add testing of pkg-config file
build: add libbsd to pkg-config file if enabled

Quoted 6 lines
 config/meson.build            | 10 ++++------
 devtools/test-meson-builds.sh | 17 +++++++++++++++++
 examples/meson.build          | 17 +++++++++++++----
 meson.build                   |  2 ++
 4 files changed, 36 insertions(+), 10 deletions(-)
Quoted 4 lines

My experiments with Ubuntu 18.04 showed the default version of meson was too old (broken) and generated bad cflags.

Getting later one for Debian stable-backports worked.

Thanks for the info - it's things like that that this scripting is needed to catch.

How old was the older version and what was the diff from it vs newer? Personally, to update meson, I'd always just use pip to get the latest.

/Bruce

#21 lore raw
Quoted 27 lines · Bruce Richardson
Quoted 26 lines · Stephen Hemminger
Quoted 22 lines · On Tue, 23 Apr 2019 23:06:40 +0100 Bruce Richardson…

As part of the meson build, a pkg-config file for libdpdk is created, which allows apps to be compiled and linked against DPDK by taking the cflags and lib parameter from pkgconfig. The example app makefiles have been reworked to take account of this support, but the build of them against the .pc file was not regularly tested.

To rectify this, and give us greater confidence in the correctness of the .pc file, this set adds in the sample apps to the installation set for "ninja install" and then builds a subset of those apps against the pkg-config file to test it. In the process a small error when compiling the cmdline sample app using the .pc file was fixed.

Bruce Richardson (4): examples: install examples as part of ninja install examples: simplify getting list of all examples devtools/test-meson-builds: add testing of pkg-config file build: add libbsd to pkg-config file if enabled

 config/meson.build            | 10 ++++------
 devtools/test-meson-builds.sh | 17 +++++++++++++++++

examples/meson.build | 17 +++++++++++++---- meson.build
| 2 ++ 4 files changed, 36 insertions(+), 10 deletions(-)

Quoted 8 lines
Quoted 4 lines

My experiments with Ubuntu 18.04 showed the default version of meson was too old (broken) and generated bad cflags.

Getting later one for Debian stable-backports worked.

Thanks for the info - it's things like that that this scripting is needed to catch.

And as further followup on the need for this, checking this patchset on FreeBSD shows that the "ninja install" step is broken there due to "ln", used by "symlink-drivers-solibs.sh" postinstall script, not supporting the "-r" flag on FreeBSD. :-(

I'll try and do up a V2 set to include that fix also.

/Bruce

#22 lore raw
In reply to Bruce Richardson
Quoted 25 lines · Bruce Richardson

As part of the meson build, a pkg-config file for libdpdk is created, which
allows apps to be compiled and linked against DPDK by taking the cflags and
lib parameter from pkgconfig. The example app makefiles have been reworked
to take account of this support, but the build of them against the .pc file
was not regularly tested.

To rectify this, and give us greater confidence in the correctness of the
.pc file, this set adds in the sample apps to the installation set for
"ninja install" and then builds a subset of those apps against the pkg-config file to test it. In the process a small error when compiling
the cmdline sample app using the .pc file was fixed.

Bruce Richardson (4):
examples: install examples as part of ninja install
examples: simplify getting list of all examples devtools/test-meson-builds: add testing of pkg-config file
build: add libbsd to pkg-config file if enabled

Quoted 5 lines
 config/meson.build            | 10 ++++------
 devtools/test-meson-builds.sh | 17 +++++++++++++++++
 examples/meson.build          | 17 +++++++++++++----
 meson.build                   |  2 ++
 4 files changed, 36 insertions(+), 10 deletions(-)
Signature · 2 lines
Kind regards,
Luca Boccassi
#23 lore raw
Quoted 26 lines · Luca Boccassi
Quoted 25 lines · Bruce Richardson

As part of the meson build, a pkg-config file for libdpdk is created, which
allows apps to be compiled and linked against DPDK by taking the cflags and
lib parameter from pkgconfig. The example app makefiles have been reworked
to take account of this support, but the build of them against the .pc file
was not regularly tested.

To rectify this, and give us greater confidence in the correctness of the
.pc file, this set adds in the sample apps to the installation set for
"ninja install" and then builds a subset of those apps against the pkg-config file to test it. In the process a small error when compiling
the cmdline sample app using the .pc file was fixed.

Bruce Richardson (4):
examples: install examples as part of ninja install
examples: simplify getting list of all examples devtools/test-meson-builds: add testing of pkg-config file
build: add libbsd to pkg-config file if enabled

Quoted 8 lines
Quoted 5 lines
 config/meson.build            | 10 ++++------
 devtools/test-meson-builds.sh | 17 +++++++++++++++++
 examples/meson.build          | 17 +++++++++++++----
 meson.build                   |  2 ++
 4 files changed, 36 insertions(+), 10 deletions(-)

Thanks. I've actually got a V2 coming soon with some more rework, since I found some more issues (especially on BSD) as I worked through the changes.

/Bruce

#24 lore raw

v2 cover add testing of libdpdk pkg-config file

In reply to Bruce Richardson

As part of the meson build, a pkg-config file for libdpdk is created, which allows apps to be compiled and linked against DPDK by taking the cflags and lib parameter from pkgconfig. The example app makefiles have been reworked to take account of this support, but the build of them against the .pc file was not regularly tested.

To rectify this, and give us greater confidence in the correctness of the .pc file, this set adds in the sample apps to the installation set for "ninja install" and then builds a subset of those apps against the pkg-config file to test it. In the process a small error when compiling the cmdline sample app using the .pc file was fixed.

V2: Fixed a number of other problems encountered on FreeBSD. Replaced
patch 2 of the original set, which broke on BSD with patch 6 of this set, which is the more correct solution.

Bruce Richardson (6):
examples/l3fwd: fix compile on freebsd
examples: install examples as part of ninja install
build: fix ninja install on FreeBSD devtools/test-meson-builds: add testing of pkg-config file
build: add libbsd to pkg-config file if enabled
examples: remove auto-generation of examples list

 buildtools/symlink-drivers-solibs.sh |  7 +++--
 config/meson.build                   | 17 ++++++++----
 devtools/test-meson-builds.sh        | 27 ++++++++++++++++++
 examples/l3fwd/l3fwd_lpm.c           |  1 +
 examples/meson.build                 | 41 +++++++++++++++++++++++++---
 meson.build                          |  9 ++----
 6 files changed, 82 insertions(+), 20 deletions(-)
Signature · 1 line
2.21.0
#25 lore raw

PATCH v2 1/6 examples/l3fwd: fix compile on FreeBSD

On freebsd we need to include sys/socket.h to get the definition of AF_INET in order to compile.

Fixes d5ceea4ab160 ("examples/l3fwd: format IP addresses for printing")
---
 examples/l3fwd/l3fwd_lpm.c | 1 +
 1 file changed, 1 insertion(+)
examples/l3fwd/l3fwd_lpm.c +1 -0
index 172a036b2..60a00639e 100644
@@ -13,6 +13,7 @@
 #include <errno.h>
 #include <getopt.h>
 #include <stdbool.h>
+#include <sys/socket.h>
 #include <arpa/inet.h>
 
 #include <rte_debug.h>
Signature · 1 line
2.21.0
Acked-by on v2 1/6
#26 lore raw

Re: [dpdk-dev] [PATCH v2 1/6] examples/l3fwd: fix compile on FreeBSD

Quoted 4 lines · Bruce Richardson

On freebsd we need to include sys/socket.h to get the definition of AF_INET in order to compile.

Fixes d5ceea4ab160 ("examples/l3fwd: format IP addresses for
Quoted 9 lines

printing")

Signed-off-by: Bruce Richardson <
[email protected]

Quoted 1 line
---
 examples/l3fwd/l3fwd_lpm.c | 1 +
 1 file changed, 1 insertion(+)
Signature · 2 lines
Kind regards,
Luca Boccassi
#27 lore raw

PATCH v2 2/6 examples: install examples as part of ninja install

In reply to Bruce Richardson

When we install dpdk onto a system, we want to put the examples into the /usr/share/dpdk (or /usr/local/share) directory for reference.

---
 examples/meson.build | 16 ++++++++++++----
 1 file changed, 12 insertions(+), 4 deletions(-)
examples/meson.build +12 -4
index e4babf6bf..1a6134f12 100644
@@ -8,12 +8,20 @@ endif
 
 execinfo = cc.find_library('execinfo', required: false)
 
-allow_skips = true # don't flag an error if we can't build an app
+all_examples = run_command('sh', '-c',
+	'cd $MESON_SOURCE_ROOT/$MESON_SUBDIR && for d in * ; do if [ -d $d ] ; then echo $d ; fi ; done'
+	).stdout().split()
+# install all example code on install - irrespective of whether the example in
+# question is to be built as part of this build or not.
+foreach ex:all_examples
+	install_subdir(ex,
+			install_dir: get_option('datadir') + '/dpdk/examples',
+			exclude_files: 'meson.build')
+endforeach
 
 if get_option('examples').to_lower() == 'all'
-	dirs = run_command('sh', '-c',
-		'cd $MESON_SOURCE_ROOT/$MESON_SUBDIR && for d in * ; do if [ -d $d ] ; then echo $d ; fi ; done')
-	examples = dirs.stdout().split()
+	examples = all_examples
+	allow_skips = true # don't flag an error if we can't build an app
 else
 	examples = get_option('examples').split(',')
 	allow_skips = false # error out if we can't build a requested app
Signature · 1 line
2.21.0
#28 lore raw

PATCH v2 3/6 build: fix ninja install on FreeBSD

In reply to Bruce Richardson

The post-install script to symlink the PMDs from their own PMD directory to the regular lib directory (so they would be found by ld at runtime) was using the "-r" flag to ln to create relative symlinks. This flag is unsupported by ln on FreeBSD causing the ninja install step to fail.

Reworking the script to take the relative driver path as parameter removes the need for ln to calculate the relative path ensuring compatibility with FreeBSD.

As part of the fix, we move the registration of the install script to the config/meson.build file, from the top level one. This improves readability as the script takes as parameters the variables set in that file.

Fixes ed4d43d73e2b ("build: symlink drivers to library directory")
---
 buildtools/symlink-drivers-solibs.sh | 7 ++++---
 config/meson.build                   | 7 +++++++
 meson.build                          | 7 -------
 3 files changed, 11 insertions(+), 10 deletions(-)
buildtools/symlink-drivers-solibs.sh +4 -3
index 9826c6ae3..42985e855 100644
@@ -7,6 +7,7 @@
 # others, e.g. PCI device PMDs depending on the PCI bus driver.
 
 # parameters to script are paths relative to install prefix:
-# 1. directory containing driver files e.g. lib64/dpdk/drivers
-# 2. directory for installed regular libs e.g. lib64
-ln -rsf ${DESTDIR}/${MESON_INSTALL_PREFIX}/$1/* ${DESTDIR}/${MESON_INSTALL_PREFIX}/$2
+# 1. directory for installed regular libs e.g. lib64
+# 2. subdirectory of libdir where the pmds are
+
+cd ${MESON_INSTALL_DESTDIR_PREFIX}/$1 && ln -sfv $2/librte_*.so* .
config/meson.build +7 -0
index f8aded6ed..3678348de 100644
@@ -42,6 +42,13 @@ endif
 driver_install_path = join_paths(get_option('libdir'), pmd_subdir_opt)
 eal_pmd_path = join_paths(get_option('prefix'), driver_install_path)
 
+# driver .so files often depend upon the bus drivers for their connect bus,
+# e.g. ixgbe depends on librte_bus_pci. This means that the bus drivers need
+# to be in the library path, so symlink the drivers from the main lib directory.
+meson.add_install_script('../buildtools/symlink-drivers-solibs.sh',
+		get_option('libdir'),
+		pmd_subdir_opt)
+
 # set the machine type and cflags for it
 if meson.is_cross_build()
 	machine = host_machine.cpu()
meson.build +0 -7
index a96486597..d1e8e5239 100644
@@ -63,13 +63,6 @@ configure_file(output: build_cfg,
 # them.
 dpdk_drivers = ['-Wl,--whole-archive'] + dpdk_drivers + ['-Wl,--no-whole-archive']
 
-# driver .so files often depend upon the bus drivers for their connect bus,
-# e.g. ixgbe depends on librte_bus_pci. This means that the bus drivers need
-# to be in the library path, so symlink the drivers from the main lib directory.
-meson.add_install_script('buildtools/symlink-drivers-solibs.sh',
-		driver_install_path,
-		get_option('libdir'))
-
 pkg = import('pkgconfig')
 pkg.generate(name: meson.project_name(),
 	filebase: 'lib' + meson.project_name().to_lower(),
Signature · 1 line
2.21.0
Acked-by on v2 3/6
#29 lore raw

Re: [dpdk-dev] [PATCH v2 3/6] build: fix ninja install on FreeBSD

Quoted 21 lines · Bruce Richardson

The post-install script to symlink the PMDs from their own PMD directory to
the regular lib directory (so they would be found by ld at runtime) was
using the "-r" flag to ln to create relative symlinks. This flag is unsupported by ln on FreeBSD causing the ninja install step to fail.

Reworking the script to take the relative driver path as parameter removes
the need for ln to calculate the relative path ensuring compatibility with
FreeBSD.

As part of the fix, we move the registration of the install script to the
config/meson.build file, from the top level one. This improves readability
as the script takes as parameters the variables set in that file.

Fixes ed4d43d73e2b ("build: symlink drivers to library directory")
Quoted 8 lines

Signed-off-by: Bruce Richardson <
[email protected]

Quoted 1 line
---
 buildtools/symlink-drivers-solibs.sh | 7 ++++---
 config/meson.build                   | 7 +++++++
 meson.build                          | 7 -------
 3 files changed, 11 insertions(+), 10 deletions(-)
Signature · 2 lines
Kind regards,
Luca Boccassi
#30 lore raw

PATCH v2 4/6 devtools/test-meson-builds: add testing of pkg-config file

In reply to Bruce Richardson

The pkg-config file generated as part of the build of DPDK should allow applications to be built with an installed DPDK. We can test this as part of the build by doing an install of DPDK to a temporary directory within the build folder, and by then compiling up a few sample apps using make working off that directory.

---
 devtools/test-meson-builds.sh | 27 +++++++++++++++++++++++++++
 1 file changed, 27 insertions(+)
devtools/test-meson-builds.sh +27 -0
index 630a1a6fe..73f993b98 100755
@@ -15,6 +15,11 @@ srcdir=$(dirname $(readlink -f $0))/..
 MESON=${MESON:-meson}
 use_shared="--default-library=shared"
 
+if command -v gmake >/dev/null 2>&1 ; then
+	MAKE=gmake
+else
+	MAKE=make
+fi
 if command -v ninja >/dev/null 2>&1 ; then
 	ninja_cmd=ninja
 elif command -v ninja-build >/dev/null 2>&1 ; then
@@ -90,3 +95,25 @@ if command -v $c >/dev/null 2>&1 ; then
 			$use_shared --cross-file $f
 	done
 fi
+
+##############
+# Test installation of the x86-default target, to be used for checking
+# the sample apps build using the pkg-config file for cflags and libs
+###############
+build_path=build-x86-default
+DESTDIR=`pwd`/$build_path/install-root ; export DESTDIR
+$ninja_cmd -C $build_path install
+
+pc_file=$(find $DESTDIR -name libdpdk.pc)
+PKG_CONFIG_PATH=$(dirname $pc_file) ; export PKG_CONFIG_PATH
+
+# rather than hacking our environment, just edit the .pc file prefix value
+sed -i -e "s|prefix=|prefix=$DESTDIR|" $pc_file
+
+for example in helloworld l2fwd l3fwd skeleton timer; do
+	echo "## Building $example"
+	$MAKE -C $DESTDIR/usr/local/share/dpdk/examples/$example
+done
+
+echo ""
+echo "## $0: Completed OK"
Signature · 1 line
2.21.0
#31 lore raw

Re: [dpdk-dev] [PATCH v2 4/6] devtools/test-meson-builds: add testing of pkg-config file

Hi,

I will probably have a ton of comments about adding a new compilation tests, and I think it is a bit late for such an addition.
However, all the fixes should go in 19.05.

26/04/2019 18:50, Bruce Richardson:

Quoted 11 lines

The pkg-config file generated as part of the build of DPDK should allow applications to be built with an installed DPDK. We can test this as part of the build by doing an install of DPDK to a temporary directory within the build folder, and by then compiling up a few sample apps using make working off that directory.

---

devtools/test-meson-builds.sh if command -v $c >/dev/null 2>&1 ; then · line 102

Quoted 4 lines
devtools/test-meson-builds.sh +4 -0
+##############
+# Test installation of the x86-default target, to be used for checking
+# the sample apps build using the pkg-config file for cflags and libs
+###############

I would prefer simpler comment formatting.
It makes this test very special.

devtools/test-meson-builds.sh if command -v $c >/dev/null 2>&1 ; then · line 104

Quoted 2 lines
devtools/test-meson-builds.sh +2 -0
+build_path=build-x86-default
+DESTDIR=`pwd`/$build_path/install-root ; export DESTDIR

export DESTDIR=... is not supported everywhere?
I prefer new shell substitution syntax $() instead of backquotes.

devtools/test-meson-builds.sh if command -v $c >/dev/null 2>&1 ; then · line 111

Quoted 7 lines

+$ninja_cmd -C $build_path install

devtools/test-meson-builds.sh +6 -0
+
+pc_file=$(find $DESTDIR -name libdpdk.pc)
+PKG_CONFIG_PATH=$(dirname $pc_file) ; export PKG_CONFIG_PATH
+
+# rather than hacking our environment, just edit the .pc file prefix value
+sed -i -e "s|prefix=|prefix=$DESTDIR|" $pc_file

What is the alternative?
Cannot we configure meson with the right prefix?

devtools/test-meson-builds.sh if command -v $c >/dev/null 2>&1 ; then · line 119

Quoted 7 lines
devtools/test-meson-builds.sh +7 -0
+for example in helloworld l2fwd l3fwd skeleton timer; do
+	echo "## Building $example"
+	$MAKE -C $DESTDIR/usr/local/share/dpdk/examples/$example
+done
+
+echo ""
+echo "## $0: Completed OK"

This last log is uncommon.

#32 lore raw

Re: [dpdk-dev] [PATCH v2 4/6] devtools/test-meson-builds: add testing of pkg-config file

Quoted 11 lines · Thomas Monjalon
Quoted 8 lines

+$ninja_cmd -C $build_path install

devtools/test-meson-builds.sh +5 -0
+
+pc_file=$(find $DESTDIR -name libdpdk.pc)
+PKG_CONFIG_PATH=$(dirname $pc_file) ; export PKG_CONFIG_PATH
+
+# rather than hacking our environment, just edit the .pc file

prefix value
+sed -i -e "s|prefix=|prefix=$DESTDIR|" $pc_file

What is the alternative?
Cannot we configure meson with the right prefix?

Please see the email exchange between myself and Bruce on v1 3/4 - the short version is that it can, but it gets messy due to the kernel modules.

Signature · 2 lines
Kind regards,
Luca Boccassi
#33 lore raw

Re: [dpdk-dev] [PATCH v2 4/6] devtools/test-meson-builds: add testing of pkg-config file

In reply to Thomas Monjalon
Quoted 13 lines · Thomas Monjalon

Hi,

I will probably have a ton of comments about adding a new compilation tests, and I think it is a bit late for such an addition.
However, all the fixes should go in 19.05.

26/04/2019 18:50, Bruce Richardson:

Quoted 6 lines

The pkg-config file generated as part of the build of DPDK should allow applications to be built with an installed DPDK. We can test this as part of the build by doing an install of DPDK to a temporary directory within the build folder, and by then compiling up a few sample apps using make working off that directory.

Quoted 12 lines

devtools/test-meson-builds.sh if command -v $c >/dev/null 2>&1 ; then · line 102

Quoted 9 lines

---

devtools/test-meson-builds.sh +4 -0
+##############
+# Test installation of the x86-default target, to be used for checking
+# the sample apps build using the pkg-config file for cflags and libs
+###############

I would prefer simpler comment formatting.
It makes this test very special.

Your comment implies that it is not :-)
Sure, I can reduce the highlighting.

Quoted 5 lines

devtools/test-meson-builds.sh if command -v $c >/dev/null 2>&1 ; then · line 104

Quoted 3 lines
devtools/test-meson-builds.sh +2 -0
+build_path=build-x86-default
+DESTDIR=`pwd`/$build_path/install-root ; export DESTDIR

export DESTDIR=... is not supported everywhere?

No 100% sure, so left it like this just in case.

Quoted 2 lines

I prefer new shell substitution syntax $() instead of backquotes.

Sure, I agree it's more readable.

Quoted 11 lines

devtools/test-meson-builds.sh if command -v $c >/dev/null 2>&1 ; then · line 111

Quoted 7 lines

+$ninja_cmd -C $build_path install

devtools/test-meson-builds.sh +6 -0
+
+pc_file=$(find $DESTDIR -name libdpdk.pc)
+PKG_CONFIG_PATH=$(dirname $pc_file) ; export PKG_CONFIG_PATH
+
+# rather than hacking our environment, just edit the .pc file prefix value
+sed -i -e "s|prefix=|prefix=$DESTDIR|" $pc_file

What is the alternative?
Cannot we configure meson with the right prefix?

See previous discussion.
Short answer, yes we can, but the prefix does not apply to kernel modules as they always install in /lib/modules folder.

Quoted 10 lines

devtools/test-meson-builds.sh if command -v $c >/dev/null 2>&1 ; then · line 119

Quoted 7 lines
devtools/test-meson-builds.sh +7 -0
+for example in helloworld l2fwd l3fwd skeleton timer; do
+	echo "## Building $example"
+	$MAKE -C $DESTDIR/usr/local/share/dpdk/examples/$example
+done
+
+echo ""
+echo "## $0: Completed OK"

This last log is uncommon.

Yes, it is. However, due to parallelism, sometimes there is an error message printed out that scrolls off-screen as the other build processes come to a halt. I felt it worthwhile to add this at the end to ensure it was clear whether things succeeded or not. If you prefer, I can change it to a print on failure?

/Bruce

#34 lore raw

Re: [dpdk-dev] [PATCH v2 4/6] devtools/test-meson-builds: add testing of pkg-config file

02/05/2019 15:21, Bruce Richardson:

Quoted 6 lines · Thomas Monjalon
Quoted 5 lines

Hi,

I will probably have a ton of comments about adding a new compilation tests, and I think it is a bit late for such an addition.
However, all the fixes should go in 19.05.

It would be more reasonnable to leave it for 19.08 and re-spin with fixes only.

Quoted 24 lines
Quoted 7 lines

26/04/2019 18:50, Bruce Richardson:

Quoted 6 lines

The pkg-config file generated as part of the build of DPDK should allow applications to be built with an installed DPDK. We can test this as part of the build by doing an install of DPDK to a temporary directory within the build folder, and by then compiling up a few sample apps using make working off that directory.

Quoted 12 lines

devtools/test-meson-builds.sh if command -v $c >/dev/null 2>&1 ; then · line 102

Quoted 9 lines

---

devtools/test-meson-builds.sh +4 -0
+##############
+# Test installation of the x86-default target, to be used for checking
+# the sample apps build using the pkg-config file for cflags and libs
+###############

I would prefer simpler comment formatting.
It makes this test very special.

Your comment implies that it is not :-)
Sure, I can reduce the highlighting.

Quoted 1 line
Quoted 6 lines
Quoted 4 lines

devtools/test-meson-builds.sh if command -v $c >/dev/null 2>&1 ; then · line 104

Quoted 2 lines
devtools/test-meson-builds.sh +2 -0
+build_path=build-x86-default
+DESTDIR=`pwd`/$build_path/install-root ; export DESTDIR

export DESTDIR=... is not supported everywhere?

No 100% sure, so left it like this just in case.

Hmmm, i would prefer "export DESTDIR="

Quoted 15 lines
Quoted 2 lines

I prefer new shell substitution syntax $() instead of backquotes.

Sure, I agree it's more readable.

Quoted 10 lines

devtools/test-meson-builds.sh if command -v $c >/dev/null 2>&1 ; then · line 111

Quoted 7 lines

+$ninja_cmd -C $build_path install

devtools/test-meson-builds.sh +6 -0
+
+pc_file=$(find $DESTDIR -name libdpdk.pc)
+PKG_CONFIG_PATH=$(dirname $pc_file) ; export PKG_CONFIG_PATH
+
+# rather than hacking our environment, just edit the .pc file prefix value
+sed -i -e "s|prefix=|prefix=$DESTDIR|" $pc_file

What is the alternative?
Cannot we configure meson with the right prefix?

Quoted 3 lines

See previous discussion.
Short answer, yes we can, but the prefix does not apply to kernel modules as they always install in /lib/modules folder.

We could do better by providing this ability in our build system without hack.

Quoted 10 lines
Quoted 10 lines

devtools/test-meson-builds.sh if command -v $c >/dev/null 2>&1 ; then · line 119

Quoted 7 lines
devtools/test-meson-builds.sh +7 -0
+for example in helloworld l2fwd l3fwd skeleton timer; do
+	echo "## Building $example"
+	$MAKE -C $DESTDIR/usr/local/share/dpdk/examples/$example
+done
+
+echo ""
+echo "## $0: Completed OK"

This last log is uncommon.

Quoted 5 lines

Yes, it is. However, due to parallelism, sometimes there is an error message printed out that scrolls off-screen as the other build processes come to a halt. I felt it worthwhile to add this at the end to ensure it was clear whether things succeeded or not. If you prefer, I can change it to a print on failure?

Failures are caught by 'set -e'.
Personnally I don't need such log because my shell prints me an error when $? is not 0, but I can understand the need.
If you want to print $0, basename may render prettier.

#35 lore raw

PATCH v2 5/6 build: add libbsd to pkg-config file if enabled

In reply to Bruce Richardson

If libbsd is enabled in DPDK, the strlcpy and strlcat functions in rte_string_fns.h redirect to the varients in libbsd, only using the fallbacks if it is not enabled. Therefore, if libbsd is enabled, it needs to be called out as a DPDK dependency in the pkgconfig file.

To ensure that we don't have undefined variables on non-Linux platforms, we can remove the linux condition around the libbsd check - no harm comes in looking for it on other OS, since it's an optional dependency.

To verify this builds correctly, the cmdline sample app is added to the list of examples compiled by test-meson-builds. Without this change compilation fails if libbsd is present on the system

---
 config/meson.build            | 10 ++++------
 devtools/test-meson-builds.sh |  2 +-
 meson.build                   |  2 ++
 3 files changed, 7 insertions(+), 7 deletions(-)
config/meson.build +4 -6
index 3678348de..0d25646f5 100644
@@ -132,12 +132,10 @@ if numa_dep.found() and cc.has_header('numaif.h')
 	dpdk_extra_ldflags += '-lnuma'
 endif
 
-# check for strlcpy
-if is_linux
-	libbsd = dependency('libbsd', required: false)
-	if libbsd.found()
-		dpdk_conf.set('RTE_USE_LIBBSD', 1)
-	endif
+# check for libbsd
+libbsd = dependency('libbsd', required: false)
+if libbsd.found()
+	dpdk_conf.set('RTE_USE_LIBBSD', 1)
 endif
 
 # add -include rte_config to cflags
devtools/test-meson-builds.sh +1 -1
index 73f993b98..a9fdce81f 100755
@@ -110,7 +110,7 @@ PKG_CONFIG_PATH=$(dirname $pc_file) ; export PKG_CONFIG_PATH
 # rather than hacking our environment, just edit the .pc file prefix value
 sed -i -e "s|prefix=|prefix=$DESTDIR|" $pc_file
 
-for example in helloworld l2fwd l3fwd skeleton timer; do
+for example in cmdline helloworld l2fwd l3fwd skeleton timer; do
 	echo "## Building $example"
 	$MAKE -C $DESTDIR/usr/local/share/dpdk/examples/$example
 done
meson.build +2 -0
index d1e8e5239..46f9c5683 100644
@@ -70,6 +70,8 @@ pkg.generate(name: meson.project_name(),
 	libraries: dpdk_libraries,
 	libraries_private: dpdk_drivers + dpdk_static_libraries +
 			['-Wl,-Bdynamic'] + dpdk_extra_ldflags,
+	requires: libbsd, # apps using rte_string_fns.h may need this if enabled
+	                  # if libbsd is not enabled, then this is blank
 	description: '''The Data Plane Development Kit (DPDK).
 Note that CFLAGS might contain an -march flag higher than typical baseline.
 This is required for a number of static inline functions in the public headers.''',
Signature · 1 line
2.21.0
#36 lore raw

PATCH v2 6/6 examples: remove auto-generation of examples list

In reply to Bruce Richardson

The examples/meson.build file scanned the filesystem to find all examples to build (for examples=all option) and install. However, using run_command and scanning the filesystem prevented ninja from properly detecting the addition or removal of any examples - one had to recreate the build directory from scratch to guarantee correct detection of all examples. This patch replaces this generated list with a static list of examples, thereby allowing proper tracking by ninja/meson, but at the cost of having to update this file when a new example is added or removed.

---
 examples/meson.build | 31 ++++++++++++++++++++++++++++---
 1 file changed, 28 insertions(+), 3 deletions(-)
examples/meson.build +28 -3
index 1a6134f12..8b6577cf7 100644
@@ -8,9 +8,34 @@ endif
 
 execinfo = cc.find_library('execinfo', required: false)
 
-all_examples = run_command('sh', '-c',
-	'cd $MESON_SOURCE_ROOT/$MESON_SUBDIR && for d in * ; do if [ -d $d ] ; then echo $d ; fi ; done'
-	).stdout().split()
+# list of all example apps. Keep 1-3 per line, in alphabetical order.
+all_examples = [
+	'bbdev_app', 'bond',
+	'bpf', 'cmdline',
+	'distributor', 'ethtool',
+	'eventdev_pipeline', 'exception_path',
+	'fips_validation', 'flow_classify',
+	'flow_filtering', 'helloworld',
+	'ip_fragmentation', 'ip_pipeline',
+	'ip_reassembly', 'ipsec-secgw',
+	'ipv4_multicast', 'kni',
+	'l2fwd', 'l2fwd-cat',
+	'l2fwd-crypto', 'l2fwd-jobstats',
+	'l2fwd-keepalive', 'l3fwd',
+	'l3fwd-acl', 'l3fwd-power',
+	'l3fwd-vf', 'link_status_interrupt',
+	'load_balancer', 'multi_process',
+	'netmap_compat', 'packet_ordering',
+	'performance-thread', 'ptpclient',
+	'qos_meter', 'qos_sched',
+	'quota_watermark', 'rxtx_callbacks',
+	'server_node_efd', 'service_cores',
+	'skeleton', 'tep_termination',
+	'timer', 'vdpa',
+	'vhost', 'vhost_crypto',
+	'vhost_scsi', 'vm_power_manager',
+	'vmdq', 'vmdq_dcb',
+]
 # install all example code on install - irrespective of whether the example in
 # question is to be built as part of this build or not.
 foreach ex:all_examples
Signature · 1 line
2.21.0
Acked-by on v2 6/6
#37 lore raw

Re: [dpdk-dev] [PATCH v2 6/6] examples: remove auto-generation of examples list

Quoted 14 lines · Bruce Richardson

The examples/meson.build file scanned the filesystem to find all examples
to build (for examples=all option) and install. However, using run_command
and scanning the filesystem prevented ninja from properly detecting the
addition or removal of any examples - one had to recreate the build directory from scratch to guarantee correct detection of all examples. This
patch replaces this generated list with a static list of examples, thereby
allowing proper tracking by ninja/meson, but at the cost of having to update this file when a new example is added or removed.

Quoted 6 lines

Signed-off-by: Bruce Richardson <
[email protected]

Quoted 1 line
---
 examples/meson.build | 31 ++++++++++++++++++++++++++++---
 1 file changed, 28 insertions(+), 3 deletions(-)
Signature · 2 lines
Kind regards,
Luca Boccassi
#38 lore raw

v4 cover file meson compilation and install issues

In reply to Bruce Richardson

Split off from the previous set to test the pkg-config file, this set includes only the fixes encountered during that work.

V4: Removed the changes to the test script and to hard-code the list of
example apps, so that only the more important patches are included.
V3: Minor style changes to patch 4 following review
V2: Fixed a number of other problems encountered on FreeBSD. Replaced
patch 2 of the original set, which broke on BSD with patch 6 of this set, which is the more correct solution.

Bruce Richardson (4):
examples/l3fwd: fix compile on FreeBSD
examples: install examples as part of ninja install
build: fix ninja install on FreeBSD
build: add libbsd to pkg-config file if enabled

 buildtools/symlink-drivers-solibs.sh |  7 ++++---
 config/meson.build                   | 17 +++++++++++------
 examples/l3fwd/l3fwd_lpm.c           |  1 +
 examples/meson.build                 | 16 ++++++++++++----
 meson.build                          |  9 ++-------
 5 files changed, 30 insertions(+), 20 deletions(-)
Signature · 1 line
2.20.1
#39 lore raw

PATCH v4 1/4 examples/l3fwd: fix compile on FreeBSD

On freebsd we need to include sys/socket.h to get the definition of AF_INET in order to compile.

Fixes d5ceea4ab160 ("examples/l3fwd: format IP addresses for printing")
---
 examples/l3fwd/l3fwd_lpm.c | 1 +
 1 file changed, 1 insertion(+)
examples/l3fwd/l3fwd_lpm.c +1 -0
index 172a036b2..60a00639e 100644
@@ -13,6 +13,7 @@
 #include <errno.h>
 #include <getopt.h>
 #include <stdbool.h>
+#include <sys/socket.h>
 #include <arpa/inet.h>
 
 #include <rte_debug.h>
Signature · 1 line
2.20.1
#40 lore raw

PATCH v4 2/4 examples: install examples as part of ninja install

In reply to Bruce Richardson

When we install dpdk onto a system, we want to put the examples into the /usr/share/dpdk (or /usr/local/share) directory for reference.

---
 examples/meson.build | 16 ++++++++++++----
 1 file changed, 12 insertions(+), 4 deletions(-)
examples/meson.build +12 -4
index e4babf6bf..1a6134f12 100644
@@ -8,12 +8,20 @@ endif
 
 execinfo = cc.find_library('execinfo', required: false)
 
-allow_skips = true # don't flag an error if we can't build an app
+all_examples = run_command('sh', '-c',
+	'cd $MESON_SOURCE_ROOT/$MESON_SUBDIR && for d in * ; do if [ -d $d ] ; then echo $d ; fi ; done'
+	).stdout().split()
+# install all example code on install - irrespective of whether the example in
+# question is to be built as part of this build or not.
+foreach ex:all_examples
+	install_subdir(ex,
+			install_dir: get_option('datadir') + '/dpdk/examples',
+			exclude_files: 'meson.build')
+endforeach
 
 if get_option('examples').to_lower() == 'all'
-	dirs = run_command('sh', '-c',
-		'cd $MESON_SOURCE_ROOT/$MESON_SUBDIR && for d in * ; do if [ -d $d ] ; then echo $d ; fi ; done')
-	examples = dirs.stdout().split()
+	examples = all_examples
+	allow_skips = true # don't flag an error if we can't build an app
 else
 	examples = get_option('examples').split(',')
 	allow_skips = false # error out if we can't build a requested app
Signature · 1 line
2.20.1
#41 lore raw

PATCH v4 3/4 build: fix ninja install on FreeBSD

In reply to Bruce Richardson

The post-install script to symlink the PMDs from their own PMD directory to the regular lib directory (so they would be found by ld at runtime) was using the "-r" flag to ln to create relative symlinks. This flag is unsupported by ln on FreeBSD causing the ninja install step to fail.

Reworking the script to take the relative driver path as parameter removes the need for ln to calculate the relative path ensuring compatibility with FreeBSD.

As part of the fix, we move the registration of the install script to the config/meson.build file, from the top level one. This improves readability as the script takes as parameters the variables set in that file.

Fixes ed4d43d73e2b ("build: symlink drivers to library directory")
---
 buildtools/symlink-drivers-solibs.sh | 7 ++++---
 config/meson.build                   | 7 +++++++
 meson.build                          | 7 -------
 3 files changed, 11 insertions(+), 10 deletions(-)
buildtools/symlink-drivers-solibs.sh +4 -3
index 9826c6ae3..42985e855 100644
@@ -7,6 +7,7 @@
 # others, e.g. PCI device PMDs depending on the PCI bus driver.
 
 # parameters to script are paths relative to install prefix:
-# 1. directory containing driver files e.g. lib64/dpdk/drivers
-# 2. directory for installed regular libs e.g. lib64
-ln -rsf ${DESTDIR}/${MESON_INSTALL_PREFIX}/$1/* ${DESTDIR}/${MESON_INSTALL_PREFIX}/$2
+# 1. directory for installed regular libs e.g. lib64
+# 2. subdirectory of libdir where the pmds are
+
+cd ${MESON_INSTALL_DESTDIR_PREFIX}/$1 && ln -sfv $2/librte_*.so* .
config/meson.build +7 -0
index f8aded6ed..3678348de 100644
@@ -42,6 +42,13 @@ endif
 driver_install_path = join_paths(get_option('libdir'), pmd_subdir_opt)
 eal_pmd_path = join_paths(get_option('prefix'), driver_install_path)
 
+# driver .so files often depend upon the bus drivers for their connect bus,
+# e.g. ixgbe depends on librte_bus_pci. This means that the bus drivers need
+# to be in the library path, so symlink the drivers from the main lib directory.
+meson.add_install_script('../buildtools/symlink-drivers-solibs.sh',
+		get_option('libdir'),
+		pmd_subdir_opt)
+
 # set the machine type and cflags for it
 if meson.is_cross_build()
 	machine = host_machine.cpu()
meson.build +0 -7
index a96486597..d1e8e5239 100644
@@ -63,13 +63,6 @@ configure_file(output: build_cfg,
 # them.
 dpdk_drivers = ['-Wl,--whole-archive'] + dpdk_drivers + ['-Wl,--no-whole-archive']
 
-# driver .so files often depend upon the bus drivers for their connect bus,
-# e.g. ixgbe depends on librte_bus_pci. This means that the bus drivers need
-# to be in the library path, so symlink the drivers from the main lib directory.
-meson.add_install_script('buildtools/symlink-drivers-solibs.sh',
-		driver_install_path,
-		get_option('libdir'))
-
 pkg = import('pkgconfig')
 pkg.generate(name: meson.project_name(),
 	filebase: 'lib' + meson.project_name().to_lower(),
Signature · 1 line
2.20.1
#42 lore raw

PATCH v4 4/4 build: add libbsd to pkg-config file if enabled

In reply to Bruce Richardson

If libbsd is enabled in DPDK, the strlcpy and strlcat functions in rte_string_fns.h redirect to the varients in libbsd, only using the fallbacks if it is not enabled. Therefore, if libbsd is enabled, it needs to be called out as a DPDK dependency in the pkgconfig file.

To ensure that we don't have undefined variables on non-Linux platforms, we can remove the linux condition around the libbsd check - no harm comes in looking for it on other OS, since it's an optional dependency.

---
 config/meson.build | 10 ++++------
 meson.build        |  2 ++
 2 files changed, 6 insertions(+), 6 deletions(-)
config/meson.build +4 -6
index 3678348de..0d25646f5 100644
@@ -132,12 +132,10 @@ if numa_dep.found() and cc.has_header('numaif.h')
 	dpdk_extra_ldflags += '-lnuma'
 endif
 
-# check for strlcpy
-if is_linux
-	libbsd = dependency('libbsd', required: false)
-	if libbsd.found()
-		dpdk_conf.set('RTE_USE_LIBBSD', 1)
-	endif
+# check for libbsd
+libbsd = dependency('libbsd', required: false)
+if libbsd.found()
+	dpdk_conf.set('RTE_USE_LIBBSD', 1)
 endif
 
 # add -include rte_config to cflags
meson.build +2 -0
index d1e8e5239..46f9c5683 100644
@@ -70,6 +70,8 @@ pkg.generate(name: meson.project_name(),
 	libraries: dpdk_libraries,
 	libraries_private: dpdk_drivers + dpdk_static_libraries +
 			['-Wl,-Bdynamic'] + dpdk_extra_ldflags,
+	requires: libbsd, # apps using rte_string_fns.h may need this if enabled
+	                  # if libbsd is not enabled, then this is blank
 	description: '''The Data Plane Development Kit (DPDK).
 Note that CFLAGS might contain an -march flag higher than typical baseline.
 This is required for a number of static inline functions in the public headers.''',
Signature · 1 line
2.20.1
#43 lore raw

Re: [dpdk-dev] [PATCH v4 0/4] file meson compilation and install issues

In reply to Bruce Richardson

02/05/2019 18:51, Bruce Richardson:

Quoted 10 lines

Split off from the previous set to test the pkg-config file, this set includes only the fixes encountered during that work.

V4: Removed the changes to the test script and to hard-code the list of
example apps, so that only the more important patches are included.
V3: Minor style changes to patch 4 following review
V2: Fixed a number of other problems encountered on FreeBSD. Replaced
patch 2 of the original set, which broke on BSD with patch 6 of this set, which is the more correct solution.

Quoted 5 lines

Bruce Richardson (4):
examples/l3fwd: fix compile on FreeBSD
examples: install examples as part of ninja install
build: fix ninja install on FreeBSD
build: add libbsd to pkg-config file if enabled

Applied, thanks